Using a Layer Mask to 'fix' a photo.

A feature that's been in Photoshop for years are layer masks, but most people I know seldom use these incredibly useful tools.

If you have used layers on Photoshop, you know that you can adjust the opacity of a layer - making it semi-transparent - but this feature affects the entire layer. There are times where you may want only part of a layer to be transparent, for this - there are layer masks. The concept is incredibly simple - a layer mask is attached to a layer - the layer mask is gray scale overlay where anything white in the layer mask is completely transparent and anything black is completely opaque, and anything in the grays is going to be either more or less transparent based on how light or dark this section is. You can use a layer mask to simply mask out a background, but to stop there leaves so much unexplored. Let's start by demonstrating how to mask out a flower in a layer.

Rose Heat

This is a photo of Pat Stamile's Hemoerocallis 'Rose Heat' taken at Benvenuti Garden in Houston at the AHS National convention in Houston in 2008.

The lawn behind and to the left does distracts from the overall image - competing somewhat for your attention.

One way to remedy this is to remove the bloom from the background entirely. This is one common use for a layer mask.

I'm going to start by using the Quick Select Tool to select the bloom on the right side.

Once I have that flower selected, I will create a layer mask from this selection. There may be a number of ways to do this but I'm going to show you how to do this in the layers palette.

The problem right now is that this image is on the background layer, which is the only layer that cannot have a layer mask applied to it, you must first convert that background layer to a layer that will allow the alpha channel which permits transparency levels and layer masks. The process is simple. Just double click the background layer in the layers palette and answer "OK"

Palette 1

When you double click the Background Layer and answer "OK" to convert it to a Layer, it renames it by default to Layer 0. Once you have converted the layer, you will see that at the bottom of the Layers Palette the Add Layer Mask icon is converted from a pale, unselectable gray to a darker, selectable icon.

This is the way to add a layer mask to a layer. You can add a layer mask to any layer except the background layer. If you choose this option with nothing selected it will add a layer mask and fill the layer mask with white - making the entire layer mask transparent, however, if you have something selected as we do now, the layer mask created will create an instant mask, making what's selected black in the layer mask, the rest white.

  Palette 2

first mask

The layer mask shows the flower as white, which makes it show through the mask, the background is rendered as black which makes the background transparent. Since there is nothing below this layer, the checkerboard pattern indicates that this area is now transparent.

This is the classic use of a layer mask, but you can do much more with them - let's try some tricks...

With Mask Applied

First, let's create a new layer by clicking on the New Layer icon at the bottom of the Layers palette - it's just to the left of the trash can icon. This creates a new layer above our layer with the flower - click on this layer in the layer palette, keep the mouse button held down and drag it below the flower. Set your foreground color to black and locate the Paint Bucket tool in the tool palette and fill this background layer with black. This makes the flower stand out nicely, but we've removed the background entirely - let's get some of that back now, because we're using a layer mask that's completely possible, the background is still there but masked out.

with out gradient  

Here's the image with a black filled layer positioned beneath it.

The next step is to click on the Paint Bucket tool and switch to the Gradient Tool. Choose white as your foreground color and in the options above for the Gradient Tool set the tool to paint the foreground color to transparent. We are going to paint a gradient of white on the layer mask starting on the right edge and moving left.

As this happens you'll notice that the background starts to appear again, if you want more background to show, apply the Gradient again.

I applied it twice, the image on the left below shows what I got. Below you can see the Layers palette and see the gradient in the mask area of the layer.

With Gradiant  
palette 4

We've now masked out the grass and attention is drawn to the bloom. This is one other way to use a layer mask - let's look at one more.

radial blur  

For this image, I filled the lower layer with white and applied the Radial Blur Filter (look under Filters-Blur-Radial Blur) to the mask itself.

I set the filter amount to 93 and shifted the center of the blur to be roughly over the throat of the flower.
radial blur control

I applied the blur multiple times observing the effect until I got what I wanted. Try this with a spider or a round bloom for very different effects as well..

The key here is that a mask can hide or reveal anything on a layer based on whether the pixels in the mask portion are white (transparent), black (opaque) or various shades of gray which can create various levels of transparency all on the same layer.

This is one to simply play with until you know what you can do with it, then keep it as a tool in your bag of tricks to be called on when you need it. Draw or paint on the layer mask itself with black and white brushes, try radial gradients, try applying textures to the mask, lots of interesting things can happen. Enjoy and have fun!

Tim Fehr - Eau Claire, WI

© 2008 by Tim Fehr - all rights reserved.