It is generally accepted that using diffusion filters on the camera makes cutting a clean key more difficult, since they reduce edge sharpness. It is best to add any diffusion effect in post production.

Having your subject very close to the background, or lighting the green background with too much light, can cause the background color to reflect back onto your subject. If the green reflection is seen by the camera it can make it difficult for your keyer to distinguish between the background and the subject.

Here, in a screen capture from the editor, you can see the green fringing in the model's hair.

It has been found helpful to counter this effect by using hair and edge lights with gels of the opposing color to the background. Magenta will counter the spill from a green background, and shades of yellow will counter a blue background.

Shadows in the final image may actually be desired to help complete the illusion. If you replace the background color with the image of a close wall for example, it would be natural to have the shadow from a motivated light source, hitting the subject, to fall on it in real life.

Some keying systems can give you the choice of having no shadow (appropriate for mixing with backgrounds shot on an overcast day), or integrating the real shadow from your light on the subject, or even synthesizing a fake shadow in the keyer and adding it to the final combined shot.

The first shot shows a real shadow, created in lighting, projected on the green screen. Note the brighter right side of the subjects face is lit by the 300 Watt Omni-light used to make the shadow. The keyer must have enough sensitivity to accept both the light values of the shadow and the light filling the rest of the green screen. The second image shows the resulting keyed image, which is unnatural since the sunlight on the brick wall creates small shadows (most visible under the window sill) that seem to come from a different angle than the shadow added by the key.

The third image shows a shadow created synthetically in the keyer. The shadow can be moved to whatever axis gives the most realistic placement / position, which is an advantage. However, because there was no specific light source hitting the subject to create the shadow, the lighting on the subject can be unconvincing.

Having a clear idea of what you want the finished product to be will help you choose and shoot background and foreground images that work well together. Here you see an example of the light on the subject being motivated by a source shown in the background image that it will be combines with. Notice that the face of the subject appears to be lit by the glow of the light posts near the house, and her hair is lit by the suggestion of another nearby light post, behind her & out of the frame.

Since the background image is a night scene we wanted very little fill light. What stray illumination was bouncing off the light colored walls of our garage studio wound up being all that was needed.

-The green screen is again lit by two 300 Watt Omni-lights bouncing into Special / White Tota-brellas.

-The key light is a 250watt Pro-light with some CTO (Color Temperature Orange) gel across the barndoors to warm the color further and match the color of the light coming from the light posts in the background. It is positioned at an angle that would suggest the light is coming from the direction of the building.

-The edge highlight on the models hair is a Pro-light with a 125watt bulb. A scrim or Neutral Density gel could have been used for the same 'reduced level' effect if it had a 250watt bulb. This light did not have orange gel added. Besides suggesting that the light from this side might be coming from a street light, the light provides some separation from the background.

To further enhance the illusion try throwing the background image out of focus, either while shooting or in the effects of your editor, to imply depth.