Restoring Details When Edge Extending In Nuke

Avoid losing details when applying edge extensions, by using these simple techniques… 

Light And Dark Edges

When extracting a subject or an object from a scan, and then putting them/it over a new background, the edges of the subject/object may get contaminated by the original background.

– For example, when doing keying work, or when restoring parts of the scan back on top of a CG render. 

The softer the edge is (e.g. due to motion blur or defocus), the more of the original background will show through. And, usually, the contamination will appear as light or dark edges along the outline of your subject/object.

(1) Original image. (2) Subject placed over a new background, with the motion blurred edges being contaminated by the original background. 

Sometimes, you can go in and surgically grade away these issues. But often, there will be complex detail in the contamination, with a mix of both light and dark edges, and you’ll have to rebuild the new edge instead. 

In those cases, edge extension is a great technique for removing the edge contamination. 

Most edge extension tools work by eroding in the RGB edge until the contamination disappears, and then in one way or another push or stretch the ‘clean’ pixels back outward to fill in the gaps from the erosion. 

But edge extensions are typically quite rough-handed. As they stretch/smear/warp the pixels from the inner parts of the subject/object outward to cover the contaminated edge, they destroy any (RGB) texture and detail in the process. 

Often, we want to keep all of the nice, dark details, but just remove the light, contaminated edges – or vice versa. 

And there’s a very easy way to do that: 

Min/Max

After you have applied the edge extension (killing both light and dark details), merge back the wanted details with a min or a max operation.

(1) Original footage. (2) Subject placed over a new background with its original, bright edge. (3) Dark details near the edge getting destroyed by the edge extension. (4) Dark details recovered using a Merge (min) operation. 

For clarity, you would simply add a Merge (min or max) node with the A-pipe connected before the edge extension, and the B-pipe connected after the extension:

Using a Merge (min) to restore dark edge details after an edge extension. 

To keep the most amount of details when extending edges, remember: 

Edge Extend Only As Much As Necessary

Just grading an edge will mostly keep the original detail, but applying an edge extension to it won’t. 

Extending an edge means that you extrapolate and ‘invent’ new pixels. And so, always use edge extensions sparingly: 

Try to find the minimum amount that you have to stretch/smear/warp the edge, in order to retain as much detail as possible while still removing bright or dark edges. The min/max recovery operation should essentially only bring back the last little bits of detail. 

It’s a delicate balance. You’ll just want to nudge the values right inside of the edge slightly outwards, but avoid creating visible edge artefacts or losing too much detail.

And, make sure to mask the edge extension to only the areas which need it: 

Plate-On-Plate Edge Extensions

An easy trap to fall into when extending edges is to accidentally add plate-on-plate edge extensions. 

– And thus stretching or warping pixels in areas that aren’t bordering with the new background.

(1) Plate-on-plate edge extension – see the inner part of the shoulder getting warped (the extension is exaggerated to more clearly show the issue). (2) Masking the edge extension to avoid plate-on-plate edge extension – again, the extension is exaggerated (i.e. ignore the outer edge). 

So, ensure that you mask the edge extension by the (properly held out) CG, DMP, or other replaced BG. That way, you’re only extending edges that go over the areas of the background that have been replaced. 

This video explains it well: 

Alternative Method For Detail Restoration

Another way to restore lost detail after edge extending is to make use of frequency separation.  

By dividing the non-edge extended image by a slightly blurred version of itself, you can isolate the high frequency details in the image. 

Then, multiply that with the extended edges, masked by an edge matte, to restore the details in those areas:

Restoring detail by multiplying original high frequency detail back over the extended edges. In this case, using the Perimeter tool to create an edge matte. Mix back the Merge (multiply) as needed. 

You could also combine this method with another technique for extending edges: using a Rank filter.  

This video explains how the rank filter works: 

(It basically lets you suppress bright or dark edges based on a weight value). 

Finally, keep in mind that edge extensions may not always solve edge problems – in fact, they can be a source for new problems, like artefacts in the areas where the extension fails. 

So, make sure to thoroughly check your edges after edge extending them. 

I hope you found this tutorial useful. For more Nuke tips & tricks, see Nuke.