Combining visible and near-infrared images for realistic skin smoothing

Skin tone images, portraits in particular, are of tremendous importance in digital photography, but a number of factors, such as pigmentation irregularities (e.g., moles, freckles), irritation, roughness, or wrinkles can reduce their appeal. Moreover, such “defects” are oftentimes enhanced by lighting conditions, e.g., when a flash is used. Starting with the observations that melanin and hemoglobin, the key components of skin colour, have little absorption in the near-infrared part of the spectrum, and that the depth of light penetration in the epidermis is proportional to the incident light’s wavelength, we propose that near-infrared images provide information that can be used to automatically smooth skin tones in a physically realistic manner. Specifically, we develop a framework that consists of capturing a pair of visible/near-infrared images and separating both of them into base and detail layers (akin to a low/high frequency decomposition) with the fast bilateral filter. We show that a smooth, realistic, output image can be obtained by fusing the base layer of the visible image with the near-infrared detail layer. This method not only outperforms equivalent decomposition in the wavelet domain, but the results also look more realistic than with a simple luminance transfer. Moreover, the proposed method delivers consistently good results across various skin types.