Lensless polarization camera for single-shot full-Stokes imaging

A. Device construction

Our lensless camera is composed of a CMOS image sensor (CIS), a phase mask, and a polarization-encoded aperture composed of three LPs and a QWP [Fig. 2(a)]. In accordance with the mask-based lensless imaging scheme,29,3029. N. Antipa, G. Kuo, R. Heckel, B. Mildenhall, E. Bostan, R. Ng, and L. Waller, “Diffusercam: Lensless single-exposure 3d imaging,” Optica 5, 1–9 (2018). https://doi.org/10.1364/optica.5.00000130. V. Boominathan, J. K. Adams, J. T. Robinson, and A. Veeraraghavan, “Phlatcam: Designed phase-mask based thin lensless camera,” IEEE Trans. Pattern Anal. Mach. Intell. 42, 1618–1629 (2020). https://doi.org/10.1109/tpami.2020.2987489 we use a transparent, weak-diffuser film as the phase mask, which creates a pseudo-random 2D PSF, placed in front of an image sensor. The phase mask is fabricated with polydimethylsiloxane (PDMS) elastomer by casting PDMS on a commercial diffuser (0.5° Light Shaping Diffuser Sheet, Luminit) via a simple soft-lithography process.3737. Y. Lee, H. Chae, K. C. Lee, N. Baek, T. Kim, J. Jung, and S. A. Lee, “Fabrication of integrated lensless cameras via UV-imprint lithography,” IEEE Photonics J. 14, 1–8 (2022). https://doi.org/10.1109/jphot.2022.3157373 The casted PDMS-based phase mask does not have any birefringent properties as opposed to the commercial polycarbonate film diffusers, which may affect the polarization-dependent PSFs of the FS-LPC (see Sec. S1 of the supplementary material). Below the phase mask, we place a polarization-encoded aperture with 0°, 45°, and 90° wire-grid LP films (No. 33-084, Edmund Optics) and a right-handed circular polarizer (RHCP) made with a 45° wire-grid LP and a polymer-based QWP film (No. 88-253, Edmund Optics) with its fast-axis oriented at 0°. The polarizers and the waveplate are carefully aligned such that there is no gap between each component and the shift-invariance of the PSF is maintained [Fig. S2(b)]. The polarizer assembly is masked by a blackout aluminum tape to a square aperture of 3 × 3 mm2 with a total thickness of 240 μm. The size of the aperture is designed in consideration of the lateral field-of-view (FOV) of the camera (see supplementary material Sec. S2 for design details). Combined with the PDMS phase mask, this aperture essentially divides the phase mask into four sections and encodes each polarization light coming from the scene with distinct PSF patterns [Fig. 1(c)]. The phase mask creates a sharp, caustic PSF ∼6 mm from the mask surface; so, we use a 3D-printed spacer to align and assemble the polarization-encoded phase mask with a commercial image sensor (IMX477, 12.3 M pixels, 1.55-μm pixel size, Sony). Figure 2(b) shows our prototype device; the effective thickness of optics is ∼400 μm, and the total thickness of the device is 6.4 mm.The polarization-dependent PSFs of the camera are measured prior to imaging and used in the image reconstruction process. We measured the PSF of the camera with unpolarized point source illumination (a white LED, MCWHL7, Thorlabs, with a 75 μm-pinhole) placed at the working distance (WD) of 40 cm from the camera. Then, we spatially divided the unpolarized PSF into four regions according to the position of the polarization-encoded apertures, creating a four-channel stack of polarization-dependent PSFs [Fig. 1(c)]. The measured PSFs show shift invariance in all three dimensions, within the designed lateral imaging FOV of ∼45° and the axial range of the WD of 20–60 cm. The overall imaging FOV and the spatial resolution of the camera are evaluated experimentally (see Secs. S2 and S3 of the supplementary material). In short, the camera can image an angular FOV of 47° × 39°, and the measured two-point angular resolution, which represents the highest achievable spatial resolution in our lensless camera, is 0.135° for the same polarization and 0.105° for different (90° LP and RHCP) polarizations.

B. Image reconstruction

In lensless cameras using weak-diffuser phase masks, the imaging forward model is formulated using a simple 2D convolution between the scene and the PSF. In FS-LPC, the raw measurement captured by the image sensor can be modeled as a summation of 2D-convolutions between the PSF of each polarization channel and the intensity of the scene at the corresponding polarization. This forward model can be expressed as b(x, y) = ∑jpj(x, y)*v(x, y; j), where b is the sensor’s measurement, pj is the PSF of the jth polarization channel, v(x, y;j) is the polarization-sensitive intensities of the scene, and * denotes 2D convolution over (x, y). Using the pre-measured PSF stack, the four polarization-dependent intensity images can be recovered from a single snapshot measurement of b. This multiplexed image reconstruction can be performed by solving the following minimization problem with the total variation regularization3838. L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D 60, 259–268 (1992). https://doi.org/10.1016/0167-2789(92)90242-f and the non-negativity prior on the scene, v.v̂=argminv≥012b−∑jpj*vj22+τΨ(v)1.(1)Here, τ is the weight of total variation regularization in the spatial domain. Equation (1) can be solved using optimization algorithms such as alternating direction method of multipliers (ADMM).3939. S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein et al., “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learn. 3, 1–122 (2011). https://doi.org/10.1561/2200000016 All images are reconstructed using an ADMM-based reconstruction algorithm with fine-tuning of the total variation regularizer, and the reconstruction time for 507 × 380 × 4 images is up to 24 s using Nvidia RTX3090 with Intel Xeon GOLD 6242 CPU under Ubuntu 20.04.3 LTS. After image reconstruction, the Stokes parameter images and other polarization parameters, such as degree of polarization (DoP), angle of linear polarization (AoLP), and ellipticity angle (EA), were computed to visualize the polarization information of the scene.To test the full-Stokes imaging capabilities of our system, we made custom polarization targets where six 30 × 30 mm2 polarizing films were placed in front of bright, diffusely reflecting surfaces [Fig. 3(a)]. The targets include four LP targets designed to have 0°, 45°, 90°, and 135° LP and two CP targets with RHCP and LHCP, made with linear and circular polarizing films (No. 14-344, No. 11-047, No. 88-084, Edmund Optics). For imaging, we illuminated polarization targets with an unpolarized, white LED (MCWHL7, Thorlabs) and captured the raw image in the sensor at the WD of 40 cm, as shown in Fig. 3(b). Figure 3(c) shows reconstructed intensity images of the polarization target objects corresponding to the intensities of the targets in 0°, 45°, 90° LP and RHCP channels. According to the arrangement of the polarizers in the aperture of the lensless camera, the reconstructed pixel intensity values appear close to 1 on the target objects with the same polarization as the channel, while the objects with polarization perpendicular to the channel do not appear in the image.To convert the recovered intensity images into Stokes parameter images, we use the following Stokes intensity formula, which represents the polarization-dependent intensity images according to the angle of the polarizer (θ) and the phase delay (ϕ) of the retarder.I(θ,ϕ)=12S0+S1⁡cos⁡2θ+S2⁡cos⁡ϕ⁡sin⁡2θ+S3⁡sin⁡ϕ⁡sin⁡2θ.(2)Specifically, I(θ, ϕ) is to represent pixel-wise intensities of the restored 2D image stack v̂(x,y;j) from Eq. (1). From Eq. (2), the 0°, 45°, and 90° LP, and RHCP intensities can be written as a simple, linear combination of Stokes parameters with θ = (0°, 45°, 90°, and 45°) and ϕ = (0°, 0°, 0°, and 90°), respectively. Then, we can define the ideal analysis matrix W that converts the four polarized intensities, I, into the Stokes vector as S = WI, whereW=101010−10−12−10−10−12.(3)This conversion is applied on all pixels of the images to reconstruct four Stokes parameter images as shown in Fig. 3(d). All Stokes parameter images show high-contrast values between (+1 and −1), highlighting a pair of perpendicular polarizers in each channel. Note that the Stokes parameter images reveal unwanted fluctuations in the background regions near the polarization objects that arise from the image reconstruction artifacts that cause small errors in the background regions (close to 0) of the intensity images. These reconstruction artifacts are more evident in the polarization target images because of the high contrast of the objects, and are less problematic in real-object scenes, where the polarization-dependent features are rather smooth.From the calculated Stokes parameters, we can also compute the DoP, AoLP, and EA using the formulas below:1212. D. H. Goldstein, Polarized Light (CRC Press, 2017).DoP=S12+S22+S32S0,AoLP=12tan−1S2S1,EA=12sin−1S3S0+S12+S22.(4)Specifically, DoP indicates the amount of polarization in the total reflected intensity, AoLP specifies the azimuth angle of the linear polarization, and EA indicates the degree and the direction of circular polarization. In Fig. 3(e), the DoP shows intensity close to +1 in all target objects, AoLP shows close to 45°, 90°, and −45° values, indicating the direction of each LP, and the EA shows +45° and −45° values, indicating the handedness of the CP targets.

C. Stokes parameter calibration

The ideal analysis matrix W in Eq. (3) is derived assuming that the polarizers and the wave plates mounted in the cameras are ideal, and each channel measures perfect intensity corresponding to the 0°, 45°, and 90° LP or RHCP components of the scene. However, the transmittance and the extinction coefficients of the polarizers and the wave plates are not perfect, and, thus, the analysis matrix needs to be calibrated in order to obtain the correct Stokes parameters of the scene. In the imaging process, the ground-truth Stokes vector of the scene, SGT, is modified by the polarizer assembly in the lensless camera to produce SLensless, the measured Stokes vector from our lensless polarization camera. The polarization-modifying behavior of the lensless camera can be expressed in terms of a Mueller matrix of the FS-LPC, M, and can be written as SLensless = MSGT.Instead of measuring each element of M of the polarizer assembly, we calibrated the analysis matrix directly by finding the relationship between SGT and ILensless, the reconstructed intensity measurements from our FS-LPC. Subsequently, the calibrated analysis matrix W′ can be obtained as follows:SGT=M−1SLensless,=M−1WILensless,=W′ILensless.(5)Given the lack of an ideal polarization target with known SGT, we calibrated our lensless polarization camera against a separate lensed full-Stokes polarization imaging system using a commercial polarization camera. A lensed polarization camera using a 5 MP monochromatic polarization image sensor (Blackfly 3, FLIR) with a C-Mount lens (TV Lens f = 16 mm, 1:1.4, Pentax) was used to image the scene with and without a high-performance QWP (No. 88-198, Edmund Optics) to measure what we consider as the ground-truth Stokes parameter images of the scene. SGT is calculated based on the eight intensity measurements (θ = 0°, 45°, 90°, and 135°and ϕ = 0° and 90°) from the lensed polarization camera with an 8 × 4 ideal analysis matrix. At least four independent measurements of SGT and ILensless are required in order to fit W′ from the measured values. The six-target object in Fig. 3(a) was imaged with both systems to obtain six average SGT and ILensless values, from which we find the calibrated analysis matrix W′ as below:W′=1.12550.14951.0786−0.15160.9878−0.3806−0.65670.2752−0.88141.7652−0.89860.0747−1.1177−0.0769−0.68691.9497.(6)Compared to the ideal analysis matrix W in Eq. (3), we can identify some discrepancies in the element-wise values of the analysis matrix, caused by the non-ideal polarization properties, possible misalignment of the polarizers and the QWPs, and the cross-talk between different polarization channels in our lensless camera. However, using W′ for Stokes imaging corrects for the effect of these imperfections and produces calibrated Stokes parameters close to the ground-truth values.Using the calibrated analysis matrix, we verified the accuracy of the Stokes parameter measurements using a separate set of images of the rearranged polarization targets (Fig. 4). We compared the calibrated Stokes parameter images, as well as the polarization parameter images, against the ground truth images measured with the lensed camera. Images computed using the ideal analysis matrix [Figs. 4(a) and 4(d)] and the calibrated analysis matrix [Figs. 4(b) and 4(e)] are compared against the images taken with the ground-truth polarization camera [Figs. 4(c) and 4(f)]. After the calibration, the discrepancies between SGT and SLensless are reduced. For example, 135° LP and RHCP objects in the S1 images (marked with yellow arrows in Fig. 4) are adjusted to have small positive S1 values similar to the ground-truth Stokes parameters. S3 values of 90° and 0° LPs (marked with green arrows in Fig. 4), as well as the EA values (black arrows in Fig. 4), also become close to the ground-truth values. As a result, the DoP, AoLP, and EA images show small adjustments, as shown in Figs. 4(d)4(f). Overall, the root mean square error (RMSE) between SGT and SLensless was reduced from 0.12 to 0.078 before and after the calibration.It is worth noting that the Stokes parameters measured with our FS-LPC may not be perfect because SGT used in the calibration process is also measured from a separate polarization imaging system, which may also contain imperfections in its own measurements. Repeating the above process with known, ideal polarization targets (if available) or calibrating our measurement against another fully-calibrated polarization imaging system can improve the accuracy of our Stokes parameters. In addition, the reconstructed image quality, as well as the accuracy of the Stokes parameters, will be dependent on the amount of noise in the measurement and the amount of regularization applied in the reconstruction. The analysis on the effect of noise and scene complexity on the Stokes parameter measurement based on image simulations can be found in Sec. S5 of the supplementary material.

留言 (0)

沒有登入
gif