Month: August 2020

Liquid lenses are a helpful technology to overcome depth of field. Many cameras are often limited by the aperture size and resolution of the lens system, however with continuous focusability over a specified focal range, tunable liquid lenses allow for custom-tuning and optimal performance. Applications of tunable lenses include machine vision inspection systems, unmanned aerial vehicles, and measurement and dimensional rendering.

There are two main types of liquid lenses: focus-tunable lenses and variable focus liquid lenses. Focus-tunable lenses consist of a liquid optical-grade polymer that is mounted in a robust casing and is protected by two windows. These focus-tunable lenses can be either manually-focused by rotating the outer ring and manipulating the focal length, or optical power, which changes the shape from convex to concave, or they can be electrically-focused by applying a control voltage and changing the lens’ radius, which adjusts the focal length within a range of +50mm to +120mm. Variable focus liquid lenses operate via a slightly different process, called electrowetting, which is the use of electric fields to control the wetting properties by applying a voltage to increase the overall curvature, shape, and optical power of the liquid lens. Nevertheless, variable focus liquid lenses are not ideal to work with, since they are small and difficult to integrate with existing objectives., which is why I will focus on the former in this post.

Electrically-focused tunable lenses (Figure 1) provide a single-lens solution for focus and zoom objectives and eliminates the need for a multi-lens system. The lens costs between $300 and $500 and according to Edmund Optics, producer of optics and imaging technology, this device can replace the functionality of a complete lens kit in the laboratory setting. The lenses quickly adjust focus to accommodate objects located at differing working distances and thus are an ideal innovation for imaging applications that require rapid focusing, high throughput, and depth of field. 

As an example, in Stay Focused! Auto Focusing Lenses for Correcting Presbyopia, Elias Wu develops active glasses for attenuating presbyopia, far-sightedness that causes the loss of the eye’s ability to focus actively on nearby objects, by employing auto-focusing lenses (ie. focus-tunable lenses) alongside a calibration-free eye-tracking method (ie. TrackingNet). The setup consists of a ski goggle-like headset with an adjustable lens holder and adjustable eye tracking mount. This device takes in data from the eye tracking cameras and runs it through TrackingNet, a deep convolutional neural network, that implements pretrained models and a multi-layer fully-connected module. This software helps to streamline the auto-focusing lens procedure.

The goal of the paper is threefold: stabilizing the frame and mount so that users could wear it securely, rendering the lens and camera within the frame adjustable, and managing the cables needed to properly operate the headset. In these regards, Wu reports that overall, the headset satisfies these three criteria, however, in regards to the software, it does not yield the best results, which may be attributed to the use of the MSE loss function, when a simple L2 loss may be better. Moreover, the ResNet-50 features convolutional layers that are optimized for classification and thus may not be suited for this type of application in which the gaze position from images of the eyes needs to be found. Most notably, Wu found that the fluid-filled tunable lenses are quite bulky and also have visible distortion around the edges of the lens (Figure 2).

Figure 1: Focus-tunable liquid lenses (Edmund Optics)

Figure 2: Distortion along edges of auto-focusing lenses (Elias Wu)

Computational imaging-  involves the joint design of imaging system hardware and software to indirectly form images from measurements using algorithms and utilizing a significant amount of computing, overcoming hardware limitations of optics and sensors, such as resolution and noise [1].

Deep learning- a subset of machine learning. Allows machines to solve complex problems even when using a data set that is very diverse, unstructured and inter-connected. Involves two phases: training and evaluation of deep neural network [2].

TensorFlow- an interface for expressing machine learning algorithms, and an implementation for executing such algorithms [3].

Convolutional Neural Network- a class of deep neural networks, most commonly applied to analyzing visual imagery.

Ground Truth- In machine learning, the term “ground truth” refers to the accuracy of the training set’s classification for supervised learning techniques. The term “ground truthing” refers to the process of gathering the proper objective (provable) data for this test. Evaluation campaigns or benchmarking events require a consistent ground truth for accurate and reliable evaluation of the participating systems or methods [4].

Physical pre-processing technique- the function relating low-resolution images to their high-resolution counterparts is learned from a given dataset, which is then used to modify the imaging hardware. By using prior knowledge gained from static images, we avoid redundancy in information collection by physically preprocessing the data [5].

Super-resolution problem- converting low-resolution images to higher resolution.

Space–bandwidth-product (SBP)- a measure of the information a signal contains and also the rendering capacity of a system. Is proportional to the product of the field-of-view and resolution [2]. Includes spatial resolution, field-of-view and temporal resolution.

Spatial resolution– refers to the number of pixels within a digital image. The greater the number of pixels, the higher the spatial resolution.

Field-of-view- the maximum area visible through a camera lens or microscope eyepiece at a particular position and orientation in space.

Temporal resolution- the time to take the multiple measurements of the cross-section and then reconstruct the image.

High-throughput imaging- automated microscopy and image analysis to visualize and quantitatively capture specimens; or real-time live imaging.

Fourier ptychographic microscopy- a computational imaging technique in which variable illumination patterns are used to collect multiple low-resolution images which are then computationally combined to create an image with resolution exceeding that of any single image from the microscope, but has poor temporal resolution [5].

Light field microscopy- a technique that records four-dimensional slices of the five-dimensional plenoptic function by applying an appropriate ray-space parametrization. In many cases, microlens arrays (MLAs) are used in the intermediate image plane of optical instruments to multiplex 2D spatial and 2D directional information on the same image sensor [6].

3D phase microscopy: interferometry-based technique- images are first taken interferometrically to directly record both phase and amplitude information of the scattered field. Next, a tomographic reconstruction algorithm is devised to recover the sample’s 3D refractive index distribution.This typically requires additional specialized and expensive hardware, which is not always compatible to standard microscopes [7].

3D phase microscopy : intensity-only technique- employ tomographic phase reconstruction from intensity-only measurements.  However, changing focus not only requires mechanical scanning, but also increases the acquisition time and data size, both of which are undesirable for high-throughput applications [7].

Optical multiplexing- way of sending multiple signals or streams of information over a communications link at the same time in the form of a single, complex signal. A single LED illumination pattern that allows the information from multiple focal planes to be multiplexed into a single image [8].

References

[1] Waller, L. & Van Duzer, T. (2017). Computational Microscopy. CITRIS.

[2] Cheng, Y., Strachan, M., Weiss, Z., Deb, M., Carone, D., & Ganapati, V. (2018). Illumination Pattern Design with Deep Learning for Single-Shot Fourier Ptychographic Microscopy.

[3] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., … Zheng, X. (2016). TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems.

[4] Foncubierta Rodríguez, A., & Müller, H. (2012, October). Ground truth generation in medical imaging: a crowdsourcing-based iterative approach. In Proceedings of the ACM multimedia 2012 workshop on Crowdsourcing for multimedia, 9-14.

[5] Robey, A., & Ganapati, V. (2018). Optimal Physical Preprocessing for Example-Based Super-Resolution. https://doi.org/10.1364/OE.26.031333

[6] Bimber, O. & Schedl, D. (2019). Light-Field Microscopy: A Review. Journal of Neurology & Neuromedicine, (4)1, 1-6. https://doi.org/10.29245/2572.942X/2019/1.1237. 

[7] Ling, R., Tahir, W., Lin, H., Lee, H., & Tian, L. (2018). High-throughput intensity diffraction tomography with a computational microscope. Biomedical Optics Express, 9(5), 2130–2141. https://doi.org/10.1364/boe.9.002130

[8] Cheng, Y., Sabry, Z., Strachan, M., Cornell, S., Chanenson, J., Collins, E., & Ganapati, V. (2019). Deep Learned Optical Multiplexing for Multi-Focal Plane Microscopy.