Notes on “Deep Learned Optical Multiplexing for Multi-Focal Plane Microscopy”

As was addressed in the last paper about FPM, it takes a long time and usually costly to scan a specimen and produce a high quality picture. This can be solved by reconstructing 69 photos taken under different LED pattern and using deep learning, the microscope can be trained to be able to produce a high-quality picture by taking one best photo, which shortens the time by a factor of 69.

However, such methods only satisfy situations where the sample given is thin enough so that the camera only needs to focus once. When encountering with circumstances where the sample is thick, the picture taken can be blurred due to the different distance each layer is from the camera. To improve the microscope’s functionality, digital refocusing is required during the process of reconstructing and training.

With assumptions of geometric optics, the 3-dimensional is assumed to modulate only the intensity instead of the phase. Instead of adding all the photos taken from which the obtained image would simply be that when all LED lights are on, the LED stack was sheared and resulting images were then added together. The shift-add algorithm would produce an example as shown below.

After completing the shift-add algorithm, the microscope is trained by using deep learning to achieve the goal of producing high-resolution picture in one shot of the photo, increasing the speed of scanning. Deep learning is divided up into training and evaluation and the algorithm is shown below.

Using this type of training and fine-tuning, the experiment is able to produce a result as shown in the graph.

References: