Robust optical autofocus system utilizing neural networks trained for extended range and time-course and automated multiwell plate imaging including single molecule localization microscopy

We present a robust, long-range optical autofocus system for microscopy utilizing machine learning. This can be useful for experiments with long image data acquisition times that may be impacted by defocusing resulting from drift of components, e.g. due to changes in temperature or mechanical drift. It is also useful for automated slide scanning or multiwell plate imaging where the sample(s) to be imaged may not be in the same horizontal plane throughout the image data acquisition. To address the impact of (thermal or mechanical) fluctuations over time in the optical autofocus system itself, we utilise a convolutional neural network (CNN) that is trained over multiple days to account for such fluctuations. To address the trade-off between axial precision and range of the autofocus, we implement orthogonal optical readouts with separate CNN training data, thereby achieving an accuracy well within the 600 nm depth of field of our 1.3 numerical aperture objective lens over a defocus range of up to approximately +/-100 μm. We characterise the performance of this autofocus system and demonstrate its application to automated multiwell plate single molecule localisation microscopy.

Download Full-text

Optical Projection Tomography Using a Commercial Microfluidic System

Micromachines ◽

10.3390/mi11030293 ◽

2020 ◽

Vol 11 (3) ◽

pp. 293

Author(s):

Wenhao Du ◽

Cheng Fei ◽

Junliang Liu ◽

Yongfu Li ◽

Zhaojun Liu ◽

...

Keyword(s):

Three Dimensional ◽

Numerical Aperture ◽

Microfluidic System ◽

Depth Of Field ◽

Objective Lens ◽

Flow Mode ◽

Wide Field ◽

Optical Projection Tomography ◽

Microfluidic Systems ◽

Optical Projection

Optical projection tomography (OPT) is the direct optical equivalent of X-ray computed tomography (CT). To obtain a larger depth of field, traditional OPT usually decreases the numerical aperture (NA) of the objective lens to decrease the resolution of the image. So, there is a trade-off between sample size and resolution. Commercial microfluidic systems can observe a sample in flow mode. In this paper, an OPT instrument is constructed to observe samples. The OPT instrument is combined with commercial microfluidic systems to obtain a three-dimensional and time (3D + T)/four-dimensional (4D) video of the sample. “Focal plane scanning” is also used to increase the images’ depth of field. A series of two-dimensional (2D) images in different focal planes was observed and compared with images simulated using our program. Our work dynamically monitors 3D OPT images. Commercial microfluidic systems simulate blood flow, which has potential application in blood monitoring and intelligent drug delivery platforms. We design an OPT adaptor to perform OPT on a commercial wide-field inverted microscope (Olympusix81). Images in different focal planes are observed and analyzed. Using a commercial microfluidic system, a video is also acquired to record motion pictures of samples at different flow rates. To our knowledge, this is the first time an OPT setup has been combined with a microfluidic system.

Download Full-text

Gaze Tracking Based on Concatenating Spatial-Temporal Features

Sensors ◽

10.3390/s22020545 ◽

2022 ◽

Vol 22 (2) ◽

pp. 545

Author(s):

Bor-Jiunn Hwang ◽

Hui-Hui Chen ◽

Chaur-Heh Hsieh ◽

Deng-Yu Huang

Keyword(s):

Short Term Memory ◽

Image Data ◽

Training Model ◽

Point Estimation ◽

Training Data ◽

Gaze Tracking ◽

Spatial Features ◽

Temporal Features ◽

Lstm Network ◽

The Impact

Based on experimental observations, there is a correlation between time and consecutive gaze positions in visual behaviors. Previous studies on gaze point estimation usually use images as the input for model trainings without taking into account the sequence relationship between image data. In addition to the spatial features, the temporal features are considered to improve the accuracy in this paper by using videos instead of images as the input data. To be able to capture spatial and temporal features at the same time, the convolutional neural network (CNN) and long short-term memory (LSTM) network are introduced to build a training model. In this way, CNN is used to extract the spatial features, and LSTM correlates temporal features. This paper presents a CNN Concatenating LSTM network (CCLN) that concatenates spatial and temporal features to improve the performance of gaze estimation in the case of time-series videos as the input training data. In addition, the proposed model can be optimized by exploring the numbers of LSTM layers, the influence of batch normalization (BN) and global average pooling layer (GAP) on CCLN. It is generally believed that larger amounts of training data will lead to better models. To provide data for training and prediction, we propose a method for constructing datasets of video for gaze point estimation. The issues are studied, including the effectiveness of different commonly used general models and the impact of transfer learning. Through exhaustive evaluation, it has been proved that the proposed method achieves a better prediction accuracy than the existing CNN-based methods. Finally, 93.1% of the best model and 92.6% of the general model MobileNet are obtained.

Download Full-text

High-numerical-aperture cryogenic light microscopy for increased precision of superresolution reconstructions

Proceedings of the National Academy of Sciences ◽

10.1073/pnas.1618206114 ◽

2017 ◽

Vol 114 (15) ◽

pp. 3832-3836 ◽

Cited By ~ 15

Author(s):

Marc Nahmani ◽

Conor Lanahan ◽

David DeRosier ◽

Gina G. Turrigiano

Keyword(s):

Single Molecule ◽

Room Temperature ◽

Fluorescent Protein ◽

Numerical Aperture ◽

Fluorescent Imaging ◽

Objective Lens ◽

High Numerical Aperture ◽

Superresolution Microscopy ◽

Superresolution Imaging ◽

Photoactivated Localization Microscopy

Superresolution microscopy has fundamentally altered our ability to resolve subcellular proteins, but improving on these techniques to study dense structures composed of single-molecule-sized elements has been a challenge. One possible approach to enhance superresolution precision is to use cryogenic fluorescent imaging, reported to reduce fluorescent protein bleaching rates, thereby increasing the precision of superresolution imaging. Here, we describe an approach to cryogenic photoactivated localization microscopy (cPALM) that permits the use of a room-temperature high-numerical-aperture objective lens to image frozen samples in their native state. We find that cPALM increases photon yields and show that this approach can be used to enhance the effective resolution of two photoactivatable/switchable fluorophore-labeled structures in the same frozen sample. This higher resolution, two-color extension of the cPALM technique will expand the accessibility of this approach to a range of laboratories interested in more precise reconstructions of complex subcellular targets.

Download Full-text

Super-resolving light fields in microscopy: depth from disparity

10.1101/527432 ◽

2019 ◽

Author(s):

Ruth R. Sims ◽

Sohaib Abdul Rehman ◽

Martin O. Lenz ◽

Leila Mureşan ◽

Kevin O'Holleran

Keyword(s):

Single Molecule ◽

Light Field ◽

Super Resolution ◽

Depth Of Field ◽

Source Position ◽

Multiple Perspective ◽

Point Emitter ◽

Simulation And Experiment ◽

Resolution Imaging ◽

Localisation Microscopy

Single molecule localisation microscopy (SMLM) has opened a new window for imaging fluorescently labelled biological specimens. Common 3D SMLM techniques enable data collection across an axial range of 1 - 5μm with high precision. Despite the success of 3D single molecule imaging there is a real need to image larger volumes. Here we demonstrate, through simulation and experiment, the potential of Single Molecule Light Field Microscopy (SMLFM) for extended depth-of-field super-resolution imaging, extracting 3D point source position by measuring the disparity between localizations of a point emitter in multiple perspective views.

Download Full-text

Nanometric plasmonic optical trapping on gold nanostructures

The European Physical Journal Applied Physics ◽

10.1051/epjap/2019180347 ◽

2019 ◽

Vol 86 (3) ◽

pp. 30501

Author(s):

Domna G. Kotsifaki ◽

Mersini Makropoulou ◽

Alexander A. Searfetinides

Keyword(s):

Optical Tweezers ◽

Single Molecule ◽

Optical Trapping ◽

Numerical Aperture ◽

Resonance Wavelength ◽

Objective Lens ◽

Plasmonic Nanostructures ◽

Theoretical Support ◽

Trapping Force ◽

Trapping Forces

The precise noninvasive optical manipulation of nanometer-sized particles by evanescent fields, instead of the conventional optical tweezers, has recently awaken an increasing interest, opening a way for investigating phenomena relevant to both fundamental and applied science. In this work, the optical trapping force exerted on trapped dielectric nanoparticle was theoretically investigated as a function on the trapping beam wavelength and as a function of several plasmonic nanostructures schemes based on numerical simulation. The maximum optical trapping forces are obtained at the resonance wavelength for each plasmonic nanostructure geometry. Prominent tunabilities, such as radius and separation of gold nanoparticles as well as the numerical aperture of objective lens were examined. This work will provide theoretical support for developing new types of plasmonic sensing substrates for exciting biomedical applications such as single-molecule fluorescence.

Download Full-text

High-speed brightfield 3-D imaging and 3-D image analysis of thick and overlapped cell clusters in cytological preparations

Proceedings, annual meeting, Electron Microscopy Society of America ◽

10.1017/s0424820100168827 ◽

1994 ◽

Vol 52 ◽

pp. 218-219

Author(s):

Robert W. Mackin

Keyword(s):

Image Analysis ◽

High Speed ◽

Three Dimensional ◽

Image Data ◽

Ccd Camera ◽

Objective Lens ◽

Array Processor ◽

Vaginal Smears ◽

Low Contrast ◽

Computer Controlled

This paper presents two advances towards the automated three-dimensional (3-D) analysis of thick and heavily-overlapped regions in cytological preparations such as cervical/vaginal smears. First, a high speed 3-D brightfield microscope has been developed, allowing the acquisition of image data at speeds approaching 30 optical slices per second. Second, algorithms have been developed to detect and segment nuclei in spite of the extremely high image variability and low contrast typical of such regions. The analysis of such regions is inherently a 3-D problem that cannot be solved reliably with conventional 2-D imaging and image analysis methods.High-Speed 3-D imaging of the specimen is accomplished by moving the specimen axially relative to the objective lens of a standard microscope (Zeiss) at a speed of 30 steps per second, where the stepsize is adjustable from 0.2 - 5μm. The specimen is mounted on a computer-controlled, piezoelectric microstage (Burleigh PZS-100, 68/μm displacement). At each step, an optical slice is acquired using a CCD camera (SONY XC-11/71 IP, Dalsa CA-D1-0256, and CA-D2-0512 have been used) connected to a 4-node array processor system based on the Intel i860 chip.

Download Full-text

Tandem scanning confocal light microscopy as compared with x-ray diffraction for characterizing the behavior of clays in fluid-saturated petroleum reservoir rock

Proceedings, annual meeting, Electron Microscopy Society of America ◽

10.1017/s0424820100152719 ◽

1989 ◽

Vol 47 ◽

pp. 148-149

Author(s):

C.J. Stuart ◽

B.E. Viani ◽

J. Walker ◽

T.H. Levesque

Keyword(s):

Light Microscopy ◽

Image Plane ◽

Reservoir Rock ◽

Depth Of Field ◽

Objective Lens ◽

Scanning System ◽

Light Sources ◽

Petroleum Reservoir ◽

Image Recording ◽

Scan Speed

Many techniques of imaging used to characterize petroleum reservoir rocks are applied to dehydrated specimens. In order to directly study behavior of fines in reservoir rock at conditions similar to those found in-situ these materials need to be characterized in a fluid saturated state.Standard light microscopy can be used on wet specimens but depth of field and focus cannot be obtained; by using the Tandem Scanning Confocal Microscope (TSM) images can be produced from thin focused layers with high contrast and resolution. Optical sectioning and extended focus images are then produced with the microscope. The TSM uses reflected light, bulk specimens, and wet samples as opposed to thin section analysis used in standard light microscopy. The TSM also has additional advantages: the high scan speed, the ability to use a variety of light sources to produce real color images, and the simple, small size scanning system. The TSM has frame rates in excess of normal TV rates with many more lines of resolution. This is accomplished by incorporating a method of parallel image scanning and detection. The parallel scanning in the TSM is accomplished by means of multiple apertures in a disk which is positioned in the intermediate image plane of the objective lens. Thousands of apertures are distributed in an annulus, so that as the disk is spun, the specimen is illuminated simultaneously by a large number of scanning beams with uniform illumination. The high frame speeds greatly simplify the task of image recording since any of the normally used devices such as photographic cameras, normal or low light TV cameras, VCR or optical disks can be used without modification. Any frame store device compatible with a standard TV camera may be used to digitize TSM images.

Download Full-text

Microspectroscopy Using a Solid Immersion Lens

Microscopy and Microanalysis ◽

10.1017/s1431927600026817 ◽

2001 ◽

Vol 7 (S2) ◽

pp. 148-149

Author(s):

C.D. Poweleit ◽

J Menéndez

Keyword(s):

Spatial Resolution ◽

Focal Plane ◽

Numerical Aperture ◽

Normal Incidence ◽

Spot Size ◽

Index Of Refraction ◽

Objective Lens ◽

Improvement Factor ◽

Oil Immersion ◽

Long Time

Oil immersion lenses have been used in optical microscopy for a long time. The light’s wavelength is decreased by the oil’s index of refraction n and this reduces the minimum spot size. Additionally, the oil medium allows a larger collection angle, thereby increasing the numerical aperture. The SIL is based on the same principle, but offers more flexibility because the higher index material is solid. in particular, SILs can be deployed in cryogenic environments. Using a hemispherical glass the spatial resolution is improved by a factor n with respect to the resolution obtained with the microscope’s objective lens alone. The improvement factor is equal to n2 for truncated spheres.As shown in Fig. 1, the hemisphere SIL is in contact with the sample and does not affect the position of the focal plane. The focused rays from the objective strike the lens at normal incidence, so that no refraction takes place.

Download Full-text

Mechanically-Tunable Quantum Interference in Ferrocene-Based Single-Molecule Junctions

10.26434/chemrxiv.12252059 ◽

2020 ◽

Author(s):

María Camarasa-Gómez ◽

Daniel Hernangómez-Pérez ◽

Michael S. Inkpen ◽

Giacomo Lovat ◽

E-Dean Fung ◽

...

Keyword(s):

Single Molecule ◽

Quantum Interference ◽

Degrees Of Freedom ◽

Building Blocks ◽

Ferrocene Derivative ◽

Single Molecule Junctions ◽

Molecule Junction ◽

Single Molecule Junction ◽

The Impact ◽

D Orbitals

Ferrocenes are ubiquitous organometallic building blocks that comprise a Fe atom sandwiched between two cyclopentadienyl (Cp) rings that rotate freely at room temperature. Of widespread interest in fundamental studies and real-world applications, they have also attracted some interest as functional elements of molecular-scale devices. Here we investigate the impact of the configurational degrees of freedom of a ferrocene derivative on its single-molecule junction conductance. Measurements indicate that the conductance of the ferrocene derivative, which is suppressed by two orders of magnitude as compared to a fully conjugated analog, can be modulated by altering the junction configuration. Ab initio transport calculations show that the low conductance is a consequence of destructive quantum interference effects that arise from the hybridization of metal-based d-orbitals and the ligand-based π-system. By rotating the Cp rings, the hybridization, and thus the quantum interference, can be mechanically controlled, resulting in a conductance modulation that is seen experimentally.

Download Full-text

Monitoring the Structural Dynamics of U2 snRNA Via Single-Molecule Förster Resonance Energy Transfer

10.31237/osf.io/9j8gq ◽

2018 ◽

Author(s):

Alexander Carl DeHaven

Keyword(s):

Energy Transfer ◽

Structural Dynamics ◽

Single Molecule ◽

Conformational Dynamics ◽

Resonance Energy Transfer ◽

Resonance Energy ◽

Förster Resonance Energy Transfer ◽

U2 Snrna ◽

Folding Dynamics ◽

The Impact

This thesis contains four topic areas: a review of single-molecule microscropy methods and splicing, conformational dynamics of stem II of the U2 snRNA, the impact of post-transcriptional modifications on U2 snRNA folding dynamics, and preliminary findings on Mango aptamer folding dynamics.

Download Full-text