3D fluorescence microscopy data synthesis for segmentation and benchmarking

Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general lack of those annotated data sets. In this work, we propose how conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy from annotation masks of 3D cellular structures. In combination with mask simulation approaches, we demonstrate the generation of fully-annotated 3D microscopy data sets that we make publicly available for training or benchmarking. An additional positional conditioning of the cellular structures enables the reconstruction of position-dependent intensity characteristics and allows to generate image data of different quality levels. A patch-wise working principle and a subsequent full-size reassemble strategy is used to generate image data of arbitrary size and different organisms. We present this as a proof-of-concept for the automated generation of fully-annotated training data sets requiring only a minimum of manual interaction to alleviate the need of manual annotations.

Correlative Organelle Microscopy: fluorescence guided volume electron microscopy of intracellular processes

Intracellular processes depend on a strict spatial and temporal organization of proteins and organelles. Directly linking molecular to nanoscale ultrastructural information is therefore crucial to understand cellular physiology. Volume or 3-dimensional (3D) correlative light and electron microscopy (volume-CLEM) holds unique potential to explore cellular physiology at high-resolution ultrastructural detail across cell volumes. Application of volume-CLEM is however hampered by limitations in throughput and 3D correlation efficiency. Addressing these limitations, we here describe a novel pipeline for volume-CLEM that provides high-precision (<100nm) registration between 3D fluorescence microscopy (FM) and 3D electron microscopy (EM) data sets with significantly increased throughput. Using multi-modal fiducial nanoparticles that remain fluorescent in epoxy resins and a 3D confocal fluorescence microscope integrated in a Focused Ion Beam Scanning Electron Microscope (FIB.SEM), our approach uses FM to target extremely small volumes of even single organelles for imaging in volume-EM, and obviates the need for post correlation of big 3D datasets. We extend our targeted volume-CLEM approach to include live-cell imaging, adding information on the motility of intracellular membranes selected for volume-CLEM. We demonstrate the power of our approach by targeted imaging of rare and transient contact sites between endoplasmic reticulum (ER) and lysosomes within hours rather than days. Our data suggest that extensive ER-lysosome and mitochondria-lysosome interactions restrict lysosome motility, highlighting the unique capabilities of our integrated CLEM pipeline for linking molecular dynamic data to high-resolution ultrastructural detail in 3D.

A dynamic, ring-forming MucB / RseB-like protein influences spore shape in Bacillus subtilis

How organisms develop into specific shapes is a central question in biology. The maintenance of bacterial shape is connected to the assembly and remodelling of the cell envelope. In endospore-forming bacteria, the pre-spore compartment (the forespore) undergoes morphological changes that result in a spore of defined shape, with a complex, multi-layered cell envelope. However, the mechanisms that govern spore shape remain poorly understood. Here, using a combination of fluorescence microscopy, quantitative image analysis, molecular genetics and transmission electron microscopy, we show that SsdC (formerly YdcC), a poorly-characterized new member of the MucB / RseB family of proteins that bind lipopolysaccharide in diderm bacteria, influences spore shape in the monoderm Bacillus subtilis. Sporulating cells lacking SsdC fail to adopt the typical oblong shape of wild-type forespores and are instead rounder. 2D and 3D-fluorescence microscopy suggest that SsdC forms a discontinuous, dynamic ring-like structure in the peripheral membrane of the mother cell, near the mother cell proximal pole of the forespore. A synthetic sporulation screen identified genetic relationships between ssdC and genes involved in the assembly of the spore coat. Phenotypic characterization of these mutants revealed that spore shape, and SsdC localization, depend on the coat basement layer proteins SpoVM and SpoIVA, the encasement protein SpoVID and the inner coat protein SafA. Importantly, we found that the ΔssdC mutant produces spores with an abnormal-looking cortex, and abolishing cortex synthesis in the mutant largely suppresses its shape defects. Thus, SsdC appears to play a role in the proper assembly of the spore cortex, through connections to the spore coat. Collectively, our data suggest functional diversification of the MucB / RseB protein domain between diderm and monoderm bacteria and identify SsdC as an important factor in spore shape development.

Miniscope3D: optimized single-shot miniature 3D fluorescence microscopy

Miniature fluorescence microscopes are a standard tool in systems biology. However, widefield miniature microscopes capture only 2D information, and modifications that enable 3D capabilities increase the size and weight and have poor resolution outside a narrow depth range. Here, we achieve the 3D capability by replacing the tube lens of a conventional 2D Miniscope with an optimized multifocal phase mask at the objective’s aperture stop. Placing the phase mask at the aperture stop significantly reduces the size of the device, and varying the focal lengths enables a uniform resolution across a wide depth range. The phase mask encodes the 3D fluorescence intensity into a single 2D measurement, and the 3D volume is recovered by solving a sparsity-constrained inverse problem. We provide methods for designing and fabricating the phase mask and an efficient forward model that accounts for the field-varying aberrations in miniature objectives. We demonstrate a prototype that is 17 mm tall and weighs 2.5 grams, achieving 2.76 μm lateral, and 15 μm axial resolution across most of the 900 × 700 × 390 μm3 volume at 40 volumes per second. The performance is validated experimentally on resolution targets, dynamic biological samples, and mouse brain tissue. Compared with existing miniature single-shot volume-capture implementations, our system is smaller and lighter and achieves a more than 2× better lateral and axial resolution throughout a 10× larger usable depth range. Our microscope design provides single-shot 3D imaging for applications where a compact platform matters, such as volumetric neural imaging in freely moving animals and 3D motion studies of dynamic samples in incubators and lab-on-a-chip devices.

Network-enabled efficient image restoration for 3D microscopy of turbid biological specimens

Though three-dimensional (3D) fluorescence microscopy has been an essential tool for modern life science research, the light scattering by biological specimens fundamentally prevents its more widespread applications in live imaging. We hereby report a deep-learning approach, termed ScatNet, that enables reversion of 3D fluorescence microscopy from high-resolution targets to low-quality, light-scattered measurements, thereby allowing restoration for a single blurred and light-scattered 3D image of deep tissue, with achieving improved resolution and signal-to-noise ratio. Our approach can computationally extend the imaging depth for current 3D fluorescence microscopes, without the addition of complicated optics. Combining ScatNet approach with cutting-edge light-sheet fluorescence microscopy, we demonstrate that the image restoration of cell nuclei in the deep layer of live Drosophila melanogaster embryos at single-cell resolution. Applying our approach to two-photon excitation microscopy, we could improve the signal and resolution of neurons in mouse brain beyond the photon ballistic region.