Automated Detection, Classification and Counting of Fish in Fish Passages With Deep Learning

The Ocean Aware project, led by Innovasea and funded through Canada's Ocean Supercluster, is developing a fish passage observation platform to monitor fish without the use of traditional tags. This will provide an alternative to standard tracking technology, such as acoustic telemetry fish tracking, which are often not appropriate for tracking at-risk fish species protected by legislation. Rather, the observation platform uses a combination of sensors including acoustic devices, visual and active sonar, and optical cameras. This will enable more in-depth scientific research and better support regulatory monitoring of at-risk fish species in fish passages or marine energy sites. Analysis of this data will require a robust and accurate method to automatically detect fish, count fish, and classify them by species in real-time using both sonar and optical cameras. To meet this need, we developed and tested an automated real-time deep learning framework combining state of the art convolutional neural networks and Kalman filters. First, we showed that an adaptation of the widely used YOLO machine learning model can accurately detect and classify eight species of fish from a public high resolution DIDSON imaging sonar dataset captured from the Ocqueoc River in Michigan, USA. Although there has been extensive research in the literature identifying particular fish such as eel vs. non-eel and seal vs. fish, to our knowledge this is the first successful application of deep learning for classifying multiple fish species with high resolution imaging sonar. Second, we integrated the Norfair object tracking framework to track and count fish using a public video dataset captured by optical cameras from the Wells Dam fish ladder on the Columbia River in Washington State, USA. Our results demonstrate that deep learning models can indeed be used to detect, classify species, and track fish using both high resolution imaging sonar and underwater video from a fish ladder. This work is a first step toward developing a fully implemented system which can accurately detect, classify and generate insights about fish in a wide variety of fish passage environments and conditions with data collected from multiple types of sensors.

A specific hybridisation internalisation probe (SHIP) enables precise live-cell and super-resolution imaging of internalized cargo

Facilitated by the advancements in microscopy, our understanding of the complexity of intracellular vesicle traffic has dramatically increased in recent years. However, distinguishing between plasma membrane-bound or internalised ligands remains a major challenge for the studies of cargo sorting to endosomal compartments, especially in small and round cells such as lymphocytes. The specific hybridization internalisation probe (SHIP) assay, developed for flow cytometry studies, employs a ssDNA fluorescence internalisation probe and a complementary ssDNA quenching probe to unambiguously detect the internalized receptors/cargo. Here, we adopted the SHIP assay to study the trafficking of receptor/ligand complexes using B lymphocytes and B cell receptor-mediated antigen internalization as a model system. Our study demonstrates the potential of the SHIP assay for improving the imaging of internalized receptor/ligand complexes and establishes the compatibility of this assay with multiple imaging modalities, including live-cell imaging and super-resolution microscopy.

Integrated Satellite System for Fire Detection and Prioritization

Several studies have shown the relevance of satellite systems in detecting, monitoring, and characterizing fire events as support to fire management activities. On the other hand, up to now, only a few satellite-based platforms provide immediately and easily usable information about events in progress, in terms of both hotspots, which identify and localize active fires, and the danger conditions of the affected area. However, this kind of information is usually provided through separated layers, without any synthetic indicator which, indeed, could be helpful, if timely provided, for planning the priority of the intervention of firefighting resources in case of concurrent fires. In this study, we try to fill these gaps by presenting an Integrated Satellite System (ISS) for fire detection and prioritization, mainly based on the Robust Satellite Techniques (RST), and the Fire Danger Dynamic Index (FDDI), an original re-structuration of the Índice Combinado de Risco de Incêndio Florestal (ICRIF), for the first time presented here. The system, using Moderate Resolution Imaging Spectroradiometer (MODIS), Advanced Very High Resolution Radiometer (AVHRR), and Spinning Enhanced Visible and InfraRed Imager (SEVIRI) data, provides near real-time integrated information about both the fire presence and danger over the affected area. These satellite-based products are generated in common formats, ready to be ingested in Geographic Information System (GIS) technologies. Results shown and discussed here, on the occasion of concurrent winter and summer fires in Italy, in agreement with information from independent sources, demonstrate that the ISS system, operating at a regional/national scale, may provide an important contribution to fire prioritization. This may result in the mitigation of fire impact in populated areas, infrastructures, and the environment.

Clouds and the Earth’s Radiant Energy System (CERES) FluxByCldTyp Edition 4 Data Product

The Clouds and the Earth’s Radiant Energy System (CERES) project has provided the climate community 20 years of globally observed top of the atmosphere (TOA) fluxes critical for climate and cloud feedback studies. The CERES Flux By Cloud Type (FBCT) product contains radiative fluxes by cloud-type, which can provide more stringent constraints when validating models and also reveal more insight into the interactions between clouds and climate. The FBCT product provides 1° regional daily and monthly shortwave (SW) and longwave (LW) cloud-type fluxes and cloud properties sorted by 7 pressure layers and 6 optical depth bins. Historically, cloud-type fluxes have been computed using radiative transfer models based on observed cloud properties. Instead of relying on radiative transfer models, the FBCT product utilizes Moderate Resolution Imaging Spectroradiometer (MODIS) radiances partitioned by cloud-type within a CERES footprint to estimate the cloud-type broadband fluxes. The MODIS multi-channel derived broadband fluxes were compared with the CERES observed footprint fluxes and were found to be within 1% and 2.5% for LW and SW, respectively, as well as being mostly free of cloud property dependencies. These biases are mitigated by constraining the cloud-type fluxes within each footprint with the CERES Single Scanner Footprint (SSF) observed flux. The FBCT all-sky and clear-sky monthly averaged fluxes were found to be consistent with the CERES SSF1deg product. Several examples of FBCT data are presented to highlight its utility for scientific applications.

Research on Sparse Representation Method of Acoustic Microimaging Signals

Acoustic microimaging (AMI), a technology for high-resolution imaging of materials using a scanning acoustic microscope, has been widely used for non-destructive testing and evaluation of electronic packages. Recently, the internal features and defects of electronic packages have reached the resolution limits of conventional time domain or frequency domain AMI methods with the miniaturization of electronic packages. Various time-frequency domain AMI methods have been developed to achieve super-resolution. In this paper, the sparse representation of AMI signals is studied, and a constraint dictionary-based sparse representation (CD-SR) method is proposed. First, the time-frequency parameters of the atom dictionary are constrained according to the AMI signal to constitute a constraint dictionary. Then, the AMI signal is sparsely decomposed using the matching pursuit algorithm, and echoes selection and echoes reconstruction are performed. The performance of CD-SR was quantitatively evaluated by simulated and experimental ultrasonic A-scan signals. The results demonstrated that CD-SR has superior longitudinal resolution and robustness.

Hyperbolic metamaterials: fusing artificial structures to natural 2D materials

Optical metamaterials have presented an innovative method of manipulating light. Hyperbolic metamaterials have an extremely high anisotropy with a hyperbolic dispersion relation. They are able to support high-k modes and exhibit a high density of states which produce distinctive properties that have been exploited in various applications, such as super-resolution imaging, negative refraction, and enhanced emission control. Here, state-of-the-art hyperbolic metamaterials are reviewed, starting from the fundamental principles to applications of artificially structured hyperbolic media to suggest ways to fuse natural two-dimensional hyperbolic materials. The review concludes by indicating the current challenges and our vision for future applications of hyperbolic metamaterials.