Digital coded exposure formation of frames from event-based imagery

Event-driven neuromorphic imagers have a number of attractive properties including low-power consumption, high dynamic range, the ability to detect fast events, low memory consumption and low band-width requirements. One of the biggest challenges with using event-driven imagery is that the field of event data processing is still embryonic. In contrast, decades worth of effort have been invested in the analysis of frame-based imagery. Hybrid approaches for applying established frame-based analysis techniques to event-driven imagery have been studied since event-driven imagers came into existence. However, the process for forming frames from event-driven imagery has not been studied in detail. This work presents a principled digital coded exposure approach for forming frames from event-driven imagery that is inspired by the physics exploited in a conventional camera featuring a shutter. The technique described in this work provides a fundamental tool for understanding the temporal information content that contributes to the formation of a frame from event-driven imagery data. Event-driven imagery allows for the application of arbitrary virtual digital shutter functions to form the final frame on a pixel-by-pixel basis. The proposed approach allows for the careful control of the spatio-temporal information that is captured in the frame. Furthermore, unlike a conventional physical camera, event-driven imagery can be formed into any variety of possible frames in post-processing after the data is captured. Furthermore, unlike a conventional physical camera, coded-exposure virtual shutter functions can assume arbitrary values including positive, negative, real, and complex values. The coded exposure approach also enables the ability to perform applications of industrial interest such as digital stroboscopy without any additional hardware. The ability to form frames from event-driven imagery in a principled manner opens up new possibilities in the ability to use conventional frame-based image processing techniques on event-driven imagery.

SI-STSAR-7: A Large SAR Images Dataset with Spatial and Temporal Information for Classification of Winter Sea Ice in Hudson Bay

Remote sensing satellites have been broadly applied to sea ice monitoring. The substantial increase in satellite imagery provides a large amount of data support for deep learning methods in the sea ice classification field. However, there is a lack of public remote sensing datasets to facilitate sea ice classification with spatial and temporal information and to benchmark the deep learning methods. In this paper, we provide a labeled large sea ice dataset derived from time-series sentinel-1 SAR images, dubbed SI-STSAR-7, and a validated dataset construction method for sea ice classification research. The SI-STSAR-7 dataset includes seven different sea ice types corresponding to different sea ice development stages in Hudson Bay during winter, and its samples are time sequences of SAR image patches in order to embody the differences of backscattering intensity and textures between different sea ice types, as well as the change of sea ice with time. We construct the dataset by first performing noise reduction and mitigation of incidence angle dependence on SAR images, and then producing data samples and labeling them based on our proposed sample-producing principles and the weekly regional ice charts provided by Canadian Ice Service. Three baseline classification methods are developed on SI-STSAR-7 to establish benchmarks, which are evaluated with accuracy and kappa coefficient. The sample-producing principles are verified through experiments. Based on the experimental results, sea ice classification can be implemented well on SI-STSAR-7.

Spectral Temporal Information for Missing Data Reconstruction (STIMDR) of Landsat Reflectance Time Series

The number of Landsat time-series applications has grown substantially because of its approximately 50-year history and relatively high spatial resolution for observing long term changes in the Earth’s surface. However, missing observations (i.e., gaps) caused by clouds and cloud shadows, orbit and sensing geometry, and sensor issues have broadly limited the development of Landsat time-series applications. Due to the large area and temporal and spatial irregularity of time-series gaps, it is difficult to find an efficient and highly precise method to fill them. The Missing Observation Prediction based on Spectral-Temporal Metrics (MOPSTM) method has been proposed and delivered good performance in filling large-area gaps of single-date Landsat images. However, it can be less practical for a time series longer than one year due to the lack of mechanics that exclude dissimilar data in time series (e.g., different phenology or changes in land cover). To solve this problem, this study proposes a new gap-filling method, Spectral Temporal Information for Missing Data Reconstruction (STIMDR), and examines its performance in Landsat reflectance time series. Two groups of experiments, including 2000 × 2000 pixel Landsat single-date images and Landsat time series acquired from four sites (Kenya, Finland, Germany, and China), were performed to test the new method. We simulated artificial gaps to evaluate predicted pixel values with real observations. Quantitative and qualitative evaluations of gap-filled images through comparisons with other state-of-the-art methods confirmed the more robust and accurate performance of the proposed method. In addition, the proposed method was also able to fill gaps contaminated by extreme cloud cover for a period (e.g., winter in high-latitude areas). A down-stream task of random forest supervised classification through both gap-filled simulated datasets and the original valid datasets verified that STIMDR-generated products are relevant to the user community for land cover applications.

Advertising Popularity Feature Collaborative Recommendation Algorithm Based on Attention-LSTM Model

To accurately predict the click-through rate (CTR) and use it for ad recommendation, we propose a deep attention AD popularity prediction model (DAFCT) based on label recommendation technology and collaborative filtering method, which integrates content features and temporal information. First, we construct an Attention-LSTM model to capture the popularity trends and exploit the temporal information based on users’ feedback; finally, we use the concatenate method to fuse temporal information and content features and design a Deep Attention Popularity Prediction (DAVPP) algorithm to solve DAFCT. We experimentally adjust the weighted composite similarity metric parameters of Query pages and verify the scalability of the algorithm. Experimental results on the KDDCUP2012 dataset show that this model collaborative filtering and recommendation algorithm has better scalability and better recommendation quality. Compared with the Attention-LSTM model and the NFM model, the F1 score of DAFCT is improved by 9.80 and 3.07 percentage points, respectively.