scholarly journals Data processing pipeline for serial femtosecond crystallography at SACLA

2016 ◽  
Vol 49 (3) ◽  
pp. 1035-1041 ◽  
Author(s):  
Takanori Nakane ◽  
Yasumasa Joti ◽  
Kensuke Tono ◽  
Makina Yabashi ◽  
Eriko Nango ◽  
...  
Keyword(s):  
Data Processing ◽  
Data Acquisition ◽  
Real Time ◽  
Data Set ◽  
Hit Rate ◽  
Processing Pipeline ◽  
Structure Solution ◽  
Two Stages ◽  
Serial Femtosecond Crystallography ◽  
Detector Saturation

A data processing pipeline for serial femtosecond crystallography at SACLA was developed, based onCheetah[Bartyet al.(2014).J. Appl. Cryst.47, 1118–1131] andCrystFEL[Whiteet al.(2016).J. Appl. Cryst.49, 680–689]. The original programs were adapted for data acquisition through the SACLA API, thread and inter-node parallelization, and efficient image handling. The pipeline consists of two stages: The first, online stage can analyse all images in real time, with a latency of less than a few seconds, to provide feedback on hit rate and detector saturation. The second, offline stage converts hit images into HDF5 files and runsCrystFELfor indexing and integration. The size of the filtered compressed output is comparable to that of a synchrotron data set. The pipeline enables real-time feedback and rapid structure solution during beamtime.

Distributed real time data processing architecture for the TJ-II data acquisition system

2004 ◽  
Vol 75 (10) ◽  
pp. 4261-4264 ◽  
Author(s):  
M. Ruiz ◽  
E. Barrera ◽  
S. López ◽  
D. Machón ◽  
J. Vega ◽  
...  
Keyword(s):  
Data Processing ◽  
Data Acquisition ◽  
Real Time ◽  
Data Acquisition System ◽  
Acquisition System ◽  
Time Data ◽  
Real Time Data ◽  
Real Time Data Processing ◽  
Processing Architecture

Real-time data acquisition and parallel data processing solution for TJ-II Bolometer arrays diagnostic

2006 ◽  
Vol 81 (15-17) ◽  
pp. 1863-1867
Author(s):  
E. Barrera ◽  
M. Ruiz ◽  
S. López ◽  
D. Machón ◽  
J. Vega ◽  
...  
Keyword(s):  
Data Processing ◽  
Data Acquisition ◽  
Real Time ◽  
Time Data ◽  
Parallel Data ◽  
Real Time Data ◽  
Bolometer Arrays ◽  
Processing Solution

scholarly journals A library for fMRI real-time processing systems in python (RTPSpy) with comprehensive online noise reduction, fast and accurate anatomical image processing, and online processing simulation

2021 ◽  
Author(s):  
Masaya Misaki ◽  
Jerzy Bodurka ◽  
Martin P Paulus
Keyword(s):  
Image Processing ◽  
Data Processing ◽  
Real Time ◽  
Easy Access ◽  
External Application ◽  
Real Time Processing ◽  
Time Processing ◽  
Target Signal ◽  
Processing Pipeline ◽  
Anatomical Image

We introduce a python library for real-time fMRI (rtfMRI) data processing systems, Real-Time Processing System in python (RTPSpy), to provide building blocks for a custom rtfMRI application with extensive and advanced functionalities. RTPSpy is a library package including 1) a fast, comprehensive, and flexible online fMRI denoising pipeline comparable to offline processing, 2) utilities for fast and accurate anatomical image processing to define a target region on-site, 3) a simulation system of online fMRI processing to optimize a pipeline and target signal calculation, 4) interface to an external application for feedback presentation, and 5) a boilerplate graphical user interface (GUI) integrating operations with RTPSpy library. Since online fMRI data processing cannot be equivalent to offline, we discussed the limitations of online analysis and their solutions in the RTPSpy implementation. We developed a fast and accurate anatomical image processing script with fast tissue segmentation (FastSeg), image alignment, and spatial normalization, utilizing the FastSurfer, AFNI, and ANTs. We confirmed that the FastSeg output was comparable with FreeSurfer, and could complete all the anatomical image processing in a few minutes. Thanks to its highly modular architecture, RTPSpy can easily be used for a simulation analysis to optimize a processing pipeline and target signal calculation. We present a sample script for building a real-time processing pipeline and running a simulation using RTPSpy. The library also offers a simple signal exchange mechanism with an external application. An external application can receive a real-time neurofeedback signal from RTPSpy in a background thread with a few lines of script. While the main components of the RTPSpy are the library modules, we also provide a GUI class for easy access to the RTPSpy functions. The boilerplate GUI application provided with the package allows users to develop a customized rtfMRI application with minimum scripting labor. Finally, we discussed the limitations of the package regarding environment-specific implementations. We believe that RTPSpy is an attractive option for developing rtfMRI applications highly optimized for individual purposes. The package is available from GitHub (https://github.com/mamisaki/RTPSpy) with GPL3 license.

Smart Factory of Industry 4.0: Connection Infrastructure, Data Acquisition, Data Processing, and Applications

2019 ◽  
Author(s):  
Zhao Zhiqiang ◽  
Chua Wei Quan ◽  
Ding Xiaoming ◽  
Prabhu Vinayak Ashok
Keyword(s):  
Data Processing ◽  
Data Acquisition ◽  
Real Time ◽  
Industry 4.0 ◽  
Manufacturing Systems ◽  
Discrete Systems ◽  
Smart Factory ◽  
Hierarchical Architecture ◽  
Support Functions ◽  
Control Functions

Abstract Smart factory adopts cyber-physical technologies integrating independent discrete systems into a context-sensitive manufacturing environment to optimize manufacturing processes using decentralized information and real-time communication. This paper presents our work in the realization of a smart factory, which comprises of a four-layer hierarchical architecture, i.e. connection infrastructure, data acquisition, data processing and smart applications. In the connection infrastructure layer, all shopfloor machines are connected through diverse protocols, IoT sensors, PLC interfaces and DNC connectors. A centralized IoT gateway supports such a scalable and adaptable connection and ensures a reliable communication among all heterogeneous manufacturing systems. In the data acquisition layer, the real-time machine and job data are acquired from shopfloor systems. Machine data indicates machines’ working condition and job data reveals the production information. The data processing layer comprises of three modules, i.e. shopfloor monitoring, data visualization and data analytics, which monitor and visualize shopfloor activities and analyze the semantics of various data using AI-based TPM engines providing the scientific indicators for next-step decisions. The smart application layer provides with several decision-making and remote control functions for shopfloor productivity and efficiency, such as predictive maintenance, shopfloor management, machine & job optimization and digital twin. The smart factory system has been implemented in the manufacturing shopfloor at Nanyang Polytechnic. The results and validation show that the system can simultaneously collect and analyze the manufacturing data from shopfloor systems, and further communicate with and control the shopfloor systems with decision-support functions. The overall shopfloor efficiency and flexibility can be significantly improved towards a smart factory of Industry 4.0.

Hardware Implementation of a Real-Time Data Processing Algorithm in Marine Engineering Data Acquisition

2011 ◽  
Vol 268-270 ◽  
pp. 110-115
Author(s):  
Ling Ma ◽  
Ke Zhu Song ◽  
Jun Feng Yang ◽  
Ping Cao
Keyword(s):  
Data Processing ◽  
Data Acquisition ◽  
Real Time ◽  
Hardware Implementation ◽  
Sequence Data ◽  
Time Sequence ◽  
Time Data ◽  
Real Time Data ◽  
Marine Seismic ◽  
Real Time Data Processing

According to the architecture characteristics of the mass data acquisition system in marine seismic exploration, this paper designed a real-time data processing algorithm which can convert the collected time-sequence data to channel-sequence data. A hardware implementation of the algorithm based on FPGA+DDR SDRAM is developed to complete the whole conversion process. Here, FPGA is used to achieve time sequence data receiving, analyzing, preliminary processing and the interface to DDR SDRAM. Two DDR SDRAM’s are used in ping-pang mode to store time-sequence data and to cooperate with FPGA in realizing time-to-channel sequence data conversion. Test results showed that, after applying the algorithm to the FCI in high-precision marine seismic data acquisition and recording system, this arithmetic could realize caching collected data without redundancy and converting data from time sequence to channel sequence without dead time, besides, this algorithm also greatly improved the efficiency and reliability of data processing.

In-house high-energy-remote SAD phasing using the magic triangle: how to tackle the P1 low symmetry using multiple orientations of the same crystal of human IBA57 to increase the multiplicity

2019 ◽  
Vol 75 (3) ◽  
pp. 317-324 ◽  
Author(s):  
Spyridon Gourdoupis ◽  
Veronica Nasta ◽  
Simone Ciofi-Baffoni ◽  
Lucia Banci ◽  
Vito Calderone
Keyword(s):  
Data Processing ◽  
Model Building ◽  
Real Data ◽  
Threshold Value ◽  
High Energy ◽  
Software Developers ◽  
Data Set ◽  
Structure Solution ◽  
Sad Phasing ◽  
Low Symmetry

This article describes the approach used to solve the structure of human IBA57 in-house by 5-amino-2,4,6-triiodoisophthalic acid (I3C) high-energy-remote single-wavelength anomalous dispersion (SAD) phasing. Multiple orientations of the same triclinic crystal were exploited to acquire sufficient real data multiplicity for phasing. How the collection of an in-house native data set and its joint use with the I3C derivative through a SIRAS approach decreases the data multiplicity needed by almost 50% is described. Furthermore, it is illustrated that there is a clear data-multiplicity threshold value for success and failure in phasing, and how adding further data does not significantly affect substructure solution and model building. To our knowledge, this is the only structure present in the PDB that has been solved in-house by remote SAD phasing in space group P1 using only one crystal. All of the raw data used, derived from the different orientations, have been uploaded to Zenodo in order to enable software developers to improve methods for data processing and structure solution, and for educational purposes.

Sign in / Sign up

Export Citation Format

Share Document