scholarly journals MxDC & MxLIVE:Software for Macromolecular Crystallography Experiments at the CLS

2014 ◽  
Vol 70 (a1) ◽  
pp. C792-C792
Author(s):  
Kathryn Janzen ◽  
Michel Fodje ◽  
Shaun Labiuk ◽  
James Gorin ◽  
Pawel Grochulski
Keyword(s):  
Data Processing ◽  
Anomalous Dispersion ◽  
Data Sets ◽  
Macromolecular Crystallography ◽  
Fully Integrated ◽  
Level Of Automation ◽  
Multiple Wavelength ◽  
Experiment Management ◽  
High Level ◽  
Integrated Software

The Canadian Macromolecular Crystallography Facility (CMCF) is a suite of two beamlines 08ID-1 and 08B1-1. Beamline 08B1-1 is a bending-magnet beamline for high-throughput macromolecular crystallography enabling Multiple-Wavelength Anomalous Dispersion (MAD) and Single-Wavelength Anomalous Dispersion (SAD) experiments with a high level of automation. We have developed an integrated software system with modules for beamline control, experiment management, and automated data processing for both on-side and remote users. The experiment management module, also known as MxLIVE (Macromolecular Crystallography Laboratory Information Virtual Environment) is responsible for managing the storage of information about samples, sample shipments, experiment requests, experiment results and data sets. It provides a web-based interface for users to submit sample information and experiment requests, track shipments en route to the CLS and review experiment results and data sets as they are completed on site, and for beamline staff to manage Mail-In data acquisition sessions, reducing the need for user travel to the synchrotron. The beamline control module includes a user-friendly interface for data collection, MxDC (Macromolecular Crystallography Data Collector). MxDC is fully integrated with beamline hardware as well as software applications such as MxLIVE and AutoProcess, an innovative data processing pipeline. This makes MxDC a hub for all experiment-focused activities at CMCF beamlines, including sample auto-mounting, centering and screening crystals, diffraction experiments, and automated data reduction.

scholarly journals The predictive power of data-processing statistics

IUCrJ ◽  
2020 ◽  
Vol 7 (2) ◽  
pp. 342-354
Author(s):  
Melanie Vollmar ◽  
James M. Parkhurst ◽  
Dominic Jaques ◽  
Arnaud Baslé ◽  
Garib N. Murshudov ◽  
...  
Keyword(s):  
Data Processing ◽  
Anomalous Dispersion ◽  
Data Sets ◽  
Predictive Tool ◽  
Data Set ◽  
X Ray Crystallography ◽  
Multiple Wavelength ◽  
Data Rates ◽  
Starting Point ◽  
Computational Resources

This study describes a method to estimate the likelihood of success in determining a macromolecular structure by X-ray crystallography and experimental single-wavelength anomalous dispersion (SAD) or multiple-wavelength anomalous dispersion (MAD) phasing based on initial data-processing statistics and sample crystal properties. Such a predictive tool can rapidly assess the usefulness of data and guide the collection of an optimal data set. The increase in data rates from modern macromolecular crystallography beamlines, together with a demand from users for real-time feedback, has led to pressure on computational resources and a need for smarter data handling. Statistical and machine-learning methods have been applied to construct a classifier that displays 95% accuracy for training and testing data sets compiled from 440 solved structures. Applying this classifier to new data achieved 79% accuracy. These scores already provide clear guidance as to the effective use of computing resources and offer a starting point for a personalized data-collection assistant.

scholarly journals Serial Electron Diffraction Data Processing With diffractem and CrystFEL

2021 ◽  
Vol 8 ◽  
Author(s):  
Robert Bücker ◽  
Pascal Hogan-Lamarre ◽  
R. J. Dwayne Miller
Keyword(s):  
Electron Diffraction ◽  
Data Processing ◽  
Three Dimensional ◽  
Electron Diffraction Data ◽  
New Techniques ◽  
X Ray ◽  
X Ray Crystallography ◽  
Level Of Automation ◽  
Rotation Method ◽  
High Level

Serial electron diffraction (SerialED) is an emerging technique, which applies the snapshot data-collection mode of serial X-ray crystallography to three-dimensional electron diffraction (3D Electron Diffraction), forgoing the conventional rotation method. Similarly to serial X-ray crystallography, this approach leads to almost complete absence of radiation damage effects even for the most sensitive samples, and allows for a high level of automation. However, SerialED also necessitates new techniques of data processing, which combine existing pipelines for rotation electron diffraction and serial X-ray crystallography with some more particular solutions for challenges arising in SerialED specifically. Here, we introduce our analysis pipeline for SerialED data, and its implementation using the CrystFEL and diffractem program packages. Detailed examples are provided in extensive supplementary code.

Aquarium: an automatic data-processing and experiment information management system for biological macromolecular crystallography beamlines

2019 ◽  
Vol 52 (2) ◽  
pp. 472-477 ◽  
Author(s):  
Feng Yu ◽  
Qisheng Wang ◽  
Minjun Li ◽  
Huan Zhou ◽  
Ke Liu ◽  
...  
Keyword(s):  
Data Processing ◽  
Information Management ◽  
Web Site ◽  
Management System ◽  
Information Management System ◽  
Automatic Data Processing ◽  
Data Sets ◽  
Macromolecular Crystallography ◽  
Automatic Data ◽  
Processing Module

With the popularity of hybrid pixel array detectors, hundreds of diffraction data sets are collected at a biological macromolecular crystallography (MX) beamline every day. Therefore, the manual processing and recording procedure will be a very time-consuming and error-prone task. Aquarium is an automatic data processing and experiment information management system designed for synchrotron radiation source MX beamlines. It is composed of a data processing module, a daemon module and a web site module. Before experiments, the sample information can be registered into a database. The daemon module will submit data processing jobs to a high-performance cluster as soon as the data set collection is completed. The data processing module will automatically process data sets from data reduction to model building if the anomalous signal is available. The web site module can be used to monitor and inspect the data processing results.

scholarly journals The use of workflows for macromolecular crystallography experiments at the ESRF.

2014 ◽  
Vol 70 (a1) ◽  
pp. C340-C340
Author(s):  
Olof Svensson ◽  
Sandor Brockhauser ◽  
Matthew Bowler ◽  
Max Nanao ◽  
Matias Guijarro ◽  
...  
Keyword(s):  
Data Analysis ◽  
High Performance ◽  
Radiation Sensitivity ◽  
Damage Effect ◽  
Macromolecular Crystallography ◽  
Quality Improvements ◽  
Software Environment ◽  
Fully Integrated ◽  
Current Implementation ◽  
High Level

The high performance of modern synchrotron facilities means there is an increasing reliance on automated data analysis and collection methods. The EMBL and ESRF are actively involved in designing and implementing such automated methods. However, as these methods are evolving there is also a need to continually integrate newer and more sophisticated data analysis and collection protocols with experimental control. This integration often poses a challenge, requiring a high level software environment to automatically coordinate beamline control with data acquisition and analysis. This is why we have extended the Eclipse RCP version of the workflow tool Passerelle into a user friendly GUI for experiment design by scientists and programmers [1], which is now part of the Data Analysis WorkbeNch (DAWN) collaboration (http://www.dawnsci.org). The execution of several complex workflows using this technology are now fully integrated in the new version of MxCuBE [2] and deployed on the ESRF macromolecular crystallography beamlines. Here, I will present their current implementation and the data quality improvements that can be achieved. In particular we have developed automated crystal re-orientation workflows that can improve the success of ab initio phasing experiments and help mitigate against radiation damage effect [3]. Other protocols implemented include a 3D diffraction based centring routine, a dehydration protocol and the automated measurement of a crystals radiation sensitivity. Lastly I will present our future plans for other new advanced diffraction based workflow routines, including automated crystal screening and data collection protocols.

FORMATION OF METASUBJECT COMPETENCIES IN THE COURSE “INFORMATION TECHNOLOGIES” BY MEANS OF THE BIG DATA PROCESSING LANGUAGE R

2019 ◽  
pp. 12-22
Author(s):  
D. M. Nazarov
Keyword(s):  
Data Processing ◽  
Information Technologies ◽  
Data Sets ◽  
Training Methods ◽  
Economic Knowledge ◽  
R Language ◽  
Currency Exchange ◽  
Business Informatics ◽  
Exchange Data ◽  
Processing Language

The article describes the training methods in the course “Information Technologies” for the future bachelors of the directions “Economics”, “Management”, “Finance”, “Business Informatics”, the development of metasubject competencies of the student while his use of tools for data processing by means of the language R. The metasubject essence of the work is to update traditional economic knowledge and skills through various presentation forms of the same data sets. As part of the laboratory work described in the article, future bachelors learn to use the basic tools of the R language and acquire specific skills and abilities in R-Studio using the example of processing currency exchange data. The description of the methods is presented in the form of the traditional Key-by-Key technology, which is widely used in teaching information technologies.

Exploring The Effect of Visual Information Degradation on Human Perception and Performance In A Human-Telerobot System

2020 ◽  
Vol 64 (1) ◽  
pp. 1302-1307
Author(s):  
Richard Stone ◽  
Minglu Wang ◽  
Thomas Schnieders ◽  
Esraa Abdelall
Keyword(s):  
Visual Information ◽  
Human Perception ◽  
Low Level ◽  
Hit Rate ◽  
Word Level ◽  
Level Of Automation ◽  
Significant Difference ◽  
Human Operators ◽  
And Performance ◽  
High Level

Human-robotic interaction system are increasingly becoming integrated into industrial, commercial and emergency service agencies. It is critical that human operators understand and trust automation when these systems support and even make important decisions. The following study focused on human-in-loop telerobotic system performing a reconnaissance operation. Twenty-four subjects were divided into groups based on level of automation (Low-Level Automation (LLA), and High-Level Automation (HLA)). Results indicated a significant difference between low and high word level of control in hit rate when permanent error occurred. In the LLA group, the type of error had a significant effect on the hit rate. In general, the high level of automation was better than the low level of automation, especially if it was more reliable, suggesting that subjects in the HLA group could rely on the automatic implementation to perform the task more effectively and more accurately.

Survey Data Quality in Analyzing Harmonized Indicators of Protest Behavior: A Survey Data Recycling Approach

2021 ◽  
pp. 000276422110216
Author(s):  
Kazimierz M. Slomczynski ◽  
Irina Tomescu-Dubrow ◽  
Ilona Wysmulek
Keyword(s):  
Data Processing ◽  
Data Quality ◽  
Survey Data ◽  
A Priori ◽  
Data Sets ◽  
New Approach ◽  
Survey Quality ◽  
Survey Error ◽  
Ex Post ◽  
The Impact

This article proposes a new approach to analyze protest participation measured in surveys of uneven quality. Because single international survey projects cover only a fraction of the world’s nations in specific periods, researchers increasingly turn to ex-post harmonization of different survey data sets not a priori designed as comparable. However, very few scholars systematically examine the impact of the survey data quality on substantive results. We argue that the variation in source data, especially deviations from standards of survey documentation, data processing, and computer files—proposed by methodologists of Total Survey Error, Survey Quality Monitoring, and Fitness for Intended Use—is important for analyzing protest behavior. In particular, we apply the Survey Data Recycling framework to investigate the extent to which indicators of attending demonstrations and signing petitions in 1,184 national survey projects are associated with measures of data quality, controlling for variability in the questionnaire items. We demonstrate that the null hypothesis of no impact of measures of survey quality on indicators of protest participation must be rejected. Measures of survey documentation, data processing, and computer records, taken together, explain over 5% of the intersurvey variance in the proportions of the populations attending demonstrations or signing petitions.

scholarly journals Evolution of the ALICE Software Framework for Run 3

2019 ◽  
Vol 214 ◽  
pp. 05010 ◽  
Author(s):  
Giulio Eulisse ◽  
Piotr Konopka ◽  
Mikolaj Krzewicki ◽  
Matthias Richter ◽  
David Rohr ◽  
...  
Keyword(s):  
Data Processing ◽  
Data Model ◽  
Message Passing ◽  
Software Framework ◽  
Distributed Software ◽  
Central Collisions ◽  
Modular Software ◽  
Level Trigger ◽  
Data Throughput ◽  
High Level

ALICE is one of the four major LHC experiments at CERN. When the accelerator enters the Run 3 data-taking period, starting in 2021, ALICE expects almost 100 times more Pb-Pb central collisions than now, resulting in a large increase of data throughput. In order to cope with this new challenge, the collaboration had to extensively rethink the whole data processing chain, with a tighter integration between Online and Offline computing worlds. Such a system, code-named ALICE O2, is being developed in collaboration with the FAIR experiments at GSI. It is based on the ALFA framework which provides a generalized implementation of the ALICE High Level Trigger approach, designed around distributed software entities coordinating and communicating via message passing. We will highlight our efforts to integrate ALFA within the ALICE O2 environment. We analyze the challenges arising from the different running environments for production and development, and conclude on requirements for a flexible and modular software framework. In particular we will present the ALICE O2 Data Processing Layer which deals with ALICE specific requirements in terms of Data Model. The main goal is to reduce the complexity of development of algorithms and managing a distributed system, and by that leading to a significant simplification for the large majority of the ALICE users.

scholarly journals Reconciliation of anti-money laundering instruments and European data protection requirements in permissionless blockchain spaces

2021 ◽  
Vol 7 (1) ◽  
Author(s):  
Iwona Karasek-Wojciechowicz
Keyword(s):  
Data Processing ◽  
Money Laundering ◽  
Data Protection ◽  
Task Force ◽  
Policy Instruments ◽  
Personal Data ◽  
General Data Protection Regulation ◽  
Terrorist Financing ◽  
The Government ◽  
High Level

AbstractThis article is an attempt to reconcile the requirements of the EU General Data Protection Regulation (GDPR) and anti-money laundering and combat terrorist financing (AML/CFT) instruments used in permissionless ecosystems based on distributed ledger technology (DLT). Usually, analysis is focused only on one of these regulations. Covering by this research the interplay between both regulations reveals their incoherencies in relation to permissionless DLT. The GDPR requirements force permissionless blockchain communities to use anonymization or, at the very least, strong pseudonymization technologies to ensure compliance of data processing with the GDPR. At the same time, instruments of global AML/CFT policy that are presently being implemented in many countries following the recommendations of the Financial Action Task Force, counteract the anonymity-enhanced technologies built into blockchain protocols. Solutions suggested in this article aim to induce the shaping of permissionless DLT-based networks in ways that at the same time would secure the protection of personal data according to the GDPR rules, while also addressing the money laundering and terrorist financing risks created by transactions in anonymous blockchain spaces or those with strong pseudonyms. Searching for new policy instruments is necessary to ensure that governments do not combat the development of all privacy-blockchains so as to enable a high level of privacy protection and GDPR-compliant data processing. This article indicates two AML/CFT tools which may be helpful for shaping privacy-blockchains that can enable the feasibility of such tools. The first tool is exceptional government access to transactional data written on non-transparent ledgers, obfuscated by advanced anonymization cryptography. The tool should be optional for networks as long as another effective AML/CFT measures are accessible for the intermediaries or for the government in relation to a given network. If these other measures are not available and the network does not grant exceptional access, the regulations should allow governments to combat the development of those networks. Effective tools in that scope should target the value of privacy-cryptocurrency, not its users. Such tools could include, as a tool of last resort, state attacks which would undermine the trust of the community in a specific network.

Determination of the site of incorporation of cobalt in CoZnPO-CZP by multiple-wavelength anomalous-dispersion crystallography

1999 ◽  
Vol 55 (3) ◽  
pp. 327-332 ◽  
Author(s):  
M. Helliwell ◽  
J. R. Helliwell ◽  
V. Kaucic ◽  
N. Zabukovec Logar ◽  
L. Barba ◽  
...  
Keyword(s):  
Metal Atom ◽  
Peak Height ◽  
Data Sets ◽  
White Line ◽  
Data Set ◽  
Small Peak ◽  
Multiple Wavelength ◽  
Subsequent Calculation ◽  
Difference Fourier ◽  
Two Peaks

Data were collected from a crystal of CoZnPO-CZP {sodium cobalt–zinc phosphate hydrate, Na6[Co0.2Zn0.8PO4]6.6H2O} using synchrotron radiation at ELETTRA at the inflection point and `white line' for both the cobalt and zinc K edges, and at 1.45 Å, a wavelength remote from the K edges of both metals. The data were processed using the programs DENZO and SCALEPACK. The CCP4 program suite was used for the scaling of data sets and the subsequent calculation of dispersive difference Fourier maps. Optimal scaling was achieved by using a subset of reflections with little or no contribution from the metal atoms (i.e. which were essentially wavelength independent in their intensities) and using weights based on the σ's to obtain an overall scale factor in each case. Phases were calculated with SHELXL97 based on the refined structure using a much higher resolution and complete Cu Kα data set. An occupancy of 100% by zinc at the two metal-atom sites was assumed. The dispersive difference Fourier map calculated for zinc gave two peaks above the background of similar heights at the expected metal-atom sites. The peak height at the Zn1 site was a little higher than at the Zn2 site. The dispersive difference Fourier map calculated for cobalt gave just one peak above the background, at the Zn1 site, and only a small peak at the Zn2 site, thus indicating that incorporation of cobalt takes place mainly at one site. Refinement of the zinc occupancies using MLPHARE reinforces this conclusion. The chemical environment of each site is discussed.

Sign in / Sign up

Export Citation Format

Share Document