A new batch-processing data-reduction application for X-ray diffraction data

2006 ◽  
Vol 39 (2) ◽  
pp. 267-272 ◽  
Author(s):  
R. J. Davies
Keyword(s):  
Diffraction Data ◽  
Data Reduction ◽  
Large Data ◽  
Treatment Session ◽  
Large Data Sets ◽  
Data Series ◽  
Data Sets ◽  
X Ray Diffraction ◽  
X Ray ◽  
Software Application

Modern synchrotron radiation facility beamlines offer high-brilliance beams and sensitive area detectors. Consequently, experiments such as scanning X-ray microdiffraction can generate large data sets within relatively short time periods. In these specialist fields there are currently very few automated data-treatment solutions to tackle the large data sets produced. Where there is existing software, it is either insufficiently specialized or cannot be operated in a batch-wise processing mode. As a result, a large gap exists between the rate at which X-ray diffraction data can be generated and the rate at which they can be realistically analysed. This article describes a new software application to perform batch-wise data reduction. It is designed to operate in combination with the commonly usedFit2Dprogram. Through the use of intuitive file selection, numerous processing lists and a generic operation sequence, it is capable of the batch-wise reduction of up to 60 000 diffraction patterns during each treatment session. It can perform automated intensity corrections to large data series, perform advanced background-subtraction operations and automatically organizes results. Integration limits can be set graphically on-screen, uniquely derived from existing peak positions or globally calculated from user-supplied values. The software represents a working solution to a hitherto unsolved problem.

A batch-wise non-linear fitting and analysis tool for treating large X-ray diffraction data sets

2006 ◽  
Vol 39 (2) ◽  
pp. 262-266 ◽  
Author(s):  
R. J. Davies
Keyword(s):  
Diffraction Data ◽  
Operation Mode ◽  
Large Data ◽  
Scattering Data ◽  
Data Sets ◽  
Analysis Tool ◽  
Data Set ◽  
X Ray ◽  
Linear Fitting ◽  
Non Linear

Synchrotron sources offer high-brilliance X-ray beams which are ideal for spatially and time-resolved studies. Large amounts of wide- and small-angle X-ray scattering data can now be generated rapidly, for example, during routine scanning experiments. Consequently, the analysis of the large data sets produced has become a complex and pressing issue. Even relatively simple analyses become difficult when a single data set can contain many thousands of individual diffraction patterns. This article reports on a new software application for the automated analysis of scattering intensity profiles. It is capable of batch-processing thousands of individual data files without user intervention. Diffraction data can be fitted using a combination of background functions and non-linear peak functions. To compliment the batch-wise operation mode, the software includes several specialist algorithms to ensure that the results obtained are reliable. These include peak-tracking, artefact removal, function elimination and spread-estimate fitting. Furthermore, as well as non-linear fitting, the software can calculate integrated intensities and selected orientation parameters.

SGTools: a suite of tools for processing and analyzing large data sets from in situ X-ray scattering experiments

2022 ◽  
Vol 55 (1) ◽  
Author(s):  
Nie Zhao ◽  
Chunming Yang ◽  
Fenggang Bian ◽  
Daoyou Guo ◽  
Xiaoping Ouyang
Keyword(s):  
Data Processing ◽  
Small Angle ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
X Ray ◽  
Intensity Mapping ◽  
X Ray Scattering ◽  
Ray Scattering

In situ synchrotron small-angle X-ray scattering (SAXS) is a powerful tool for studying dynamic processes during material preparation and application. The processing and analysis of large data sets generated from in situ X-ray scattering experiments are often tedious and time consuming. However, data processing software for in situ experiments is relatively rare, especially for grazing-incidence small-angle X-ray scattering (GISAXS). This article presents an open-source software suite (SGTools) to perform data processing and analysis for SAXS and GISAXS experiments. The processing modules in this software include (i) raw data calibration and background correction; (ii) data reduction by multiple methods; (iii) animation generation and intensity mapping for in situ X-ray scattering experiments; and (iv) further data analysis for the sample with an order degree and interface correlation. This article provides the main features and framework of SGTools. The workflow of the software is also elucidated to allow users to develop new features. Three examples are demonstrated to illustrate the use of SGTools for dealing with SAXS and GISAXS data. Finally, the limitations and future features of the software are also discussed.

scholarly journals AUSPEX: a graphical tool for X-ray diffraction data analysis

2017 ◽  
Vol 73 (9) ◽  
pp. 729-737 ◽  
Author(s):  
Andrea Thorn ◽  
James Parkhurst ◽  
Paul Emsley ◽  
Robert A. Nicholls ◽  
Melanie Vollmar ◽  
...  
Keyword(s):  
Data Analysis ◽  
Diffraction Data ◽  
Software Tool ◽  
Data Conversion ◽  
Data Sets ◽  
X Ray Diffraction ◽  
X Ray ◽  
Pixel Detectors ◽  
Graphical Tool ◽  
Data Acquisition And Processing

In this paper,AUSPEX, a new software tool for experimental X-ray data analysis, is presented. Exploring the behaviour of diffraction intensities and the associated estimated uncertainties facilitates the discovery of underlying problems and can help users to improve their data acquisition and processing in order to obtain better structural models. The program enables users to inspect the distribution of observed intensities (or amplitudes) against resolution as well as the associated estimated uncertainties (sigmas). It is demonstrated howAUSPEXcan be used to visually and automatically detect ice-ring artefacts in integrated X-ray diffraction data. Such artefacts can hamper structure determination, but may be difficult to identify from the raw diffraction images produced by modern pixel detectors. The analysis suggests that a significant portion of the data sets deposited in the PDB contain ice-ring artefacts. Furthermore, it is demonstrated how other problems in experimental X-ray data caused, for example, by scaling and data-conversion procedures can be detected byAUSPEX.

scholarly journals Experience with exchange and archiving of raw data: comparison of data from two diffractometers and four software packages on a series of lysozyme crystals

2012 ◽  
Vol 46 (1) ◽  
pp. 108-119 ◽  
Author(s):  
Simon W. M. Tanley ◽  
Antoine M. M. Schreurs ◽  
John R. Helliwell ◽  
Loes M. J. Kroon-Batenburg
Keyword(s):  
Diffraction Data ◽  
Data Sets ◽  
X Ray Diffraction ◽  
Noticeable Effect ◽  
Raw Data ◽  
X Ray ◽  
Data Comparison ◽  
Data Processing Software ◽  
Software Packages ◽  
Lysozyme Crystals

The International Union of Crystallography has for many years been advocating archiving of raw data to accompany structural papers. Recently, it initiated the formation of the Diffraction Data Deposition Working Group with the aim of developing standards for the representation of these data. A means of studying this issue is to submit exemplar publications with associated raw data and metadata. A recent study on the effects of dimethyl sulfoxide on the binding of cisplatin and carboplatin to histidine in 11 different lysozyme crystals from two diffractometers led to an investigation of the possible effects of the equipment and X-ray diffraction data processing software on the calculated occupancies andBfactors of the bound Pt compounds. 35.3 Gb of data were transferred from Manchester to Utrecht to be processed withEVAL. A systematic comparison shows that the largest differences in the occupancies andBfactors of the bound Pt compounds are due to the software, but the equipment also has a noticeable effect. A detailed description of and discussion on the availability of metadata is given. By making these raw diffraction data sets availableviaa local depository, it is possible for the diffraction community to make their own evaluation as they may wish.

XRD Instrument Sensitivity Results from a Round Robin Study

1991 ◽  
Vol 35 (A) ◽  
pp. 333-340 ◽  
Author(s):  
W.N. Schreiner ◽  
R. Jenkins ◽  
P.F. Dismore
Keyword(s):  
Diffraction Data ◽  
Round Robin ◽  
Data Sets ◽  
X Ray Diffraction ◽  
Diffraction Intensity ◽  
Generic Type ◽  
X Ray ◽  
The Past ◽  
Instrument Sensitivity

During the course of the past ten years the International Centre for Diffraction Data has sponsored a number of “Round Robin” tests to evaluate the quality of experimental X-ray diffraction data [1-5]. The latest of this series, called the Instrument Parameter Round Robin, was designed to evaluate, among other things, relative angularly-dependent sensitivity differences between diffractometers. Previous experiments have indicated that even perfectly aligned diffractometers of the same generic type, do not necessarily give the same set of relative intensities. One objective of the round robin was to quantify the magnitude of the experimental differences between data sets, and to demonstrate a means for external calibration of diffractometers, so that digitized diffraction intensity data obtained from different instruments could be directly compared.

Preparation and Crystal Structures of the Isotypic Compounds CdXO4 · 2 HgO (X = S, Se)

2004 ◽  
Vol 59 (3) ◽  
pp. 281-285 ◽  
Author(s):  
Matthias Weil
Keyword(s):  
Single Crystals ◽  
Crystal Structures ◽  
Diffraction Data ◽  
Building Blocks ◽  
Data Sets ◽  
X Ray Diffraction ◽  
Hydrothermal Conditions ◽  
Space Group ◽  
X Ray ◽  
Structure Factors

Colourless single crystals of the compounds CdXO4 · 2 HgO (X = S, Se) were obtained under hydrothermal conditions (250 °C, 5 d), starting from stoichiometric amounts of HgO, CdSO4 ·7H2O and CdSeO4 ·2H2O, respectively. The crystal structures were determined from X-ray diffraction data sets. The CdXO4 · 2HgO compounds crystallise isotypically with two formula units in space group P1̅ (# 2) [CdSO4 · 2HgO (CdSeO4 · 2HgO): a = 6.793(2) (6.9097(5)) Å , b = 7.205(2) (7.1786(6)) Å , c=7.359(2) (7.4556(6)) Å ,α =73.224(6) (74.586(2))°, β =66.505(6) (68.229(1))°, γ =63.054(5) (63.886(1))°, 1670 (1786) structure factors, 92 parameters, R[F2 > 2σ(F2)] = 0.0379 (0.0244)] and are made up from zig-zag [O-Hg-O]∞ chains with very short bonds of d̅(Hg-O) 2.025 Å , distorted [CdO6] octahedra (d̅(Cd-O)= 2.297 Å ), and XO4 tetrahedra (d̅(S-O)= 1.458 Å , d̅(Se-O)= 1.633 Å ) as the main building blocks. The CdXO4 ·2HgO compounds reveal no structural relationship with the corresponding HgXO4 ·2HgO phases

On the quality of the continuous rotation electron diffraction data for accurate atomic structure determination of inorganic compounds

2018 ◽  
Vol 51 (4) ◽  
pp. 1094-1101 ◽  
Author(s):  
Yunchen Wang ◽  
Taimin Yang ◽  
Hongyi Xu ◽  
Xiaodong Zou ◽  
Wei Wan
Keyword(s):  
Electron Diffraction ◽  
Single Crystal ◽  
Diffraction Data ◽  
Structural Models ◽  
Electron Diffraction Data ◽  
Data Sets ◽  
X Ray Diffraction ◽  
X Ray ◽  
Continuous Rotation

The continuous rotation electron diffraction (cRED) method has the capability of providing fast three-dimensional electron diffraction data collection on existing and future transmission electron microscopes; unknown structures could be potentially solved and refined using cRED data collected from nano- and submicrometre-sized crystals. However, structure refinements of cRED data using SHELXL often lead to relatively high R1 values when compared with those refined against single-crystal X-ray diffraction data. It is therefore necessary to analyse the quality of the structural models refined against cRED data. In this work, multiple cRED data sets collected from different crystals of an oxofluoride (FeSeO3F) and a zeolite (ZSM-5) with known structures are used to assess the data consistency and quality and, more importantly, the accuracy of the structural models refined against these data sets. An evaluation of the precision and consistency of the cRED data by examination of the statistics obtained from the data processing software DIALS is presented. It is shown that, despite the high R1 values caused by dynamical scattering and other factors, the refined atomic positions obtained from the cRED data collected for different crystals are consistent with those of the reference models refined against single-crystal X-ray diffraction data. The results serve as a reference for the quality of the cRED data and the achievable accuracy of the structural parameters.

scholarly journals Femtosecond free-electron laser x-ray diffraction data sets for algorithm development

2012 ◽  
Vol 20 (4) ◽  
pp. 4149 ◽  
Author(s):  
Stephan Kassemeyer ◽  
Jan Steinbrener ◽  
Lukas Lomb ◽  
Elisabeth Hartmann ◽  
Andrew Aquila ◽  
...  
Keyword(s):  
Diffraction Data ◽  
Free Electron ◽  
Free Electron Laser ◽  
Data Sets ◽  
X Ray Diffraction ◽  
Algorithm Development ◽  
X Ray

Accounting for unknown systematic errors in Rietveld refinements: a Bayesian statistics approach

2015 ◽  
Vol 48 (4) ◽  
pp. 1201-1211 ◽  
Author(s):  
Anton Gagin ◽  
Igor Levin
Keyword(s):  
Bayesian Statistics ◽  
Diffraction Data ◽  
Structural Parameters ◽  
Simulated Data ◽  
Systematic Errors ◽  
Data Sets ◽  
X Ray Diffraction ◽  
Rietveld Refinements ◽  
X Ray ◽  
Minimization Algorithms

A method has been developed to address the effects of systematic errors in Rietveld refinements using powder diffraction data. Relevant errors were categorized into multiplicative, additive and peak-shape types. Corrections for these errors were incorporated into structural refinements using a Bayesian statistics approach, with the corrections themselves treated as nuisance parameters and marginalized out of the analysis. Structural parameters refined using the proposed method represent probability-weighted averages over all possible error corrections. The developed formalism has been adapted to least-squares minimization algorithms and implemented as an extension to the Rietveld software packageGSAS-II. The technique was first tested using neutron and X-ray diffraction data simulated for PbSO4and then applied to the equivalent experimental data sets for the same compound. The results obtained using the simulated data confirmed that the proposed method yields significantly more accurate estimates of structural parameters and their uncertainties than standard refinements. The benefits were particularly significant for joint refinements using neutron and X-ray diffraction data because accounting for systematic errors enabled more adequate weighting of the individual data sets.

scholarly journals A convolutional neural network-based screening tool for X-ray serial crystallography

2018 ◽  
Vol 25 (3) ◽  
pp. 655-670 ◽  
Author(s):  
Tsung-Wei Ke ◽  
Aaron S. Brewster ◽  
Stella X. Yu ◽  
Daniela Ushizima ◽  
Chao Yang ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Large Data ◽  
Large Data Sets ◽  
Training Data ◽  
Data Sets ◽  
X Ray ◽  
X Ray Crystallography ◽  
Automatic Image Processing

A new tool is introduced for screening macromolecular X-ray crystallography diffraction images produced at an X-ray free-electron laser light source. Based on a data-driven deep learning approach, the proposed tool executes a convolutional neural network to detect Bragg spots. Automatic image processing algorithms described can enable the classification of large data sets, acquired under realistic conditions consisting of noisy data with experimental artifacts. Outcomes are compared for different data regimes, including samples from multiple instruments and differing amounts of training data for neural network optimization.

Sign in / Sign up

Export Citation Format

Share Document