Design and Implementation of a Metadata Repository about UML Class Diagrams. A Software Tool Supporting the Automatic Feeding of the Repository

Model-Driven Engineering is largely recognized as the most powerful method for the design of complex software. This study deals with the automated archival of metadata about the content of UML class diagrams (a particularly relevant category of models) into a pre-existing repository. To define the structure of the repository, we started from the definition of a UML metamodel. From the latter, we derived the schema of the metadata repository. Then, a parser was developed that is responsible for extracting the useful information from the XMI file about class diagrams and enters it as metadata into the repository. The parser has been implemented as a Java web interface, while the metadata repository has been implemented as a PostgreSQL database based on the JSONB data type. The metadata repository is thought to support modelers in the initial phase of the process of the development of new models when looking for artifacts to start from. The schema of the metadata repository and the Java code of the parser are available from the authors.

The Collaborative Metadata Repository (CoMetaR) Web App: Quantitative and Qualitative Usability Evaluation

Background In the field of medicine and medical informatics, the importance of comprehensive metadata has long been recognized, and the composition of metadata has become its own field of profession and research. To ensure sustainable and meaningful metadata are maintained, standards and guidelines such as the FAIR (Findability, Accessibility, Interoperability, Reusability) principles have been published. The compilation and maintenance of metadata is performed by field experts supported by metadata management apps. The usability of these apps, for example, in terms of ease of use, efficiency, and error tolerance, crucially determines their benefit to those interested in the data. Objective This study aims to provide a metadata management app with high usability that assists scientists in compiling and using rich metadata. We aim to evaluate our recently developed interactive web app for our collaborative metadata repository (CoMetaR). This study reflects how real users perceive the app by assessing usability scores and explicit usability issues. Methods We evaluated the CoMetaR web app by measuring the usability of 3 modules: core module, provenance module, and data integration module. We defined 10 tasks in which users must acquire information specific to their user role. The participants were asked to complete the tasks in a live web meeting. We used the System Usability Scale questionnaire to measure the usability of the app. For qualitative analysis, we applied a modified think aloud method with the following thematic analysis and categorization into the ISO 9241-110 usability categories. Results A total of 12 individuals participated in the study. We found that over 97% (85/88) of all the tasks were completed successfully. We measured usability scores of 81, 81, and 72 for the 3 evaluated modules. The qualitative analysis resulted in 24 issues with the app. Conclusions A usability score of 81 implies very good usability for the 2 modules, whereas a usability score of 72 still indicates acceptable usability for the third module. We identified 24 issues that serve as starting points for further development. Our method proved to be effective and efficient in terms of effort and outcome. It can be adapted to evaluate apps within the medical informatics field and potentially beyond.

Linking a Consortium-Wide Data Quality Assessment Tool with the MIRACUM Metadata Repository

Background Many research initiatives aim at using data from electronic health records (EHRs) in observational studies. Participating sites of the German Medical Informatics Initiative (MII) established data integration centers to integrate EHR data within research data repositories to support local and federated analyses. To address concerns regarding possible data quality (DQ) issues of hospital routine data compared with data specifically collected for scientific purposes, we have previously presented a data quality assessment (DQA) tool providing a standardized approach to assess DQ of the research data repositories at the MIRACUM consortium's partner sites. Objectives Major limitations of the former approach included manual interpretation of the results and hard coding of analyses, making their expansion to new data elements and databases time-consuming and error prone. We here present an enhanced version of the DQA tool by linking it to common data element definitions stored in a metadata repository (MDR), adopting the harmonized DQA framework from Kahn et al and its application within the MIRACUM consortium. Methods Data quality checks were consequently aligned to a harmonized DQA terminology. Database-specific information were systematically identified and represented in an MDR. Furthermore, a structured representation of logical relations between data elements was developed to model plausibility-statements in the MDR. Results The MIRACUM DQA tool was linked to data element definitions stored in a consortium-wide MDR. Additional databases used within MIRACUM were linked to the DQ checks by extending the respective data elements in the MDR with the required information. The evaluation of DQ checks was automated. An adaptable software implementation is provided with the R package DQAstats. Conclusion The enhancements of the DQA tool facilitate the future integration of new data elements and make the tool scalable to other databases and data models. It has been provided to all ten MIRACUM partners and was successfully deployed and integrated into their respective data integration center infrastructure.

Pragmatic MDR: a metadata repository with bottom-up standardization of medical metadata through reuse

Background The variety of medical documentation often leads to incompatible data elements that impede data integration between institutions. A common approach to standardize and distribute metadata definitions are ISO/IEC 11179 norm-compliant metadata repositories with top-down standardization. To the best of our knowledge, however, it is not yet common practice to reuse the content of publicly accessible metadata repositories for creation of case report forms or routine documentation. We suggest an alternative concept called pragmatic metadata repository, which enables a community-driven bottom-up approach for agreeing on data collection models. A pragmatic metadata repository collects real-world documentation and considers frequent metadata definitions as high quality with potential for reuse. Methods We implemented a pragmatic metadata repository proof of concept application and filled it with medical forms from the Portal of Medical Data Models. We applied this prototype in two use cases to demonstrate its capabilities for reusing metadata: first, integration into a study editor for the suggestion of data elements and, second, metadata synchronization between two institutions. Moreover, we evaluated the emergence of bottom-up standards in the prototype and two medical data managers assessed their quality for 24 medical concepts. Results The resulting prototype contained 466,569 unique metadata definitions. Integration into the study editor led to a reuse of 1836 items and item groups. During the metadata synchronization, semantic codes of 4608 data elements were transferred. Our evaluation revealed that for less complex medical concepts weak bottom-up standards could be established. However, more diverse disease-related concepts showed no convergence of data elements due to an enormous heterogeneity of metadata. The survey showed fair agreement (Kalpha = 0.50, 95% CI 0.43–0.56) for good item quality of bottom-up standards. Conclusions We demonstrated the feasibility of the pragmatic metadata repository concept for medical documentation. Applications of the prototype in two use cases suggest that it facilitates the reuse of data elements. Our evaluation showed that bottom-up standardization based on a large collection of real-world metadata can yield useful results. The proposed concept shall not replace existing top-down approaches, rather it complements them by showing what is commonly used in the community to guide other researchers.

The Collaborative Metadata Repository (CoMetaR) Web App: Quantitative and Qualitative Usability Evaluation (Preprint)

BACKGROUND In the field of medicine and medical informatics, the importance of comprehensive metadata has long been recognized, and the composition of metadata has become its own field of profession and research. To ensure sustainable and meaningful metadata are maintained, standards and guidelines such as the FAIR (Findability, Accessibility, Interoperability, Reusability) principles have been published. The compilation and maintenance of metadata is performed by field experts supported by metadata management apps. The usability of these apps, for example, in terms of ease of use, efficiency, and error tolerance, crucially determines their benefit to those interested in the data. OBJECTIVE This study aims to provide a metadata management app with high usability that assists scientists in compiling and using rich metadata. We aim to evaluate our recently developed interactive web app for our collaborative metadata repository (CoMetaR). This study reflects how real users perceive the app by assessing usability scores and explicit usability issues. METHODS We evaluated the CoMetaR web app by measuring the usability of 3 modules: <i>core module</i>, <i>provenance module</i>, and <i>data integration module</i>. We defined 10 tasks in which users must acquire information specific to their user role. The participants were asked to complete the tasks in a live web meeting. We used the System Usability Scale questionnaire to measure the usability of the app. For qualitative analysis, we applied a modified think aloud method with the following thematic analysis and categorization into the ISO 9241-110 usability categories. RESULTS A total of 12 individuals participated in the study. We found that over 97% (85/88) of all the tasks were completed successfully. We measured usability scores of 81, 81, and 72 for the 3 evaluated modules. The qualitative analysis resulted in 24 issues with the app. CONCLUSIONS A usability score of 81 implies very good usability for the 2 modules, whereas a usability score of 72 still indicates acceptable usability for the third module. We identified 24 issues that serve as starting points for further development. Our method proved to be effective and efficient in terms of effort and outcome. It can be adapted to evaluate apps within the medical informatics field and potentially beyond.

Establishing Diagnostic Reference Levels for CT Through a Provincial Medical Informatics Metadata Repository in Ontario

Purpose: To determine whether computed tomography radiation dose data could be captured electronically across hospitals to derive regional diagnostic reference levels for quality improvement. Methods: Data on consecutive computed tomography examinations from 8 hospitals were collected automatically in a central database (Repository) from April 2017 to September 2017. The most frequently performed examinations were used to determine the standard protocols for each hospital. Diagnostic reference levels across hospitals were derived using statistical distribution for 2 radiation dose metrics. These values were compared between hospitals, within and between hospitals by scanner and against national Health Canada achievable doses and diagnostic reference levels. Results: Three master protocol groups, Head, Abdomen–Pelvis, and Chest–Abdomen–Pelvis, accounted for 43% of all valid studies (N = 40 277). For the Repository, 11 of 12 mean values and 75th percentile diagnostic reference levels were below the Health Canada mean and 75th percentile values, and one was the same as the Health Canada value. Mean radiation dose by protocol varied by as much as 97% between hospitals. There was no consistent pattern in the difference between mean doses between large and small hospitals. Conclusion: This electronic data acquisition process could be used to continually update achievable doses for frequently used computed tomography examinations in Ontario and eliminate the need for nationwide manual surveys. Results compared across institutions will allow hospitals to maintain achievable doses and lower patient exposure.

SWx TREC: Further Developments on an Integrative Space Weather (SWx) Data Portal

&lt;p&gt;Working under the Space Weather Technology, Research and Education Center (SWx-TREC https://www.colorado.edu/spaceweather/).&amp;#160; The Laboratory for Atmospheric and Space Physics (LASP) is developing a Space Weather (SWx) Data Portal to provide unified access to disparate datasets to help close the Research to Operations (R2O) and Operations to Research (O2R) gap.&amp;#160;&lt;/p&gt;&lt;p&gt;LASP is building the SWx Portal leveraging technologies developed in support of spacecraft operations (WEBTCAD), Irradiance Dataset viewing and downloading (LISIRD: http://lasp.colorado.edu/lisird/ ) and the MAVEN and MMS Science Data Portals.&amp;#160; The primary technologies include a data model and software library that enables data interoperability known as LaTiS&amp;#160;(https://github.com/latis-data) and the LASP Extended Metadata Repository (LEMR) which is developed as ontologies that not only represent the datasets, but also the front-end elements which are used to display them. &amp;#160;Additionally, we have developed a JavaScript science data display technology that leverages off LaTiS server instances to allow for consistent and straightforward display of datasets. &amp;#160;These technologies together facilitate a common interface to myriad datasets and formats which will enable us to expand the offerings quickly and provide consistent visualization, access to metadata, and download capabilities across them.&lt;/p&gt;&lt;p&gt;This presentation will discuss advancements in the portal development in the last year to both in terms of available datasets and in terms of new functionality.&amp;#160; We will also provide a demonstration of the released system that will include datasets demonstrating a solar event, its progression toward Earth and its Earth affect perspective of Space Weather Data centering on the 2015 St. Patrick&amp;#8217;s day storm.&lt;/p&gt;