scholarly journals A Case Study of Publishing Internal APIs to External Users

2021 ◽  
Vol 5 ◽  
Author(s):  
Max Patiiuk
Keyword(s):  
Data Management ◽  
Service Integration ◽  
Lessons Learned ◽  
Management Platform ◽  
Industry Standards ◽  
Service Oriented ◽  
Api Documentation ◽  
Machine Readable ◽  
Data Management Platform ◽  
Automatic Validation

External service integration and adherence to industry standards has become ever more important for collections data management platforms. External APIs (Application Programming Interfaces), allow for the development of bi-directional data flows critical to service integration. In contrast to service-oriented backend APIs, public APIs must have continually up-to-date, comprehensive documentation that covers common use cases, on-the-fly request validation, and meaningful error messages. OpenAPI (OpenAPI Initiative 2021), a machine-readable API documentation specification can help significantly with testing and maintenance, and libraries can be used to automate common maintenance tasks. Specify 7 is a biological collections data management platform developed by the Specify Collections Consortium (Specify Software Consortium 2021). This presentation summarizes the challenges and lessons learned with publishing the existing backend Specify 7 API to a public-facing external API. Each Specify 7 API is composed of 200 resources. A standard set of CRUD (Create, Read, Update, Delete) operations is provided for each resource for client interaction with a group of service-based endpoints for bulk operations such as file uploads, file-based data imports, and attachment manipulation. To support the migration, we developed a custom library to enhance request validation. Parameter validation is extended through a real-time comparison against the existing schema and data. The library is available to the community under a MIT license on GitHub (https://github.com/specify/open_api_tools/). In this presentation, we will close with an overview of the next steps for the Specify 7 public API. These include: An update to the latest OpenAPI specification, version 3.1. The latest version aims to increase compatibility with the Javascript Object Notation (JSON) Schema specification, and thus would allow us to use JSON Schema (IETF Trust 2021) validation frameworks. An in-depth evaluation of GraphQL for its ability to force all endpoints to be strongly typed and automatic validation of request parameters and response objects. An update to the latest OpenAPI specification, version 3.1. The latest version aims to increase compatibility with the Javascript Object Notation (JSON) Schema specification, and thus would allow us to use JSON Schema (IETF Trust 2021) validation frameworks. An in-depth evaluation of GraphQL for its ability to force all endpoints to be strongly typed and automatic validation of request parameters and response objects.

Developing a Data Management Platform for the Ocean Science Community

2018 ◽  
Vol 52 (3) ◽  
pp. 28-32 ◽  
Author(s):  
Chris Turner ◽  
Ian Gill
Keyword(s):  
Data Collection ◽  
Data Management ◽  
High Performance ◽  
Project Planning ◽  
Data Sets ◽  
Management Platform ◽  
Ocean Science ◽  
Science Community ◽  
Oceanographic Data ◽  
Data Management Platform

AbstractThe management of oceanographic data is particularly challenging given the variety of protocols for the analysis of data collection and model output, the vast range of environmental conditions studied, and the potentially enormous extent and volume of the resulting data sets and model results. Here, we describe the Research Workspace (the Workspace), a web platform designed around data management best practices to meet the challenges of managing oceanographic data throughout the research life cycle. The Workspace features secure user accounts and automatic file versioning to assist with the early stages of project planning and data collection. Jupyter Notebooks have been integrated into the Workspace to support reproducible numerical analysis and data visualization while making use of high-performance computer resources collocated with data assets. An ISO-compliant metadata editor has also been integrated into the Workspace to support data synthesis, publication, and reuse. The Workspace currently supports stakeholders across the ocean science community, from funding agencies to individual investigators, by providing a data management platform to meet the needs of big ocean data.

scholarly journals Experimental Pipeline (Expipe): A Lightweight Data Management Platform to Simplify the Steps From Experiment to Data Analysis

2020 ◽  
Vol 14 ◽  
Author(s):  
Mikkel Elle Lepperød ◽  
Svenn-Arne Dragly ◽  
Alessio Paolo Buccino ◽  
Milad Hobbi Mobarhan ◽  
Anders Malthe-Sørenssen ◽  
...  
Keyword(s):  
Data Analysis ◽  
Data Management ◽  
Management Platform ◽  
Data Management Platform
Sign in / Sign up

Export Citation Format

Share Document