Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebooks

Computational notebooks allow data scientists to express their ideas through a combination of code and documentation. However, data scientists often pay attention only to the code, and neglect creating or updating their documentation during quick iterations. Inspired by human documentation practices learned from 80 highly-voted Kaggle notebooks, we design and implement Themisto, an automated documentation generation system to explore how human-centered AI systems can support human data scientists in the machine learning code documentation scenario. Themisto facilitates the creation of documentation via three approaches: a deep-learning-based approach to generate documentation for source code, a query-based approach to retrieve online API documentation for source code, and a user prompt approach to nudge users to write documentation. We evaluated Themisto in a within-subjects experiment with 24 data science practitioners, and found that automated documentation generation techniques reduced the time for writing documentation, reminded participants to document code they would have ignored, and improved participants’ satisfaction with their computational notebook.

An interview with Joanna Suau from Infobip on the design of application programming interface (API) documentation

About Joanna Suau Joanna studied English literature and culture at the University of Silesia in Poland, where she was born. She did a technical writing postgraduate degree in the picturesque city of Krakow and moved to the U.K. in 2012, to work for shipping solutions provider Pierbridge, where she mainly focused on user guides and walkthroughs of various types of shipping applications. Interested in what makes an app tick, Joanna started learning programming language (JavaScript) and explored CSS and HTML in more detail. This is when she discovered her passion for writing clean and appealing developer-oriented documentation, and moved to the start-up company Moltin, to become a part of the Developer Success team. Joanna has changed industry, and currently works in the field of telecommunication. She works for a messaging services provider, Infobip, contributing content to their robust API solutions.

A Case Study of Publishing Internal APIs to External Users

External service integration and adherence to industry standards has become ever more important for collections data management platforms. External APIs (Application Programming Interfaces), allow for the development of bi-directional data flows critical to service integration. In contrast to service-oriented backend APIs, public APIs must have continually up-to-date, comprehensive documentation that covers common use cases, on-the-fly request validation, and meaningful error messages. OpenAPI (OpenAPI Initiative 2021), a machine-readable API documentation specification can help significantly with testing and maintenance, and libraries can be used to automate common maintenance tasks. Specify 7 is a biological collections data management platform developed by the Specify Collections Consortium (Specify Software Consortium 2021). This presentation summarizes the challenges and lessons learned with publishing the existing backend Specify 7 API to a public-facing external API. Each Specify 7 API is composed of 200 resources. A standard set of CRUD (Create, Read, Update, Delete) operations is provided for each resource for client interaction with a group of service-based endpoints for bulk operations such as file uploads, file-based data imports, and attachment manipulation. To support the migration, we developed a custom library to enhance request validation. Parameter validation is extended through a real-time comparison against the existing schema and data. The library is available to the community under a MIT license on GitHub (https://github.com/specify/open_api_tools/). In this presentation, we will close with an overview of the next steps for the Specify 7 public API. These include: An update to the latest OpenAPI specification, version 3.1. The latest version aims to increase compatibility with the Javascript Object Notation (JSON) Schema specification, and thus would allow us to use JSON Schema (IETF Trust 2021) validation frameworks. An in-depth evaluation of GraphQL for its ability to force all endpoints to be strongly typed and automatic validation of request parameters and response objects. An update to the latest OpenAPI specification, version 3.1. The latest version aims to increase compatibility with the Javascript Object Notation (JSON) Schema specification, and thus would allow us to use JSON Schema (IETF Trust 2021) validation frameworks. An in-depth evaluation of GraphQL for its ability to force all endpoints to be strongly typed and automatic validation of request parameters and response objects.

Applying Model-Driven Engineering to Stimulate the Adoption of DevOps Processes in Small and Medium-Sized Development Organizations

Microservice architecture (MSA) denotes an increasingly popular architectural style in which business capabilities are wrapped into autonomously developable and deployable software components called microservices. Microservice applications are developed by multiple DevOps teams each owning one or more services. In this article, we explore the state of how DevOps teams in small and medium-sized organizations (SMOs) cope with MSA and how they can be supported. We show through a secondary analysis of an exploratory interview study comprising six cases, that the organizational and technological complexity resulting from MSA poses particular challenges for small and medium-sized organizations (SMOs). We apply model-driven engineering to address these challenges. As results of the second analysis, we identify the challenge areas of building and maintaining a common architectural understanding, and dealing with deployment technologies. To support DevOps teams of SMOs in coping with these challenges, we present a model-driven workflow based on LEMMA—the Language Ecosystem for Modeling Microservice Architecture. To implement the workflow, we extend LEMMA with the functionality to (i) generate models from API documentation; (ii) reference remote models owned by other teams; (iii) generate deployment specifications; and (iv) generate a visual representation of the overall architecture. We validate the model-driven workflow and our extensions to LEMMA through a case study showing that the added functionality to LEMMA can bring efficiency gains for DevOps teams. To develop best practices for applying our workflow to maximize efficiency in SMOs, we plan to conduct more empirical research in the field in the future.

Graph-Augmented Code Summarization in Computational Notebooks

Computational notebooks allow data scientists to express their ideas through a combination of code and documentation. However, data scientists often pay attention only to the code and neglect the creation of the documentation in a notebook. In this work, we present a human-centered automation system, Themisto, that can support users to easily create documentation via three approaches: 1) We have developed and reported a GNN-augmented code documentation generation algorithm in a previous paper, which can generate documentation for a given source code; 2) Themisto also implements a query-based approach to retrieve the online API documentation as the summary for certain types of source code; 3) Lastly, Themistoalso enables a user prompt approach to motivate users to write documentation for some use cases that automation does not work well.

Scout-bot: Leveraging API Community Knowledge for Exploration and Discovery of API Learning Resources

Application Programming Interface (API) is a core technology that facilitates developers’ productivity by enabling the reuse of software components. Understanding APIs and gaining knowledge about their usage are therefore fundamental needs for developers. Here, API documentation plays a pivotal role in enabling developers to take full advantage of the benefits brought by APIs. The quality of API documentation has therefore become an important concern given the celerity and dynamics at which APIs are now being made available to users. This article aims at exploring existing research in the area of API documentation in order to identify the associated quality dimensions addressed by the literature. The research is carried out as a systematic mapping study where 103 research papers selected from the literature were reviewed and a total of 5 core quality dimensions were identified and analyzed. By focusing on the two most relevant quality dimensions (understandability and completeness), this article presents an approach to enable API users to explore, discover and learn about APIs through API topic issues discussed in Stack Overflow (SO). We demonstrate the feasibility of our approach through Scout-bot, our tool for exploration and discovery of API topic issues.

Automatic API Usage Scenario Documentation from Technical Q&A Sites

The online technical Q&A site Stack Overflow (SO) is popular among developers to support their coding and diverse development needs. To address shortcomings in API official documentation resources, several research works have thus focused on augmenting official API documentation with insights (e.g., code examples) from SO. The techniques propose to add code examples/insights about APIs into its official documentation. Recently, surveys of software developers find that developers in SO consider the combination of code examples and reviews about APIs as a form of API documentation, and that they consider such a combination to be more useful than official API documentation when the official resources can be incomplete, ambiguous, incorrect, and outdated. Reviews are opinionated sentences with positive/negative sentiments. However, we are aware of no previous research that attempts to automatically produce API documentation from SO by considering both API code examples and reviews. In this article, we present two novel algorithms that can be used to automatically produce API documentation from SO by combining code examples and reviews towards those examples. The first algorithm is called statistical documentation, which shows the distribution of positivity and negativity around the code examples of an API using different metrics (e.g., star ratings). The second algorithm is called concept-based documentation, which clusters similar and conceptually relevant usage scenarios. An API usage scenario contains a code example, a textual description of the underlying task addressed by the code example, and the reviews (i.e., opinions with positive and negative sentiments) from other developers towards the code example. We deployed the algorithms in Opiner, a web-based platform to aggregate information about APIs from online forums. We evaluated the algorithms by mining all Java JSON-based posts in SO and by conducting three user studies based on produced documentation from the posts. The first study is a survey, where we asked the participants to compare our proposed algorithms against a Javadoc-syle documentation format (called as Type-based documentation in Opiner). The participants were asked to compare along four development scenarios (e.g., selection, documentation). The participants preferred our proposed two algorithms over type-based documentation. In our second user study, we asked the participants to complete four coding tasks using Opiner and the API official and informal documentation resources. The participants were more effective and accurate while using Opiner. In a subsequent survey, more than 80% of participants asked the Opiner documentation platform to be integrated into the formal API documentation to complement and improve the API official documentation.