Deep Understanding of Technical Documents: An Enhancement on Diagrams Understanding

Humans are capable of understanding the knowledge that is included in technical documents automatically by consciously combining the information that is presented in the document’s individual modalities. These modalities are mathematical formulas, charts, tables, diagram images and etc. In this paper, we significantly enhance a previously presented technical document understanding methodology3 that emulates the way that humans also perceive information. More specifically, we make the original diagram understanding methodology adaptive to larger architectures with more complex structures and modules. The overall understanding methodology results in the generation of a Stochastic Petri-net (SPN) graph that describes the system’s functionality. Finally, we conclude with the introduction of the hierarchical association of different diagram images from the same technical document. This processing step aims to provide a holistic understanding of all illustrated diagram information.

Decontextualization: Making Sentences Stand-Alone

Models for question answering, dialogue agents, and summarization often interpret the meaning of a sentence in a rich context and use that meaning in a new context. Taking excerpts of text can be problematic, as key pieces may not be explicit in a local window. We isolate and define the problem of sentence decontextualization: taking a sentence together with its context and rewriting it to be interpretable out of context, while preserving its meaning. We describe an annotation procedure, collect data on the Wikipedia corpus, and use the data to train models to automatically decontextualize sentences. We present preliminary studies that show the value of sentence decontextualization in a user-facing task, and as preprocessing for systems that perform document understanding. We argue that decontextualization is an important subtask in many downstream applications, and that the definitions and resources provided can benefit tasks that operate on sentences that occur in a richer context.