Structural Markup of Official Documents in Diachronic Linguistic Corpus: Problems and Solutions

The research relevance is determined by the need to annotate official documents of Don Cossack Host written in the middle of the 18 th century and kept in "Mikhailovsky Stanitsa Ataman" archive fund of the State Archive of the Volgograd Region (SAVR, fund 332, inventory 1), so as to compile a linguistic corpus. The authors characterize the problems of the deposited documentary text structural division. These difficulties occur due to the specifics of the form, the dynamics of genres and the syntactical peculiarities of business communication in the middle of the 18 th century. It is revealed that the complexity of documentary text division depends on the degree of its narrativity. The choice of a structural-semantic segment that coincides with a sentence or several closely connected sentences as a layout unit is motivated. A complex method of document segmentation for the structural markup is justified. The approach is based on genre parameterization of documents and their syntactic segmentation. It has been established that the segment boundaries can be indicated by the complex of graphic symbols, speech formulas that perform the function of details of payments, lexical and grammatical means. As a result of the study, it has been shown that the succession of procedures implemented for text segmentation, and targeted at genre and speech organization of the document identification, makes it possible to present in the diachronic corpus the information, which is necessary and sufficient for the user to conclude about the properties of the document text and its units.

MUSED: A multimedia multi-document dataset for topic segmentation

Research on topic segmentation has recently focused on segmenting documents by taking advantage of documents covering the same topics. In order to properly evaluate such approaches, a dataset of related documents is needed. However, existing datasets are limited in the number of related documents per domain. In addition, most of the available datasets do not consider documents from different media sources (PowerPoints, videos, etc.), which pose specific challenges to segmentation. We fill this gap with the MUltimedia SEgmentation Dataset (MUSED), a collection of documents manually segmented, from different media sources, in seven different domains, with an average of twenty related documents per domain. In this paper, we describe the process of building MUSED. A multi-annotator study is carried out to determine if it is possible to observe agreement among human judges and characterize their disagreement patterns. In addition, we use MUSED to compare the state-of-the-art topic segmentation techniques, including the ones that take advantage of related documents. Moreover, we study the impact of having documents from different media sources in the dataset. To the best of our knowledge, MUSED is the first dataset that allows a straightforward evaluation of both single- and multiple-documents topic segmentation techniques, as well as to study how these behave in the presence of documents from different media sources. Results show that some techniques are, indeed, sensitive to different media sources, and also that current multi-document segmentation models do not outperform previous models, pointing to a research line that needs to be boosted.