Data preprocessing for machine analysis of sales representatives’ key performance indicators

Significant transformation of the operational activity of product and service distributors is driven by changes in data-receiving and processing technology. At present, the work of these companies’ representatives is digitized to a large extent: for example, the road time, the number and places of meetings with customers are automatically recorded. At the same time, the productivity of managers who do not make direct sales is usually evaluated with the help of surveys, experts and costly double visits, although the existence of large data samples makes possible the use of statistical analysis to identify both insufficient and inflated values of performance indicators. Source data: a relational database that accumulates information about 28 categorical, quantitative, geolocation and temporal parameters of sale representatives’ activities for the last year. Based on available data, we created synthetic features (the latitude and longitude features produced the index, region, street, and house features; based upon identifiers we calculated the sum of activities of sales representatives; according to temporary features we defined the season of the year, the day of the week and the period of day features). The methodology for statistical analysis consists of three main stages: collection and processing of primary data; summary and grouping processed information; setting statistical hypotheses and interpreting the results. A probabilistic approach was used to model the level of distortion of sale representatives’ activities. As a result, with the built tag cloud we highlighted: the most popular season for advertising campaigns; the most productive departments and sale representatives; days of the week with the largest number of contacts to customers. We established a significant number of records about meetings with clients at the weekends. As a result of the data mining, we made a statistical hypothesis about the possibility of identifying the sale representatives who distort the number and parameters of meetings. A set of synthetic integer, real and categorical features was created to identify hidden relationships. Doubtful data (such as working at weekends or at night) were revealed. The resulting aggregated dataset is grouped by a sale representative’s activity ID and the distribution of this feature is plotted. For each sale representative, integer and real features are summarized and outliers that characterize inefficient performance or distortion of data have been detected. Thus, the presence of a large sample of data on the history of movements and activities allowed us to evaluate the productivity of the distribution company’s sales representatives based upon indirect features.

On Building and Evaluating a Medical Records Exploration Interface Using Text Mining Techniques

Medical records contain many terms that are difficult to process. Our aim in this study is to allow visual exploration of the information in medical databases where texts present a large number of syntactic variations and abbreviations by using an interface that facilitates content identification, navigation, and information retrieval. We propose the use of multi-term tag clouds as content representation tools and as assistants for browsing and querying tasks. The tag cloud generation is achieved by using a novelty mathematical method that allows related terms to remain grouped together within the tags. To evaluate this proposal, we have carried out a survey over a spanish database with 24,481 records. For this purpose, 23 expert users in the medical field were tasked to test the interface and answer some questions in order to evaluate the generated tag clouds properties. In addition, we obtained a precision of 0.990, a recall of 0.870, and a F1-score of 0.904 in the evaluation of the tag cloud as an information retrieval tool. The main contribution of this approach is that we automatically generate a visual interface over the text capable of capturing the semantics of the information and facilitating access to medical records, obtaining a high degree of satisfaction in the evaluation survey.

Visual Exploration of Medical Records

Medical records contain many terms which are difficult to process. Our aim in this study is to allow the visual exploration of the information in medical databases where the texts presents a large number of syntactic variations and abbreviations, through an interface which facilitates content identification, navigation and information retrieval. We propose the use of multi-term tag clouds as content representation tools and as assistants for the browsing and querying tasks. The tag cloud generation is achieved through a novelty mathematical method that allows related terms to remain grouped together within the tags To evaluate this proposal, we have used a database with 24,481 records. 23 expert users in the medical field were tasked to complete a survey to evaluate the generated tag clouds properties and we obtained a precision of 0.990, a recall of 0.870 and a F1score of 0.904 in the evaluation of the tag cloud as an information retrieval tool. The main contribution of this approach is that we automatically generate a visual interface over the text capable of capturing the semantics of the information and facilitating access to medical records.

The Glossaryfication Web Service – an automated glossary creation tool to support One Health communication

Efficient communication and collaboration across sectors is an important precondition for true One Health Surveillance (OHS) activities. Despite the overall willingness to embrace the One Health paradigm, it is still challenging to accomplish this in day-to-day practice due to the differences in terminology and interpretation of sector-specific terms. In this sense, simple interventions like the inclusion of integrative glossaries in OHS documents (e.g. reports, research papers and guidelines) would help to reduce misunderstandings and could significantly improve the written communication in OHS. Here, we present the Glossaryfication Web Service that generates a document-specific glossary for any text file provided by the user. The web service automatically adds the available definitions with their corresponding references for the words in the document that match with terms in the user-selected glossaries. The Glossaryfication Web Service was developed to provide added value to the OHEJP Glossary that was developed within the OHEJP project ORION. The OHEJP Glossary improves the communication and collaboration among OH sectors by providing an online resource that lists relevant OH terms and sector-specific definitions. The Glossaryfication Web Service supports the practical use of the curated OHEJP Glossary and can also source information from other glossaries relevant for OH professionals (currently supporting the online CDC, WHO and EFSA glossaries). The Glossaryfication Web Service was created using the open-source software KNIME and the KNIME Text Processing extension (https://www.knime.com/knime-text-processing). The Glossaryfication KNIME workflow is deployed on BfR’s KNIME Server infrastructure providing an easy-to-use web interface where the users can upload their documents (any text-type file e.g. PDF, Word, Excel) and select the desired glossary to compare with. The Glossaryfication KNIME workflow reads in the document provided via the web interface and applies natural language processing (e.g. text cleaning, stemming), transforming (bag-of-words generation) and information retrieval methods to identify the matching terms in the selected glossaries. The Glossaryfication Web Service generates as an output a table containing all the terms that match with the selected glossaries. It also provides the available definitions, corresponding references and additional meta-information, e.g. the term frequency, i.e., how often each term appears in the given text, and the sectoral classification (only for the OHEJP Glossary terms). Furthermore, the workflow generates a tag cloud where the terms are categorized as: (i) exact match when the term in the text matches exactly with the entry of this term in the glossary; (ii) inexact match when the term appears in the text slightly modified (e.g. plural forms or suffixes) and (iii) non-matching that corresponds to all the other words appearing in the text that do not match with any glossary term. Through the user interface, the users can then choose if they want to download the whole list of terms, select only the exact/inexact matching terms, or just choose those terms and definitions that match with the meaning intended for this term in the user-provided document. The resulting table of terms can be downloaded as an Excel file and added to the user’s document as a document-specific glossary. The Glossaryfication Web Service provides an easy-to-adopt solution to enrich documents and reports with more comprehensive and unambiguous glossaries. Furthermore, it improves the referentiality of terms and definitions from different OH sectors. An additional feature provided by the Glossaryfication Web Service is the possibility of extending its use to other glossaries from other national or international institutions allowing the user to customize this glossary creation service.

A survey of tag clouds as tools for information retrieval and content representation

Tag clouds are tools that have been widely used on the Internet since their conception. The main applications of these textual visualizations are information retrieval, content representation and browsing of the original text from which the tags are generated. Despite the extensive use of tag clouds, their enormous popularity and the amount of research related to different aspects of them, few studies have summarized their most important features when they work as tools for information retrieval and content representation. In this paper we present a summary of the main characteristics of tag clouds found in the literature, such as their different functions, designs and negative aspects. We also present a summary of the most popular metrics used to capture the structural properties of a tag cloud generated from the query results, as well as other measures for evaluating the goodness of the tag cloud when it works as a tool for content representation. The different methods for tagging and the semantic association processes in tag clouds are also considered. Finally we give a list of alternative for visual interfaces, which makes this study a useful first help for researchers who want to study the content representation and information retrieval interfaces in greater depth.

Artificial Intelligence Visual Metaphors in E-Learning Interfaces for Learning Analytics

This work proposes an innovative visual tool for real-time continuous learners analytics. The purpose of the work is to improve the design, functionality, and usability of learning management systems to monitor user activity to allow educators to make informed decisions on e-learning design, usually limited to dashboards graphs, tables, and low-usability user logs. The standard visualisation is currently scarce, and often inadequate to inform educators about the design quality and students engagement on their learning objects. The same low usability can be found in learning analytics tools, which mostly focus on post-course analysis, demanding specific skills to be effectively used, e.g., for statistical analysis and database queries. We propose a tool for student analytics embedded in a Learning Management System, based on the innovative visual metaphor of interface morphing. Artificial intelligence provides in remote learning immediate feedback, crucial in a face-to-face setting, highlighting the students’ engagement in each single learning object. A visual metaphor is the representation of a person, group, learning object, or concept through a visual image that suggests a particular association or point of similarity. The basic idea is that elements of the application interface, e.g., learning objects’ icons and student avatars, can be modified in colour and dimension to reflect key performance indicators of learner’s activities. The goal is to provide high-affordance information on the student engagement and usage of learning objects, where aggregation functions on subsets of users allow a dynamic evaluation of cohorts with different granularity. The proposed visual metaphors (i.e., thermometer bar, dimensional morphing, and tag cloud morphing) have been implemented and experimented within academic-level courses. Experimental results have been evaluated with a comparative analysis of user logs and a subjective usability survey, which show that the tool obtains quantitative, measurable effectiveness and the qualitative appreciation of educators. Among metaphors, the highest success is obtained by Dimensional morphing and Tag cloud transformation.

Using a multimedia semantic graph for web document visualization and summarization

The synthesis process of document content and its visualization play a basic role in the context of knowledge representation and retrieval. Existing methods for tag-clouds generations are mostly based on text content of documents, others also consider statistical or semantic information to enrich the document summary, while precious information deriving from multimedia content is often neglected. In this paper we present a document summarization and visualization technique based on both statistical and semantic analysis of textual and visual contents. The result of our framework is a Visual Semantic Tag Cloud based on the highlighting of relevant terms in a document using some features (font size, color, etc.) showing the importance of a term compared to other ones. The semantic information is derived from a knowledge base where concepts are represented through several multimedia items. The Visual Semantic Tag Cloud can be used not only to synthesize a document but also to represent a set of documents grouped by categories using a topic detection technique based on textual and visual analysis of multimedia features. Our work aims at demonstrating that with the help of semantic analysis and the combination of textual and visual features it is possible to improve the user knowledge acquisition by means of a synthesized visualization. The whole strategy has been evaluated by means of a ground truth and compared with similar approaches. Experimental results show the effectiveness of our approach, which outperforms state-of-art algorithms in topic detection combining both visual and semantic information.

Tag Clouds for Software Documents Visualization

Legacy software documents are hard to understand and visualize. The tag cloud technique helps software developers to visualize the contents of software documents. A tag cloud is a well-known and simple visualization technique. This paper proposes a new method to visualize software documents, using a tag cloud. In this paper, tags visualize in the cloud based on their frequency in an alphabetical order. The most important tags are displayed with a larger font size. The originality of this method is that it visualizes the contents of JavaDoc as a tag cloud. To validate the JavaDocCloud method, it was applied to NanoXML case study, the results of these experiments display the most common and uncommon tags used in the software documents.