Evaluating Human versus Machine Learning Performance in a LegalTech Problem

Many machine learning-based document processing applications have been published in recent years. Applying these methodologies can reduce the cost of labor-intensive tasks and induce changes in the company’s structure. The artificial intelligence-based application can replace the application of trainees and free up the time of experts, which can increase innovation inside the company by letting them be involved in tasks with greater added value. However, the development cost of these methodologies can be high, and usually, it is not a straightforward task. This paper presents a survey result, where a machine learning-based legal text labeler competed with multiple people with different legal domain knowledge. The machine learning-based application used binary SVM-based classifiers to resolve the multi-label classification problem. The used methods were encapsulated and deployed as a digital twin into a production environment. The results show that machine learning algorithms can be effectively utilized for monotonous but domain knowledge- and attention-demanding tasks. The results also suggest that embracing the machine learning-based solution can increase discoverability and enrich the value of data. The test confirmed that the accuracy of a machine learning-based system matches up with the long-term accuracy of legal experts, which makes it applicable to automatize the working process.

Capturing the Basics of the GDPR in a Well-Founded Legal Domain Modular Ontology

The primary goal of the General Data Protection Regulation (GDPR) is to regulate the rights and duties of citizens and organizations over personal data protection. Implementing the GDPR is recently gaining much importance for legal reasoning and compliance checking purposes. In this work, we aim to capture the basics of GDPR in a well-founded legal domain modular ontology named OPPD (Ontology for the Protection of Personal Data). Ontology-Driven Conceptual Modeling (ODCM), ontology layering, modularization, and reuse processes are applied. These processes aim to support the ontology engineer in overcoming the complexity of the legal knowledge and developing an ontology model faithful to reality. ODCM is used for grounding OPPD in the Unified Foundational Ontology (UFO). Ontology modularization and layering aim to simplify the ontology building process. Ontology reuse focuses on selecting and reusing Conceptual Ontology Patterns (COPs) from UFO and the legal core ontology UFO-L. OPPD intends to overcome the lack of a representation of legal procedures that most ontologies encountered. The potential use of OPPD is proposed to formalize the GDPR rules by combining ontological reasoning and Logic Programming.

Argumentation and Defeasible Reasoning in the Law

Different formalisms for defeasible reasoning have been used to represent knowledge and reason in the legal field. In this work, we provide an overview of the following logic-based approaches to defeasible reasoning: defeasible logic, Answer Set Programming, ABA+, ASPIC+, and DeLP. We compare features of these approaches under three perspectives: the logical model (knowledge representation), the method (computational mechanisms), and the technology (available software resources). On top of that, two real examples in the legal domain are designed and implemented in ASPIC+ to showcase the benefit of an argumentation approach in real-world domains. The CrossJustice and Interlex projects are taken as a testbed, and experiments are conducted with the Arg2P technology.

The Unreasonable Effectiveness of the Baseline: Discussing SVMs in Legal Text Classification

We aim to highlight an interesting trend to contribute to the ongoing debate around advances within legal Natural Language Processing. Recently, the focus for most legal text classification tasks has shifted towards large pre-trained deep learning models such as BERT. In this paper, we show that a more traditional approach based on Support Vector Machine classifiers reaches competitive performance with deep learning models. We also highlight that error reduction obtained by using specialised BERT-based models over baselines is noticeably smaller in the legal domain when compared to general language tasks. We discuss some hypotheses for these results to support future discussions.

Data-Centric Machine Learning: Improving Model Performance and Understanding Through Dataset Analysis

Machine learning research typically starts with a fixed data set created early in the process. The focus of the experiments is finding a model and training procedure that result in the best possible performance in terms of some selected evaluation metric. This paper explores how changes in a data set influence the measured performance of a model. Using three publicly available data sets from the legal domain, we investigate how changes to their size, the train/test splits, and the human labelling accuracy impact the performance of a trained deep learning classifier. Our experiments suggest that analyzing how data set properties affect performance can be an important step in improving the results of trained classifiers, and leads to better understanding of the obtained results.

Accounting for Sentence Position and Legal Domain Sentence Embedding in Learning to Classify Case Sentences

In this paper, we treat sentence annotation as a classification task. We employ sequence-to-sequence models to take sentence position information into account in identifying case law sentences as issues, conclusions, or reasons. We also compare the legal domain specific sentence embedding with other general purpose sentence embeddings to gauge the effect of legal domain knowledge, captured during pre-training, on text classification. We deployed the models on both summaries and full-text decisions. We found that the sentence position information is especially useful for full-text sentence classification. We also verified that legal domain specific sentence embeddings perform better, and that meta-sentence embedding can further enhance performance when sentence position information is included.

Towards Reducing the Pendency of Cases at Court: Automated Case Analysis of Supreme Court Judgments in India

The Indian court system generates huge amounts of data relating to administration, pleadings, litigant behaviour, and court decisions on a regular basis. But the existing Judiciary is incapable of managing these vast troves of data efficiently that causes delays and pendency of a large volume of cases in the courts. Some of these time-consuming tasks involve case briefing, examining the legal issues, facts, legal principles, observations, and other significant aspects submitted by the contending parties in the court. In other words, computational methods to understand the underlying structure of a case document will directly aid the lawyers to perform these tasks efficiently and improve the overall efficiency of the Justice delivery system. Application of Computational techniques (such as Natural Language Processing) can help to gather and sift through these vast troves of information, identify patterns, extract the document structure, draft documents and make the information available online. Traditionally lawyers are trained to examine cases using the Case Law Analysis approach for case briefing. In this article, the authors aim to establish the importance and relevance of the automated case analysis problem in the legal domain. They introduce a novel case analysis structure for the supreme court judgment documents and define twelve different case law labels that are used by legal professionals to identify the structure. Finally the authors propose a method for automated case analysis, which will directly aid the lawyers to prepare speedy and efficient case briefs and drastically reduce the time taken by them in litigation.

Social Media-Tion: A Constructive Approach to Dispute Resolution?

In the twenty-first century which is regarded as the dawn of the social media age, the disputants, as well as the legal professionals such as advocates and judges, embrace the information available at their disposal on several social media platforms. It has altered the conduct of arbitration by changing the way disputants communicate. Being the modern tool for communication, it has elevated the speed and dissemination of information, which allows audiences to follow the dispute and express their support or dissatisfaction towards the disputants. As a consequence, the parties seeking redressal of their grievances through ADR get influenced due to the formation of ‘unconscious bias’. Communication is the epitome of the dispute resolution process, and the intervention of social media in the process generates a ghost syndrome, thus, resulting in the fading of such epitome. Its impact is not restricted to the parties but has the potential to undermine the independence, integrity, and impartiality of the judge or the mediator. Social Media has become significant within the legal domain as technology penetrates all ambits of individual endeavors. Looking towards the positive contributions, it acts as a source of evidence, especially in employment and labor disputes. Transformations in communication technologies have altered the definition of power in international arbitration, the class of individuals participating in the process, and strategies employed to mediate the conflict. The paper intends to discuss the elite usage and manipulation of social media impacting ADR, the cases influenced by it, and the theoretical framework required for its conduct.

An overview of information extraction techniques for legal document analysis and processing

<span>In an Indian law system, different courts publish their legal proceedings every month for future reference of legal experts and common people. Extensive manual labor and time are required to analyze and process the information stored in these lengthy complex legal documents. Automatic legal document processing is the solution to overcome drawbacks of manual processing and will be very helpful to the common man for a better understanding of a legal domain. In this paper, we are exploring the recent advances in the field of legal text processing and provide a comparative analysis of approaches used for it. In this work, we have divided the approaches into three classes NLP based, deep learning-based and, KBP based approaches. We have put special emphasis on the KBP approach as we strongly believe that this approach can handle the complexities of the legal domain well. We finally discuss some of the possible future research directions for legal document analysis and processing.</span>