scholarly journals Iteratively Questioning and Answering for Interpretable Legal Judgment Prediction

2020 ◽  
Vol 34 (01) ◽  
pp. 1250-1257 ◽  
Author(s):  
Haoxi Zhong ◽  
Yuzhong Wang ◽  
Cunchao Tu ◽  
Tianyang Zhang ◽  
Zhiyuan Liu ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Gender Bias ◽  
Ethical Issues ◽  
State Of The Art ◽  
Presumption Of Innocence ◽  
The World ◽  
Comparable Performance ◽  
Reward Functions ◽  
Real World Datasets ◽  
The Given

Legal Judgment Prediction (LJP) aims to predict judgment results according to the facts of cases. In recent years, LJP has drawn increasing attention rapidly from both academia and the legal industry, as it can provide references for legal practitioners and is expected to promote judicial justice. However, the research to date usually suffers from the lack of interpretability, which may lead to ethical issues like inconsistent judgments or gender bias. In this paper, we present QAjudge, a model based on reinforcement learning to visualize the prediction process and give interpretable judgments. QAjudge follows two essential principles in legal systems across the world: Presumption of Innocence and Elemental Trial. During inference, a Question Net will select questions from the given set and an Answer Net will answer the question according to the fact description. Finally, a Predict Net will produce judgment results based on the answers. Reward functions are designed to minimize the number of questions asked. We conduct extensive experiments on several real-world datasets. Experimental results show that QAjudge can provide interpretable judgments while maintaining comparable performance with other state-of-the-art LJP models. The codes can be found from https://github.com/thunlp/QAjudge.

scholarly journals Multi-Task Reinforcement Learning in Humans

2019 ◽  
Author(s):  
Momchil S. Tomov ◽  
Eric Schulz ◽  
Samuel J. Gershman
Keyword(s):  
Reinforcement Learning ◽  
State Of The Art ◽  
Task Demands ◽  
Human Intelligence ◽  
Value Functions ◽  
Complex Environments ◽  
Multiple Features ◽  
Model Free ◽  
Reward Functions ◽  
Study Participants

ABSTRACTThe ability to transfer knowledge across tasks and generalize to novel ones is an important hallmark of human intelligence. Yet not much is known about human multi-task reinforcement learning. We study participants’ behavior in a novel two-step decision making task with multiple features and changing reward functions. We compare their behavior to two state-of-the-art algorithms for multi-task reinforcement learning, one that maps previous policies and encountered features to new reward functions and one that approximates value functions across tasks, as well as to standard model-based and model-free algorithms. Across three exploratory experiments and a large preregistered experiment, our results provide strong evidence for a strategy that maps previously learned policies to novel scenarios. These results enrich our understanding of human reinforcement learning in complex environments with changing task demands.

scholarly journals What Makes A Good Story? Designing Composite Rewards for Visual Storytelling

2020 ◽  
Vol 34 (05) ◽  
pp. 7969-7976
Author(s):  
Junjie Hu ◽  
Yu Cheng ◽  
Zhe Gan ◽  
Jingjing Liu ◽  
Jianfeng Gao ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
State Of The Art ◽  
Quality Criteria ◽  
High Quality ◽  
Visual Storytelling ◽  
Learning Framework ◽  
Good Story ◽  
Human Evaluation ◽  
Reward Functions ◽  
New Criteria

Previous storytelling approaches mostly focused on optimizing traditional metrics such as BLEU, ROUGE and CIDEr. In this paper, we re-examine this problem from a different angle, by looking deep into what defines a natural and topically-coherent story. To this end, we propose three assessment criteria: relevance, coherence and expressiveness, which we observe through empirical analysis could constitute a “high-quality” story to the human eye. We further propose a reinforcement learning framework, ReCo-RL, with reward functions designed to capture the essence of these quality criteria. Experiments on the Visual Storytelling Dataset (VIST) with both automatic and human evaluation demonstrate that our ReCo-RL model achieves better performance than state-of-the-art baselines on both traditional metrics and the proposed new criteria.

scholarly journals LTL and Beyond: Formal Languages for Reward Function Specification in Reinforcement Learning

2019 ◽  
Author(s):  
Alberto Camacho ◽  
Rodrigo Toro Icarte ◽  
Toryn Q. Klassen ◽  
Richard Valenzano ◽  
Sheila A. McIlraith
Keyword(s):  
Reinforcement Learning ◽  
Normal Form ◽  
State Of The Art ◽  
Formal Languages ◽  
Function Structure ◽  
Q Learning ◽  
Reward Function ◽  
Form Representation ◽  
Reward Shaping ◽  
Reward Functions

In Reinforcement Learning (RL), an agent is guided by the rewards it receives from the reward function. Unfortunately, it may take many interactions with the environment to learn from sparse rewards, and it can be challenging to specify reward functions that reflect complex reward-worthy behavior. We propose using reward machines (RMs), which are automata-based representations that expose reward function structure, as a normal form representation for reward functions. We show how specifications of reward in various formal languages, including LTL and other regular languages, can be automatically translated into RMs, easing the burden of complex reward function specification. We then show how the exposed structure of the reward function can be exploited by tailored q-learning algorithms and automated reward shaping techniques in order to improve the sample efficiency of reinforcement learning methods. Experiments show that these RM-tailored techniques significantly outperform state-of-the-art (deep) RL algorithms, solving problems that otherwise cannot reasonably be solved by existing approaches.

scholarly journals Rebalancing Expanding EV Sharing Systems with Deep Reinforcement Learning

2020 ◽  
Author(s):  
Man Luo ◽  
Wenzhe Zhang ◽  
Tianyou Song ◽  
Kun Li ◽  
Hongming Zhu ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
State Of The Art ◽  
Charging Time ◽  
Novel Approach ◽  
The World ◽  
Net Revenue ◽  
Multi Agent ◽  
User Demand ◽  
Policy Optimization ◽  
Over Time

Electric Vehicle (EV) sharing systems have recently experienced unprecedented growth across the world. One of the key challenges in their operation is vehicle rebalancing, i.e., repositioning the EVs across stations to better satisfy future user demand. This is particularly challenging in the shared EV context, because i) the range of EVs is limited while charging time is substantial, which constrains the rebalancing options; and ii) as a new mobility trend, most of the current EV sharing systems are still continuously expanding their station networks, i.e., the targets for rebalancing can change over time. To tackle these challenges, in this paper we model the rebalancing task as a Multi-Agent Reinforcement Learning (MARL) problem, which directly takes the range and charging properties of the EVs into account. We propose a novel approach of policy optimization with action cascading, which isolates the non-stationarity locally, and use two connected networks to solve the formulated MARL. We evaluate the proposed approach using a simulator calibrated with 1-year operation data from a real EV sharing system. Results show that our approach significantly outperforms the state-of-the-art, offering up to 14% gain in order satisfied rate and 12% increase in net revenue.

scholarly journals CURIOSITY-DRIVEN REINFORCEMENT LEARNING AGENT FOR MAPPING UNKNOWN INDOOR ENVIRONMENTS

2021 ◽  
Vol V-1-2021 ◽  
pp. 129-136
Author(s):  
N. Botteghi ◽  
R. Schulte ◽  
B. Sirmacek ◽  
M. Poel ◽  
C. Brune
Keyword(s):  
Artificial Intelligence ◽  
Reinforcement Learning ◽  
Mobile Robot ◽  
Indoor Environments ◽  
Reward Function ◽  
Learning Framework ◽  
Learning Agent ◽  
The World ◽  
Reward Functions ◽  
Slam Algorithm

Abstract. Autonomously exploring and mapping is one of the open challenges of robotics and artificial intelligence. Especially when the environments are unknown, choosing the optimal navigation directive is not straightforward. In this paper, we propose a reinforcement learning framework for navigating, exploring, and mapping unknown environments. The reinforcement learning agent is in charge of selecting the commands for steering the mobile robot, while a SLAM algorithm estimates the robot pose and maps the environments. The agent, to select optimal actions, is trained to be curious about the world. This concept translates into the introduction of a curiosity-driven reward function that encourages the agent to steer the mobile robot towards unknown and unseen areas of the world and the map. We test our approach in explorations challenges in different indoor environments. The agent trained with the proposed reward function outperforms the agents trained with reward functions commonly used in the literature for solving such tasks.

scholarly journals Representation Learning for Grounded Spatial Reasoning

2018 ◽  
Vol 6 ◽  
pp. 49-61 ◽  
Author(s):  
Michael Janner ◽  
Karthik Narasimhan ◽  
Regina Barzilay
Keyword(s):  
Reinforcement Learning ◽  
Spatial Reasoning ◽  
State Of The Art ◽  
Representation Learning ◽  
Localization Error ◽  
Value Iteration ◽  
Joint Inference ◽  
Simulated Environment ◽  
Proposed Model ◽  
The World

The interpretation of spatial references is highly contextual, requiring joint inference over both language and the environment. We consider the task of spatial reasoning in a simulated environment, where an agent can act and receive rewards. The proposed model learns a representation of the world steered by instruction text. This design allows for precise alignment of local neighborhoods with corresponding verbalizations, while also handling global references in the instructions. We train our model with reinforcement learning using a variant of generalized value iteration. The model outperforms state-of-the-art approaches on several metrics, yielding a 45% reduction in goal localization error.

scholarly journals Exploring Clustering-Based Reinforcement Learning for Personalized Book Recommendation in Digital Library

Information ◽  
2021 ◽  
Vol 12 (5) ◽  
pp. 198
Author(s):  
Xinhua Wang ◽  
Yuchen Wang ◽  
Lei Guo ◽  
Liancheng Xu ◽  
Baozhong Gao ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Digital Library ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Decision Making Process ◽  
Learning Method ◽  
Large Collection ◽  
Recommendation Algorithms ◽  
Small Set ◽  
Real World Datasets

Digital library as one of the most important ways in helping students acquire professional knowledge and improve their professional level has gained great attention in recent years. However, its large collection (especially the book resources) hinders students from finding the resources that they are interested in. To overcome this challenge, many researchers have already turned to recommendation algorithms. Compared with traditional recommendation tasks, in the digital library, there are two challenges in book recommendation problems. The first is that users may borrow books that they are not interested in (i.e., noisy borrowing behaviours), such as borrowing books for classmates. The second is that the number of books in a digital library is usually very large, which means one student can only borrow a small set of books in history (i.e., data sparsity issue). As the noisy interactions in students’ borrowing sequences may harm the recommendation performance of a book recommender, we focus on refining recommendations via filtering out data noises. Moreover, due to the the lack of direct supervision information, we treat noise filtering in sequences as a decision-making process and innovatively introduce a reinforcement learning method as our recommendation framework. Furthermore, to overcome the sparsity issue of students’ borrowing behaviours, a clustering-based reinforcement learning algorithm is further developed. Experimental results on two real-world datasets demonstrate the superiority of our proposed method compared with several state-of-the-art recommendation methods.

A Keyphrase-Based Approach to Text Summarization for English and Bengali Documents

2014 ◽  
Vol 5 (2) ◽  
pp. 28-38 ◽  
Author(s):  
Kamal Sarkar
Keyword(s):  
World Wide Web ◽  
Rapid Growth ◽  
World Wide ◽  
Information Overload ◽  
State Of The Art ◽  
Text Summarization ◽  
Text Document ◽  
Web Information ◽  
The World ◽  
The Given

With the rapid growth of the World Wide Web, information overload is becoming a problem for an increasingly large number of people. Since summarization helps human to digest the main contents of a text document very rapidly, there is a need for an effective and powerful tool that can automatically summarize text. In this paper, we present a keyphrase based approach to single document summarization that extracts first a set of keyphrases from a document, use the extracted keyphrases to choose sentences from the document and finally form an extractive summary with the chosen sentences. We view keyphrases (single or multi-word) as the important concepts and we assume that an extractive summary of a document is an elaboration of the important concepts contained in the document to some permissible extent and it is controlled by the given summary length. We have tested our proposed keyphrase-based summarization approach on two different datasets: one for English and another for Bengali. The experimental results show that the performance of the proposed system is comparable to some state-of-the art summarization systems.

Ambivalenci Pemikiran Mahmud Syaltut tentang Fikih Perempuan

1970 ◽  
Vol 5 (2) ◽  
pp. 197-214
Author(s):  
Mahmud Arif
Keyword(s):  
Gender Equality ◽  
Gender Bias ◽  
Legal Thought ◽  
Islamic Thought ◽  
Islamic Jurisprudence ◽  
The World

In general, we know about Egypt very well, because of all this time, Egypt, especially Kairo, has been viewed as one of the centers of Islamic thought in the world. Naturally this country had a lot of Islamic thinkers, like Mahmud Syaltut (d. 1963) that has become the Rector of al-Azhar University. The influence of his thought overstepped the bounds of time and political territory. The Islamic jurisprudence is an inseparable legal thought from the fulfillment of social demands. One of the evidences is its’ response to actual issues, like gender equality represented in his opinions about domestical duty, women testimony, girl marriage, and poligamy. As a thinker in the Islamic jurisprudence, Syaltut has endeavored to respond such issues, including gender. As a reformer in the turbulent time, his reflection on such matters expressed critical preference, so frequently looked different from the prevalent opinion. In one side, his reflection was “liberal” because of his bravery in stepping beyond the Islamic orthodoxy and the modernity, but in another side, his thought was “conservative”if it was viewed from his endorsement to the old Islamic thought that reflected a gender bias. This showed the uniqueness and the ambivalence of his thought, so very interesting to being studied.

The Facticity of Things – Reframing Slotawa's Practice with Meillassoux and Harman

2015 ◽  
Vol 60 (1) ◽  
pp. 55-79
Author(s):  
Robert Jackson
Keyword(s):  
Ordinary Objects ◽  
Speculative Realism ◽  
The World ◽  
The Given

Robert Jackson examines the work of the German artist Florian Slotawa. Beginning with his first works, “Hotelarbeiten”, Slotawa recomposes and reconfigures the order of ordinary objects – in this case, the furniture of hotel rooms. In reconstructing these rooms in another order without altering these objects in any way, photographing them, and then subsequently restoring them to their previous configuration, the artist reveals the ordinary function of the objects and by withdrawing from their function shows their material and factual character. To elucidate the specificity of Slotawa’s intervention, Jackson critiques Heidegger’s conception of facticity in its exclusive account of Dasein and its being-in-the world, in contrast to the factuality of “things-within-the world.” Drawing on Harman’s extension of finitude beyond Dasein to all things, he encourages us to see Slotawa as engaged in “facticity of things” that is characterized by dispossession, lack of reason, and radical contingency. As Jackson argues, Slotawa is trying to find a way to dwell in a world that has no room or possibility for the given coordinates of dwelling; a world that is a fact without reason. In concluding he explores a reading of Slotawa that explores the intersecting yet radically different approaches to thinking about a speculative realism in the work of Harman and Meillassoux, and their differing attitudes to the finite and the infinite, facticity and factiality, contingency and necessity, without presuming to assume that either of these accounts cover the speculative facticity of things revealed in Slotawa’s work.

Sign in / Sign up

Export Citation Format

Share Document