scholarly journals Hierarchical learning from human preferences and curiosity

2021 ◽  
Author(s):  
Nicolas Bougie ◽  
Ryutaro Ichise
Keyword(s):  
High Dimensional ◽  
Learning Approaches ◽  
Robotic Control ◽  
Complex State ◽  
Hierarchical Learning ◽  
Hierarchical Reinforcement Learning ◽  
Recent Success ◽  
Speed Up ◽  
Human Effort ◽  
High Level

AbstractRecent success in scaling deep reinforcement algorithms (DRL) to complex problems has been driven by well-designed extrinsic rewards, which limits their applicability to many real-world tasks where rewards are naturally extremely sparse. One solution to this problem is to introduce human guidance to drive the agent’s learning. Although low-level demonstrations is a promising approach, it was shown that such guidance may be difficult for experts to demonstrate since some tasks require a large amount of high-quality demonstrations. In this work, we explore human guidance in the form of high-level preferences between sub-goals, leading to drastic reductions in both human effort and cost of exploration. We design a novel hierarchical reinforcement learning method that introduces non-expert human preferences at the high-level, and curiosity to drastically speed up the convergence of subpolicies to reach any sub-goals. We further propose a strategy based on curiosity to automatically discover sub-goals. We evaluate the proposed method on 2D navigation tasks, robotic control tasks, and image-based video games (Atari 2600), which have high-dimensional observations, sparse rewards, and complex state dynamics. The experimental results show that the proposed method can learn significantly faster than traditional hierarchical RL methods and drastically reduces the amount of human effort required over standard imitation learning approaches.

MMIR

2011 ◽  
pp. 371-387 ◽  
Author(s):  
Min Chen ◽  
Shu-Ching Chen
Keyword(s):  
Multiple Instance Learning ◽  
Image Features ◽  
Large Set ◽  
Hierarchical Learning ◽  
Retrieval Accuracy ◽  
The Neural Network ◽  
Semantic Concepts ◽  
Major Bottleneck ◽  
Speed Up ◽  
High Level

This chapter introduces an advanced content-based image retrieval (CBIR) system, MMIR, where Markov model mediator (MMM) and multiple instance learning (MIL) techniques are integrated seamlessly and act coherently as a hierarchical learning engine to boost both the retrieval accuracy and efficiency. It is well-understood that the major bottleneck of CBIR systems is the large semantic gap between the low-level image features and the high-level semantic concepts. In addition, the perception subjectivity problem also challenges a CBIR system. To address these issues and challenges, the proposed MMIR system utilizes the MMM mechanism to direct the focus on the image level analysis together with the MIL technique (with the neural network technique as its core) to real-time capture and learn the object-level semantic concepts with some help of the user feedbacks. In addition, from a long-term learning perspective, the user feedback logs are explored by MMM to speed up the learning process and to increase the retrieval accuracy for a query. The comparative studies on a large set of real-world images demonstrate the promising performance of our proposed MMIR system.

MMIR

2009 ◽  
pp. 1189-1204
Author(s):  
Min Chen ◽  
Shu-Ching Chen
Keyword(s):  
Multiple Instance Learning ◽  
Image Features ◽  
Large Set ◽  
Hierarchical Learning ◽  
Retrieval Accuracy ◽  
The Neural Network ◽  
Semantic Concepts ◽  
Major Bottleneck ◽  
Speed Up ◽  
High Level

This chapter introduces an advanced content-based image retrieval (CBIR) system, MMIR, where Markov model mediator (MMM) and multiple instance learning (MIL) techniques are integrated seamlessly and act coherently as a hierarchical learning engine to boost both the retrieval accuracy and efficiency. It is well-understood that the major bottleneck of CBIR systems is the large semantic gap between the low-level image features and the high-level semantic concepts. In addition, the perception subjectivity problem also challenges a CBIR system. To address these issues and challenges, the proposed MMIR system utilizes the MMM mechanism to direct the focus on the image level analysis together with the MIL technique (with the neural network technique as its core) to real-time capture and learn the objectlevel semantic concepts with some help of the user feedbacks. In addition, from a long-term learning perspective, the user feedback logs are explored by MMM to speed up the learning process and to increase the retrieval accuracy for a query. The comparative studies on a large set of real-world images demonstrate the promising performance of our proposed MMIR system.

scholarly journals Efficient Rank-Based Diffusion Process with Assured Convergence

2021 ◽  
Vol 7 (3) ◽  
pp. 49
Author(s):  
Daniel Carlos Guimarães Pedronette ◽  
Lucas Pascotti Valem ◽  
Longin Jan Latecki
Keyword(s):  
Diffusion Process ◽  
Learning Strategies ◽  
State Of The Art ◽  
Representation Learning ◽  
Theoretical Background ◽  
High Dimensional ◽  
Visual Features ◽  
Learning Approaches ◽  
Previous Decade ◽  
Asymptotic Complexity

Visual features and representation learning strategies experienced huge advances in the previous decade, mainly supported by deep learning approaches. However, retrieval tasks are still performed mainly based on traditional pairwise dissimilarity measures, while the learned representations lie on high dimensional manifolds. With the aim of going beyond pairwise analysis, post-processing methods have been proposed to replace pairwise measures by globally defined measures, capable of analyzing collections in terms of the underlying data manifold. The most representative approaches are diffusion and ranked-based methods. While the diffusion approaches can be computationally expensive, the rank-based methods lack theoretical background. In this paper, we propose an efficient Rank-based Diffusion Process which combines both approaches and avoids the drawbacks of each one. The obtained method is capable of efficiently approximating a diffusion process by exploiting rank-based information, while assuring its convergence. The algorithm exhibits very low asymptotic complexity and can be computed regionally, being suitable to outside of dataset queries. An experimental evaluation conducted for image retrieval and person re-ID tasks on diverse datasets demonstrates the effectiveness of the proposed approach with results comparable to the state-of-the-art.

scholarly journals A Survey of Graphical Page Object Detection with Deep Neural Networks

2021 ◽  
Author(s):  
Jwalin Bhatt ◽  
Khurram Azeem Hashmi ◽  
Muhammad Zeshan Afzal ◽  
Didier Stricker
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Conceptual Understanding ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Learning Approaches ◽  
Document Images ◽  
Essential Information ◽  
Current State ◽  
High Level

In any document, graphical elements like tables, figures, and formulas contain essential information. The processing and interpretation of such information require specialized algorithms. Off-the-shelf OCR components cannot process this information reliably. Therefore, an essential step in document analysis pipelines is to detect these graphical components. It leads to a high-level conceptual understanding of the documents that makes digitization of documents viable. Since the advent of deep learning, the performance of deep learning-based object detection has improved many folds. In this work, we outline and summarize the deep learning approaches for detecting graphical page objects in the document images. Therefore, we discuss the most relevant deep learning-based approaches and state-of-the-art graphical page object detection in document images. This work provides a comprehensive understanding of the current state-of-the-art and related challenges. Furthermore, we discuss leading datasets along with the quantitative evaluation. Moreover, it discusses briefly the promising directions that can be utilized for further improvements.

scholarly journals An Open-Source Cloud-FPGA Gene Regulatory Accelerator

2021 ◽  
Author(s):  
Lucas Bragança ◽  
Jeronimo Penha ◽  
Michael Canesche ◽  
Dener Ribeiro ◽  
José Augusto M. Nacif ◽  
...  
Keyword(s):  
Open Source ◽  
Gene Regulatory Networks ◽  
High Performance ◽  
Regulatory Networks ◽  
Cloud Services ◽  
On Demand ◽  
Speed Up ◽  
Gene Regulatory ◽  
High Level ◽  
Better Than

FPGAs are suitable to speed up gene regulatory network (GRN) algorithms with high throughput and energy efficiency. In addition, virtualizing FPGA using hardware generators and cloud resources increases the computing ability to achieve on-demand accelerations across multiple users. Recently, Amazon AWS provides high-performance Cloud's FPGAs. This work proposes an open source accelerator generator for Boolean gene regulatory networks. The generator automatically creates all hardware and software pieces from a high-level GRN description. We evaluate the accelerator performance and cost for CPU, GPU, and Cloud FPGA implementations by considering six GRN models proposed in the literature. As a result, the FPGA accelerator is at least 12x faster than the best GPU accelerator. Furthermore, the FPGA reaches the best performance per dollar in cloud services, at least 5x better than the best GPU accelerator.

scholarly journals EXPLORING ALS AND DIM DATA FOR SEMANTIC SEGMENTATION USING CNNS

2018 ◽  
Vol XLII-1 ◽  
pp. 347-354 ◽  
Author(s):  
F. Politz ◽  
M. Sester
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Good Alternative ◽  
Aerial Images ◽  
Learning Approaches ◽  
Advantages And Disadvantages ◽  
Sensing Applications ◽  
High Level

<p><strong>Abstract.</strong> Over the past years, the algorithms for dense image matching (DIM) to obtain point clouds from aerial images improved significantly. Consequently, DIM point clouds are now a good alternative to the established Airborne Laser Scanning (ALS) point clouds for remote sensing applications. In order to derive high-level applications such as digital terrain models or city models, each point within a point cloud must be assigned a class label. Usually, ALS and DIM are labelled with different classifiers due to their varying characteristics. In this work, we explore both point cloud types in a fully convolutional encoder-decoder network, which learns to classify ALS as well as DIM point clouds. As input, we project the point clouds onto a 2D image raster plane and calculate the minimal, average and maximal height values for each raster cell. The network then differentiates between the classes ground, non-ground, building and no data. We test our network in six training setups using only one point cloud type, both point clouds as well as several transfer-learning approaches. We quantitatively and qualitatively compare all results and discuss the advantages and disadvantages of all setups. The best network achieves an overall accuracy of 96<span class="thinspace"></span>% in an ALS and 83<span class="thinspace"></span>% in a DIM test set.</p>

scholarly journals PENENTUAN LOKASI TOWER CRANE PADA PROYEK KONSTRUKSI BERBASIS SIMULASI

2020 ◽  
Vol 3 (4) ◽  
pp. 1305
Author(s):  
Gerwyn Persulessy ◽  
Basuki Anondho
Keyword(s):  
Travel Time ◽  
Construction Projects ◽  
Tower Crane ◽  
Location Determination ◽  
Heavy Equipment ◽  
Total Travel Time ◽  
Speed Up ◽  
High Level ◽  
Map Data ◽  
Geometric Layout

Development of high-level building construction projects that require complex equipment that can be used in high-level construction, equipment used to help complete construction projects called heavy equipment. One of the heavy equipment used in high-rise buildings is a tower crane. The use and layout of tower cranes can speed up the schedule and save on project costs. Therefore many methods have been developed to determine the tower crane layout. This study will discuss determining the location of tower cranes by discussing simulations. The location will be determined based on the site map data which is processed in the form of a geometric arrangement and tower crane data specifications. Location determination is done by comparing the total travel time of several simulated locations according to several different speed criteria in a construction project. Speed criteria are divided into four times the jib speed and trolley speed. Location of the location with the total travel time will be taken as the final result. Different speed criteria will make the total travel time change. ABSTRAKPerkembangan proyek pembangunan gedung bertingkat tinggi yang semakin kompleks menyebabkan diperlukannya peralatan yang dapat mempermudah pembangunan gedung bertingkat, peralatan yang digunakan untuk membantu menyelesaikan tugas konstruksi disebut alat berat. Salah satu peralatan berat yang digunakan pada gedung bertingkat tinggi adalah tower crane. Penggunaan dan tata letak tower crane yang baik dapat mempercepat jadwal dan menghemat biaya proyek. Oleh karena itu banyak dikembangkan metode-metode untuk menentukan tata letak tower crane. Penelitian ini akan membahas penetapan letak lokasi tower crane dengan pendekatan  simulasi. Letak lokasi akan ditetapkan berdasarkan data site map yang diolah dalam bentuk geometric layout dan data spesifikasi tower crane. Penetapan lokasi dilakukan dengan cara membandingkan total travel time dari beberapa lokasi yang disimulasi sesuai dengan beberapa kriteria kecepatan yang berbeda-beda pada suatu proyek konstruksi. Kriteria kecepatan terbagi menjadi empat berdasarkan besarnya kecepatan jib dan kecepatan trolley. Letak lokasi dengan total travel time terkecil akan diambil sebagai hasil akhir. Kriteria-kriteria kecepatan yang berbeda disimulasi akan membuat total travel time berubah.

scholarly journals Deep Learning Strategies for ProtoDUNE Raw Data Denoising

2022 ◽  
Vol 6 (1) ◽  
Author(s):  
Marco Rossi ◽  
Sofia Vallecorsa
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Learning Strategies ◽  
Simulation Data ◽  
Raw Data ◽  
Digital Detector ◽  
Speed Up ◽  
Neural Network Hardware ◽  
Graph Neural Networks ◽  
High Level

AbstractIn this work, we investigate different machine learning-based strategies for denoising raw simulation data from the ProtoDUNE experiment. The ProtoDUNE detector is hosted by CERN and it aims to test and calibrate the technologies for DUNE, a forthcoming experiment in neutrino physics. The reconstruction workchain consists of converting digital detector signals into physical high-level quantities. We address the first step in reconstruction, namely raw data denoising, leveraging deep learning algorithms. We design two architectures based on graph neural networks, aiming to enhance the receptive field of basic convolutional neural networks. We benchmark this approach against traditional algorithms implemented by the DUNE collaboration. We test the capabilities of graph neural network hardware accelerator setups to speed up training and inference processes.

scholarly journals Fast Drivable Areas Estimation with Multi-Task Learning for Real-Time Autonomous Driving Assistant

2021 ◽  
Vol 11 (22) ◽  
pp. 10713
Author(s):  
Dong-Gyu Lee
Keyword(s):  
Real Time ◽  
Computational Efficiency ◽  
Autonomous Driving ◽  
Learning Approaches ◽  
Practical Applications ◽  
Task Learning ◽  
Lane Line ◽  
High Level ◽  
Time Autonomous

Autonomous driving is a safety-critical application that requires a high-level understanding of computer vision with real-time inference. In this study, we focus on the computational efficiency of an important factor by improving the running time and performing multiple tasks simultaneously for practical applications. We propose a fast and accurate multi-task learning-based architecture for joint segmentation of drivable area, lane line, and classification of the scene. An encoder-decoder architecture efficiently handles input frames through shared representation. A comprehensive understanding of the driving environment is improved by generalization and regularization from different tasks. The proposed method learns end-to-end through multi-task learning on a very challenging Berkeley Deep Drive dataset and shows its robustness for three tasks in autonomous driving. Experimental results show that the proposed method outperforms other multi-task learning approaches in both speed and accuracy. The computational efficiency of the method was over 93.81 fps at inference, enabling execution in real-time.

scholarly journals Enhancing Students’ Assessment Feedback Skills Within Higher Education

2020 ◽  
Author(s):  
Carol Evans ◽  
Michael Waring
Keyword(s):  
Higher Education ◽  
Individual Differences ◽  
Industrial Revolution ◽  
Integrated Approach ◽  
Learning Approaches ◽  
Learning And Teaching ◽  
Self Regulated Learning ◽  
Assessment Feedback ◽  
High Level ◽  
Coming To Know

In higher education (HE) considerable attention is focused on the skills sets students need to meet the requirements of the fourth industrial revolution. The acquisition of high-level assessment feedback skills is fundamental to lifelong learning. HE has made significant investment in developing assessment feedback practices over the last 30 years; however, far less attention has been given to the development of inclusive agentic integrated assessment systems that promote student agency and autonomy in assessment feedback, and from an individual differences perspective. “Inside the Black Box,” a seminal work, opened the potential of assessment as a supportive process in facilitating students in coming to know (understanding the requirements of a task and context, and their own learning) through the development of formative assessment. However, overall, the assessment for learning movement has not changed students’ perceptions, on entering HE, that feedback is something they receive rather than something they can generate and orchestrate despite being predicated on a self-regulatory approach. HE promotes students’ use of self-regulated learning approaches although these are not sufficiently integrated into curriculum systems. In moving forward assessment feedback, it is important to adopt a theoretically integrated approach that draws on self-regulatory frameworks, agentic engagement concepts, understanding of individual differences, and the situated nature of assessment. Current emphases in HE focus on how we engage students as active participants in assessment, in coming to know assessment requirements as part of sustainable practices with students as co-constructors of assessment inputs and outputs. Assessment design should be challenging students to maximize their selective and appropriate use of assessment feedback skills for both immediate and longer-term learning gains. Addressing the professional development of lecturers and students in the acquisition and development of essential fourth industrial age assessment feedback competencies is fundamental to enhancing the quality of learning and teaching in HE.

Sign in / Sign up

Export Citation Format

Share Document