scholarly journals Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning

Author(s):  
Marc Deisenroth ◽  
Carl Rasmussen ◽  
Dieter Fox

Author(s):  
Jun Long ◽  
Yueyi Luo ◽  
Xiaoyu Zhu ◽  
Entao Luo ◽  
Mingfeng Huang

AbstractWith the developing of Internet of Things (IoT) and mobile edge computing (MEC), more and more sensing devices are widely deployed in the smart city. These sensing devices generate various kinds of tasks, which need to be sent to cloud to process. Usually, the sensing devices do not equip with wireless modules, because it is neither economical nor energy saving. Thus, it is a challenging problem to find a way to offload tasks for sensing devices. However, many vehicles are moving around the city, which can communicate with sensing devices in an effective and low-cost way. In this paper, we propose a computation offloading scheme through mobile vehicles in IoT-edge-cloud network. The sensing devices generate tasks and transmit the tasks to vehicles, then the vehicles decide to compute the tasks in the local vehicle, MEC server or cloud center. The computation offloading decision is made based on the utility function of the energy consumption and transmission delay, and the deep reinforcement learning technique is adopted to make decisions. Our proposed method can make full use of the existing infrastructures to implement the task offloading of sensing devices, the experimental results show that our proposed solution can achieve the maximum reward and decrease delay.



Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3864
Author(s):  
Tarek Ghoul ◽  
Tarek Sayed

Speed advisories are used on highways to inform vehicles of upcoming changes in traffic conditions and apply a variable speed limit to reduce traffic conflicts and delays. This study applies a similar concept to intersections with respect to connected vehicles to provide dynamic speed advisories in real-time that guide vehicles towards an optimum speed. Real-time safety evaluation models for signalized intersections that depend on dynamic traffic parameters such as traffic volume and shock wave characteristics were used for this purpose. The proposed algorithm incorporates a rule-based approach alongside a Deep Deterministic Policy Gradient reinforcement learning technique (DDPG) to assign ideal speeds for connected vehicles at intersections and improve safety. The system was tested on two intersections using real-world data and yielded an average reduction in traffic conflicts ranging from 9% to 23%. Further analysis was performed to show that the algorithm yields tangible results even at lower market penetration rates (MPR). The algorithm was tested on the same intersection with different traffic volume conditions as well as on another intersection with different physical constraints and characteristics. The proposed algorithm provides a low-cost approach that is not computationally intensive and works towards optimizing for safety by reducing rear-end traffic conflicts.



2014 ◽  
Vol 7 (5) ◽  
pp. 1153-1167 ◽  
Author(s):  
K. Van Tricht ◽  
I. V. Gorodetskaya ◽  
S. Lhermitte ◽  
D. D. Turner ◽  
J. H. Schween ◽  
...  

Abstract. Optically thin ice and mixed-phase clouds play an important role in polar regions due to their effect on cloud radiative impact and precipitation. Cloud-base heights can be detected by ceilometers, low-power backscatter lidars that run continuously and therefore have the potential to provide basic cloud statistics including cloud frequency, base height and vertical structure. The standard cloud-base detection algorithms of ceilometers are designed to detect optically thick liquid-containing clouds, while the detection of thin ice clouds requires an alternative approach. This paper presents the polar threshold (PT) algorithm that was developed to be sensitive to optically thin hydrometeor layers (minimum optical depth τ ≥ 0.01). The PT algorithm detects the first hydrometeor layer in a vertical attenuated backscatter profile exceeding a predefined threshold in combination with noise reduction and averaging procedures. The optimal backscatter threshold of 3 × 10−4 km−1 sr−1 for cloud-base detection near the surface was derived based on a sensitivity analysis using data from Princess Elisabeth, Antarctica and Summit, Greenland. At higher altitudes where the average noise level is higher than the backscatter threshold, the PT algorithm becomes signal-to-noise ratio driven. The algorithm defines cloudy conditions as any atmospheric profile containing a hydrometeor layer at least 90 m thick. A comparison with relative humidity measurements from radiosondes at Summit illustrates the algorithm's ability to significantly discriminate between clear-sky and cloudy conditions. Analysis of the cloud statistics derived from the PT algorithm indicates a year-round monthly mean cloud cover fraction of 72% (±10%) at Summit without a seasonal cycle. The occurrence of optically thick layers, indicating the presence of supercooled liquid water droplets, shows a seasonal cycle at Summit with a monthly mean summer peak of 40 % (±4%). The monthly mean cloud occurrence frequency in summer at Princess Elisabeth is 46% (±5%), which reduces to 12% (±2.5%) for supercooled liquid cloud layers. Our analyses furthermore illustrate the importance of optically thin hydrometeor layers located near the surface for both sites, with 87% of all detections below 500 m for Summit and 80% below 2 km for Princess Elisabeth. These results have implications for using satellite-based remotely sensed cloud observations, like CloudSat that may be insensitive for hydrometeors near the surface. The decrease of sensitivity with height, which is an inherent limitation of the ceilometer, does not have a significant impact on our results. This study highlights the potential of the PT algorithm to extract information in polar regions from various hydrometeor layers using measurements by the robust and relatively low-cost ceilometer instrument.



2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Johnathan Kongoletos ◽  
Ethan Munden ◽  
Jennifer Ballew ◽  
Daniel J. Preston

AbstractVentilation, including fume hoods, consumes 40–70% of the total energy used by modern laboratories. Energy-conscious fume hood usage—for example, closing the sash when a hood is unused—can significantly reduce energy expenditures due to ventilation. Prior approaches to promote such behaviors among lab users have primarily relied on passive feedback methods. In this work, we developed a low-cost fume hood monitoring device with active feedback to alert lab users when a fume hood is left open and unused. Using data collected by the building management system, we observed a 75.6% decrease in the average sash height after installation of these “Motion and Sash Height” (MASH) alarms, which would result in a reduction roughly equal to 43% of the annual carbon emissions of a typical American vehicle, for each fume hood. The MASH alarm presented here reduced energy costs by approximately $1,159 per year, per hood, at MIT.



2021 ◽  
Vol 101 (5) ◽  
pp. 553-574
Author(s):  
Eric G. Lambert ◽  
Emily Berthelot ◽  
Weston Morrow ◽  
Lauren Block ◽  
Nancy Hogan

Research examining the effect of organizational justice on the correctional environment is typically limited to its consequences on various outcomes. Absent from this body of literature is how perceptions of organizational justice are formed among correctional staff. Filling this void and using data from a Midwestern correctional facility, the current study examines the impact of instrumental communication, integration, formalization, and input into decision-making on the distributive and procedural justice perceptions of correctional staff. With the exception of integration, all organizational structure variables were significantly related to both forms of organizational justice. These findings offer correctional administrators a low cost and practical solution for enhancing organizational justice through organizational structure.



Author(s):  
Jonathan Becker ◽  
Aveek Purohit ◽  
Zheng Sun

USARSim group at NIST developed a simulated robot that operated in the Unreal Tournament 3 (UT3) gaming environment. They used a software PID controller to control the robot in UT3 worlds. Unfortunately, the PID controller did not work well, so NIST asked us to develop a better controller using machine learning techniques. In the process, we characterized the software PID controller and the robot’s behavior in UT3 worlds. Using data collected from our simulations, we compared different machine learning techniques including linear regression and reinforcement learning (RL). Finally, we implemented a RL based controller in Matlab and ran it in the UT3 environment via a TCP/IP link between Matlab and UT3.



2003 ◽  
Vol 15 (1) ◽  
pp. 47-54 ◽  
Author(s):  
TINA TIN ◽  
MARTIN O. JEFFRIES ◽  
MIKKO LENSU ◽  
JUKKA TUHKURI

Ship-based observations of sea ice thickness using the Antarctic Sea Ice Processes and Climate (ASPeCt) protocol provide information on ice thickness distribution at relatively low cost. This protocol uses a simple formula to calculate the mass of ice in ridges based on surface observations. We present two new formulae and compare these with results from the “Original” formula using data obtained in the Ross Sea in autumn and winter. The new “r-star” formula uses a more realistic ratio of sail and keel areas to transform dimensions of sails to estimates of mean keel areas. As a result, estimates of “equivalent thickness” (i.e. mean thickness of ice in ridged areas) increased by over 200%. The new “Probability” formula goes one step further, by incorporating the probability that a sail is associated with a keel underwater, and the probability that keels may be found under level surfaces. This resulted in estimates of equivalent thickness comparable with the Original formula. Estimates of equivalent thickness at one or two degree latitude resolution are sufficiently accurate for validating sea ice models. Although ridges are small features in the Ross Sea, we have shown that they constitute a significant fraction of the total ice mass.



ILR Review ◽  
1994 ◽  
Vol 47 (2) ◽  
pp. 173-188 ◽  
Author(s):  
Paul Osterman

The author, using data on 694 U.S. manufacturing establishments from a 1992 survey, examines the incidence of innovative work practices (teams, job rotation, quality circles, and Total Quality Management) and investigates what variables, including human resource practices, are associated with the adoption of these practices. He finds that about 35% of private sector establishments with 50 or more employees made substantial use of flexible work organization in 1992. Some factors associated with an establishment's adoption of these practices are being in an internationally competitive product market, having a technology that requires high levels of skill, following a “high road” strategy that emphasizes variety, service, and quality rather than low cost, and using such human resource practices as high levels of training and innovative pay systems.



Author(s):  
Saon Banerjee ◽  
Sawon Pratiher ◽  
Subhankar Chattoraj ◽  
Rishabh Gupta ◽  
Parthasarathi Patra ◽  
...  


Sign in / Sign up

Export Citation Format

Share Document