A Comprehensive Sensitivity Analysis Framework for Model Evaluation and Improvement Using a Case Study of the Rangeland Hydrology and Erosion Model

Abstract Background Image segmentation is a common task in medical imaging e.g., for volumetry analysis in cardiac MRI. Artificial neural networks are used to automate this task with performance similar to manual operators. However, this performance is only achieved in the narrow tasks networks are trained on. Performance drops dramatically when data characteristics differ from the training set properties. Moreover, neural networks are commonly considered black boxes, because it is hard to understand how they make decisions and why they fail. Therefore, it is also hard to predict whether they will generalize and work well with new data. Here we present a generic method for segmentation model interpretation. Sensitivity analysis is an approach where model input is modified in a controlled manner and the effect of these modifications on the model output is evaluated. This method yields insights into the sensitivity of the model to these alterations and therefore to the importance of certain features on segmentation performance. Results We present an open-source Python library (misas), that facilitates the use of sensitivity analysis with arbitrary data and models. We show that this method is a suitable approach to answer practical questions regarding use and functionality of segmentation models. We demonstrate this in two case studies on cardiac magnetic resonance imaging. The first case study explores the suitability of a published network for use on a public dataset the network has not been trained on. The second case study demonstrates how sensitivity analysis can be used to evaluate the robustness of a newly trained model. Conclusions Sensitivity analysis is a useful tool for deep learning developers as well as users such as clinicians. It extends their toolbox, enabling and improving interpretability of segmentation models. Enhancing our understanding of neural networks through sensitivity analysis also assists in decision making. Although demonstrated only on cardiac magnetic resonance images this approach and software are much more broadly applicable.

Download Full-text

The Potential of GRASP/GARRLiC Retrievals for Dust Aerosol Model Evaluation: Case Study during the PreTECT Campaign

Remote Sensing ◽

10.3390/rs13050873 ◽

2021 ◽

Vol 13 (5) ◽

pp. 873

Author(s):

Dimitra Konsta ◽

Alexandra Tsekeri ◽

Stavros Solomos ◽

Nikolaos Siomos ◽

Anna Gialitaki ◽

...

Keyword(s):

Model Evaluation ◽

Dust Concentration ◽

Saharan Dust ◽

North Coast ◽

Concentration Profiles ◽

Future Studies ◽

Aerosol Model ◽

The North ◽

Sun Photometer

We use the Generalized Retrieval of Aerosol Surface Properties algorithm (GRASP) to compare with dust concentration profiles derived from the NMME-DREAM model for a specific dust episode. The GRASP algorithm provides the possibility of deriving columnar and vertically-resolved aerosol properties from a combination of lidar and sun-photometer observations. Herein, we apply GRASP for analysis of a Saharan dust outburst observed during the “PREparatory: does dust TriboElectrification affect our ClimaTe” campaign (PreTECT) that took place at the North coast of Crete, at the Finokalia ACTRIS station. GRASP provides column-averaged and vertically resolved microphysical and optical properties of the particles. The retrieved dust concentration profiles are compared with modeled concentration profiles derived from the NMME-DREAM dust model. To strengthen the results, we use dust concentration profiles from the POlarization-LIdar PHOtometer Networking method (POLIPHON). A strong underestimation of the maximum dust concentration is observed from the NMME-DREAM model. The reported differences between the retrievals and the model indicate a high potential of the GRASP algorithm for future studies of dust model evaluation.

Download Full-text

Semantic and knowledge based support to business model evaluation to stimulate green behaviour of electric vehicles’ drivers and energy prosumers

Journal of Ambient Intelligence and Humanized Computing ◽

10.1007/s12652-021-03243-4 ◽

2021 ◽

Author(s):

Beniamino Di Martino ◽

Dario Branco ◽

Luigi Colucci Cante ◽

Salvatore Venticinque ◽

Reinhard Scholten ◽

...

Keyword(s):

Business Model ◽

Model Evaluation ◽

Business Models ◽

Real Case ◽

Sustainable Mobility ◽

Semantic Framework ◽

Knowledge Based ◽

Survey Results ◽

Energy Prosumers

AbstractThis paper proposes a semantic framework for Business Model evaluation and its application to a real case study in the context of smart energy and sustainable mobility. It presents an ontology based representation of an original business model and examples of inferential rules for knowledge extraction and automatic population of the ontology. The real case study belongs to the GreenCharge European Project, that in these last years is proposing some original business models to promote sustainable e-mobility plans. An original OWL Ontology contains all relevant Business Model concepts referring to GreenCharge’s domain, including a semantic description of TestCards, survey results and inferential rules.

Download Full-text

4B.004 Time series spatial analysis framework to identify prominent crash locations: Case study Delhi

10.1136/injuryprev-2021-safety.103 ◽

2021 ◽

Author(s):

Richa Ahuja ◽

Geetam Tiwari

Keyword(s):

Time Series ◽

Spatial Analysis ◽

Analysis Framework

Download Full-text

Sensitivity Analysis of Fuzzy Activity-Based Life Cycle Costing

MATEC Web of Conferences ◽

10.1051/matecconf/201822505002 ◽

2018 ◽

Vol 225 ◽

pp. 05002

Author(s):

Freselam Mulubrhan ◽

Ainul Akmar Mokhtar ◽

Masdi Muhammad

Keyword(s):

Sensitivity Analysis ◽

Life Cycle ◽

Interest Rate ◽

Life Cycle Costing ◽

High Impact ◽

Final Output ◽

Left And Right ◽

The Interest Rate ◽

Operation Cost

A sensitivity analysis is typically conducted to identify how sensitive the output is to changes in the input. In this paper, the use of sensitivity analysis in the fuzzy activity based life cycle costing (LCC) is shown. LCC is the most frequently used economic model for decision making that considers all costs in the life of a system or equipment. The sensitivity analysis is done by varying the interest rate and time 15% and 45%, respectively, to the left and right, and varying 25% of the maintenance and operation cost. It is found that the operation cost and the interest rate give a high impact on the final output of the LCC. A case study of pumps is used in this study.

Download Full-text

Improving Casthouse Throughput Using Discrete-Event Modelling

Materials Science Forum ◽

10.4028/www.scientific.net/msf.693.3 ◽

2011 ◽

Vol 693 ◽

pp. 3-9 ◽

Cited By ~ 2

Author(s):

Bruce Gunn ◽

Yakov Frayman

Keyword(s):

Sensitivity Analysis ◽

Discrete Event ◽

Casting Machine ◽

Production Data ◽

Model Output ◽

Arrival Times ◽

Event Simulation ◽

Modelling System ◽

Key Parameter

The scheduling of metal to different casters in a casthouse is a complicated problem, attempting to find the balance between pot-line, crucible carrier, furnace and casting machine capacity. In this paper, a description will be given of a casthouse modelling system designed to test different scenarios for casthouse design and operation. Using discrete-event simulation, the casthouse model incorporates variable arrival times of metal carriers, crucible movements, caster operation and furnace conditions. Each part of the system is individually modelled and synchronised using a series of signals or semaphores. In addition, an easy to operate user interface allows for the modification of key parameters, and analysis of model output. Results from the model will be presented for a case study, which highlights the effect different parameters have on overall casthouse performance. The case study uses past production data from a casthouse to validate the model outputs, with the aim to perform a sensitivity analysis on the overall system. Along with metal preparation times and caster strip-down/setup, the temperature evolution within the furnaces is one key parameter in determining casthouse performance.

Download Full-text