scholarly journals A New Knowledge Distillation Method for Object Detection Based on EMD

2021 ◽  
Vol 2083 (4) ◽  
pp. 042028
Author(s):  
Zhihao Liang
Keyword(s):  
Object Detection ◽  
Poor Performance ◽  
Learning Ability ◽  
Distillation Method ◽  
Model Compression ◽  
Knowledge Distillation ◽  
Small Model ◽  
And Performance ◽  
Almost All ◽  
The Impact

Abstract As a common method of model compression, the knowledge distillation method can distill the knowledge from the complex large model with strong learning ability to student small model with weak learning ability in the training process, to improve the accuracy and performance of the small model. At present, there has been much knowledge distillation methods specially designed for object detection and achieved good results. However, almost all methods failed to solve the problem of performance degradation caused by the high noise in the current detection framework. In this study, we proposed a feature automatic weight learning method based on EMD to solve these two problems. That is, the EMD method is used to process the space vector to reduce the impact of negative transfer and noise as much as possible, and at the same time, the weights are allocated adaptive to reduce student model’s learning from the teacher model with poor performance and make students more inclined to learn from good teachers. The loss (EMD Loss) was redesigned, and the HEAD was improved to fit our approach. We have carried out different comprehensive performance tests on multiple datasets, including PASCAL, KITTI, ILSVRC, and MS-COCO, and obtained encouraging results, which can not only be applied to the one-stage and two-stage detectors but also can be used radiatively with some other methods.

scholarly journals Self-boosting for Feature Distillation

2021 ◽  
Author(s):  
Yulong Pei ◽  
Yanyun Qu ◽  
Junping Zhang
Keyword(s):  
Poor Performance ◽  
Feature Integration ◽  
Large Network ◽  
Distillation Method ◽  
Memory Consumption ◽  
Training Process ◽  
Model Compression ◽  
Teacher Student ◽  
Knowledge Distillation ◽  
The Difference

Knowledge distillation is a simple but effective method for model compression, which obtains a better-performing small network (Student) by learning from a well-trained large network (Teacher). However, when the difference in the model sizes of Student and Teacher is large, the gap in capacity leads to poor performance of Student. Existing methods focus on seeking simplified or more effective knowledge from Teacher to narrow the Teacher-Student gap, while we address this problem by Student's self-boosting. Specifically, we propose a novel distillation method named Self-boosting Feature Distillation (SFD), which eases the Teacher-Student gap by feature integration and self-distillation of Student. Three different modules are designed for feature integration to enhance the discriminability of Student's feature, which leads to improving the order of convergence in theory. Moreover, an easy-to-operate self-distillation strategy is put forward to stabilize the training process and promote the performance of Student, without additional forward propagation or memory consumption. Extensive experiments on multiple benchmarks and networks show that our method is significantly superior to existing methods.

Revisiting knowledge distillation for light-weight visual object detection

2021 ◽  
Vol 43 (13) ◽  
pp. 2888-2898
Author(s):  
Tianze Gao ◽  
Yunfeng Gao ◽  
Yu Li ◽  
Peiyuan Qin
Keyword(s):  
Object Detection ◽  
Essential Element ◽  
Detection Algorithm ◽  
Positive Sample ◽  
Detection Methods ◽  
Visual Object ◽  
Light Weight ◽  
Model Compression ◽  
Novel Approach ◽  
Knowledge Distillation

An essential element for intelligent perception in mechatronic and robotic systems (M&RS) is the visual object detection algorithm. With the ever-increasing advance of artificial neural networks (ANN), researchers have proposed numerous ANN-based visual object detection methods that have proven to be effective. However, networks with cumbersome structures do not befit the real-time scenarios in M&RS, necessitating the techniques of model compression. In the paper, a novel approach to training light-weight visual object detection networks is developed by revisiting knowledge distillation. Traditional knowledge distillation methods are oriented towards image classification is not compatible with object detection. Therefore, a variant of knowledge distillation is developed and adapted to a state-of-the-art keypoint-based visual detection method. Two strategies named as positive sample retaining and early distribution softening are employed to yield a natural adaption. The mutual consistency between teacher model and student model is further promoted through a hint-based distillation. By extensive controlled experiments, the proposed method is testified to be effective in enhancing the light-weight network’s performance by a large margin.

scholarly journals An Experimental Analysis of Model Compression Techniques for Object Detection

2020 ◽  
Author(s):  
Andrey De Aguiar Salvi ◽  
Rodrigo Coelho Barros
Keyword(s):  
Object Detection ◽  
Experimental Analysis ◽  
State Of The Art ◽  
Neural Architecture ◽  
Model Compression ◽  
Processing Power ◽  
Benchmark Datasets ◽  
The Difference ◽  
And Performance ◽  
Consumption Constraints

Recent research on Convolutional Neural Networks focuses on how to create models with a reduced number of parameters and a smaller storage size while keeping the model’s ability to perform its task, allowing the use of the best CNN for automating tasks in limited devices, with reduced processing power, memory, or energy consumption constraints. There are many different approaches in the literature: removing parameters, reduction of the floating-point precision, creating smaller models that mimic larger models, neural architecture search (NAS), etc. With all those possibilities, it is challenging to say which approach provides a better trade-off between model reduction and performance, due to the difference between the approaches, their respective models, the benchmark datasets, or variations in training details. Therefore, this article contributes to the literature by comparing three state-of-the-art model compression approaches to reduce a well-known convolutional approach for object detection, namely YOLOv3. Our experimental analysis shows that it is possible to create a reduced version of YOLOv3 with 90% fewer parameters and still outperform the original model by pruning parameters. We also create models that require only 0.43% of the original model’s inference effort.

scholarly journals Object Detection Network Based on Feature Fusion and Attention Mechanism

2019 ◽  
Vol 11 (1) ◽  
pp. 9 ◽  
Author(s):  
Ying Zhang ◽  
Yimin Chen ◽  
Chen Huang ◽  
Mingke Gao
Keyword(s):  
Object Detection ◽  
Feature Fusion ◽  
Empirical Evaluation ◽  
Attention Mechanism ◽  
Detection Accuracy ◽  
Small Object ◽  
Art Object ◽  
Pascal Voc ◽  
Almost All ◽  
The Impact

In recent years, almost all of the current top-performing object detection networks use CNN (convolutional neural networks) features. State-of-the-art object detection networks depend on CNN features. In this work, we add feature fusion in the object detection network to obtain a better CNN feature, which incorporates well deep, but semantic, and shallow, but high-resolution, CNN features, thus improving the performance of a small object. Also, the attention mechanism was applied to our object detection network, AF R-CNN (attention mechanism and convolution feature fusion based object detection), to enhance the impact of significant features and weaken background interference. Our AF R-CNN is a single end to end network. We choose the pre-trained network, VGG-16, to extract CNN features. Our detection network is trained on the dataset, PASCAL VOC 2007 and 2012. Empirical evaluation of the PASCAL VOC 2007 dataset demonstrates the effectiveness and improvement of our approach. Our AF R-CNN achieves an object detection accuracy of 75.9% on PASCAL VOC 2007, six points higher than Faster R-CNN.

scholarly journals The Impact of Optimized Daylight and Views on the Sleep Duration and Cognitive Performance of Office Workers

2020 ◽  
Vol 17 (9) ◽  
pp. 3219
Author(s):  
Mohamed Boubekri ◽  
Jaewook Lee ◽  
Piers MacNaughton ◽  
May Woo ◽  
Lauren Schuyler ◽  
...  
Keyword(s):  
Cognitive Function ◽  
Sleep Quality ◽  
Cognitive Performance ◽  
Sleep Duration ◽  
Office Workers ◽  
And Performance ◽  
Almost All ◽  
Office Environments ◽  
Daylight Exposure ◽  
The Impact

A growing awareness has recently emerged on the health benefits of exposure to daylight and views. Daylight exposure is linked to circadian rhythm regulation, which can have significant impacts on sleep quality and cognitive function. Views of nature have also been shown to impact emotional affect and performance. This study explores the impact of optimized daylight and views on the sleep and cognitive performance of office workers. Thirty knowledge workers spent one week working in each of two office environments with identical layouts, furnishings, and orientations; however, one was outfitted with electrochromic glass and the other with traditional blinds, producing lighting conditions of 40.6 and 316 equivalent melanopic lux, respectively. Participants in the optimized daylight and views condition slept 37 min longer as measured by wrist-worn actigraphs and scored 42% higher on cognitive simulations designed to test their higher order decision-making performance. Both sleep and cognitive function were impacted after one day in the space, yet the impacts became more significant over the course of the week. The positive effect of optimized daylight and views on cognitive function was comparable for almost all participants, while increases in sleep duration were significantly greater for those with the lowest baseline sleep duration. This study stresses the significance of designing with daylight in order to optimize the sleep quality and performance of office workers.

scholarly journals Nursing Students’ Perceptions of Menstrual Distress during Clinical Practice: A Q-Methodology Study

2021 ◽  
Vol 18 (6) ◽  
pp. 3160
Author(s):  
Ya-Lin Fu ◽  
Chia-Ling Yang ◽  
Shu-Chuan Yu ◽  
Yun-Hsuan Lin ◽  
Hsiao-Pei Hsu ◽  
...  
Keyword(s):  
Nursing Education ◽  
Clinical Practice ◽  
Nursing Students ◽  
Q Methodology ◽  
Negative Impact ◽  
Learning Ability ◽  
Clinical Learning ◽  
And Performance ◽  
Menstrual Distress ◽  
The Impact

This study aimed to explore the cluster patterns of female nursing students’ perceptions of the effects of menstrual distress during clinical practice. This study adopted the Q-methodology study design. We recruited female nursing students from a college in northern Taiwan. Forty-seven Q-statements were constructed to explore participants’ experiences of the impact of menstrual distress on clinical learning. In total, 58 participants subjectively ranked Q-statements concerning menstrual distress experiences during clinical practice and were classified. After Q-sorting, the subjective ranking process PQ Method (version 2.35, Schmolck, Emmendingen, Germany) was employed for factor analysis. Four patterns of shared perspectives, accounting for 46.6% of the total variance, were identified: (a) influencing clinical learning and making good use of painkillers; (b) responsible attitudes and diversified relief of discomfort; (c) seeking peer support and effect on mood; (d) negative impact on learning ability and conservative self-care. Clinical practice is a major component of nursing education; menstrual distress affects female nursing students’ clinical learning and performance. The exploration of clustering different nursing students’ perceptions may facilitate customized strategies to enable more appropriate assistance.

scholarly journals The Impact and Efficiency of Electronic Libraries In Various Districts of Punjab, Pakistan

2021 ◽  
Vol 38 (1) ◽  
Author(s):  
Ehsan Bhutta ◽  
Yasir Rasool ◽  
Chaudhry Abdul Rehman
Keyword(s):  
Structural Equation ◽  
Positive Influence ◽  
Vital Role ◽  
Learning Ability ◽  
Equation Modeling ◽  
Digital Learning ◽  
Learning Capacity ◽  
Dynamic Methods ◽  
And Performance ◽  
The Impact

The present research study is conducted with the aim to assess and analyze the impact of electronic libraries (EL) by using usability criteria which include consistency, efficiency, learning and satisfaction in digital learning and reading stimulus among the general public and youth in specific. The structural equation modeling (SEM) of variables like effectiveness (EEF), efficiency (EFT), learning ability (LER) and performance & satisfaction (PES) was followed by research design. Survey was conducted in divisional headquarters of the Punjab province to collect data. The population was N=270 persons from 9 out of 20 districts having EL facilities. The findings revealed that E-Libraries have a positive correlation between productivity, effectiveness, learning and success. Performance, efficacy and learning capacity had a substantial and positive influence on user’s satisfaction. The study found that the provision of a conducive atmosphere that ensures productivity, effectiveness and learning capacity plays a vital role in enhancing performance of EL. It is proposed that we follow more efficient and dynamic methods in order to support the concept of EL for promotion of culture of digital learning philosophy among the public.

Behavior of investor flows in Israeli provident funds

2019 ◽  
Vol 16 (3) ◽  
pp. 334-356
Author(s):  
Ofer Arbaa ◽  
Eva Varon
Keyword(s):  
Poor Performance ◽  
Fixed Effect ◽  
Fund Flows ◽  
Past Performance ◽  
Content Type ◽  
Fund Managers ◽  
The Usa ◽  
And Performance ◽  
The Impact ◽  
Provident Fund

Purpose The purpose of this paper is to study the sensitivity of provident fund investors to past performance and how market conditions, changes in risk and liquidity levels influence the net flows into provident funds by using a unique sample from Israel. Design/methodology/approach The study checks the impact of different levels of fund performance on provident fund flows using three alternative proxies for performance: raw return and the risk adjusted returns based on the Sharpe ratio and the Jensen’s α. The analysis relies on the time fixed effect and fund fixed effect regression models. Findings Results reveal that there exists an approximately concave flow–performance relationship and performance persistence among Israeli provident funds. Israeli provident fund investors are risk averse so they overreact to bad performance both in bull and bear markets. Moreover, liquidity is an important factor to influence the flow–performance curve. The investors’ strong negative response to poor performance and relative insensitivity to outperformance show that provident fund managers are not rewarded for their risk-shifting activities as in mutual funds. Originality/value The authors explore the behavior of investor flows in non-institutional retirement savings funds specifically outside of the USA, which is a topic not properly investigated in literature. Moreover, examining inflows and outflows separately gives the authors a richer understanding of investors in pension schemes. This study also enhances the understanding of the impact of fund liquidity on the flow–performance relationship for the retirement funds segment.

Empirical Investigation of Organisational Learning Ability as a Performance Driver in Construction

2011 ◽  
pp. 166-184 ◽  
Author(s):  
Irem Dikmen ◽  
M. Talat Birgonul ◽  
Tunca Ataoglu
Keyword(s):  
Firm Performance ◽  
Conceptual Model ◽  
Organisational Learning ◽  
Learning Ability ◽  
Learning Mechanisms ◽  
Construction Companies ◽  
And Performance ◽  
Main Components ◽  
Learning Factors ◽  
The Impact

In this chapter, the impact of organisational learning competency on the performance of construction companies is investigated. A conceptual model is proposed for the measurement of organisational learning competency. The main components of the model are learning sources, learning mechanisms, and the organisational setting. Organisational learning competency is assumed to be high only if an appropriate organisational setting exists, as well as the mechanisms used for management of knowledge acquired from various sources. A questionnaire is designed to collect data about organisational learning factors and performance. Findings of the questionnaire answered by 85 Turkish contractors demonstrate that there are statistically significant differences between the performances of contractors grouped according to their learning ability. It is empirically proved that as the organisational learning ability increases, firm performance also increases.

Sign in / Sign up

Export Citation Format

Share Document