Mango Fruit Load Estimation Using a Video Based MangoYOLO—Kalman Filter—Hungarian Algorithm Method

: Pre-harvest fruit yield estimation is useful to guide harvesting and marketing resourcing, but machine vision estimates based on a single view from each side of the tree (“dual-view”) underestimates the fruit yield as fruit can be hidden from view. A method is proposed involving deep learning, Kalman filter, and Hungarian algorithm for on-tree mango fruit detection, tracking, and counting from 10 frame-per-second videos captured of trees from a platform moving along the inter row at 5 km/h. The deep learning based mango fruit detection algorithm, MangoYOLO, was used to detect fruit in each frame. The Hungarian algorithm was used to correlate fruit between neighbouring frames, with the improvement of enabling multiple-to-one assignment. The Kalman filter was used to predict the position of fruit in following frames, to avoid multiple counts of a single fruit that is obscured or otherwise not detected with a frame series. A “borrow” concept was added to the Kalman filter to predict fruit position when its precise prediction model was absent, by borrowing the horizontal and vertical speed from neighbouring fruit. By comparison with human count for a video with 110 frames and 192 (human count) fruit, the method produced 9.9% double counts and 7.3% missing count errors, resulting in around 2.6% over count. In another test, a video (of 1162 frames, with 42 images centred on the tree trunk) was acquired of both sides of a row of 21 trees, for which the harvest fruit count was 3286 (i.e., average of 156 fruit/tree). The trees had thick canopies, such that the proportion of fruit hidden from view from any given perspective was high. The proposed method recorded 2050 fruit (62% of harvest) with a bias corrected Root Mean Square Error (RMSE) = 18.0 fruit/tree while the dual-view image method (also using MangoYOLO) recorded 1322 fruit (40%) with a bias corrected RMSE = 21.7 fruit/tree. The video tracking system is recommended over the dual-view imaging system for mango orchard fruit count.

Download Full-text

Kalman filter and Iterative-Hungarian Algorithm implementation for low complexity point tracking as part of fast multiple object tracking system

2016 6th International Conference on System Engineering and Technology (ICSET) ◽

10.1109/icsengt.2016.7849633 ◽

2016 ◽

Cited By ~ 13

Author(s):

Bima Sahbani ◽

Widyawardana Adiprawita

Keyword(s):

Kalman Filter ◽

Object Tracking ◽

Tracking System ◽

Low Complexity ◽

Multiple Object Tracking ◽

Hungarian Algorithm ◽

Point Tracking ◽

Multiple Object ◽

Algorithm Implementation

Download Full-text

Building a Multiple Object Tracking System with Occlusion Handling in Surveillance Videos

Image Processing ◽

10.4018/978-1-4666-3994-2.ch053 ◽

2013 ◽

pp. 1051-1063

Author(s):

Raed Almomani ◽

Ming Dong

Keyword(s):

Kalman Filter ◽

Tracking System ◽

Video Tracking ◽

Appearance Model ◽

Partial Occlusion ◽

Security Monitoring ◽

Robotic Vision ◽

Objects Tracking ◽

Full Occlusion ◽

New Location

Video tracking systems are increasingly used day in and day out in various applications such as surveillance, security, monitoring, and robotic vision. In this chapter, the authors propose a novel multiple objects tracking system in video sequences that deals with occlusion issues. The proposed system is composed of two components: An improved KLT tracker, and a Kalman filter. The improved KLT tracker uses the basic KLT tracker and an appearance model to track objects from one frame to another and deal with partial occlusion. In partial occlusion, the appearance model (e.g., a RGB color histogram) is used to determine an object’s KLT features, and the authors use these features for accurate and robust tracking. In full occlusion, a Kalman filter is used to predict the object’s new location and connect the trajectory parts. The system is evaluated on different videos and compared with a common tracking system.

Download Full-text

Kalman filter and iterative-hungarian algorithm implementation for low complexity point tracking as part of fast multiple object tracking system

2016 International Conference on Frontiers of Information Technology (FIT) ◽

10.1109/fit.2016.7857548 ◽

2016 ◽

Cited By ~ 1

Author(s):

Bima Sahbani ◽

Widyawardana Adiprawita

Keyword(s):

Kalman Filter ◽

Object Tracking ◽

Tracking System ◽

Low Complexity ◽

Multiple Object Tracking ◽

Hungarian Algorithm ◽

Point Tracking ◽

Multiple Object ◽

Algorithm Implementation

Download Full-text

Building a Multiple Object Tracking System with Occlusion Handling in Surveillance Videos

Robotic Vision ◽

10.4018/978-1-4666-2672-0.ch006 ◽

2013 ◽

pp. 98-111

Author(s):

Raed Almomani ◽

Ming Dong

Keyword(s):

Kalman Filter ◽

Tracking System ◽

Video Tracking ◽

Appearance Model ◽

Partial Occlusion ◽

Security Monitoring ◽

Robotic Vision ◽

Objects Tracking ◽

Full Occlusion ◽

New Location

Download Full-text

Deep-Learning-Based Pupil Center Detection and Tracking Technology for Visible-Light Wearable Gaze Tracking Devices

Applied Sciences ◽

10.3390/app11020851 ◽

2021 ◽

Vol 11 (2) ◽

pp. 851

Author(s):

Wei-Liang Ou ◽

Tzu-Ling Kuo ◽

Chin-Chieh Chang ◽

Chih-Peng Fan

Keyword(s):

Deep Learning ◽

Visible Light ◽

Tracking System ◽

Recall Rate ◽

Detection Accuracy ◽

Learning Technology ◽

Tracking Errors ◽

Gaze Tracking ◽

Detection Technology ◽

Pupil Tracking

In this study, for the application of visible-light wearable eye trackers, a pupil tracking methodology based on deep-learning technology is developed. By applying deep-learning object detection technology based on the You Only Look Once (YOLO) model, the proposed pupil tracking method can effectively estimate and predict the center of the pupil in the visible-light mode. By using the developed YOLOv3-tiny-based model to test the pupil tracking performance, the detection accuracy is as high as 80%, and the recall rate is close to 83%. In addition, the average visible-light pupil tracking errors of the proposed YOLO-based deep-learning design are smaller than 2 pixels for the training mode and 5 pixels for the cross-person test, which are much smaller than those of the previous ellipse fitting design without using deep-learning technology under the same visible-light conditions. After the combination of calibration process, the average gaze tracking errors by the proposed YOLOv3-tiny-based pupil tracking models are smaller than 2.9 and 3.5 degrees at the training and testing modes, respectively, and the proposed visible-light wearable gaze tracking system performs up to 20 frames per second (FPS) on the GPU-based software embedded platform.

Download Full-text

Synthetic polarization-sensitive optical coherence tomography by deep learning

npj Digital Medicine ◽

10.1038/s41746-021-00475-8 ◽

2021 ◽

Vol 4 (1) ◽

Author(s):

Yi Sun ◽

Jianfeng Wang ◽

Jindou Shi ◽

Stephen A. Boppart

Keyword(s):

Optical Coherence Tomography ◽

Deep Learning ◽

Imaging System ◽

Imaging Modality ◽

Similarity Index ◽

Polarization Sensitivity ◽

Imaging Method ◽

Optical Coherence ◽

The Real ◽

Oct Imaging

AbstractPolarization-sensitive optical coherence tomography (PS-OCT) is a high-resolution label-free optical biomedical imaging modality that is sensitive to the microstructural architecture in tissue that gives rise to form birefringence, such as collagen or muscle fibers. To enable polarization sensitivity in an OCT system, however, requires additional hardware and complexity. We developed a deep-learning method to synthesize PS-OCT images by training a generative adversarial network (GAN) on OCT intensity and PS-OCT images. The synthesis accuracy was first evaluated by the structural similarity index (SSIM) between the synthetic and real PS-OCT images. Furthermore, the effectiveness of the computational PS-OCT images was validated by separately training two image classifiers using the real and synthetic PS-OCT images for cancer/normal classification. The similar classification results of the two trained classifiers demonstrate that the predicted PS-OCT images can be potentially used interchangeably in cancer diagnosis applications. In addition, we applied the trained GAN models on OCT images collected from a separate OCT imaging system, and the synthetic PS-OCT images correlate well with the real PS-OCT image collected from the same sample sites using the PS-OCT imaging system. This computational PS-OCT imaging method has the potential to reduce the cost, complexity, and need for hardware-based PS-OCT imaging systems.

Download Full-text

Possibility of Autonomous Estimation of Shiba Goat’s Estrus and Non-Estrus Behavior by Machine Learning Methods

Animals ◽

10.3390/ani10050771 ◽

2020 ◽

Vol 10 (5) ◽

pp. 771

Author(s):

Toshiya Arakawa

Keyword(s):

Neural Network ◽

Machine Learning ◽

Random Forest ◽

Markov Models ◽

Tracking System ◽

Video Tracking ◽

Training Data ◽

Support Vector ◽

Learning Methods ◽

Machine Learning Methods

Mammalian behavior is typically monitored by observation. However, direct observation requires a substantial amount of effort and time, if the number of mammals to be observed is sufficiently large or if the observation is conducted for a prolonged period. In this study, machine learning methods as hidden Markov models (HMMs), random forests, support vector machines (SVMs), and neural networks, were applied to detect and estimate whether a goat is in estrus based on the goat’s behavior; thus, the adequacy of the method was verified. Goat’s tracking data was obtained using a video tracking system and used to estimate whether they, which are in “estrus” or “non-estrus”, were in either states: “approaching the male”, or “standing near the male”. Totally, the PC of random forest seems to be the highest. However, The percentage concordance (PC) value besides the goats whose data were used for training data sets is relatively low. It is suggested that random forest tend to over-fit to training data. Besides random forest, the PC of HMMs and SVMs is high. However, considering the calculation time and HMM’s advantage in that it is a time series model, HMM is better method. The PC of neural network is totally low, however, if the more goat’s data were acquired, neural network would be an adequate method for estimation.

Download Full-text

Measurement Feedback Self-Tuning Weighted Measurement Fusion Kalman Filter for Systems with Correlated Noises

Journal of Applied Mathematics ◽

10.1155/2012/324296 ◽

2012 ◽

Vol 2012 ◽

pp. 1-16 ◽

Cited By ~ 4

Author(s):

Xin Wang ◽

Shu-Li Sun

Keyword(s):

Kalman Filter ◽

Stochastic Systems ◽

Tracking System ◽

Weighted Least Squares ◽

Full Rank ◽

Global Optimality ◽

Measurement Feedback ◽

Innovation Model ◽

Self Tuning ◽

Noise Statistics

For the linear discrete stochastic systems with multiple sensors and unknown noise statistics, an online estimators of the noise variances and cross-covariances are designed by using measurement feedback, full-rank decomposition, and weighted least squares theory. Further, a self-tuning weighted measurement fusion Kalman filter is presented. The Fadeeva formula is used to establish ARMA innovation model with unknown noise statistics. The sampling correlated function of the stationary and reversible ARMA innovation model is used to identify the noise statistics. It is proved that the presented self-tuning weighted measurement fusion Kalman filter converges to the optimal weighted measurement fusion Kalman filter, which means its asymptotic global optimality. The simulation result of radar-tracking system shows the effectiveness of the presented algorithm.

Download Full-text