Obtaining the effective gripper dimensions for a kiwifruit harvesting robot using kinematic calibration procedures

Purpose A harvesting robot is developed as part of kiwifruit industry automation in New Zealand. This kiwifruit harvester is currently not economically viable, as it drops and damages too many kiwifruit in the harvesting task due to the positional inaccuracy of the gripper. This is due to the difficulties in measuring the exact effective dimensions of the gripper from the manipulator. The purpose of this study is to obtain the effective gripper dimensions using kinematic calibration procedures. Design/methodology/approach A setup of a constraint plate with a dial gauge is proposed to acquire the calibration data. The constraint plate is positioned above the robot. The data is obtained by using a dial gauge and a permanent marker. The effective dimensions of the gripper are used as error parameters in the calibration process. Calibration is exercised by minimizing the difference between target positions and measured positions iteratively. Findings The robot with the obtained effective dimensions is tested in the field. It is found that the fruit drops due to positional inaccuracy of the gripper are greatly reduced after calibration. Practical implications The kiwifruit industry in New Zealand is growing rapidly and announced plans in 2017 to double global sales by 2025. This growth will put extra pressure on the labour supply for harvesting. Furthermore, the Covid pandemic and resulting border restrictions have dramatically reduced seasonal imported labour availability. A robotic system is a potential solution to address the labour shortages for harvesting kiwifruit. Originality/value For kiwifruit harvesting, the picking envelope is well above the robot; the experimental data points obtained by placing a constraint plate above the robot are at similar positions to the target positions of kiwifruit. Using this set of data points for calibration yields a good effect of obtaining the effective dimension of the gripper, which reduces the positional inaccuracy as shown in the field test results.

Application of a computer vision system for recognizing tomato fruits and determining their position relative to the gripper device of the harvesting robot

The article presents a method for recognizing tomato fruits covered with foliage, de-termining their centers and boundaries using the OpenCV computer vision library and a hardware complex based on Raspberry Pi 4. The methods for solving the inverse kinematics problem for the five-link robotic manipulator designed by the authors, installed on a mobile plat-form, in order to create a robot for collecting fruits are considered. The simulation of the manipulator movement in the Scilab environment is performed.

Ablation studies on YOLOFruit detection algorithm for fruit harvesting robot using deep learning

Fruit detection algorithm as an integral part of harvesting robot is expected to be robust, accurate, and fast against environmental factors such as occlusion by stem and leaves, uneven illumination, overlapping fruit and many more. For this reason, this paper explored and compared ablation studies on proposed YOLOFruit, YOLOv4, and YOLOv5 detection algorithms. The final selected YOLOFruit algorithm used ResNet43 backbone with Combined activation function for feature extraction, Spatial Pyramid Pooling Network (SPPNet) for detection accuracies, Feature Pyramid Network (FPN) for feature pyramids, Distance Intersection Over Union-Non Maximum Suppression (DIoU-NMS) for detection efficiency and accuracy, and Complete Intersection Over Union (CIoU) loss for faster and better performance. The obtained results showed that the average detection accuracy of YOLOFruit at 86.2% is 1% greater than YOLOv4 at 85.2% and 4.3% higher than YOLOv5 at 81.9%, while the detection time of YOLOFruit at 11.9ms is faster than YOLOv4 at 16.6ms, but not with YOLOv5 at 2.7ms. Hence, the YOLOFruit detection algorithm is highly prospective for better generalization and real-time fruit detection.

Investigation on Recognition Performance of Harvesting Robot Using Regions of Interest Histogram of Oriented Gradients Feature Based on Improved Fuzzy Least Square Support Vector Machine

In order to solve the problems such as big errors, lack of universality, and too much time consuming occurred in the recognition of overlapped fruits, an improved fuzzy least square support vector machine (FLS-SVM) is established based on the fruit ROI-HOG feature. First, the RGB image is transformed into saturation and value (HSV) image, and then the regions of interest (ROI) are detected from HSV color information. Finally, the histogram of oriented gradients (HOG) feature of ROI will be used as the input of FLS-SVM pattern recognizer to realize the recognition of picking fruit. In addition, the verified FLS-SVM is used to investigate the recognition performance of harvesting robot using regions of interest histogram of oriented gradients feature. The results reveal that the vector sizes are effectively reduced and a higher detection speed is achieved without compromising accuracy relative to conventional approaches. Similarly, the detection accuracy for the learning samples, the isolated fruit, the overlapped fruit, and the background can achieve 99.50%, 96.0%, 89.9%, and 97.0%, respectively, which shows the good performance of the proposed improved ROI-HOG feature recognition method.

Target Recognition and Trajectory Planning of Apple Harvesting Robot considering Color Multimedia Image Segmentation Algorithm

For the purpose of significantly reducing the processing time of the apple harvesting robot during the harvesting process, it is highly necessary to carry out the corresponding studies on the methods for rapid recognition and trajectory planning. Through the comprehensive application of information relevance, the image processing area can be reduced. For image recognition and trajectory planning, the related template matching algorithm for removing the mean value and normalization product can be adopted, and segmentation methods based on different threshold values can be used for the realization of the effect. Subsequently, the comparative experiments are properly carried out to verify the effectiveness of the method used.