A hierarchical approach to apple identification for robotic harvesting

Abstract

Identifying fruits with limited visibility and fruits in clusters has been a challenging problem in machine vision systems used in robotic tree fruit harvesting. Issues with clusters and partial to full occlusion of fruit could be minimized by strategically harvesting the most visible fruits first. This work presents a hierarchical method for apple identification suitable for robotic harvesting. Test images were acquired from six overlapping sections of tree canopies trained in fruiting wall architecture. An image processing method consisting of circular Hough transform (CHT) and blob analysis was applied in an iterative fashion. Clearly visible fruit identified by CHT were preferred to partially visible apples for initial harvesting. These prioritized apples were then manually picked to prove the concept of hierarchical fruit identification. As images were taken again after harvesting the well-exposed apples, partially or fully occluded apples were better exposed in successive iterations. This iterative process was applied over every image acquired from both sides of tree canopies until no apples were identified. In total, 740 images were taken from 240 canopy sections of 20 trees where 1789 apples were identified out of 1827 manual counts. On average, this method achieved 98% identification accuracy. In addition, 80% of the apples were detected and harvested with images taken from one side of the canopy, and the remaining apples were harvested from the opposite side. Although this process is simple and intuitive, the work provided a unique and novel insight into the fruit identification and harvesting accuracy achievable with such an approach in a field environment and showed the huge potential of this machine vision system for robotic harvesting of apples trained in fruiting wall architectures.