Strawberry picking point localization ripeness and weight estimation

Alessandra Tafuro, Adeayo Adewumi, Soran Parsa, Ghalamzan E. Amir, Bappaditya Debnath

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

9 Citations (Scopus)

Abstract

Labour shortage, difficulties in labour management, the digitalization of fruit production pipeline to reduce the fruit production costs have made robotic systems for selective harvesting of strawberries an important industry and academic research. One of the important components of such technologies yet to be developed is fruit picking perception. For picking strawberries, a robot needs to infer the location of picking points from the images of strawberries. Moreover, the size and weight of strawberries to be picked can help the robot to place the picked strawberries in proper punnets directly to be delivered to customers in supermarkets. This can save significant time and packing costs in packhouses. Geometry-based approaches are the most common approach to determine the picking point but they suffer from inaccuracies due to noise, occlusion, and varying shape and orientation of the berries. In contrast, we present two novel datasets of strawberries annotated with picking points, key-points (such as the shoulder points, the contact point between the calyx and flesh, and the point on the flesh farthest from the calyx), and the weight and size of the berries. We performed experiments with Detectron-2, which is an extended version of Mask-RCNN with key-points detection capability. The results show that the key-points detection approach works well for picking and grasping point localization. The second dataset also presents the dimensions and weight of strawberries. Our novel baseline model for weight estimation outperforms many state-of-the-art deep networks. The datasets and annotations are available at https://github.com/imanlab/strawberry-pp-w-r-dataset.

Original languageEnglish
Title of host publication2022 IEEE International Conference on Robotics and Automation
Subtitle of host publicationICRA 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2295-2302
Number of pages8
ISBN (Electronic)9781728196817
ISBN (Print)9781728196824
DOIs
Publication statusPublished - 12 Jul 2022
Externally publishedYes
Event39th IEEE International Conference on Robotics and Automation - Philadelphia, United States
Duration: 23 May 202227 May 2022
Conference number: 39

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
PublisherIEEE
ISSN (Print)1050-4729

Conference

Conference39th IEEE International Conference on Robotics and Automation
Abbreviated titleICRA 2022
Country/TerritoryUnited States
CityPhiladelphia
Period23/05/2227/05/22

Cite this