Robotics and automation have significantly advanced modern industry in many aspects. In particular, automation has enabled companies to increase their outputs, improve the quality of finished work, and enhance production efficiency. Agriculture is one industry yet to be successfully automated, and is still highly dependent on labour. Currently, there is a rapid deterioration in the number of people working in the agricultural sector, at a time when growing global populations are causing an ever increasing demand for food. This is profoundly concerning, and clearly there is a need for research on agricultural automation.

Automating agricultural tasks has historically proven challenging, with complexities arising from dynamic outdoor environments, difficult terrains, and wide variety in plants morphologies. Moreover, previous studies on automating agricultural tasks have failed to deliver solutions broadly considered reliable by industry. Current research has indicated that full automation of agricultural activities is unrealistic. This thesis discusses how human intelligence and flexibility are still important factors which can be integrated with automation technology as part of a multi-agent system, and proposes the use of a Human – Robot Hybrid Control Multi-agent System to overcome outdoor automation challenges. A collaborative multi-agent model is a novel solution, and is developed here to reliably and optimally automate fruit harvest.

The Hybrid Control Multi-agent System aims to successfully automate fruit harvesting and yield management processes at a minimum economic cost. In this case, it is expected that a well-designed collaborative fruit harvesting system can reduce the yearly economic cost of a harvest by 60%. The hybrid control system’s architecture is designed to be reliable by evaluating the limitations of current and previous automation applications. The system architecture has a central controller to master plan the harvest process, while decentralized robotic transporting agents collaboratively serve fruit pickers and compensate for the absence of other robotic transporting agents and the central controller.

The proposed system architecture is modelled and simulated in realistic fruit harvesting scenarios to demonstrate its feasibility and behaviour. The model includes a virtual fruit orchard block represented by an undirected weighted graph; human picking agent models who pick fruit; mobile robot models who collect picked fruit from workers; and a tractor model which collects fruit bins located near workers. A Monte-Carlo simulation enhanced with a convergence analysis was used to ensure the reliability of results within the simulated framework. The model can be used for different kind of fruit, however, this study simulated apple fruit orchard. A first simulation was made by replacing the conventional setup of a human driven tractor and fixed picking bins, with a single tractor-equivalent robotic transporting agent which acts as a mobile fruit bin. The simulation found that the robotic transporting agent reduced a workers unproductive time by 41.3% when compared to a conventional apple harvesting setup. However, it was found that the single robot transporting agent resulted in high waiting times between a worker having a full bag and being served by the robot transporting agent. Replacing the tractor-equivalent robotic transporting agent with two smaller robotic transporting agents reduced the simulated service waiting time by 43.1%. These simulations both used a basic First Available First Served (FAFS) dispatching algorithm which assigns robotic transporting agents based on their availability.

To further reduce service waiting times and decrease RTAs utilisation, a Dynamic Distance dispatching algorithm (DD) replaced the FAFS algorithm to serve human picking agents based on both the availability and proximity of the robotic transporting agent to a service request. A simulation of 10 human picking agents and three small robotic transporting agents with the new algorithm improved the mean service waiting time by 68.1% compared to the FAFS service algorithm. Further, the DD algorithm reduced robotic transporting agent utilisation from 45.7% to 15%. Serving capacity and the robotic transporting agents’ utilisation are exponentially related, and the robotics transporting agents’ utilisation should be below 80% to prevent human picking agents waiting to empty their bags for long time. A Dynamic Distance and Best fit (DDB) dispatching algorithm for heterogeneous robotic transporting agents which assigns them base on their availability, location, and speed reduced the mean of the service waiting time even further by 73.9% when compared to FAFS, and reduced the robotic transporting agents’ utilisation to 12%. For the simulated orchard, an optimal cost is achieved when deploying three small, fast robotic transporting agents.

A smart picking bag system for fruit pickers has been prototyped, as a suitable commercial bag was not available. This technology is needed to test the system concept in a real orchard setting. The smart bag manages the fruit picking, tracks human picking agents, and allows for continuous human-machine interaction during the harvesting process. The smart bag measures the weight of fruit in the picking bag, and calls for a robotic transporting agent when required.

Technology was also developed to manage the robotic transporting agent’s visual navigation through orchards from off-the-shelf and open source object detection algorithms. Initial testing of the visualisation technology successfully detected the human picking agents and calculated their distance. However, further testing and improvement of both technologies is required.

The proposed system is capable of reducing unproductive picking time by a significant fraction, and has the potential to reduce harvesting costs by approximately 60%. Work in developing the underlying dispatching algorithms has tripled the system’s serving capacity and service time reliability from initial results. Initial prototypes of required technologies have been designed and field tested to manage the cooperative human-robot interaction, and robotic visual navigation in a GPS denied environment. The proposed Hybrid Control Multi-agent System has clear benefits to the farming, and is an innovative and promising new attempt to automate the agricultural industry.