Dr Timothy Wiley

Abstract

The utilisation of UAVs to deliver packages has steadily increased in popularity over the past few years. However, delivering multiple packages of varying weights using long-range and endurance UAVs is still an emerging technology with immense potential to become an integral part of the mainstream postal system.

This work focuses on developing a fixed-wing UAV which can travel further with heavier packages than current multi-rotor or VTOL concepts. The aircraft must remain in the air to make this concept feasible, as landing an aircraft is not always possible. To achieve this, this work investigates an innovative approach involving the package attached to the aircraft by a long tether. This delivery must be completed within current ICAO regulations that require uncrewed aircraft to operate below a ceiling of 120m. To achieve this requirement, additional features are presented in this work that work together to ensure the delivery is accurate and repeatable under various environmental conditions.

The most critical feature is the use of a Mid Tether Drag Device, which has been demonstrated in both simulations and real-world experiments to effectively control the package within ICAO regulations. Other adverse effects are presented with the operation of the system in the environment, where a reinforcement learning agent has been trained to manage the complexity of the system and carry out mission operations. The agent is capable of handling various tasks, such as managing the loitering origin and rate of the UAV, identifying the best flight path and entry velocity, and adjusting the tether length to reduce vertical oscillations. This marks the first use of an RL agent in the tethered delivery system. This research is backed by experimental results, which are compared with computational simulations. Practical experiments are discussed to highlight differences from the computational environment.

The findings show that it is possible to safely and precisely deliver packages using a fixed-wing drone, allowing heavier packages to be delivered further than current concepts in various windy environments. The system is anticipated to be a game-changer in package delivery via drones, providing a faster, safer, and more cost-effective method to deliver multiple packages of varying weights.

Abstract

Deep Reinforcement Learning (DRL) agents have shown impressive ability in mastering computer games, but notoriously take a long time to learn. As an agent progresses through a game, it will often encounter new states containing previously unencountered game entities, e.g., new enemies. In such situations, DRL agents typically struggle to generalise their prior knowledge to the new entities, owing to differences in state and object representations. In particular, even when new entities behave similarly to previously encountered ones, if they appear to be different then DRL agents can take a long time to adapt.

Policy transfer learning offers a promising approach for allowing DRL agents to adapt their knowledge; however, establishing the connection between the newly presented states (the target task) and previously encountered ones (the source task) requires guidance from a domain expert. Guidance in the form of externally constructed mapping of state-action pairings, must be continually maintained in response to new game entity encounters.

This thesis proposes an alternative approach, where policy transfer is accomplished by leveraging an intermediate state transformation, removing the need for manual mapping. Each entity is mapped to a unique entity ID, and when a new game entity is encountered, a "substitution agent" strives to learn a mapping between the new entity ID and a previously encountered one. For example, if the new entity is a type of enemy, the substitution agent will ideally learn to map the new ID to a previously encountered enemy's ID, rather than, say, the ID of a powerup item. Experimental results show that this approach is effective, allowing for rapid improvement of end-of-episode scores when encountering new entity representations in the game, Infinite Mario.

Abstract

Poor road boundary lanes and detections of road objects have been identified as some of the serious causes of road accidents, in both conventional and autonomous driving. Therefore, it is critical to develop models that could help autonomous vehicles' perception systems while accurately identifying and locating road objects from images and video frames. However, the existing models face a series of challenges due to the highly complex nature of the road traffic scene and the influence of various road objects on the manoeuvring. Most of the existing models cannot simultaneously detect all the major road objects, with some, either detecting lanes or detecting some of the road objects. To address these gaps, the combined road objects and lane detection model was developed using the You Only Look Once (YOLO) algorithm. As a first step, a model was developed to detect road objects only and the results were compared with existing studies. Next, another model was developed to detect road lanes based on YOLOv8 capability. Finally, an improved YOLOv8 model was developed to simultaneously detect road objects and lanes. To achieve this, the YOLOv8 model was tuned and optimised using various optimization approaches considering several hyperparameters such as activation functions and regularisation methods. Further, the effect of augmentation was investigated using three techniques; cut-out, rotation and rotation with noise. Also, the effect of the data stream on the performance of the model was investigated based on the obtained hyperparameter. The relevant performance metrics such as precision, F1, and recall were deployed. In addition, mean average precision calculated at an intersection over union (IoU) threshold of 0.5 and 0.95 was reported to assess the model's detection capabilities. The results from this study were further compared with some existing studies such as Feature Pyramid Networks, Task-aligned One-stage Object Detection, Dynamic R-CNN Probabilistic Anchor Assignment with IoU Prediction, Sparse R-CNN and CenterNet to demonstrate the contribution of the model. Further, the performance of the models based on different dataset (Curated data, COCO, and KITTI) showed that curated data outperformed others across all the performance metrics. Notably, curated data has the most promising results with precision, recall and F1 score of 0.68, 0.61, and 0.64, respectively. The success of the curated dataset highlights the significance of tailoring datasets to the specific nuances of the targeted application domain. Finally, the conclusion and recommendations were made based on the findings from the study.

Abstract

Macadamia nut farmers estimate that up to 15% of the macadamia nuts grown each harvest season are left on the ground due to current environmental impacts and harvesting practices. Our research is a first step in addressing and more accurately ascertaining this loss through a vision-based loss estimation system onboard a resource constrained robotic platform.

A vision-based approach confers benefits of scalability and cost-reduction over current manual count and estimate practices. However, this approach carries a number of challenges related to data sparsity and vision-based object identification. Due to the rural and often remote location of macadamia nut orchards, collecting sufficient data is inhibitive in terms of cost, time, and logistics. The inherent challenges of vision-based identification are compounded by the camouflage and occlusion of the green in-shell macadamia nuts in a predominantly green background.

Our research addresses this challenging vision problem through the use of a surrogate training technique for overcoming data sparsity limitations and a set of generalisable and robust manually-tuned heuristics for use with vision systems. The application of this training approach and our heuristics build on current deep convolution networks to produce improved classification accuracy to a standard normally reserved for simpler problems (e.g., blueberry identification). Our approach was supported by recent advances in prompt-based instance segmentation to perform annotations on a scale which would have otherwise been infeasible given time constraints, thereby enabling the utilisation of larger convolutional neural networks.

To the best of our knowledge, our novel system for loss estimation is the first public application of modern deep neural networks and artificial intelligence techniques to the Australian macadamia nut industry since 2008.

Abstract

The work presents a faster hybrid approach in order to identify different signals from human soccer referee in real-time for the "Visual Referee Challenge" which is introduced in 2022 RoboCup Standard Platform League (SPL). The main objective of the challenge is to develop a solution that could be applied to the resource constrained humanoid Nao V6 robots for better in-game human-robot interaction. Our approach consists of a lightweight machine learning model combined with hand-crafted heuristic method. Further, the performance of the devised approach is contrasted against the 2 sophisticated Convolutional Neural Networks (CNNs) models that are specifically designed for identifying different key points in human body. The result in single image shows that there is significant trade-off between the high accuracy of the CNN models and faster processing time of our approach. However, the analysis on real-time video sequence demonstrates that the proposed hybrid model produce strong performance in much shorter time and is preferred over more powerful and time-consuming CNNs, such that, the devised algorithm achieved 3rd position in the official competition.

Abstract

In this work, we present a lightweight keypoint-based Recurrent Neural Network (RNN) approach to the 2023 RoboCup Standard Platform League (SPL) Visual Referee Challenge. The goal of the challenge is to classify 13 different static and dynamic referee gestures during a robotic soccer match. As such, the developed solution must be lightweight and perform well from anywhere on the field. Multiple referees may be present on the field at a time, with the main distinguishing trait of the visual challenge referee being their red gloves. We use the lightweight Convolutional Neural Network (CNN) pose detector BlazePose Lite to extract the pose key-points. To isolate the referee in the frame, we find the red glove regions using HSV colour segmentation and mask unneeded sections of the image. We then use a lightweight Recurrent Neural Network (RNN) to classify the sequence of keypoints. We compare the results of our work to previous solutions developed for the 2022 Visual Referee Challenge. Analysis on real-time testing shows our method achieves strong performance for all gestures on many different locations on the soccer field. Though slower than previous hybrid methods, our solution is easily able to distinguish between the static and dynamic gestures.

Abstract

The alleviation of turbulence for small unmanned aerial vehicles (sUAVs) is critical, as it significantly impacts their performance, safety, and mission constraints. Previous wind tunnel experiments highlighted the remarkable abilities of the nankeen kestrel (Falco cenchroides) to windhover steadily in the face of intense turbulence, attributed to its utilization of the multiple degrees of freedom (DoF) present in its wings. The novel design of a bio-inspired aircraft wing inspired by Kestrel birds, consisting of a main body wing and a tail fin with feathers, requires a new design for a servo-level flight controller. Existing controllers, both optimisation-based and learning-based, are insufficient. The bio-inspired design significantly differs from traditional aerodynamic control surfaces, which exhibit unknown nonlinear dynamics that cannot be replicated in simulation, especially in turbulent wind flow conditions. Deep reinforcement learning (DRL) has been used for training stable controllers for non-linear systems, but there is limited investigation into online DRL for stable aircraft flight. We conduct a novel investigation into online DRL design requirements for training a closed-loop control system for a half-wing robotic replica of a Kestrel, to derive an optimal nonlinear policy for effective control and turbulence mitigation for a multi-degree of freedom, multi-objective control system. We consider the performance of DRL algorithms including Twin-Delayed Deep Deterministic and Soft-Actor-Critic, and present design requirements for online learning without simulation, including reward function shaping, state space, and action space configurations. We demonstrate the effectiveness of our DRL designs through wind-tunnel experiments, achieving gust rejections in lift by up to 72.6%, and in pitch moment by up to 68.4%.

Abstract

Swarms of unmanned aerial vehicles (UAVs) have been gaining a growing interest for civil and military applications. Utilizing UAV swarms increases flight missions' efficiency, robustness, and reliability, e.g., surveillance or monitoring missions. Fixed-wing UAVs have increased energy efficiency in horizontal flight, longer mission endurance, and better stability, making them more suitable for monitoring large areas than rotary-wing UAVs. Formation flight is an essential part of a UAV swarm. The leader-follower strategy is well-known for its easy applicability, scalability, and flexibility. This work deals with the development and set up of a UAV swarm system for the application of large-area monitoring. It analyses two approaches for utilizing a leader-follower strategy for a fixed-wing UAV swarm: a ground-station-based and a leader-based approach. The ground-station-based approach uses a ground station to calculate the commands for the follower drones and send them to the follower drone. This approach highly relies on good communication. In contrast, the leader-based approach uses a companion computer on the leader drone to calculate commands for the follower drones and send them directly to the follower drone via a mesh network. Extensive simulations using MissionPlanner and flight tests were performed with both approaches. Comparing the approaches in terms of formation accuracy and flight stability provides insights into the advantages of both approaches. The ground-station-based approach reached a final formation accuracy of 67 meters, while the leader-based approach reached a formation accuracy of 53 meters. The leader-based approach proved to be more robust to communication losses and showed higher flight stability. These results show that a leader-based approach is more suitable for application for a large-area monitoring mission. This project sets the foundation for future work in the field of fixed-wing UAV swarms.

Abstract

The main objective and outcome of the research is the development of a functioning digital twin for the prediction of the flow fields around a building exposed to a variety of wind conditions in urban environments. Unmanned aerial vehicles, also referred to as UAVs, air taxis, helicopters and a range of other aerial vehicles provides a promising platform for this technology due to an ever-expanding interest in operating within these obstacle dense and unpredictable environments. Allowing them to manoeuvre accordingly in reaction to onboard sensors comparing simulated data of the buildings flow field, aiding in the prediction of conditions along its flight path. Ability to predict wind conditions would be beneficial for flight path mapping with current studies highlighting the correlation between turbulent flow experienced by aircraft in urban environments and catastrophic crashes. Further potential is seen with the prediction of the shear layers behaviour on top of the building as these changes in velocity and induced turbulent regions have potential for sudden disastrous effects on aircrafts behaviour during taking off, landing and operation within this region. In order to develop a digital twin for the stated application, research on the behaviour of wind flow fields on isolated buildings will be conducted. Known calculations and quantitative values of estimating the position of the points of interest such as stagnation of the flow on the front of the building, separation on top of the build and wake sizes will need to be investigated to accurately place sensor points in simulations to train, test, and apply the predictive abilities of the digital twin. The proposed area of research can be carried out through the use of Computer Aided Design (CAD), Computational Fluid Dynamics (CFD), Machine Learning (ML), High Power Computational (HPC) devices, powerful computational servers, and practical validation testing from recorded data on scale or full-size models.