PRELUDIUM grant - Development of weak supervision methods for multispectral image processing in the field of precision agriculture

Przemyslaw received a PRELUDIUM 22 grant for PhD students organised National Science Centre (NCN). The topic of reseatch is Development of weak supervision methods for multispectral image processing in the field of precision agriculture. The project will start in Semptember 2024 and will last 12 months.

Abstract for the general public

Precision agriculture is an advanced approach to farming that leverages computer technology to enhance crop yields, reduce costs, and promote sustainability. One interesting aspect of precision agriculture involves the use of computer vision tools, which provide a non-intrusive means of measuring and monitoring crops. While most computer vision applications focus on the visible spectrum (red, green, and blue colours), this research delves into the realm of multispectral images. These images contain additional spectral bands that reveal information beyond what is visible to the human eye, such as near-infrared and ultraviolet ranges.

The primary objective of this research is to develop innovative methods for processing agricultural images, thereby enabling and simplifying new applications in precision agriculture, specifically within the realm of crop farming. Key tasks in this domain include object detection and segmentation. Object detection involves identifying and classifying objects visible in the image, such as distinguishing weeds among crops. Segmentation tasks entail assigning specific classes to different regions of the image, for example, determining areas in the field where crops are diseased. Both of these tasks rely on data acquired from ground-level images or airborne sources, such as drones, which have gained popularity in recent years.

Existing tools in precision agriculture often require large amounts of labelled data to accomplish specific tasks. For instance, localizing cauliflower heads in an image would necessitate manual annotation of hundreds of images, outlining each cauliflower individually. This process of data labelling is time-consuming and expensive, discouraging the widespread use of such applications. However, recent technological advancements in methods and the availability of multispectral sensors present new opportunities to reduce the manual labelling burden.

This research focuses not only on publicly available datasets but also on collecting new data using drones for selected fields. The investigation centres on minimizing the amount of manually labelled data required, without compromising the quality of results compared to traditional methods that demand more labelled data. Additionally, the research explores the utilization of algorithms that intelligently select a small number of images for annotation, where human input is most needed. Moreover, the study examines the potential of foundation models, such as the recent Segment Anything Model from Meta, which has been trained on a diverse range of labelled data. These models enable the implementation of techniques that mimic human generalization and learning abilities, requiring only a few labelled examples to automatically detect new insects in other images.

The anticipated outcomes of this research include the improvement of methods for processing multispectral images, along with their practical implementation through code. Additionally, a dataset and reference models, capable of addressing specific tasks, will be created using the developed methods. All tangible results will be made freely available as open-source resources for the benefit of the wider community.