Rasti, SanazSanazRasti2022-06-302022-06-302022 the A2022http://hdl.handle.net/10197/12947Demand for increased food production arising from steady population growth has focused attention on smart farming. Automatic crop growth monitoring is an important part of smart farming. Computer vision offers a promising approach to the problem of automated crop growth monitoring. The study herein focuses on wheat and barley growth stage (GS) estimation using machine learning algorithms and in-field proximal images. Recently, Convolutional Neural Networks (ConvNets), have achieved state of the art performance for various image processing tasks. The aim of this study is to employ ConvNets for GS classification of wheat and barley crops. In this research, extensive data collection was undertaken over three years. Video recordings were captured using a DJI Osmo+ camera in video mode and images were extracted as a post processing stage. Two camera views were recorded, downward looking and 45° angled looking. Ground truth GS was recorded by applying the Zadoks growth scale to the observed crop. The wheat dataset contains 110,000 images, in 34 classes. The barley dataset contains 106,000 images obtained in 33 classes. For comparison purposes GS classification was carried out using feature extraction and Support Vector Machine (SVM) classifier. The best results using this algorithm were achieved by training on a mix of downward and 45° angled looking images in each class. Excess Green index features were extracted from images as the most relevant index for crop growth metric image analysis. GS classification accuracy of seen field-day test data using this method was 63.8% and 59.8% for wheat and barley, respectively. Principal GS classification of unseen field-day test data resulted in 26.4% and 29.3% accuracy for wheat and barley, respectively. Two ConvNet models were studied for the GS classification task. The first model was trained by learning from scratch. For this model, two different architectures were trained: (a) without batch normalization layers and (b) with batch normalization layers. The input data was preprocessed. The best accuracy achieved for this model employed batch normalization layers, preprocessed and augmented input, and each class consisted of a mix of downward and 45° angled looking images. Employing this model and data preprocessing resulted in 95.4% and 96.5% classification accuracy for seen field-day test data of wheat and barley, respectively. The principal GS classification of unseen field-day test data achieved 73.4% and 77.2% accuracy for wheat and barley, respectively. The second ConvNet model used transfer learning based on VGG19 with ImageNet pre-trained weights. Experiments for fine-tuning the model, input preprocessing and augmentation were carried out. The best results employing this model were achieved for fine-tuned VGG19 with four fully connected layers on top of the network; with the last Conv-layer of the network and these four layers were set as trainable. This experiment included preprocessing, augmentation and a mix of downward and angled looking images in each class. The model achieved 97.3% and 97.5% GS classification accuracy for seen field-day test data of wheat and barley, respectively. The result of principal GS classification of unseen field-day test data using this model achieved 93.5% and 92.2% accuracy for wheat and barley, respectively. Of the three models investigated, transfer learning achieved the highest accuracy for both classification tasks. The main contributions of this work includes a comprehensive literature survey on high resolution image processing techniques for crop growth metric monitoring, a unique GS labeled proximal image dataset of wheat and barley crops, GS classification of cereal crops using crop colour index feature extraction and SVM classifier, GS classification of cereal crops employing ConvNet with learning from scratch and GS classification of cereal crops employing ConvNet with transfer learning.enArtificial neural networksDeep learningConvNetsCrop growth stage estimationDoctoral Thesis2022-06-04https://creativecommons.org/licenses/by-nc-nd/3.0/ie/