Plant Detection and State Classification with Machine Learning

Water deficiency in household plants can adversely affect growth. Existing solutions to monitor water stress are primarily intended for agricultural contexts where only asmall selection of plants are of interest. To date, there has been no research in household settings where the variety of plants is considerably higher and it is thus more difficult to obtain accurate measures of water stress. Furthermore, current approaches either do not detect plants in images first or use traditional feature extraction for plant detection. We develop a prototype to detect plants and classify them into water-stressed or not using deep learning based methods exclusively.Our two-stage approach consists of a detection and a classification step. In the detectionstep, plants are identified and cut out from the original image. The cut outs are passed to the classifier which outputs a probability for water stress. We use transfer learning to start from a robust base and fine-tune both models for their respective tasks. Each model is optimized using hyperparameter optimization and first evaluated individually and then in aggregate on a custom dataset. We deploy both models to an Nvidia Jetson Nano which is able to survey plants autonomously via an attached camera. The results of the pipeline are published continuously via an API. Downstream watering systems canuse the water stress predictions to water the plants without human intervention.The two models in aggregate achieve a mAP of 0.3581 for the non-optimized version.Both constituent models have robust feature extraction capabilities and are able to cope with various lighting conditions, different angles and a wide variety of household plants. The optimized pipeline achieves a mAP of 0.3838 on unseen images with higher precision for the non-stressed but lower precision for the stressed class. Recall for thenon-stressed class remains at the same level compared to the non-optimized baseline butis 12.1 percentage points higher for the stressed class. The weighted F1-score across both classes was improved by 2.4 percentage points. These results show that our two-stage approach is viable and a promising first step for plant state classification for household plants.