All Issue

2022 Vol.64, Issue 1 Preview Page
Development of 3D Crop Segmentation Model in Open-field Based on Supervised Machine Learning Algorithm

지도학습 알고리즘 기반 3D 노지 작물 구분 모델 개발

Young-Joon Jeonga
,
Jong-Hyuk Leeb
,
Sang-Ik Leec
,
Bu-Yeong Ohd
,
Fawzy Ahmede
,
Byung-Hun Seof
,
Don-Su Kimg
,
Ye-Jin Seoh
,
Won Choii†
정 영준a
,
이 종혁b
,
이 상익c
,
오 부영d
,
Ahmed Fawzyee
,
서 병훈f
,
김 동수g
,
서 예진h
,
최 원i†
aM.S. Student, Department of Rural Systems Engineering, Global Smart Farm Convergence Major, Seoul National University
bPh.D. Student, Department of Rural Systems Engineering, Global Smart Farm Convergence Major, Seoul National University
cPh.D. Student, Department of Rural Systems Engineering, Global Smart Farm Convergence Major, Seoul National University
dAgricultural Researcher, Division of Soil and Fertilizer, National Institute of Agricultural Sciences, Rural Development Administration
ePh.D. Student, Department of Rural Systems Engineering, Global Smart Farm Convergence Major, Seoul National University
fM.S. Student, Department of Rural Systems Engineering, Global Smart Farm Convergence Major, Seoul National University
gM.S. Student, Department of Rural Systems Engineering, Global Smart Farm Convergence Major, Seoul National University
hM.S. Student, Department of Rural Systems Engineering, Global Smart Farm Convergence Major, Seoul National University
iDepartment of Landscape Architecture and Rural Systems Engineering, Research Institute of Agriculture and Life Sciences, Global Smart Farm Convergence Major, College of Agriculture and Life Sciences, Seoul National University
a서울대학교 글로벌스마트팜 융합전공 농촌시스템공학과 석사과정 학생
b서울대학교 글로벌스마트팜 융합전공 농촌시스템공학과 박사과정 학생
c서울대학교 글로벌스마트팜 융합전공 농촌시스템공학과 박사과정 학생
d농촌진흥청 국립농업과학원 토양비료과 농업연구원
e서울대학교 글로벌스마트팜 융합전공 농촌시스템공학과 박사과정 학생
f서울대학교 글로벌스마트팜 융합전공 농촌시스템공학과 석사과정 학생
g서울대학교 글로벌스마트팜 융합전공 농촌시스템공학과 석사과정 학생
h서울대학교 글로벌스마트팜 융합전공 농촌시스템공학과 석사과정 학생
i서울대학교 농업생명과학대학 조경농촌시스템공학과 농업생명과학연구원 글로벌스마트팜 융합전공
† Corresponding author Tel.: +82-2-880-4715, Fax: +82-2-873-2087 E-mail: fembem@snu.ac.kr
2022. pp. 15-26
PDF XML Citation
Abstract

3D open-field farm model developed from UAV (Unmanned Aerial Vehicle) data could make crop monitoring easier, also could be an important dataset for various fields like remote sensing or precision agriculture. It is essential to separate crops from the non-crop area because labeling in a manual way is extremely laborious and not appropriate for continuous monitoring. We, therefore, made a 3D open-field farm model based on UAV images and developed a crop segmentation model using a supervised machine learning algorithm. We compared performances from various models using different data features like color or geographic coordinates, and two supervised learning algorithms which are SVM (Support Vector Machine) and KNN (K-Nearest Neighbors). The best approach was trained with 2-dimensional data, ExGR (Excess of Green minus Excess of Red) and z coordinate value, using KNN algorithm, whose accuracy, precision, recall, F1 score was 97.85, 96.51, 88.54, 92.35% respectively. Also, we compared our model performance with similar previous work. Our approach showed slightly better accuracy, and it detected the actual crop better than the previous approach, while it also classified actual non-crop points (e.g. weeds) as crops.

Keywords
Open-field farm
point cloud model
supervised learning
precision agriculture
3D image segmentation
References

REFERENCES

1
H. R. Ahn, J. I. Kim and T. J. Kim, Journal of the Korean Society of Broad Engineers, 3D accuracy analysis of mobile phone-based stereo images, 19(5); 677-686, in Korean (2014)10.5909/JBE.2014.19.5.677.
2
C. Albornoz and L. F. Giraldo, 2017 IEEE 3rd Colombian Conference on Automatic Control (CCAC); 1-6, Cartagena, Colombia. IEEE. (2018)10.1109/CCAC.2017.8276401.
3
Y. Ampatzidis, V. Partel and L. Costa, Computers and Electronics in Agriculture, Agroview: Cloud-based application to process, analyze and visualize UAV-collected data for precision agriculture applications utilizing artificial intelligence, 174; 105457 (2020)10.1016/j.compag.2020.105457
4
P. Angin, M. H. Anisi, F. Goksel, C. Gursoy and A. Buyukgulcu, Journal of Wireless Mobile Networks Ubiquitous Computing and Dependable Applications, AgriLoRa: A digital twin framework for smart agriculture, 11; 77-96 (2020)10.22667/JOWUA.2020.12.31.077.
5
M. D. Bah, A. Hafiane and R. Canals, 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA); 1-6, Montreal, QC, Canada. IEEE. (2017)10.1109/IPTA.2017. 8310102.
6
D. Bzdok, M. Krzywinski and N. Altman, Nature Methods, Machine learning: supervised methods, 15(1); 5 (2018)10.1038/nmeth.4551.
7
M. T. Chiu, X. Xu, Y. Wei, Z. Huang, A. Schwing, R. Brunner, H. Khachatrian, H. Karapetyan, I. Dozier, G. Rose, D. Wilson, A. Tudor, N. Hovakimyan, T. S. Huang and H. Shi, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Agriculture-vision: A large aerial image database for agricultural pattern analysis; 2825-2835 (2020)10.1109/cvpr42600.2020.00290
8
S. K. Choi, S. K. Lee, Y. B. Kang, D. Y. Choi and J. W. Choi, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Use of unmanned aerial vehicle imagery and deep learning UNet to classification upland crop in small scale agricultural land, 38(6); 671-679, in Korean (2020)10.7848/ksgpc.2020.38.6. 671.
9
L. Comba, A. Biglia, D. R. Aimonino and P. Gay, Computers and Electronics in Agriculture, Unsupervised detection of vineyards by 3D point-cloud UAV photogrammetry for precision agriculture, 155; 84-95 (2018)10.1016/ j.compag.2018.10.005.
10
Commercializations Promotion Agency for R&D Outcomes, Smart farm technology and market trend report; 4-14, Seoul. (2019)
11
L. Costa, L. Nunes and Y. Ampatzidis, Computers and Electronics in Agriculture, A new visible band index (vNDVI) for estimating NDVI values on RGB images utilizing genetic algorithms, 172; 105334 (2020)10.1016/ j.compag.2020.105334.
12
A. I. De Castro, F. M. Jimenez-Brenes, J. Torres-Sanchez, J. M. Pena, I. Borra-Serrano and F. Lopez-Granados, Remote Sensing, 3-D characterization of vineyards using a novel UAV imagery-based OBIA procedure for precision viticulture applications, 10(4); 584 (2018)10.3390/ rs10040584.
13
H. I. Dino and M. B. Abdulrazzaq, In 2019 International Conference on Advanced Science and Engineering (ICOASE), Facial expression classification based on SVM, KNN and MLP classifiers; 70-75, Zakho - Duhok, Iraq. IEEE. (2019)10.1109/ICOASE.2019.8723728.
14
D. Fawcett, B. Azlan, T. C. Hill, L. K. Kho, J. Bennie and K. Anderson, International Journal of Remote Sensing, Unmanned aerial vehicle (UAV) derived structure-from-motion photogrammetry point clouds for oil palm (Elaeis guineensis) canopy segmentation and height estimation, 40(19); 7538-7560 (2019)10.1080/01431161.2019.1591651.
15
A. D. S. Ferreira, D. M. Freitas, G. G. D. Silva, H. Pistori and M. T. Folhes, Computers and Electronics in Agriculture, Weed detection in soybean crops using ConvNets, 143; 314-324 (2017)10.1016/j.compag.2017.10.027.
16
C. Gée, J. Bossu, G. Jones and F. Truchetet, Computers and Electronics in Agriculture, Crop/ weed discrimination in perspective agronomic images, 60(1); 49-59 (2008)10.1016/j.compag.2007.06.003.
17
E. R. Hunt, W. D. Hively, S. J. Fujikawa, D. S. Linden, C. S. Daughtry and G. W. McCarty, Remote Sensing, Acquisition of NIR-green-blue digital photographs from unmanned aircraft for crop monitoring, 2(1); 290-305 (2010)10.3390/rs2010290.
18
S. Jay, G. Rabatel, X. Hadoux, D. Moura and N. Gorretta, Computers and Electronics in Agriculture, In-field crop row phenotyping from 3D modeling performed using Structure from Motion, 110; 70-77 (2015)10.1016/j.compag. 2014.09.021.
19
F. M. Jimenez-Brenes, F. Lopez-Granados, A. I. de Castro, J. Torres-Sanchez, N. Serrano and J. M. Pena, Plant Methods, Quantifying pruning impacts on olive tree architecture and annual canopy growth by using UAV-based 3D modeling, 13(55) (2017)10.1186/s13007-017-0205-3.
20
J. M. Jurado, L. Ortega, J. J. Cubillas and F. R. Feito, Remote Sensing, Multispectral mapping on 3D models and multi- temporal monitoring for individual characterization of olive trees, 12(7); 1106 (2020)10.3390/rs12071106.
21
M. Kerkech, A. Hafiane and R. Canals, Remote Sensing, VddNet: Vine disease detection network based on multispectral images and depth map, 12(20); 3305 (2020)10.3390/rs12203305.
22
H. G. Kim, H. S. Yun and J. M. Cho, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Analysis of 3D accuracy according to determination of calibration initial value in close-range digital photogrammetry using VLBI antenna and mobile phone camera, 33(1); 31-43, in Korean (2015)10.7848/ ksgpc.2015.33.1.31.
23
K. Y. Kim and S. I. Na, Journal of Korean Society for Geospatial Information System, Vegetation map service system using UAV imagery and sample field data on major cultivation regions, 27(1); 33-41 (2019)10.7319/kogsis.2019. 27.1.033.
24
S. K. Kim and J. G. Ahn, Journal of the Korea Academia-Industrial, Data mining based forest fires prediction models using meteorological data, 21(8); 521-529, in Korean (2020)10.5762/KAIS.2020.21.8.521.
25
H. Long, Z. Zhang and Y. Su, Applied Energy, Analysis of daily solar power prediction with data-driven approaches, 126; 29-37 (2014)10.1016/j.apenergy.2014.03.084.
26
A. J. Mathews and J. L. R. Jensen, Remote Sensing, Visualizing and quantifying vineyard canopy LAI using an unmanned aerial vehicle (UAV) collected high density structure from motion point cloud, 5(5); 2164-2183 (2013)10. 3390/rs5052164.
27
F. Mesas-Carrascosa, A. I. De Castro, J. Torres-Sanchez, P. Trivino-Tarradas, F. M. Jimenez-Brenes, A. Garcia-Ferrer and F. Lopez-Granados, Remote Sensing, Classification of 3D point clouds using color vegetation indices for precision viticulture and digitizing applications, 12(2); 317 (2020)10.3390/rs12020317.
28
Ministry of Agriculture, Food and Rural Affairs, Agricultural business registration information inquiry site, https://uni.agrix.go.kr/docs7/biOlap/dashBoard.do. 2021.
29
F. S. Mohamad, M. Iqtait and F. Alsuhimat, In 2018 4th International Conference on Computer and Technology Applications (ICCTA), Age prediction on face features via multiple classifiers; 161-166, Istanbul, Turkey. IEEE. (2018)10.1109/CATA.2018.8398675.
30
A. K. Mortensen, A. Benderb, B. Whelanc, M. M. Barbourc, S. Sukkariehb, H. Karstoftd and R. Gisluma, Computers and Electronics in Agriculture, Segmentation of lettuce in coloured 3D point clouds for fresh weight estimation, 154; 373-381 (2018)10.1016/j.compag.2018. 09.010.
31
S. Na, C. Park, K. So, H. Ahn and K. Lee, Korean Journal of Remote Sensing, Selection on optimal bands to estimate yield of the chinese cabbage using drone-based hyperspectral image, 35(3); 375-387, in Korean (2019)10.7780/ kjrs.2019.35.3.3.
32
Y. Pan, Y. Han, L. Wang, J. Chen, H. Meng, G. Wang, Z. Zhang and S. Wang, IFAC-Papers On Line, 3D reconstruction of ground crops based on airborne LiDAR technology, 52(24); 35-40 (2019)10.1016/j.ifacol.2019.12.376.
33
J. K. Park, A. Das and J. H. Park, CNU Journal of Agricultural Science, Application trend of unmanned aerial vehicle (UAV) image in agricultural sector: Review and proposal, 42(3); 269-276, in Korean (2015)10.7744/cnujas. 2015.42.3.269.
34
W. Pi, J. Du, Y. Bi, X. Gao and X. Zhu, Ecological Informatics, 3D-CNN based UAV hyperspectal imagery for grassland degradation indicator ground object classification research, 62; 101278 (2021)10.1016/j.ecoinf.2021.101278.
35
C. Sager, P. Zschech and N. Kühl, labelCloud: A lightweight domain-independent labeling tool for 3D object detection in point clouds, arXiv: 2103.04970 [cs.CV] (2021)
36
R. Sarabia, A. Aquino, J. M. Ponce, G. Lopez and J. M. Andujar, Remote Sensing, Automated identification of crop tree crowns from UAV multispectral imagery by means of morphological image analysis, 12(5); 748 (2020)10.3390/rs12050748.
37
J. Sarron, E. Malezieux, C. A. B. Sane and E. Faye, Remote Sensing, Mango yield mapping at the orchard scale based on tree structure and land cover assessed by UAV, 10(12); 1900 (2018)10.3390/rs10121900.
38
M. W. Smith, J. L. Carrivick and D. J. Quincey, Progress in Physical Geography: Earth and Environment, Structure from motion photogrammetry in physical geography, 40(2); 247-275 (2015)10.1177/030913331561 5805.
39
M. Tjahjadi and F. D. Agustina, International Journal of Advances in Intelligent Informatics, Fast and Stable DirectRelative Orientation of UAV-based Stereo Pair, 5(1); 24-39 (2019)10.26555/ijain.v5i1.327.
40
J. Torres-Sanchez, A. I. De Castro, J. M. Pena, F. M. Jimenez-Brenes, O. Arquero, M. Lovera and F. Lopez- Granados, Biosystems Engineering, Mapping the 3D structure of almond trees using UAV acquired photogrammetric point clouds and object-based image analysis, 176; 172-184 (2018)10.1016/j.biosystemseng. 2018.10.018.
41
Z. Xu, K. Guan, N. Casler, B. Peng and S. Wang, ISPRS Journal of Photogrammetry and Remote Sensing, A 3D convolutional neural network method for land cover classification using LiDAR and multi-temporal Landsat imagery, 144(6); 423-434 (2018)10.1016/j.isprsjprs.2018.08.005
42
S. Yang, L. Zheng, W. Gao, B. Wang, X. Hao, J. Mi and M. Wang, Remote Sensing, An efficient processing approach for colored point cloud-based high-throughput seedling phenotyping, 12(10); 1540 (2020)10.3390/rs12101540.
43
C. L. Zhang, K. F. Zhang, L. Z. Ge, K. L. Zou, S. Wang, J. X. Zhang and W. Li, Scientia Horticulturae, A method for organs classification and fruit counting on pomegranate trees based on multi-features fusion and support vector machine by 3D point cloud, 278; 109791 (2021)10. 1016/j.scienta.2020.109791.
44
Q. Y. Zhou, J. S. Park and V. Koltun, Open3D: A modern library for 3D data processing, arXiv: 1801. 09847v1 [cs.CV] (2018)
45
M. G. Ziliani, S. D. Parkes, I. Hoteit and M. F. McCabe, Remote Sensing, Intra-season crop height variability at commercial farm scales using a fixed-wing UAV, 10(12); 2007 (2018)10.3390/rs10122007.
Information
  • Publisher :The Korean Society of Agricultural Engineers
  • Publisher(Ko) :한국농공학회
  • Journal Title :Journal of Korean Society of Agricultural Engineers
  • Journal Title(Ko) :한국농공학회논문집
  • Volume : 64
  • No :1
  • Pages :15-26
  • Received Date : 2021-06-14
  • Revised Date : 2021-09-17
  • Accepted Date : 2021-09-24
  • DOI :https://doi.org/10.5389/KSAE.2022.64.1.015
Journal Informaiton Journal of Korean Society of Agricultural Engineers Journal of Korean Society of Agricultural Engineers
  • NRF
  • KOFST
  • crossref crossmark
  • crossref cited-by
  • crossref funder-registry
  • crosscheck
  • orcid
  • open access
Journal Informaiton Journal Informaiton - close