vstispСадоводство и виноградарствоHorticulture and viticulture0235-25912618-9003Autonomous non-profit organization Editorial Board of journal «Horticulture and viticulture»10.31676/0235-2591-2023-6-41-50vstisp-1098Research ArticleИНФОРМАЦИОННЫЕ ТЕХНОЛОГИИINFORMATION TECHNOLOGYАнализ гиперспектральных снимков плодов яблони для автоматизированной сортировкиAnalysis of hyperspectral images of apple fruits for automated sortinghttps://orcid.org/0000-0001-6503-0065ХортД. О.KhortD. O.

Хорт Д. О. − доктор технических наук, ведущий научный сотрудник

1-й Институтский проезд, д. 5, Москва, 109428

Khort D. O., Dr. Sci. (Tech.), Leading Researcher

5, 1st Institute Passage, Moscow, 109428

dmitriyhort@mail.ruhttps://orcid.org/0000-0001-9650-3523ЧиликинА. Д.ChilikinA. D.

Чиликин А. Д. − аспирант, научный сотрудник

1-й Институтский проезд, д. 5, Москва, 109428

Chilikin A.D., Graduate Student, Researcher

5, 1st Institute Passage, Moscow, 109428

https://orcid.org/0000-0003-4795-6977ШурыгинБ. М.ShuryginB. M.

Шурыгин Б. М. – ведущий инженер, кафедра биоинженерии, биологический факультет

Москва

Shurygin B. M., Leading Engineer

Moscow

Федеральный научный агроинженерный центр ВИМРоссияFederal Scientifi c Agroengineering Center VIMRussian FederationМосковский государственный университет имени М. В. ЛомоносоваРоссияLomonosov Moscow State UniversityRussian Federation202320122023064150Copyright © Autonomous non-profit organization Editorial Board of journal «Horticulture and viticulture», 20232023Autonomous non-profit organization Editorial Board of journal «Horticulture and viticulture»Autonomous non-profit organization Editorial Board of journal «Horticulture and viticulture»https://www.sadivin.com/jour/about/submissions#copyrightNoticehttps://www.sadivin.com/jour/article/view/1098

Одним из неинвазивных методов обнаружения повреждений плодов является их мониторинг с использованием гиперспектрометров. Гиперспектральные изображения представляют собой набор пространственноразрешенных спектров излучения отраженного объекта. Используя эту информацию, можно выделить характеристики и параметры плодов, которые могут указывать на их повреждения. В данном исследовании анализировали спектральные и пространственные компоненты гиперспектральных снимков плодов яблони. Для классификации объектов на изображениях использовали классификаторы типа «случайный лес». В качестве входных данных для классификаторов использовались спектры отражения, вегетационные индексы и дескрипторы пространственной текстуры (локальные бинарные шаблоны). Классификаторы, основанные на спектральных признаках, работали надежнее тех, которые были обучены без использования спектральных данных. Использование спектральной информации о плодах значительно улучшало результаты классификации при наличии помех от неравномерности освещения. Комбинируя спектральные данные с методами машинного обучения, можно повысить эффективность сортировки плодов. Этот подход открывает новые возможности для разработки методов обработки данных от гиперспектральных сенсоров, установленных на сортировочных линиях с целью надежного обнаружения повреждений плодов яблони

One non-invasive method for detecting fruit damage involves monitoring them using hyper-spectrometers. Hyperspectral images comprise a set of spatially resolved radiation spectra of a reflected object. By using these data, it is possible to identify the characteristics and parameters of fruits that may indicate their damage. In this study, the spectral and spatial components of hyperspectral images of apple fruits were analysed. Random forest classifiers were used to detect objects in the images, with reflection spectra, vegetation indices and spatial texture descriptors (local binary patterns) used as input data for classifiers. Classifiers based on spectral characteristics proved to be more reliable than those trained without spectral data. Using spectral information about fruits signifi cantly improved the classifi cation results under the conditions of uneven lighting interference. By combining spectral data with machine learning methods, fruit sorting efficiency can be improved. This approach offers advanced development of methods for processing data from hyperspectral sensors installed on sorting lines in order to detect damage to apple fruits reliably

цифровой мониторингидентификация повреждений плодов яблонифизиологические расстройстваинфекционные пораженияdigital monitoringidentifi cation of damage to apple fruitphysiological disturbanceinfectious lesionsReferencesCavaco A. M., Utkin A. B., Marques da Silva J., Guerra R. Making Sense of Light: The Use of Optical Spectroscopy Techniques in Plant Sciences and Agriculture. Appl. Sci. 2022;12:997.Cavaco A. M., Utkin A. B., Marques da Silva J., Guerra R. Making Sense of Light: The Use of Optical Spectroscopy Techniques in Plant Sciences and Agriculture. Appl. Sci. 2022;12:997.Che W., Sun L., Zhang Q., Tan, W., Ye D., Zhang D., Liu Y. Pixel based bruise region extraction of apple using Vis-NIR hyperspectral imaging. Comput. Electron. Agric. 2018;146:12-21.Che W., Sun L., Zhang Q., Tan, W., Ye D., Zhang D., Liu Y. Pixel based bruise region extraction of apple using Vis-NIR hyperspectral imaging. Comput. Electron. Agric. 2018;146:12-21.Di Gennaro S. F., Toscano P., Cinat P., Berton A., Matese A. A low-cost and unsupervised image recognition methodology for yield estimation in a vineyard. Front. Plant Sci. 2019;10:559.Di Gennaro S. F., Toscano P., Cinat P., Berton A., Matese A. A low-cost and unsupervised image recognition methodology for yield estimation in a vineyard. Front. Plant Sci. 2019;10:559.Drozdov D., Kolomeychenko M., Borisov Y. Supervisely. Available online: https://www.supervise.ly (accessed on 24 November 2022).Drozdov D., Kolomeychenko M., Borisov Y. Supervisely. Available online: https://www.supervise.ly (accessed on 24 November 2022).Gamon J. A., Somers B., Malenovský, Z.; Middleton, E. M.; Rascher, U.; Schaepman, M. E. Assessing Vegetation Function with Imaging Spectroscopy. Surv. Geophys. 2019, 40, 489-513.Gamon J. A., Somers B., Malenovský, Z.; Middleton, E. M.; Rascher, U.; Schaepman, M. E. Assessing Vegetation Function with Imaging Spectroscopy. Surv. Geophys. 2019, 40, 489-513.Gilpin L. H., Bau D., Yuan B. Z., Bajwa A., Specter M., Kagal L. Explaining explanations: An overview of interpretability of machine learning. In Proceedings of the 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA), Turin, Italy, 1–3 October 2018, 80-89.Gilpin L. H., Bau D., Yuan B. Z., Bajwa A., Specter M., Kagal L. Explaining explanations: An overview of interpretability of machine learning. In Proceedings of the 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA), Turin, Italy, 1–3 October 2018, 80-89.Gitelson A., Arkebauer T., Viña A., Skakun S., Inoue Y. Evaluating plant photosynthetic traits via absorption coeffi cient in the photosynthetically active radiation region. Remote Sens. Environ. 2021;258:112401.Gitelson A., Arkebauer T., Viña A., Skakun S., Inoue Y. Evaluating plant photosynthetic traits via absorption coeffi cient in the photosynthetically active radiation region. Remote Sens. Environ. 2021;258:112401.Gitelson A., Solovchenko A. Non-invasive quantifi cation of foliar pigments: Possibilities and limitations of refl ectance-and absorbance-based approaches. J. Photochem. Photobiol. B Biol. 2018;178:537-544.Gitelson A., Solovchenko A. Non-invasive quantifi cation of foliar pigments: Possibilities and limitations of refl ectance-and absorbance-based approaches. J. Photochem. Photobiol. B Biol. 2018;178:537-544.Gitelson A., Solovchenko A., Viña A. Foliar absorption coeffi cient derived from refl ectance spectra: A gauge of the efficiency of in situ light-capture by different pigment groups. J. Plant Physiol. 2020;254:153277.Gitelson A., Solovchenko A., Viña A. Foliar absorption coeffi cient derived from refl ectance spectra: A gauge of the efficiency of in situ light-capture by different pigment groups. J. Plant Physiol. 2020;254:153277.Hill C. Learning Scientifi c Programming with Python; Cambridge University Press: Cambridge, UK, 2020.Hill C. Learning Scientifi c Programming with Python; Cambridge University Press: Cambridge, UK, 2020.Hufk ens K., Melaas E. K., Mann M. L., Foster T., Ceballos F., Robles M., Kramer B. Monitoring crop phenology using a smartphone based near-surface remote sensing approach. Agric. For. Meteorol. 2019;265:327-337.Hufk ens K., Melaas E. K., Mann M. L., Foster T., Ceballos F., Robles M., Kramer B. Monitoring crop phenology using a smartphone based near-surface remote sensing approach. Agric. For. Meteorol. 2019;265:327-337.Lu R., Van Beers R., Saeys W., Li C., Cen H. Measurement of optical properti es of fruits and vegetables: A review. Postharvest Biol. Technol. 2020;159:111003.Lu R., Van Beers R., Saeys W., Li C., Cen H. Measurement of optical properti es of fruits and vegetables: A review. Postharvest Biol. Technol. 2020;159:111003.Pan E., Ma Y., Fan F., Mei. X., Huang J. Hyperspectral image classifi cation across different datasets: A generalization to unseen categories. Remote Sens. 2021;13:1672.Pan E., Ma Y., Fan F., Mei. X., Huang J. Hyperspectral image classifi cation across different datasets: A generalization to unseen categories. Remote Sens. 2021;13:1672.Penzel M., Tsoulias N., Herppich W.B., Weltzien C., Zude-Sasse M. Mapping the fruit bearing capacity in a commercial apple (Malus x domestica BORKH) orchard. In Proceedings of the 2020 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), Trento, Italy. November 2020;4-6:283-287.Penzel M., Tsoulias N., Herppich W.B., Weltzien C., Zude-Sasse M. Mapping the fruit bearing capacity in a commercial apple (Malus x domestica BORKH) orchard. In Proceedings of the 2020 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), Trento, Italy. November 2020;4-6:283-287.Rendon. E., Alejo R., Castorena C., Isidro Ortega F. J.; Granda Gutierrez E. E. Data sampling methods to deal with the big data multi-class imbalance problem. Appl. Sci. 2020;10:1276.Rendon. E., Alejo R., Castorena C., Isidro Ortega F. J.; Granda Gutierrez E. E. Data sampling methods to deal with the big data multi-class imbalance problem. Appl. Sci. 2020;10:1276.Roitsch T.; CabreraBosquet L., Fournier A., Ghamkhar K., JiménezBerni J., Pinto F., Ober E. S. Review: New sensors and data- driven approaches-A path to next generation phenomics. Plant Sci. 2019;282:2-10. [PubMed]Roitsch T.; CabreraBosquet L., Fournier A., Ghamkhar K., JiménezBerni J., Pinto F., Ober E. S. Review: New sensors and data- driven approaches-A path to next generation phenomics. Plant Sci. 2019;282:2-10. [PubMed]Shurygin B., Chivkunova O., Solovchenko O., Solovchenko A., Dorokhov A., Smirnov I., Astashev M. E., Khort D. Comparison of the Non-Invasive Monitoring of Fresh-Cut Lettuce Condition with Imaging Refl ectance Hyperspectrometer and Imaging PAM-Fluorimeter. Photonics. 2021;8:425.Shurygin B., Chivkunova O., Solovchenko O., Solovchenko A., Dorokhov A., Smirnov I., Astashev M. E., Khort D. Comparison of the Non-Invasive Monitoring of Fresh-Cut Lettuce Condition with Imaging Refl ectance Hyperspectrometer and Imaging PAM-Fluorimeter. Photonics. 2021;8:425.Shurygin B., Smirnov I., Chilikin A., Khort D., Kutyrev A., Zhukovskaya S., Solovchenko A. Mutual Augmentation of Spectral Sensing and MachineLearning for Non-Invasive Detection of Apple Fruit Damages / B. Shurygin, I. Smirnov, A. Chilikin, D. Khort, A. Kutyrev, S. Zhukovskaya, A. Solovchenko. // E3S Web of Conferences, Tashkent, 01–03 апреля 2021 года. – Tashkent, 2021. – P. 04017.Shurygin B., Smirnov I., Chilikin A., Khort D., Kutyrev A., Zhukovskaya S., Solovchenko A. Mutual Augmentation of Spectral Sensing and MachineLearning for Non-Invasive Detection of Apple Fruit Damages / B. Shurygin, I. Smirnov, A. Chilikin, D. Khort, A. Kutyrev, S. Zhukovskaya, A. Solovchenko. // E3S Web of Conferences, Tashkent, 01–03 апреля 2021 года. – Tashkent, 2021. – P. 04017.Solovchenko A., Dorokhov A., Shurygin B., Nikolenko A., Velichko V., Smirnov I., Khort D., Aksenov A., Kuzin A. Linking tissue damage to hyperspectral refl ectance for non-invasive monitoring of apple fruit in orchards. Plants. 2021;10:310.Solovchenko A., Dorokhov A., Shurygin B., Nikolenko A., Velichko V., Smirnov I., Khort D., Aksenov A., Kuzin A. Linking tissue damage to hyperspectral refl ectance for non-invasive monitoring of apple fruit in orchards. Plants. 2021;10:310.Solovchenko A., Lukyanov A., Nikolenko A., Shurygin B., Akimov M., Gitelson A. Physiological foundations of spectral imaging-based monitoring of apple fruit ripening. Acta Hortic. 2021;1324:419-428.Solovchenko A., Lukyanov A., Nikolenko A., Shurygin B., Akimov M., Gitelson A. Physiological foundations of spectral imaging-based monitoring of apple fruit ripening. Acta Hortic. 2021;1324:419-428.Vinuesa R., Sirmacek B. Interpretable deep-learning models to help achieve the Sustainable Development Goals. Nat. Mach. Intell. 2021;3:926.Vinuesa R., Sirmacek B. Interpretable deep-learning models to help achieve the Sustainable Development Goals. Nat. Mach. Intell. 2021;3:926.Walsh K. B.; Blasco J.; Zude-Sasse, M.; Sun, X. Visible-NIR «point» spectroscopy in postharvest fruit and vegetable assessment: The science behind three decades of commercial use. Postharvest Biol. Technol. 2020;168:111246.Walsh K. B.; Blasco J.; Zude-Sasse, M.; Sun, X. Visible-NIR «point» spectroscopy in postharvest fruit and vegetable assessment: The science behind three decades of commercial use. Postharvest Biol. Technol. 2020;168:111246.Yost J. M., Sweeney P. W., Gilbert E., Nelson G., Guralnick R., Gallinat A. S., Ellwood E. R., Rossington N., Willis C. G.; Blum S. D. Digitization protocol for scoring reproductive phenology from herbarium specimens of seed plants. Appl. Plant Sci. 2018;6:e1022.Yost J. M., Sweeney P. W., Gilbert E., Nelson G., Guralnick R., Gallinat A. S., Ellwood E. R., Rossington N., Willis C. G.; Blum S. D. Digitization protocol for scoring reproductive phenology from herbarium specimens of seed plants. Appl. Plant Sci. 2018;6:e1022.

The authors declare that there are no conflicts of interest present.