Improving the accuracy of soil heavy metal concentration estimation based on spectral reflectance by minimizing the effect of moisture with the external parameter orthogonalization algorithm

Rapid global industrial development has led to a significant increase in the risk of environmental contamination with heavy metals. Soil contamination with heavy metals is a challenge due to the risks associated with human health and the environment, as well as soil-related food security. It is considered universal. Soil moisture, as one of the influencing parameters on soil spectral reflectance and its high spatio-temporal variability, is considered the most important confounding factor in using Visible and Near Infrared Reflectance Spectroscopy (VNIRS) techniques to estimate soil heavy metals. . In this study, the ability of the external parameter orthogonal (EPO) algorithm to reduce the effect of moisture on soil spectral reflectance was evaluated to improve the performance of machine learning methods for heavy metal estimation.

In the present study, a modeling approach based on spectral information obtained from VNIRS technique is used to investigate the effect of soil moisture on the estimation of heavy metal concentration (nickel and lead). Soil samples were obtained from areas suspected of heavy metal contamination. For this purpose, 129 soil samples were collected from fields contaminated with heavy metals in Tehran, Gilan and East Azerbaijan provinces. Nickel and lead concentrations in air-dried and sieved soil samples were measured in the laboratory using the ISO 11466 method. Then the soil samples were coded and transferred to the dark room for spectroscopy. Spectral reflectance of soil samples at 7 moisture levels (air-dried, 6, 12, 18, 24, 30 and 36%) using a FieldSpec-3 spectrometer and a contact probe in the spectral range of 350-2500 nm in a room Darkness was measured. After applying the necessary pre-processing, the soft and de-noised spectra related to the soil samples were randomly separated into two sets of data sets for modeling and validation. EPO using a set of calibration samples was developed. To estimate heavy metals, machine learning algorithms including PLSR and SVR were used.

The results indicate that with the increase in soil moisture, the spectral reflectance in the entire range of 2450-400 nm decreases non-linearly. This means that the amount of reduction in different wavelengths is not the same. The greatest reduction occurs in the range of absorption peaks located in the range of wavelengths of 1600-1400 nm and 1850-2000 nm. VNIR spectroscopy has a high ability to estimate nickel and lead heavy metals in dry soil. The presence of moisture in the soil, even at the level of 6%, leads to a significant decrease in the ability of this technology to accurately estimate heavy metals in the soil. At humidity greater than 24%, both machine learning models evaluated for both heavy metals are in the medium class.

In a general conclusion, it can be stated that the use of EPO algorithm significantly improves the ability of machine learning methods in the estimation of heavy metals in soil in the presence of moisture in soil samples. In general, SVR algorithm has the best performance compared to PLSR methods for modeling soil heavy metals. On the other hand, VNIR reflectance spectrum information provides more capability in estimating nickel than lead.In this study, the ability of the EPO method was evaluated in order to minimize the effect of soil moisture disturbing parameters on the ability of VNIR spectroscopy to improve the accuracy of soil heavy metal concentration modeling including nickel and lead based on PLSR and SVR machine learning methods. The results indicate that with the increase in soil moisture, the spectral reflectance in the entire range of 2450-400 nm decreases non-linearly. This means that the amount of reduction in different wavelengths is not the same. The greatest decrease occurs in the range of absorption peaks located in the wavelength range of 1600-1400 nm and 1850-2000 nm. VNIR spectroscopy has a high ability to estimate nickel and lead heavy metals in dry soil. The presence of moisture in the soil, even at the level of 6%, leads to a significant decrease in the ability of this technology to accurately estimate heavy metals in the soil. At humidity greater than 24%, both machine learning models are evaluated for both heavy metals in the medium class. Using the EPO algorithm significantly improves the ability of machine learning methods to estimate heavy metals in soil in the presence of moisture in soil samples. In general, SVR algorithm has the best performance compared to PLSR methods for modeling soil heavy metals. On the other hand, the information of VNIR reflectance spectrum offers more capability in estimating nickel than lead.