Skip to main content
SpringerLink
Log in

On the Use of Different Efficiency Criteria for the Validation of a Heavy Metal Balancing Tool

  • Original Article
  • Published:
Water, Air, & Soil Pollution Aims and scope Submit manuscript

Abstract

The validation of metal balancing tools is usually based on the comparison of simulated versus observed data. In our study, we applied a set of different relative and absolute criteria to evaluate the performance of the model Assessment Tool for Metals in Soils. In this process, the uncertainty of the model output and the sensitivity of model parameters were also assessed. The study includes data from 123 agricultural used top soils which are characterized by the application of different fertilizers (mineral and farmyard fertilizers, sewage sludge) resulting in diverse metal inputs into the soil. Although the most common validation criteria (coefficient of determination, error ratio between prediction and observation) indicated a good model performance in predicting the metal contents over a simulation period, the absolute measure (mean absolute difference between prediction and observation) showed that the informational value of the validation results was limited for several sites. Therefore sites with short simulation periods and/or low metal inputs are not suitable for validation, because the model uncertainty covers the metal concentration changes. Excluding such sites from the validation statistics led to evaluable and quite better validation results. Although the calculated output uncertainty was low, a further reduction can be realized by improving the database for the identified sensitive parameters (initial soil metal content and fertilizers metal concentration).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

a :

Regression coefficient

ATOMIS:

Assessment Tool for Metals in Soils

b :

Regression coefficient

β 0 , β H+ , β CECp :

Regression coefficients of general purpose Freundlich isotherm

c :

Regression coefficient

CECpot :

Potential cation exchange capacity

Cd:

Cadmium

C org :

Content of soil organic carbon

C sol :

Soluble heavy metal concentration

C sorb :

Sorbed heavy metal concentration

C tot :

Total heavy metal concentration

Cu:

Copper

d :

Regression coefficient

e :

Regression coefficient

ε :

Error ratio

GMER:

Geometric mean

GSDER:

Geometric standard deviation

H+ :

H+ ions concentration

HM:

Heavy metal

m :

Regression coefficient of general purpose Freundlich isotherm

M i :

ith modeled total heavy metal concentration

MAEpre-obs :

Mean absolute error (between model predictions and observations)

MAEpre-ini :

Mean absolute error (between model predictions and initial values)

n :

Number of datasets (sites)

Ni:

Nickel

O i :

ith observed total heavy metal concentration

R 2 :

Coefficient of determination

Zn:

Zinc

References

  • Andersson, A. (1992). Trace elements in agricultural soils: Fluxes, balances and background values. Report 4077. Solna, Sweden: Swedish Environmental Protection Agency.

    Google Scholar 

  • Bannick, C., Bieber, E., Boeken, H., Brach, M., Brackemann, H., Ehrmann, H., et al. (2001). UBA-Texte 59/01. Berlin: Umweltbundesamt.

    Google Scholar 

  • Christiaens, K., & Feyen, J. (2002). Use of sensitivity and uncertainty measures in distributed hydrological modeling with an application to the MIKE SHE model. Water Resources Research, 38(9), 1169.

    Article  Google Scholar 

  • Dach, J., & Starmans, D. (2005). Heavy metals balance in Polish and Dutch agronomy: Actual state and previsions for the future. Agriculture, Ecosystems and Environment, 107(4), 309–316.

    Article  CAS  Google Scholar 

  • De Meeûs, C., Eduljee, G., & Hutton, M. (2002). Assessment and management of risks arising from exposure to cadmium in fertilisers. I. Science of the Total Environment, 291(1–3), 167–187.

    Article  Google Scholar 

  • De Temmerman, L., Vanongeval, L., Boon, W., Hoenig, M., & Geypens, M. (2003). Heavy metal content of arable soils in Northern Belgium. Water, Air, and Soil Pollution, 148, 61–76.

    Article  Google Scholar 

  • De Vries, W., Römkens, P. F. A. M., Van Leeuwen, T., & Bronswijk, J. J. B. (2002). Heavy metals. Agriculture, hydrology and water quality (pp. 107–132). Wallingford: CAB International.

    Book  Google Scholar 

  • De Vries, W., Schütze, G., Lofts, S., Tipping, E., Meili, M., Römkens, P. F. A. M., et al. (2005). Calculation of critical loads for cadmium, lead and mercury; background document to a mapping manual on critical loads of cadmium, lead and mercury. Wageningen: Alterra. Alterrar Report, 1104.

    Google Scholar 

  • Elspass, R. (1988). Mobile und mobilisierbare Schwermetallfraktionen in Böden und im Bodenwasser. Dargestellt für d. Elemente Blei Cadmium Eisen Mangan Nickel u. Zink unter landwirtschaftlichen Nutzflächen. Marburger Geographische Schriften, 109, Marburg/Lahn.

  • Erhart, E., Hartl, W., & Putz, B. (2008). Total soil heavy-metal concentrations and mobile fractions after 10 years of biowaste-compost fertilization. Journal of Plant Nutrition and Soil Science, 171(3), 378–383.

    Article  CAS  Google Scholar 

  • Feuereissen, S. (1986). Belastbarkeit von Böden mit Schwermetallen aus Klärschlamm im Hinblick auf Wachstum und Mineralstoffgehalt verschiedener Getreidearten. Agrar- und Umweltforschung in Baden-Württemberg 12. Stuttgart: Ulmer.

    Google Scholar 

  • Franco, A., Schuhmacher, M., Roca, E., & Domingo, J. (2006). Application of cattle manure as fertilizer in pastureland: Estimating the incremental risk due to metal accumulation employing a multicompartment model. Environment International, 32, 724–732.

    Article  Google Scholar 

  • Gustafson, G., Salomon, E., & Jonsson, S. (2007). Barn balance calculations of Ca, Cu, K, Mg, Mn, N, P, S and Zn in a conventional and organic dairy farm in Sweden. Agriculture, Ecosystems and Environment, 119(1–2), 160–170.

    Article  CAS  Google Scholar 

  • Hamby, D. (1994). A review of techniques for parameter sensitivity analysis of environmental models. Environmental Monitoring and Assessment, 32(2), 135–154.

    Article  Google Scholar 

  • He, Z., Yang, X., & Stoffella, P. (2005). Trace elements in agroecosystems and impacts on the environment. Journal of Trace Elements in Medicine and Biology, 19(2–3), 125–140.

    Article  CAS  Google Scholar 

  • Heuvelink, G. (1998). Uncertainty analysis in environmental modelling under a change of spatial scale. Nutrient Cycling in Agroecosystems, 50(1–3), 255–264.

    Article  Google Scholar 

  • Horn, A., Reiher, W., Düring, R., & Gäth, S. (2006). Efficiency of pedotransfer functions describing cadmium sorption in soils. Water, Air, and Soil Pollution, 170, 229–247.

    Article  CAS  Google Scholar 

  • KTBL - Kuratorium für Technik und Bauwesen in der Landwirtschaft (2005). Assessment and reduction of heavy metal input into agro-ecosystems (AROMIS). KTBL-Schriften 432, Darmstadt.

  • Ingwersen, J. (2001). The environmental fate of cadmium in the soils of the waste water irrigation area of Braunschweig. Measurement, modelling and assessment. Techn. Univ., Diss.-Braunschweig, 2001. Braunschweig: UB.

  • Isukapalli, S., & Georgopoulos P. G. (1999). Computational methods for the efficient sensitivity and uncertainty analysis of models for environmental and biological systems. Technical Report CCL EDMAS-03, Piscataway, NJ.

  • Keller, A., & Schulin, R. (2003). Modelling regional-scale mass balances of phosphorus, cadmium and zinc fluxes on arable and dairy farms. European Journal of Agronomy, 20(1–2), 181–198.

    Article  CAS  Google Scholar 

  • Keller, A., Rossier, N., & Desaules, A. (2005). Schwermetallbilanzen von Landwirtschaftsparzellen der nationalen Bodenbeobachtung (p. 56). Zürich: Nationales Bodenbeachtungsnetz der Schweiz.

    Google Scholar 

  • Kördel, W., Herrchen, M., Müller, J., Kratz, S., Fleckenstein, J., Schnug, E., et al. (2007). Begrenzung von Schadstoffeinträgen bei Bewirtschaftungsmaßnahmen in der Landwirtschaft bei Düngung und Abfallverwertung (p. 122). Dessau: Umweltbundesamt. UBA-Texte 30/07.

    Google Scholar 

  • Kühnen, V., & Goldbach, H. E. (2004). Schwermetallbilanzen verschiedener Betriebstypen: Eintragswege, Flüsse, Minderungspotential. Schriftenreihe des Lehr- und Forschungsschwerpunktes USL, Landwirtschaftliche Fakultät der Universität Bonn, 118.

  • LABO (2000). Cadmiumanreicherung in Böden/einheitlich Bewertung von Düngemitteln. Bund-Länder-Arbeitsgemeinschaft Bodenschutz (LABO). Report of the UMK-AMK-LABO-AG for the 26th ACK of UMK on 11./12.10.2000.

  • LABO (2003). Hintergrundwerte für anorganische und organische Stoffe in Böden. Bund-Länder-Arbeitsgemeinschaft Bodenschutz (LABO). Schwerin.

  • Legates, D. R., & McCabe, G. (1999). Evaluating the use of “goodness-of-fit” measures in hydrologic and hydroclimatic model validation. Water Resources Research, 35(1), 233–241.

    Article  Google Scholar 

  • Louekari, K., Mäkelä-Kurtto, R., Pasanen, J., Virtanen, V., Sippola, J., & Malm, J. (2000). Cadmium in fertilizers. Risks to human health and the environment. Helsinki: Ministry of Agriculture and Forestry, Publications 4/2000.

    Google Scholar 

  • McKay, M., Beckmann, R., & Conover, W. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 21(2), 239–245.

    Google Scholar 

  • Michel, K., Roose, M., & Ludwig, B. (2007). Comparison of different approaches for modelling heavy metal transport in acidic soils. Geoderma, 140(1–2), 207–214.

    Article  CAS  Google Scholar 

  • Monte, L. (2009). Multi-model approach and evaluation of the uncertainty of model results. Rationale and applications to predict the behaviour of contaminants in the abiotic components of the fresh water environment. Ecological Modelling, 220, 1469–1480.

    CAS  Google Scholar 

  • Nicholson, F., Smith, S., Alloway, B., Carlton-Smith, C., & Chambers, B. (2003). An inventory of heavy metals inputs to agricultural soils in England and Wales. Science of the Total Environment, 311(1–3), 205–219.

    Article  CAS  Google Scholar 

  • Onnen, O. (2001). Umweltschonende Verwertung von Klärschlamm in der Landwirtschaft. P-Wirkung des Klärschlammes in Abhängigkeit von P-Fällung und vom Substrat: Forschungsbericht des Fachbereichs Agrarwirtschaft Soest, Nr. 13. Universität-Gesamthochschule-Paderborn.

  • Pampura, T., Groenenberg, J., Lofts, S., & Priputina, I. (2007). Validation of transfer functions predicting Cd and Pb free metal ion activity in soil solution as a function of soil characteristics and reactive metal content. Water, Air, and Soil Pollution, 184(1–4), 217–234.

    Article  CAS  Google Scholar 

  • Pappenberger, F., & Beven, K. (2006). Ignorance is bliss: Or seven reasons not to use uncertainty analysis. Water Resources Research, 42(5), 1–8.

    Article  Google Scholar 

  • Rebba, R., Mahadevan, S., & Huang, S. (2006). Validation and error estimation of computational models. Reliability Engineering and System Safety, 91(10–11), 1390–1397.

    Article  Google Scholar 

  • Reiher, W. (2008). Entwicklung eines regionalisierten Modells zur Projektion des landnutzungsabhängigen Schwermetallstatus von Oberböden. Boden und Landschaft, 52. Justus-Liebig-Universität, Gießen, 198 pp.

  • Römkens, P., Groenenberg, J., Bonten, L., & Bril, J. (2004). Derivation of partition relationships to calculate Cd, Cu, Ni, Pb and Zn solubiility and activity in soil solutions. Wageningen: Alterra. Alterrar Report, 305.

    Google Scholar 

  • Sager, M. (1997). Possible trace metal load from fertilizers. Bodenkultur, 48(4), 217–223.

    CAS  Google Scholar 

  • Saltelli, A., Tarantola, S., & Campolongo, F. (2000). Sensitivity analysis as an ingredient of modeling. Statistical Science, 15(4), 377–395.

    Article  Google Scholar 

  • Sauerbeck, D., & Styperek, P. (1988). Schadstoffe im Boden, insbesondere Schwermetalle und organische Schadstoffe aus langjähriger Anwendung von Siedlungsabfällen (p. 83). Berlin: Umweltbundesamt.

    Google Scholar 

  • Sieber, A., & Uhlenbrook, S. (2005). Sensitivity analyses of a distributed catchment model to verify the model structure. Journal of Hydrology, 310(1–4), 216–235.

    Article  Google Scholar 

  • Styperek, P. (1986). Die Cd-Aufnahme von Pflanzen aus verschiedenen Böden und Bindungsformen und ihre Prognose durch chemische Extraktionsverfahren. Berlin: Umweltbundesamt. UBA-Texte, 86/9.

    Google Scholar 

  • Thiele, S., & Leinweber, P. (2001). Parameterization of Freundlich adsorption isotherms for heavy metals in soils from an area with intensive livestock production. Journal of Plant Nutrition and Soil Science, 164(6), 623–629.

    Article  CAS  Google Scholar 

  • Tietje, O., & Hennings, V. (1996). Accuracy of the saturated hydraulic conductivity prediction by pedo-transfer functions compared to the variability within FAO textural classes. Geoderma, 69(1–2), 71–84.

    Article  Google Scholar 

  • Tiktak, A., Alkemade, J., Van Grinsven, J., & Makaske, G. (1998). Modelling cadmium accumulation at a regional scale in the Netherlands. Nutrient Cycling in Agroecosystems, 50(1–3), 209–222.

    Article  Google Scholar 

  • Trucano, T., Swiler, L., Igusa, T., Oberkampf, W., & Pilch, M. (2006). Calibration, validation, and sensitivity analysis: What‘s what. Reliability Engineering and System Safety, 91(10–11), 1331–1357.

    Article  Google Scholar 

  • Waldhardt, R., Bach, M., Borresch, R., Breuer, L., Diekötter, T., Frede, H.-G., et al. (2010). Evaluating today‘s landscape multifunctionality and providing an alternative future: A normative scenario approach. Ecology and Society, 15(3).

  • Willmott, C., & Matsuura, K. (2005). Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Research, 30(1), 79–82.

    Article  Google Scholar 

  • Zörner, D. (2010). Prognose des Schwermetallhaushaltes von Landschaften mit dem Modell ATOMIS. Boden und Landschaft, 53. Justus-Liebig-Universität, Gießen, 132 pp.

Download references

Acknowledgments

The development and validation of the Assessment Tool for Metals in Soils in an intensive agriculturally used landscape was part of the Collaborative Research Centre ‘Land Use Options for Peripheral Regions’ (SFB 299) granted by the German Research Foundation (DFG).

Author information

Authors and Affiliations

  1. Institute of Landscape Ecology and Resources Management, Justus-Liebig-University, Heinrich-Buff-Ring 26C, 35392, Giessen, Germany

    Dorit Julich & Stefan Gäth

  2. Resource Centre for Environmental Technologies, Public Research Centre Henri Tudor, 66, rue de Luxembourg, B.P. 144, 4022, Esch/Alzette, Luxembourg

    Stefan Julich

Authors
  1. Dorit Julich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Gäth
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefan Julich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dorit Julich.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Julich, D., Gäth, S. & Julich, S. On the Use of Different Efficiency Criteria for the Validation of a Heavy Metal Balancing Tool. Water Air Soil Pollut 223, 3589–3599 (2012). https://doi.org/10.1007/s11270-012-1083-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11270-012-1083-y

Keywords

Search

Navigation