WoSIS snapshot - July 2016

The World Soil Information Service (WoSIS) provides quality-assessed and standardised soil profile data to support digital soil mapping and environmental applications at broad scale levels. Since the release of the first ‘WoSIS snapshot’, in July 2016, many new soil data were shared with us, registered in the ISRIC data repository, and subsequently standardised in accordance with the licences specified by the data providers. Soil profile data managed in WoSIS were contributed by a wide range of data providers, therefore special attention was paid to measures for soil data quality and the standardisation of soil property definitions, soil property values (and units of measurement), and soil analytical method descriptions. We presently consider the following soil chemical properties (organic carbon, total carbon, total carbonate equivalent, total Nitrogen, Phosphorus (extractable-P, total-P, and P-retention), soil pH, cation exchange capacity, and electrical conductivity) and physical properties (soil texture (sand, silt, and clay), bulk density, coarse fragments, and water retention), grouped according to analytical procedures (aggregates) that are operationally comparable. Further, for each profile, we provide the original soil classification (FAO, WRB, USDA, and version) and horizon designations insofar as these have been specified in the source databases. Measures for geographical accuracy (i.e. location) of the point data as well as a first approximation for the uncertainty associated with the operationally defined analytical methods are presented, for possible consideration in digital soil mapping and subsequent earth system modelling. The present snapshot, referred to as ‘WoSIS snapshot - September 2019’, comprises 196,498 geo-referenced profiles originating from 173 countries. They represent over 832 thousand soil layers (or horizons), and over 6 million records. The actual number of observations for each property varies (greatly) between profiles and with depth, this generally depending on the objectives of the initial soil sampling programmes. The downloadable ZIP file has the data in TSV (tab separated values) and GeoPackage format. It contains the following files: - ReadmeFirst_WoSIS_2019dec04.pdf (546.7 KB) - wosis_201909.gpkg (2.2 GB, same data as in the tsv) - wosis_201909_attributes.tsv (8.7 KB) - wosis_201909_layers_chemical.tsv (893.5 MB) - wosis_201909_layers_physical.tsv (890.7 MB) - wosis_201909_profiles.tsv (18.8 MB) To read the data in R, please, uncompress the ZIP file and specify the uncompressed folder. Then use read_tsv to read the TSV files, specifying the data types for each column (c = character, i = integer, n = number, d = double, l = logical, f = factor, D = date, T = date time, t = time). setwd("/YourFolder/WoSIS_2019_September/") attributes = readr::read_tsv('wosis_201909_attributes.tsv', col_types='cccciicd') profiles = readr::read_tsv('wosis_201909_profiles.tsv', col_types='icccdddiicccciccccicccc') chemical = readr::read_tsv('wosis_201909_layers_chemical.tsv', col_types='iiddclcdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccc') physical = readr::read_tsv('wosis_201909_layers_physical.tsv', col_types='iiddclcdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccccdccccc') For more detailed instructions on how to read the data with R, please visit https://www.isric.org/accessing-wosis-using-r. Citation: Batjes N.H, Ribeiro E, and van Oostrum A.J.M, 2019. Standardised soil profile data for the world (WoSIS snapshot - September 2019), https://doi.org/10.17027/isric-wdcsoils.20190901. The dataset accompanies the following data paper: Batjes N.H., Ribeiro E., and van Oostrum A.J.M., 2019. Standardised soil profile data to support global mapping and modelling (WoSIS snapshot - 2019). Earth System Science Data, https://doi.org/10.5194/essd-12-299-2020.

ABSTRACT: The World Soil Information Service (WoSIS) provides quality-assessed and standardized soil profile data to support digital soil mapping and environmental applications at broad scale levels. Since the release of the ‘WoSIS snapshot 2019’ many new soil data were shared with us, registered in the ISRIC data repository, and subsequently standardized in accordance with the licenses specified by the data providers. The source data were contributed by a wide range of data providers, therefore special attention was paid to the standardization of soil property definitions, soil analytical procedures and soil property values (and units of measurement). We presently consider the following soil chemical properties (organic carbon, total carbon, total carbonate equivalent, total Nitrogen, Phosphorus (extractable-P, total-P, and P-retention), soil pH, cation exchange capacity, and electrical conductivity) and physical properties (soil texture (sand, silt, and clay), bulk density, coarse fragments, and water retention), grouped according to analytical procedures (aggregates) that are operationally comparable. For each profile we provide the original soil classification (FAO, WRB, USDA, and version) and horizon designations as far as these have been specified in the source databases. Three measures for 'fitness-for-intended-use' are provided: positional uncertainty (for site locations), time of sampling/description, and a first approximation for the uncertainty associated with the operationally defined analytical methods. These measures should be considered during digital soil mapping and subsequent earth system modelling that use the present set of soil data. DATA SET DESCRIPTION: The 'WoSIS 2023 snapshot' comprises data for 228k profiles from 217k geo-referenced sites that originate from 174 countries. The profiles represent over 900k soil layers (or horizons) and over 6 million records. The actual number of measurements for each property varies (greatly) between profiles and with depth, this generally depending on the objectives of the initial soil sampling programmes. The data are provided in TSV (tab separated values) format and as GeoPackage. The zip-file (446 Mb) contains the following files: - Readme_WoSIS_202312_v2.pdf: Provides a short description of the dataset, file structure, column names, units and category values (this file is also available directly under 'online resources'). The pdf includes links to tutorials for downloading the TSV files into R respectively Excel. See also 'HOW TO READ TSV FILES INTO R AND PYTHON' in the next section. - wosis_202312_observations.tsv: This file lists the four to six letter codes for each observation, whether the observation is for a site/profile or layer (horizon), the unit of measurement and the number of profiles respectively layers represented in the snapshot. It also provides an estimate for the inferred accuracy for the laboratory measurements. - wosis_202312_sites.tsv: This file characterizes the site location where profiles were sampled. - wosis_2023112_profiles: Presents the unique profile ID (i.e. primary key), site_id, source of the data, country ISO code and name, positional uncertainty, latitude and longitude (WGS 1984), maximum depth of soil described and sampled, as well as information on the soil classification system and edition. Depending on the soil classification system used, the number of fields will vary . - wosis_202312_layers: This file characterises the layers (or horizons) per profile, and lists their upper and lower depths (cm). - wosis_202312_xxxx.tsv : This type of file presents results for each observation (e.g. “xxxx” = “BDFIOD” ), as defined under “code” in file wosis_202312_observation.tsv. (e.g. wosis_202311_bdfiod.tsv). - wosis_202312.gpkg: Contains the above datafiles in GeoPackage format (which stores the files within an SQLite database). HOW TO READ TSV FILES INTO R AND PYTHON: A) To read the data in R, please uncompress the ZIP file and specify the uncompressed folder. setwd("/YourFolder/WoSIS_2023_December/") ## For example: setwd('D:/WoSIS_2023_December/') Then use read_tsv to read the TSV files, specifying the data types for each column (c = character, i = integer, n = number, d = double, l = logical, f = factor, D = date, T = date time, t = time). observations = readr::read_tsv('wosis_202312_observations.tsv', col_types='cccciid') observations ## show columns and first 10 rows sites = readr::read_tsv('wosis_202312_sites.tsv', col_types='iddcccc') sites profiles = readr::read_tsv('wosis_202312_profiles.tsv', col_types='icciccddcccccciccccicccci') profiles layers = readr::read_tsv('wosis_202312_layers.tsv', col_types='iiciciiilcc') layers ## Do this for each observation 'XXXX', e.g. file 'Wosis_202312_orgc.tsv': orgc = readr::read_tsv('wosis_202312_orgc.tsv', col_types='iicciilccdccddccccc') orgc Note: One may also use the following R code (example is for file 'observations.tsv'): observations <- read.table("wosis_202312_observations.tsv", sep = "\t", header = TRUE, quote = "", comment.char = "", stringsAsFactors = FALSE ) B) To read the files into python first decompress the files to your selected folder. Then in python: # import the required library import pandas as pd # Read the observations data observations = pd.read_csv("wosis_202312_observations.tsv", sep="\t") # print the data frame header and some rows observations.head() # Read the sites data sites = pd.read_csv("wosis_202312_sites.tsv", sep="\t") # Read the profiles data profiles = pd.read_csv("wosis_202312_profiles.tsv", sep="\t") # Read the layers data layers = pd.read_csv("wosis_202312_layers.tsv", sep="\t") # Read the soil property data, e.g. 'cfvo' (do this for each observation) cfvo = pd.read_csv("wosis_202312_cfvo.tsv", sep="\t") CITATION: Calisto, L., de Sousa, L.M., Batjes, N.H., 2023. Standardised soil profile data for the world (WoSIS snapshot – December 2023), https://doi.org/10.17027/isric-wdcsoils-20231130 Supplement to: Batjes N.H., Calisto, L. and de Sousa L.M., 2023. Providing quality-assessed and standardised soil data to support global mapping and modelling (WoSIS snapshot 2023). Earth System Science Data, https://doi.org/10.5194/essd-16-4735-2024.

A homogenized, global set of 1,125 soil profiles is presented. These profiles have been extracted from the database developed at ISRIC for a project on "World Inventory of Soil Emission Potentials" (WISE), as a contribution to the activities of the Global Soils Data Task Group of IGBP-DIS. The subset consists of a selection of 665 profiles originating from digital data files released by the Natural Resources Conservation Service (NRCS, Lincoln), 250 profiles obtained from the Food and Agriculture Organization (FAO, Rome), and 210 profiles from the reference collection of the International Soil Reference and Information Centre (ISRIC, Wageningen). All profiles are georeferenced and classified in the FAO-Unesco Legend whereby they can be linked to the edited and digital version of the FAO-Unesco Soil Map of the World. This data set is being released in the public domain for use by global modellers and other interested scientists. It is envisaged that the data set will be expanded by ISRIC when new, uniform soil profile data become available. Note: a) A more recent version (some 10,000 profiles) of WISE profiles is available at: http://data.isric.org/geonetwork/srv/eng/catalog.search#/metadata/a351682c-330a-4995-a5a1-57ad160e621c (2009) b) For a larger compilation see the WoSIS database: http://isric.org/explore/wosis (2017)

Version 3.1 of the ISRIC-WISE database (WISE3) was compiled from a wide range of soil profile data collected by many soil professionals worldwide. All profiles have been harmonized with respect to the original Legend (1974) and Revised Legend (1988) of FAO-Unesco. Thereby, the primary soil data ─ and any secondary data derived from them ─ can be linked using GIS to the spatial units of the digitized Soil Map of the World as well as more recent digital Soil and Terrain (SOTER) databases through the soil legend code. WISE3 holds selected attribute data for some 10,250 soil profiles, with some 47,800 horizons, from 149 countries. Individual profiles have been sampled, described, and analyzed according to methods and standards in use in the originating countries. There is no uniform set of properties for which all profiles have analytical data, generally because only selected measurements were planned during the original surveys. Methods used for laboratory determinations of specific soil properties vary between laboratories and over time; sometimes, results for the same property cannot be compared directly. WISE3 will inevitably include gaps, being a compilation of legacy soil data derived from traditional soil survey, which can be of a taxonomic, geographic, and soil analytical nature. As a result, the amount of data available for modelling is sometimes much less than expected. Adroit use of the data, however, will permit a wide range of agricultural and environmental applications at a global and continental scale (1:500 000 and broader). Preferred citation: Batjes NH 2009. Harmonized soil profile data for applications at global and continental scales: updates to the WISE database. Soil Use and Management 5:124–127, http://dx.doi.org/10.1111/j.1475-2743.2009.00202.x

This harmonized, gridded global data set of soil parameter estimates includes files listing: (1) soil parameter estimates for the component soil units of each terrestrial grid cell, in un-binned format, and (2) soil parameter estimates aggregated or binned into a number of predefined classes. The spatial data, with a resolution of ½ by ½ degree, was derived from the ISRIC-WISE soil database. The land surface between longitudes -180o W and +180o E and latitudes +90o N and -90o S has been characterized using 45948 unique map units; each of these can comprise from one to ten soil units, characterized according to the original legend of the 1:5 million scale Soil Map of the World (FAO-Unesco 1974). Soil parameter estimates for each of these units were derived from analyses of some 9600 profiles held in a working copy of WISE (ver. 2.0). Twenty-two soil variables, identified as being useful for agro-ecological zoning, land evaluation, crop growth simulation, modelling of soil gaseous emissions and analyses of global environmental change, were considered. Parameter estimates for the topsoil (0-30 cm) and the subsoil (30-100 cm) are presented for the following variables: content of organic carbon, total nitrogen, the C/N ratio, pH(H2O), CECsoil, CECclay, base saturation, total exchangeable bases, aluminum saturation, exchangeable sodium percentage (ESP), electrical conductivity of saturated paste (ECe), calcium carbonate content, gypsum content, content of sand, silt and clay, content of fragments less than 2 mm, bulk density, total porosity. For soil drainage class, effective soil depth, and available water capacity (-10 to -1500 kPa), however, parameter estimates are presented on a profile basis. The parameter estimates - median values - presented here should be seen as best estimates; possible types and sources of uncertainty are discussed in the report. The data are considered appropriate for exploratory studies at global scale (greater than 1:5 000 000). Note: A more recent assessment, at a resolution of 30arcsec (WISE30sec), is available at: http://data.isric.org/geonetwork/srv/eng/catalog.search#/metadata/dc7b283a-8f19-45e1-aaed-e9bd515119bc

This harmonized dataset of derived soil properties for the world (WISE30sec) is comprised of a soil-geographical and a soil attribute component. The GIS dataset was created using the soil map unit delineations of the broad scale Harmonised World Soil Database, version 1.21, with minor corrections, overlaid by a climate zones map (Köppen-Geiger) as co-variate, and soil property estimates derived from analyses of the ISRIC-WISE soil profile database for the respective mapped ‘soil/climate’ combinations. The dataset considers 20 soil properties that are commonly required for global agro-ecological zoning, land evaluation, crop growth simulation, modelling of soil gaseous emissions, and analyses of global environmental change. It presents ‘best’ estimates for: organic carbon content, total nitrogen, C/N ratio, pH(H2O), CECsoil, CECclay, effective CEC, total exchangeable bases (TEB), base saturation, aluminium saturation, calcium carbonate content, gypsum content, exchangeable sodium percentage (ESP), electrical conductivity, particle size distribution (content of sand, silt and clay), proportion of coarse fragments (less than 2 mm), bulk density, and available water capacity (-33 to -1500 kPa); also the dominant soil drainage class. Soil property estimates are presented for fixed depth intervals of 20 cm up to a depth of 100 cm, respectively of 50 cm between 100 cm to 200 cm (or less when appropriate) for so-called ‘synthetic’ profiles’ (as defined by their ‘soil/climate’ class). The respective soil property estimates were derived from statistical analyses of data for some 21,000 soil profiles managed in a working copy of the ISRIC-WISE database; this was done using an elaborate scheme of taxonomy-based transfer rules complemented with expert-rules that consider the ‘in-pedon’ consistency of the predictions. The type of rules used was flagged to provide an indication of the possible confidence (i.e. lineage) in the derived data. Best estimates for each attribute are given as means and standard deviations (STD), as calculated for the sample populations that remained upon application of a robust data outlier detection scheme. Results of the analyses can be linked to the spatial data through the unique map unit (grid cell) identifier, which is a combination of the soil unit and climate class code. Most map units are comprised of up to ten different components; each of these with their own range of derived soil properties and associated statistical uncertainties. Estimates of global soil organic carbon (SOC) stocks to 200 cm are presented in the technical documentation as an example of possible application. For citation use: Batjes NH 2016. Harmonised soil property values for broad-scale modelling (WISE30sec) with estimates of global soil carbon stocks. Geoderma 2016(269), 61-68 ( http://dx.doi.org/10.1016/j.geoderma.2016.01.034 ) Note: The chemical and physical soil property estimates generated for WISE30sec have been used to populate version 2.0 of the Harmonised World Soil Database (see https://data.isric.org/geonetwork/srv/eng/catalog.search#/metadata/54aebf11-ec73-4ff8-bf6c-ecff4b0725ea).