Aquastat data: Difference between revisions

From Pardee Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
(26 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<table cellspacing="0" border="1" bgcolor="#ffffff" align="left">
Most of the data used in the water preprocessor comes from the AQUASTAT database. [http://www.fao.org/nr/water/aquastat/main/index.stm http://www.fao.org/nr/water/aquastat/main/index.stm]<span style="font-size:medium;">&nbsp;</span>


<tr>
The AQUASTAT database can be queried on-line and the query results can be downloaded in CSV (table or flat) format. The current database regroups data per 5-year period and shows for each variable the value for the most recent year during that period, if available. For example, if for the period 2003-2007 data are available for the year 2004 and for the year 2006, then the value for the year 2006 is shown. Also, for many variables no time series can be made available yet due to lack of sufficient data. Data collection began in 1958.<br/><br/>Data for the variable [4313] "Area equipped for irrigation: total" is collected by and available in both AQUASTAT and FAOSTAT and harmonization between AQUASTAT and FAOSTAT takes place on a regular basis to ensure consistency between the two. The difference between the two is that AQUASTAT only enters data for the year given in the reference, while FAOSTAT fills all years from 1961 onwards by inter- and extrapolation from the common data-points. Thus, time series on this variable can be found under&nbsp;[http://faostat3.fao.org/download/E/EL/E FAOSTAT land], choosing the last variable in "items": Total area equipped for irrigation. For other non-water related time-series on food and agriculture see the&nbsp;[http://faostat3.fao.org/home/E FAOSTAT home page].
<th bgcolor="#c0c0c0"> <font style="font-size: 11pt;" face="calibri" color="#000000">Table</font>
 
</th>
= Metadata =
<th bgcolor="#c0c0c0"> <font style="font-size: 11pt;" face="calibri" color="#000000">Group</font>
 
</th>
== Update Frequency ==
<th bgcolor="#c0c0c0"> <font style="font-size: 11pt;" face="calibri" color="#000000">Definition</font>
 
</th>
==== Varies by Main Database category and sub-category: ====
<th bgcolor="#c0c0c0"> <font style="font-size: 11pt;" face="calibri" color="#000000">Years</font>
 
</th>
#Geography and population category: Every year (through FAOSTAT for land use and population and undernourishment, World Bank for GDP, UNDP for HDI and GII).
<th bgcolor="#c0c0c0" style="width: 156px;"> <font style="font-size: 11pt;" face="calibri" color="#000000">Source</font>
#Water resources category: These are long-term average annual values and therefore remain the same over the years. A comprehensive review had been undertaken in 2014.
</th>
#Water use category and Irrigation and drainage development category: For Africa, Southern and Central America & Caribbean, and Asia: between 1 and 10 years, depending on data availability which is checked at least every 5 years for each country, and with a thorough update (full dataset) every 10 years together with the country profile update. For Europe, Northern America, Australia and New Zealand: depends on data availability from Eurostat and OECD.
<th bgcolor="#c0c0c0" style="width: 252px;"> <font style="font-size: 11pt;" face="calibri" color="#000000">Last IFs Update</font>
#Updates of data for some specific sub-categories are being done in collaboration with others, as and when data become available, such as: wastewater sub-category in collaboration with IWMI; conservation agriculture sub-category in collaboration with Conservation Agriculture expert groups; access to improved drinking water source sub-category data are provided by the WHO/UNICEF Joint Monitoring Programme for Water Supply and Sanitation.
</th>
#Some of the variables are updated during major review exercises (see the other datasets below).
<th bgcolor="#c0c0c0"> <font style="font-size: 11pt;" face="calibri" color="#000000">UsedInPreprocessor</font>
 
</th></tr>
== AquaStat Questionnaires&nbsp;and Guidelines ==
<tr valign="top">
 
<td> <font style="font-size: 11pt;" face="calibri" color="#000000">SeriesDesalinatedWater</font>
AQUASTAT uses questionnaires and prepares guidelines with detailed definitions and instructions, implements projects to strengthen national capacities, and holds workshops to clarify some of the more complex data concepts. Survey information is as follows:
</td>
 
<td> <font style="font-size: 11pt;" face="calibri" color="#000000">Environment, Infrastructure, Water</font>
To reach the above objective, it has been decided to proceed as follows, with two surveys:
</td>
 
<td> <font style="font-size: 11pt;" face="calibri" color="#000000">Desalinated Water Produced</font>
*Every 5-10 years: a detailed global survey to prepare/update country profiles and regional syntheses. This document refers to this survey, which is prepared on the basis of a detailed questionnaire filled in at country level by national and regional experts, under the supervision of FAO/NRL.
</td>
*Every year: a small global survey to ensure regular updating of the existing country profiles, of key changes at country level, especially on the area equipped for irrigation and on the institutional and policy aspects. It is done by concerned institutions, which are willing to participate, and is managed in the regional and central offices of FAO.
<td> <font style="font-size: 11pt;" face="calibri" color="#000000">1962, 1965, 1970, 1975-1976, 1979-1980, 1982, 1984-1997, 1999-2010</font>
 
</td>
The 5-10 year survey aims at preparing national country profiles in a homogeneous way. It ensures participation of national experts, in charge of the information collection and report drafting at the country level, and validation by FAO
<td style="width: 156px;"> <font style="font-size: 11pt;" face="calibri" color="#000000">AQUASTAT</font>
</td>
<td align="right" style="width: 252px;"> <font style="font-size: 11pt;" face="calibri" color="#000000">1/20/2011</font>
</td>
<td> <font style="font-size: 11pt;" face="calibri" color="#000000">FALSE</font>
</td></tr>
<tr valign="top">
<td> <font style="font-size: 11pt;" face="calibri" color="#000000">SeriesLandCultivatedArea</font>
</td>
<td> <font style="font-size: 11pt;" face="calibri" color="#000000">Economic, Infrastructure</font>
</td>
<td> <font style="font-size: 11pt;"

Revision as of 21:57, 7 September 2018

Most of the data used in the water preprocessor comes from the AQUASTAT database. http://www.fao.org/nr/water/aquastat/main/index.stm 

The AQUASTAT database can be queried on-line and the query results can be downloaded in CSV (table or flat) format. The current database regroups data per 5-year period and shows for each variable the value for the most recent year during that period, if available. For example, if for the period 2003-2007 data are available for the year 2004 and for the year 2006, then the value for the year 2006 is shown. Also, for many variables no time series can be made available yet due to lack of sufficient data. Data collection began in 1958.

Data for the variable [4313] "Area equipped for irrigation: total" is collected by and available in both AQUASTAT and FAOSTAT and harmonization between AQUASTAT and FAOSTAT takes place on a regular basis to ensure consistency between the two. The difference between the two is that AQUASTAT only enters data for the year given in the reference, while FAOSTAT fills all years from 1961 onwards by inter- and extrapolation from the common data-points. Thus, time series on this variable can be found under FAOSTAT land, choosing the last variable in "items": Total area equipped for irrigation. For other non-water related time-series on food and agriculture see the FAOSTAT home page.

Metadata

Update Frequency

Varies by Main Database category and sub-category:

  1. Geography and population category: Every year (through FAOSTAT for land use and population and undernourishment, World Bank for GDP, UNDP for HDI and GII).
  2. Water resources category: These are long-term average annual values and therefore remain the same over the years. A comprehensive review had been undertaken in 2014.
  3. Water use category and Irrigation and drainage development category: For Africa, Southern and Central America & Caribbean, and Asia: between 1 and 10 years, depending on data availability which is checked at least every 5 years for each country, and with a thorough update (full dataset) every 10 years together with the country profile update. For Europe, Northern America, Australia and New Zealand: depends on data availability from Eurostat and OECD.
  4. Updates of data for some specific sub-categories are being done in collaboration with others, as and when data become available, such as: wastewater sub-category in collaboration with IWMI; conservation agriculture sub-category in collaboration with Conservation Agriculture expert groups; access to improved drinking water source sub-category data are provided by the WHO/UNICEF Joint Monitoring Programme for Water Supply and Sanitation.
  5. Some of the variables are updated during major review exercises (see the other datasets below).

AquaStat Questionnaires and Guidelines

AQUASTAT uses questionnaires and prepares guidelines with detailed definitions and instructions, implements projects to strengthen national capacities, and holds workshops to clarify some of the more complex data concepts. Survey information is as follows:

To reach the above objective, it has been decided to proceed as follows, with two surveys:

  • Every 5-10 years: a detailed global survey to prepare/update country profiles and regional syntheses. This document refers to this survey, which is prepared on the basis of a detailed questionnaire filled in at country level by national and regional experts, under the supervision of FAO/NRL.
  • Every year: a small global survey to ensure regular updating of the existing country profiles, of key changes at country level, especially on the area equipped for irrigation and on the institutional and policy aspects. It is done by concerned institutions, which are willing to participate, and is managed in the regional and central offices of FAO.

The 5-10 year survey aims at preparing national country profiles in a homogeneous way. It ensures participation of national experts, in charge of the information collection and report drafting at the country level, and validation by FAO experts. The experts are provided with:

  • A detailed questionnaire to be completed for each country at national level (and sub-national level for some variables) on the basis of existing available information
  • Detailed guidelines for preparing a country profile and gathering qualitative information
  • Explanatory notes and definitions to ensure consistency in the use of concepts and words and facilitate computation

Costs

Costs should be expressed in US$ (United States Dollars) in the questionnaire and country profiles. The conversion used to pass from the local currency to US$ should be the one that was valid in the year to which the value refers and should be indicated in the comments. 

Definitions

Definitions for every series can be found in this document: http://www.fao.org/nr/water/aquastat/sets/aq-5yr-guide_eng.pdf&nbsp;

DataDict

Variable: Names were not changed from previous years this data was pulled. New variables were assigned names that are consistent with variables of similar topics.

Table: These were not changed from previous years this data was pulled. New variables were assigned based on the variable title and consistency.

Groups: These were not changed from previous years this data was pulled. New variables were assigned groups based on consistency with variables of similar topics. New Irrigation series were given groups: Agriculture, Infrastructure, Water. 

Subgroups: These were not changed form previoys years this data was pulled. New Irrigation data was given the Subgroup: Irrigation.

Definitions and Units: This categories were not changed from previous years. New variables were given definitions based on the consise definitions provided in the AquaStat database. Units were also provided in the data base.

Years: Years for every series were changed to available data provided through the database. Years vary between different series. The aquastat database calculates years in 5 year incriments. This five year incriments go back to 1958. The most recent incriment is 2013-2017. In the datadict, these incrimental periods are not used. Instead, only actual data recorded for the earliest specific year to the last specific year is used.

Source: Source for all AquaStat series were given AQU BATCH PULL.

Original Source: The original source for all series is the AquaStat database.

Notes and Last IFs Update were updated accordingly.

Aggregation: Aggregations were not changed from pervious updates. New variables were assigned aggregations based on the type of variable and the Units used.

Disaggregation was consistently GDP.

Name in source is based on the name each variable was given in the AquaStat database.

Code in Source: All codes were changed to match the exact variable names present when downloading the data from the database. The downloaded names were different from the name in source.

Decimal places: This category was left blank in order to avoid rounding exact data.

Country Concordance: FAO countries were used.

Series Updated

DataDict
Variable
DesalinatedWater
IrrigatedCropIntensity
IrWaterReq
IrWaterWith
LandCultivatedArea
LandEquipIrActual
LandEquipIrFullControl
LandEquipIrFullControlActual
LandIr%Grain
LandIrAreaSalinized
LandIrEquipDrained
LandIrEquipGround
LandIrHarvestedCropArea
LandIrWaterLogged
TotalDamCapacity
WasterwaterTreated
WasteWaterDirectNotTreated
WastewaterIrDirectTreated
WasteWaterLandEquipDirectNotTreated
WasteWaterLandEquipDirectTreated
WastewaterProduced
WastewaterTreatedReused
WaterDependencyRatio
WaterDesalinated
WaterGroundEntering
WaterGroundLeaving
WaterGroundProdInternal
WaterGroundTotal
WaterGroundWithD
WaterResExploitGround
WaterResExploitSurface
WaterResOverlap
WaterResTotalExploit
WaterResTotalRenew
WaterResTotalRenewGround
WaterResTotalRenewSurface
WaterSurfaceWithD
WaterTotalRenewPC
WaterTotalWithd
WaterTotalWithdPC
WaterTotalWithdSector
WaterTotalWithdSources
WaterWith%Agric
WaterWith%Fresh
WaterWith%Household
WaterWith%Ind
WaterWithAgr%FreshAquastat
WaterWithdAgriculture
WaterWithdIndustrial
WaterWithdMunicipal

Series that will soon be updated

DataDict
Variable
LandIrActual%Equip
LandIrAreaEquip
LandIrEquip%Cultivated
LandIrEquip%Potential
LandIrEquip%WaterManaged
LandIrEquipActual
LandIrEquipMixed
LandIrEquipSurface
LandIrPotential
WaterTotalAgManagedArea


New Series added

  • Direct use of not treated municipal wastewater for irrigation purposes 
  • Irrigated crop intensity 
  • Area equipped for irrigation by surface water 
  • Area equipped for irrigation by groundwater 
  • Area equipped for irrigation by mixed surface water and groundwater 
  • Area equipped for irrigation by direct use of treated municipal wastewater 
  • Area equipped for irrigation by direct use of non-treated municipal wastewater 
  • Area equipped for irrigation by direct use of agricultural drainage water 
  • Area equipped for irrigation by desalinated water
  • Irrigation water withdrawal
  • Irrigation water requirement
  • Direct use of treated municipal wastewater for irrigation purposes
  • area equipped for irrigation: actually irrigated
  • area equipped for full control irrigation: total
  • Area equipped for full control irrigation: actually irrigated

Preprocessor Series

Unknown since the model is currently in development

Country Coverage

Every country, with the exception of Kosovo, is included in the database. However, available data for each country varies by series.

FAO Country Translation
IFs Countries FAO Countries
United Arab Emirates United Arab Emirates
Afghanistan Afghanistan
Albania Albania
Armenia Armenia
Angola Angola
Argentina Argentina
Austria Austria
Australia Australia
Azerbaijan Azerbaijan
Bosnia and Herzegovina Bosnia and Herzegovina
Barbados Barbados
Bangladesh Bangladesh
Belgium Belgium
Burkina Faso Burkina Faso
Bulgaria Bulgaria
Bahrain Bahrain
Burundi Burundi
Benin Benin
Brunei Brunei Darussalam
Bolivia Bolivia (Plurinational State of)
Brazil Brazil
Bahamas, The Bahamas
Bhutan Bhutan
Botswana Botswana
Belarus Belarus
Belize Belize
Canada Canada
Congo, Democratic Republic of Democratic Republic of the Congo
Central African Republic Central African Republic
Congo, Republic of Congo
Switzerland Switzerland
Cote d'Ivoire Côte d'Ivoire
Chile Chile
Cameroon Cameroon
China China
Colombia Colombia
Costa Rica Costa Rica
Cuba Cuba
Cape Verde Cabo Verde
Cyprus Cyprus
Czech Republic Czech Republic
Germany Germany
Djibouti Djibouti
Denmark Denmark
Dominican Republic Dominican Republic
Algeria Algeria
Ecuador Ecuador
Estonia Estonia
Egypt, Arab Republic of Egypt
Eritrea Eritrea
Spain Spain
Ethiopia Ethiopia
Finland Finland
Fiji Fiji
Micronesia, Fed. Sts. Micronesia (Federated States of)
France France
Gabon Gabon
United Kingdom United Kingdom
Grenada Grenada
Georgia Georgia
Ghana Ghana
Gambia, The Gambia
Guinea Guinea
Equatorial Guinea Equatorial Guinea
Greece Greece
Guatemala Guatemala
Guinea-Bissau Guinea-Bissau
Guyana Guyana
Hong Kong China, Hong Kong SAR
Honduras Honduras
Croatia Croatia
Haiti Haiti
Hungary Hungary
Indonesia Indonesia
Ireland Ireland
Israel Israel
India India
Iraq Iraq
Iran, Islamic Republic of Iran (Islamic Republic of)
Iceland Iceland
Italy Italy
Jamaica Jamaica
Jordan Jordan
Japan Japan
Kenya Kenya
Kyrgyz Republic Kyrgyzstan
Cambodia Cambodia
Comoros Comoros
Kosovo
Korea, Democratic People's Republic of Democratic People's Republic of Korea
Korea, Republic of Republic of Korea
Kuwait Kuwait
Kazakhstan Kazakhstan
Laos, People's Democratic Republic Lao People's Democratic Republic
Lebanon Lebanon
St. Lucia Saint Lucia
Sri Lanka Sri Lanka
Liberia Liberia
Lesotho Lesotho
Lithuania Lithuania
Luxembourg Luxembourg
Latvia Latvia
Libya Libya
Morocco Morocco
Moldova Republic of Moldova
Montenegro Montenegro
Madagascar Madagascar
Macedonia, Former Yugoslav Republic of The former Yugoslav Republic of Macedonia
Mali Mali
Myanmar Myanmar
Mongolia Mongolia
Mauritania Mauritania
Malta Malta
Mauritius Mauritius
Maldives Maldives
Malawi Malawi
Mexico Mexico
Malaysia Malaysia
Mozambique Mozambique
Namibia Namibia
Niger Niger
Nigeria Nigeria
Nicaragua Nicaragua
Netherlands Netherlands
Norway Norway
Nepal Nepal
New Zealand New Zealand
Oman Oman
Panama Panama
Peru Peru
Papua New Guinea Papua New Guinea
Philippines Philippines
Pakistan Pakistan
Poland Poland
Puerto Rico Puerto Rico
Palestine Occupied Palestinian Territory
Portugal Portugal
Paraguay Paraguay
Qatar Qatar
Romania Romania
Serbia Serbia
Russian Federation Russian Federation
Rwanda Rwanda
Saudi Arabia Saudi Arabia
Solomon Islands Solomon Islands
Seychelles Seychelles
Sudan Sudan
Sweden Sweden
Singapore Singapore
Slovenia Slovenia
Slovak Republic Slovakia
Sierra Leone Sierra Leone
Senegal Senegal
Somalia Somalia
Suriname Suriname
Sudan South South Sudan
Sao Tome and Principe Sao Tome and Principe
El Salvador El Salvador
Syrian Arab Republic Syrian Arab Republic
Swaziland Swaziland
Chad Chad
Togo Togo
Thailand Thailand
Tajikistan Tajikistan
Timor-Leste Timor-Leste
Turkmenistan Turkmenistan
Tunisia Tunisia
Tonga Tonga
Turkey Turkey
Trinidad and Tobago Trinidad and Tobago
Taiwan, China China, Taiwan Province of
Tanzania United Republic of Tanzania
Ukraine Ukraine
Uganda Uganda
United States United States of America
Uruguay Uruguay
Uzbekistan Uzbekistan
St. Vincent and the Grenadines Saint Vincent and the Grenadines
Venezuela Venezuela (Bolivarian Republic of)
Vietnam Viet Nam
Vanuatu Vanuatu
Samoa Samoa
Yemen, Republic of Yemen
South Africa South Africa
Zambia Zambia
Zimbabwe Zimbabwe

Instructions

  1. Download the data from the Aquastat Database. http://www.fao.org/nr/water/aquastat/data/query/index.html?lang=en. Select all countries and all years. There are too many variables to download all variables at once. Divide the variables by topics (ie geography and population, water resources, etc) download this data via the "Flat" option.
  2. Change the codes in the Datadict for every variable downloaded in this Excel file. The codes need to match to match the variables listed in the downloaded data exactly. 
  3. Check to ensure no changes are needed in the FAO Country translation. In the most recent update China, Sudan and South Sudan all needed to be updated.
  4. Upload data in IFs using the batch pool option. There will be multiple excel sheets, as the AquaStat data cannot download all variables at once. After the first excel sheet is updated, unselect data that was imported; otherwise this data will be uploaded again with a new excel sheet, and you will lose the already imported data.
  5. Vet. The most recent update reveiled errors in data in some series. You may need extensive vetting.
  6. Some series do not contain enough data in the database to be updated. These include:

LandIrEquipDesalinated

LandIrEquipDirectAg

LandIrHarvest%Equip

LandIrHarvest%FullControl

LandIrPotentialReached