Pardee Wiki - User contributions [en]

Version notes 7.46 (November 2019)

2019-11-15T17:40:28Z

JakeDubbert: Created page with "= Recent Model Updates = = Recent Data Updates (since version 7.42) = {| style="color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-..."

= Recent Model Updates =

= Recent Data Updates (since version 7.42) =

{| style="color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; width: 300px; word-spacing: 0px;" cellpadding="1" cellspacing="1" border="1"
|-
| style="text-align: center; width: 177px;" | Source 
| style="text-align: center; width: 110px;" | Number of Series 
|-
| style="text-align: center; width: 177px;" | FAO
| style="text-align: center; width: 110px;" | 11 
|-
| style="text-align: center; width: 177px;" | Aquastat
| style="text-align: center; width: 110px;" | 51
|-
| style="text-align: center; width: 177px;" | Fraser
| style="text-align: center; width: 110px;" | 6 
|-
| style="text-align: center; width: 177px;" | UNPD WPP 
| style="text-align: center; width: 110px;" | 74 
|-
| style="text-align: center; width: 177px;" | UIS
| style="text-align: center; width: 110px;" | 97 
|-
| style="text-align: center; width: 177px;" | Varieties of Democracy
| style="text-align: center; width: 110px;" | 47 
|-
| style="text-align: center; width: 177px;" | UNPD HDR
| style="text-align: center; width: 110px;" | 7 
|-
| style="text-align: center; width: 177px;" | IMF
| style="text-align: center; width: 110px;" | 1 
|}

== '''Preprocessor Updates''' ==

A total of 77 preprocessor series have been updated in this data collection cycle. The main series updated are GDP (GDP2011), land series from FAO, water series from Aquastat, economic freedom from Fraser, population series from UNPD World Population Prospects, and education series from UIS.

== '''Non-preprocessor Updates''' ==
<span style="background-color: rgb(255, 255, 255); color: rgb(37, 37, 37); display: inline; float: none; font-family: sans-serif; font-size: 13.93px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; line-height: inherit; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">
A total of 232 non preprocessor series were updated. This update includes series from FAO, UIS, Aquastat, Fraser, Varieties of Democracy, UNPD WPP, and UNPD HDR. Additionally, there were new series added from the World Bank – UCDP, Gini SSP projections from Rao, and 9 new UIS education completion rate series.
<span style="background-color: rgb(255, 255, 255); color: rgb(37, 37, 37); display: inline; float: none; font-family: sans-serif; font-size: 13.93px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; line-height: inherit; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">
== '''New Series Added''' ==

=== '''Conflict data from World Bank''' ===

There were 7 new series updated from the World Bank – Uppsala Conflict Data Program (UCDP) which cover conflict based data. The series added were:
<ul style="color: rgb(51, 51, 51); font-family: sans-serif,Arial,Verdana,"Trebuchet MS"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; padding-bottom: 0px; padding-left: 40px; padding-right: 40px; padding-top: 0px; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">
<li>ConflictOngoingUCDP – Dummy variable coded 1 for more than 25 deaths in intrastate conflict and 0 otherwise</li>
<li>ConflictDeathsUCDP – Total deaths in all categories of violence (non-state, one-sided and state-based)</li>
<li>ConflictStateDeathsUCDP – Total deaths in state-based conflict</li>
<li>ConflictPopulationAffectedUCDP – Total number of population affected by violence from conflict</li>
<li>ConflictPeaceYearsUCDP – The years since the last minor conflict since 1970</li>
<li>ConflictPopulationShareAffectedUCDP – Share of population affected by violence</li>
<li>ConflictAreaAffectedUCDP - Percent of land area affected by violence from conflict</li>
</ul>

=== '''Gini SSP Projections from RAO''' ===

<span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">There were 5 series added from Rao, ND, M. Gidden, P. Sauer, K. Riahi, 'Income inequality projections for the Shared Socioeconomic Pathways'. These series are forecasts of Gini under SSP1 through SSP 5 and cover the years 2011-2100.

== '''Fixes/updates since 7.42''' ==

'''Updated Aggregation Rules'''

<span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">Aggregation rules for EnConTotalWDI and EnProdElec were changed to POP. Additionally, all of the ForecastTFP series were updated to AVG aggregation.

'''TaxImportDuties%Imp and TaxExportDuties$Exp'''

<span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">TaxImportDuties%Imp was changed to TaxImportDuties%Rev and the definition was updated as well. Additionally, TaxExportDuties%Exp was changed to TaxExportDuties%Rev and the definition was updated accordingly.

Aquastat data

2019-08-29T18:58:45Z

JakeDubbert:

Most of the data used in the water preprocessor comes from the AQUASTAT database. [http://www.fao.org/nr/water/aquastat/main/index.stm http://www.fao.org/nr/water/aquastat/main/index.stm] 

The AQUASTAT database can be queried on-line and the query results can be downloaded in CSV (table or flat) format. The current database regroups data per 5-year period and shows for each variable the value for the most recent year during that period, if available. For example, if for the period 2003-2007 data are available for the year 2004 and for the year 2006, then the value for the year 2006 is shown. Also, for many variables no time series can be made available yet due to lack of sufficient data. Data collection began in 1958. Data for the variable [4313] "Area equipped for irrigation: total" is collected by and available in both AQUASTAT and FAOSTAT and harmonization between AQUASTAT and FAOSTAT takes place on a regular basis to ensure consistency between the two. The difference between the two is that AQUASTAT only enters data for the year given in the reference, while FAOSTAT fills all years from 1961 onwards by inter- and extrapolation from the common data-points. Thus, time series on this variable can be found under [http://faostat3.fao.org/download/E/EL/E FAOSTAT land], choosing the last variable in "items": Total area equipped for irrigation. For other non-water related time-series on food and agriculture see the [http://faostat3.fao.org/home/E FAOSTAT home page].

= Metadata =

== Update Frequency ==

==== Varies by Main Database category and sub-category: ====

#Geography and population category: Every year (through FAOSTAT for land use and population and undernourishment, World Bank for GDP, UNDP for HDI and GII).
#Water resources category: These are long-term average annual values and therefore remain the same over the years. A comprehensive review had been undertaken in 2014.
#Water use category and Irrigation and drainage development category: For Africa, Southern and Central America & Caribbean, and Asia: between 1 and 10 years, depending on data availability which is checked at least every 5 years for each country, and with a thorough update (full dataset) every 10 years together with the country profile update. For Europe, Northern America, Australia and New Zealand: depends on data availability from Eurostat and OECD.
#Updates of data for some specific sub-categories are being done in collaboration with others, as and when data become available, such as: wastewater sub-category in collaboration with IWMI; conservation agriculture sub-category in collaboration with Conservation Agriculture expert groups; access to improved drinking water source sub-category data are provided by the WHO/UNICEF Joint Monitoring Programme for Water Supply and Sanitation.
#Some of the variables are updated during major review exercises (see the other datasets below).

== AquaStat Questionnaires and Guidelines ==

AQUASTAT uses questionnaires and prepares guidelines with detailed definitions and instructions, implements projects to strengthen national capacities, and holds workshops to clarify some of the more complex data concepts. Survey information is as follows:

To reach the above objective, it has been decided to proceed as follows, with two surveys:

*Every 5-10 years: a detailed global survey to prepare/update country profiles and regional syntheses. This document refers to this survey, which is prepared on the basis of a detailed questionnaire filled in at country level by national and regional experts, under the supervision of FAO/NRL.
*Every year: a small global survey to ensure regular updating of the existing country profiles, of key changes at country level, especially on the area equipped for irrigation and on the institutional and policy aspects. It is done by concerned institutions, which are willing to participate, and is managed in the regional and central offices of FAO.

The 5-10 year survey aims at preparing national country profiles in a homogeneous way. It ensures participation of national experts, in charge of the information collection and report drafting at the country level, and validation by FAO experts. The experts are provided with:

*A detailed questionnaire to be completed for each country at national level (and sub-national level for some variables) on the basis of existing available information
*Detailed guidelines for preparing a country profile and gathering qualitative information
*Explanatory notes and definitions to ensure consistency in the use of concepts and words and facilitate computation

=== Costs ===

Costs should be expressed in US$ (United States Dollars) in the questionnaire and country profiles. The conversion used to pass from the local currency to US$ should be the one that was valid in the year to which the value refers and should be indicated in the comments. 

== Definitions ==

Definitions for every series can be found in this document: [http://www.fao.org/nr/water/aquastat/sets/aq-5yr-guide_eng.pdf&nbsp http://www.fao.org/nr/water/aquastat/sets/aq-5yr-guide_eng.pdf&nbsp];

== Series Updated ==

{| cellspacing="0" border="1" bgcolor="#ffffff"
|+ '''DataDict'''
|-
! style="text-align: left;" bgcolor="#c0c0c0" | Variable
|- valign="top"
| DesalinatedWater
|- valign="top"
| IrrigatedCropIntensity
|- valign="top"
| IrWaterReq
|- valign="top"
| IrWaterWith
|- valign="top"
| LandCultivatedArea
|- valign="top"
| LandEquipIrActual
|- valign="top"
| LandEquipIrFullControl
|- valign="top"
| LandEquipIrFullControlActual
|- valign="top"
| LandIr%Grain
|- valign="top"
| LandIrAreaSalinized
|- valign="top"
| LandIrEquipDrained
|- valign="top"
| LandIrEquipGround
|- valign="top"
| LandIrHarvestedCropArea
|- valign="top"
| LandIrWaterLogged
|- valign="top"
| TotalDamCapacity
|- valign="top"
| WasterwaterTreated
|- valign="top"
| WasteWaterDirectNotTreated
|- valign="top"
| WastewaterIrDirectTreated
|- valign="top"
| WasteWaterLandEquipDirectNotTreated
|- valign="top"
| WasteWaterLandEquipDirectTreated
|- valign="top"
| WastewaterProduced
|- valign="top"
| WastewaterTreatedReused
|- valign="top"
| WaterDependencyRatio
|- valign="top"
| WaterDesalinated
|- valign="top"
| WaterGroundEntering
|- valign="top"
| WaterGroundLeaving
|- valign="top"
| WaterGroundProdInternal
|- valign="top"
| WaterGroundTotal
|- valign="top"
| WaterGroundWithD
|- valign="top"
| WaterResExploitGround
|- valign="top"
| WaterResExploitSurface
|- valign="top"
| WaterResOverlap
|- valign="top"
| WaterResTotalExploit
|- valign="top"
| WaterResTotalRenew
|- valign="top"
| WaterResTotalRenewGround
|- valign="top"
| WaterResTotalRenewSurface
|- valign="top"
| WaterSurfaceWithD
|- valign="top"
| WaterTotalRenewPC
|- valign="top"
| WaterTotalWithd
|- valign="top"
| WaterTotalWithdPC
|- valign="top"
| WaterTotalWithdSector
|- valign="top"
| WaterTotalWithdSources
|- valign="top"
| WaterWith%Agric
|- valign="top"
| WaterWith%Fresh
|- valign="top"
| WaterWith%Household
|- valign="top"
| WaterWith%Ind
|- valign="top"
| WaterWithAgr%FreshAquastat
|- valign="top"
| WaterWithdAgriculture
|- valign="top"
| WaterWithdIndustrial
|- valign="top"
| WaterWithdMunicipal 
|}

{| cellspacing="0" border="1" bgcolor="#ffffff"
|+ 
|-
! bgcolor="#c0c0c0" | Variable
|- valign="top"
| LandIrActual%Equip
|- valign="top"
| LandIrAreaEquip
|- valign="top"
| LandIrEquip%Cultivated
|- valign="top"
| LandIrEquip%Potential
|- valign="top"
| LandIrEquip%WaterManaged
|- valign="top"
| LandIrEquipActual
|- valign="top"
| LandIrEquipMixed
|- valign="top"
| LandIrEquipSurface
|- valign="top"
| LandIrPotential
|- valign="top"
| WaterTotalAgManagedArea
|}

== New Series added ==

*Direct use of not treated municipal wastewater for irrigation purposes 
*Irrigated crop intensity 
*Area equipped for irrigation by surface water 
*Area equipped for irrigation by groundwater 
*Area equipped for irrigation by mixed surface water and groundwater 
*Area equipped for irrigation by direct use of treated municipal wastewater 
*Area equipped for irrigation by direct use of non-treated municipal wastewater 
*Area equipped for irrigation by direct use of agricultural drainage water 
*Area equipped for irrigation by desalinated water
*Irrigation water withdrawal
*Irrigation water requirement
*Direct use of treated municipal wastewater for irrigation purposes
*area equipped for irrigation: actually irrigated
*area equipped for full control irrigation: total
*Area equipped for full control irrigation: actually irrigated

== Country Coverage ==

Every country, with the exception of Kosovo, is included in the database. However, available data for each country varies by series.

{| cellspacing="0" border="1" bgcolor="#ffffff"
|+ '''FAO Country Translation'''
|-
! bgcolor="#c0c0c0" | IFs Countries
! bgcolor="#c0c0c0" | FAO Countries
|- valign="top"
| United Arab Emirates
| United Arab Emirates
|- valign="top"
| Afghanistan
| Afghanistan
|- valign="top"
| Albania
| Albania
|- valign="top"
| Armenia
| Armenia
|- valign="top"
| Angola
| Angola
|- valign="top"
| Argentina
| Argentina
|- valign="top"
| Austria
| Austria
|- valign="top"
| Australia
| Australia
|- valign="top"
| Azerbaijan
| Azerbaijan
|- valign="top"
| Bosnia and Herzegovina
| Bosnia and Herzegovina
|- valign="top"
| Barbados
| Barbados
|- valign="top"
| Bangladesh
| Bangladesh
|- valign="top"
| Belgium
| Belgium
|- valign="top"
| Burkina Faso
| Burkina Faso
|- valign="top"
| Bulgaria
| Bulgaria
|- valign="top"
| Bahrain
| Bahrain
|- valign="top"
| Burundi
| Burundi
|- valign="top"
| Benin
| Benin
|- valign="top"
| Brunei
| Brunei Darussalam
|- valign="top"
| Bolivia
| Bolivia (Plurinational State of)
|- valign="top"
| Brazil
| Brazil
|- valign="top"
| Bahamas, The
| Bahamas
|- valign="top"
| Bhutan
| Bhutan
|- valign="top"
| Botswana
| Botswana
|- valign="top"
| Belarus
| Belarus
|- valign="top"
| Belize
| Belize
|- valign="top"
| Canada
| Canada
|- valign="top"
| Congo, Democratic Republic of
| Democratic Republic of the Congo
|- valign="top"
| Central African Republic
| Central African Republic
|- valign="top"
| Congo, Republic of
| Congo
|- valign="top"
| Switzerland
| Switzerland
|- valign="top"
| Cote d'Ivoire
| Côte d'Ivoire
|- valign="top"
| Chile
| Chile
|- valign="top"
| Cameroon
| Cameroon
|- valign="top"
| China
| China
|- valign="top"
| Colombia
| Colombia
|- valign="top"
| Costa Rica
| Costa Rica
|- valign="top"
| Cuba
| Cuba
|- valign="top"
| Cape Verde
| Cabo Verde
|- valign="top"
| Cyprus
| Cyprus
|- valign="top"
| Czech Republic
| Czech Republic
|- valign="top"
| Germany
| Germany
|- valign="top"
| Djibouti
| Djibouti
|- valign="top"
| Denmark
| Denmark
|- valign="top"
| Dominican Republic
| Dominican Republic
|- valign="top"
| Algeria
| Algeria
|- valign="top"
| Ecuador
| Ecuador
|- valign="top"
| Estonia
| Estonia
|- valign="top"
| Egypt, Arab Republic of
| Egypt
|- valign="top"
| Eritrea
| Eritrea
|- valign="top"
| Spain
| Spain
|- valign="top"
| Ethiopia
| Ethiopia
|- valign="top"
| Finland
| Finland
|- valign="top"
| Fiji
| Fiji
|- valign="top"
| Micronesia, Fed. Sts.
| Micronesia (Federated States of)
|- valign="top"
| France
| France
|- valign="top"
| Gabon
| Gabon
|- valign="top"
| United Kingdom
| United Kingdom
|- valign="top"
| Grenada
| Grenada
|- valign="top"
| Georgia
| Georgia
|- valign="top"
| Ghana
| Ghana
|- valign="top"
| Gambia, The
| Gambia
|- valign="top"
| Guinea
| Guinea
|- valign="top"
| Equatorial Guinea
| Equatorial Guinea
|- valign="top"
| Greece
| Greece
|- valign="top"
| Guatemala
| Guatemala
|- valign="top"
| Guinea-Bissau
| Guinea-Bissau
|- valign="top"
| Guyana
| Guyana
|- valign="top"
| Hong Kong
| China, Hong Kong SAR
|- valign="top"
| Honduras
| Honduras
|- valign="top"
| Croatia
| Croatia
|- valign="top"
| Haiti
| Haiti
|- valign="top"
| Hungary
| Hungary
|- valign="top"
| Indonesia
| Indonesia
|- valign="top"
| Ireland
| Ireland
|- valign="top"
| Israel
| Israel
|- valign="top"
| India
| India
|- valign="top"
| Iraq
| Iraq
|- valign="top"
| Iran, Islamic Republic of
| Iran (Islamic Republic of)
|- valign="top"
| Iceland
| Iceland
|- valign="top"
| Italy
| Italy
|- valign="top"
| Jamaica
| Jamaica
|- valign="top"
| Jordan
| Jordan
|- valign="top"
| Japan
| Japan
|- valign="top"
| Kenya
| Kenya
|- valign="top"
| Kyrgyz Republic
| Kyrgyzstan
|- valign="top"
| Cambodia
| Cambodia
|- valign="top"
| Comoros
| Comoros
|- valign="top"
| Kosovo
| 
|- valign="top"
| Korea, Democratic People's Republic of
| Democratic People's Republic of Korea
|- valign="top"
| Korea, Republic of
| Republic of Korea
|- valign="top"
| Kuwait
| Kuwait
|- valign="top"
| Kazakhstan
| Kazakhstan
|- valign="top"
| Laos, People's Democratic Republic
| Lao People's Democratic Republic
|- valign="top"
| Lebanon
| Lebanon
|- valign="top"
| St. Lucia
| Saint Lucia
|- valign="top"
| Sri Lanka
| Sri Lanka
|- valign="top"
| Liberia
| Liberia
|- valign="top"
| Lesotho
| Lesotho
|- valign="top"
| Lithuania
| Lithuania
|- valign="top"
| Luxembourg
| Luxembourg
|- valign="top"
| Latvia
| Latvia
|- valign="top"
| Libya
| Libya
|- valign="top"
| Morocco
| Morocco
|- valign="top"
| Moldova
| Republic of Moldova
|- valign="top"
| Montenegro
| Montenegro
|- valign="top"
| Madagascar
| Madagascar
|- valign="top"
| Macedonia, Former Yugoslav Republic of
| The former Yugoslav Republic of Macedonia
|- valign="top"
| Mali
| Mali
|- valign="top"
| Myanmar
| Myanmar
|- valign="top"
| Mongolia
| Mongolia
|- valign="top"
| Mauritania
| Mauritania
|- valign="top"
| Malta
| Malta
|- valign="top"
| Mauritius
| Mauritius
|- valign="top"
| Maldives
| Maldives
|- valign="top"
| Malawi
| Malawi
|- valign="top"
| Mexico
| Mexico
|- valign="top"
| Malaysia
| Malaysia
|- valign="top"
| Mozambique
| Mozambique
|- valign="top"
| Namibia
| Namibia
|- valign="top"
| Niger
| Niger
|- valign="top"
| Nigeria
| Nigeria
|- valign="top"
| Nicaragua
| Nicaragua
|- valign="top"
| Netherlands
| Netherlands
|- valign="top"
| Norway
| Norway
|- valign="top"
| Nepal
| Nepal
|- valign="top"
| New Zealand
| New Zealand
|- valign="top"
| Oman
| Oman
|- valign="top"
| Panama
| Panama
|- valign="top"
| Peru
| Peru
|- valign="top"
| Papua New Guinea
| Papua New Guinea
|- valign="top"
| Philippines
| Philippines
|- valign="top"
| Pakistan
| Pakistan
|- valign="top"
| Poland
| Poland
|- valign="top"
| Puerto Rico
| Puerto Rico
|- valign="top"
| Palestine
| Occupied Palestinian Territory
|- valign="top"
| Portugal
| Portugal
|- valign="top"
| Paraguay
| Paraguay
|- valign="top"
| Qatar
| Qatar
|- valign="top"
| Romania
| Romania
|- valign="top"
| Serbia
| Serbia
|- valign="top"
| Russian Federation
| Russian Federation
|- valign="top"
| Rwanda
| Rwanda
|- valign="top"
| Saudi Arabia
| Saudi Arabia
|- valign="top"
| Solomon Islands
| Solomon Islands
|- valign="top"
| Seychelles
| Seychelles
|- valign="top"
| Sudan
| Sudan
|- valign="top"
| Sweden
| Sweden
|- valign="top"
| Singapore
| Singapore
|- valign="top"
| Slovenia
| Slovenia
|- valign="top"
| Slovak Republic
| Slovakia
|- valign="top"
| Sierra Leone
| Sierra Leone
|- valign="top"
| Senegal
| Senegal
|- valign="top"
| Somalia
| Somalia
|- valign="top"
| Suriname
| Suriname
|- valign="top"
| Sudan South
| South Sudan
|- valign="top"
| Sao Tome and Principe
| Sao Tome and Principe
|- valign="top"
| El Salvador
| El Salvador
|- valign="top"
| Syrian Arab Republic
| Syrian Arab Republic
|- valign="top"
| Swaziland
| Swaziland
|- valign="top"
| Chad
| Chad
|- valign="top"
| Togo
| Togo
|- valign="top"
| Thailand
| Thailand
|- valign="top"
| Tajikistan
| Tajikistan
|- valign="top"
| Timor-Leste
| Timor-Leste
|- valign="top"
| Turkmenistan
| Turkmenistan
|- valign="top"
| Tunisia
| Tunisia
|- valign="top"
| Tonga
| Tonga
|- valign="top"
| Turkey
| Turkey
|- valign="top"
| Trinidad and Tobago
| Trinidad and Tobago
|- valign="top"
| Taiwan, China
| China, Taiwan Province of
|- valign="top"
| Tanzania
| United Republic of Tanzania
|- valign="top"
| Ukraine
| Ukraine
|- valign="top"
| Uganda
| Uganda
|- valign="top"
| United States
| United States of America
|- valign="top"
| Uruguay
| Uruguay
|- valign="top"
| Uzbekistan
| Uzbekistan
|- valign="top"
| St. Vincent and the Grenadines
| Saint Vincent and the Grenadines
|- valign="top"
| Venezuela
| Venezuela (Bolivarian Republic of)
|- valign="top"
| Vietnam
| Viet Nam
|- valign="top"
| Vanuatu
| Vanuatu
|- valign="top"
| Samoa
| Samoa
|- valign="top"
| Yemen, Republic of
| Yemen
|- valign="top"
| South Africa
| South Africa
|- valign="top"
| Zambia
| Zambia
|- valign="top"
| Zimbabwe
| Zimbabwe
|}

= Instructions =

#Download the data from the Aquastat Database. [http://www.fao.org/nr/water/aquastat/data/query/index.html?lang=en http://www.fao.org/nr/water/aquastat/data/query/index.html?lang=en]. Select all countries and all years. There are too many variables to download all variables at once. Divide the variables by topics (ie geography and population, water resources, etc) download this data via the "Flat" option.
#Change the codes in the Datadict for every variable downloaded in this Excel file. The codes need to match to match the variables listed in the downloaded data exactly. 
#Check to ensure no changes are needed in the FAO Country translation. In the most recent update China, Sudan and South Sudan all needed to be updated.
#Upload data in IFs using the batch pool option. There will be multiple excel sheets, as the AquaStat data cannot download all variables at once. After the first excel sheet is updated, unselect data that was imported; otherwise this data will be uploaded again with a new excel sheet, and you will lose the already imported data.
#Vet. The most recent update reveiled errors in data in some series. You may need extensive vetting.
#Some series do not contain enough data in the database to be updated. These include:

LandIrEquipDesalinated

LandIrEquipDirectAg

LandIrHarvest%Equip

LandIrHarvest%FullControl

LandIrPotentialReached

Global Burdens of Disease (GBD), Institute for Health Metrics and Evaluation (IHME)

2019-08-16T16:10:16Z

JakeDubbert:

The Institute for Health Metrics and Evaluation (IHME) provides data on the Global Burdens of Disease that can be accessed here [http://ghdx.healthdata.org/gbd-results-tool http://ghdx.healthdata.org/gbd-results-tool]. IHME provides data on multiple metrics that we pull into IFs, including deaths, DALYs, incidence, prevelance, violence and forecasts for each. The latest IFs data update for this series was in August 2019, covering 189 economies over the years 1990-2017. This data pull is done using Python.

= IFs Series =

There are currently 146 series that are pulled from the IHME into IFs. 

= Instructions on Pulling IHME Series =

#The data can be accessed from the GBD tool at [http://ghdx.healthdata.org/gbd-results-tool http://ghdx.healthdata.org/gbd-results-tool]. Select "Only countries and territories", "All years", "Cause", "All Ages", "Number", the correct measure (i.e. "Deaths"), "Male" "Female" and "Both", and select all causes. This process might differ depending on the series being pulled in, so just be aware of that.
#Once data is downloaded, a Python script is used to concord and clean the data so it can be pulled into IFs. To do this, you need to have the folder "PythonOutput&Scripts" and "Pythonfiles" in the file path C/Users/Public. WIthin the Pythonfiles folder, you will put the data downloaded from the IHME website in the IHMEDownloads folder, in the respective folder. For example, for Deaths by disease series, you will put the downloaded data in the HistDeathFileData folder.
#Once the data is in the folder, you can run the corresponding Python script. The Python script will pull in the data from the corresponding Excel files, clean it, correspond it to the right IFs series, and concord the countries to the correct country concordance. All you need to do is run the Python script for the series you are trying to pull. For example, for the deaths by diesease series, you will run the DataforIFs-IHME-HistoricalDeathFile.py script.
#Once the script is done running, the completed Excel file will be located in the PythonOutput&Scripts folder.
#Pull in the data into IFs using the IFs data import tool.

= Things to Be Aware of While Pulling =

For the historical deaths by disease series, the data are in millions in IFs. So you will need to do this conversion before you import.

== August 2019 Update ==

For the 2019 update, we have added historical deaths by disease by sex data for the years 1990-2017. These are new series, as we only had totals and not aggregated by sex.

Global Burdens of Disease (GBD), Institute for Health Metrics and Evaluation (IHME)

2019-08-16T16:09:17Z

JakeDubbert:

The Institute for Health Metrics and Evaluation (IHME) provides data on the Global Burdens of Disease that can be accessed here [http://ghdx.healthdata.org/gbd-results-tool http://ghdx.healthdata.org/gbd-results-tool]. IHME provides data on multiple metrics that we pull into IFs, including deaths, DALYs, incidence, prevelance, violence and forecasts for each. The latest IFs data update for this series was in August 2019, covering 189 economies over the years 1990-2017. This data pull is done using Python.

= IFs Series =

= Instructions on Pulling IHME Series =

#The data can be accessed from the GBD tool at [http://ghdx.healthdata.org/gbd-results-tool http://ghdx.healthdata.org/gbd-results-tool]. Select "Only countries and territories", "All years", "Cause", "All Ages", "Number", the correct measure (i.e. "Deaths"), "Male" "Female" and "Both", and select all causes. This process might differ depending on the series being pulled in, so just be aware of that.
#Once data is downloaded, a Python script is used to concord and clean the data so it can be pulled into IFs. To do this, you need to have the folder "PythonOutput&Scripts" and "Pythonfiles" in the file path C/Users/Public. WIthin the Pythonfiles folder, you will put the data downloaded from the IHME website in the IHMEDownloads folder, in the respective folder. For example, for Deaths by disease series, you will put the downloaded data in the HistDeathFileData folder.
#Once the data is in the folder, you can run the corresponding Python script. The Python script will pull in the data from the corresponding Excel files, clean it, correspond it to the right IFs series, and concord the countries to the correct country concordance. All you need to do is run the Python script for the series you are trying to pull. For example, for the deaths by diesease series, you will run the DataforIFs-IHME-HistoricalDeathFile.py script.
#Once the script is done running, the completed Excel file will be located in the PythonOutput&Scripts folder.
#Pull in the data into IFs using the IFs data import tool.

= Things to Be Aware of While Pulling =

For the historical deaths by disease series, the data are in millions in IFs. So you will need to do this conversion before you import.

== August 2019 Update ==

For the 2019 update, we have added historical deaths by disease by sex data for the years 1990-2017. These are new series, as we only had totals and not aggregated by sex.

Global Burdens of Disease (GBD), Institute for Health Metrics and Evaluation (IHME)

2019-08-16T16:04:32Z

JakeDubbert:

Global Burdens of Disease (GBD), Institute for Health Metrics and Evaluation (IHME)

2019-08-16T15:56:07Z

JakeDubbert: Undo revision 9261 by JakeDubbert (talk)

Global Burdens of Disease (GBD), Institute for Health Metrics and Evaluation (IHME)

2019-08-16T15:55:49Z

JakeDubbert:

<parsererror xmlns="http://www.mozilla.org/newlayout/xml/parsererror.xml">XML Parsing Error: syntax error</parsererror>

Global Burdens of Disease (GBD), Institute for Health Metrics and Evaluation (IHME)

2019-08-16T15:53:25Z

JakeDubbert:

Global Burdens of Disease (GBD), Institute for Health Metrics and Evaluation (IHME)

2019-08-16T15:50:34Z

JakeDubbert: Replaced content with "IHME"

IHME

Global Burdens of Disease (GBD), Institute for Health Metrics and Evaluation (IHME)

2019-08-16T15:50:04Z

JakeDubbert: Created page with "<parsererror xmlns="http://www.mozilla.org/newlayout/xml/parsererror.xml">XML Parsing Error: syntax error</parsererror>"

<parsererror xmlns="http://www.mozilla.org/newlayout/xml/parsererror.xml">XML Parsing Error: syntax error</parsererror>

Version notes 7.41 (June 2019)

2019-08-09T17:33:35Z

JakeDubbert:

= Recent model updates =

= Recent data updates (since Version 7.37, October 2018) =

There have been a total of 581 series updated since Version 7.37, which was updated in October 2018. The majority of these updated series come from the WDI. There were 112 preprocessor series updated, from WDI, BGR, ITU, UIS, ILO, WGI, UNAIDS, BP Statistical Review, UNICEF/WHO/WBG, UNESCO, Center for Systemic Peace, UNDP, and SIPRI.  

{| cellpadding="0" cellspacing="0" border="1" width="471"
|-
| width="347" height="20" | '''Source'''
| width="124" | '''Number of series '''
|-
| height="20" | BGR
| align="right" | 15
|-
| height="20" | BP’s Statistical Review of World Energy 
| align="right" | 17
|-
| height="20" | Center for Systemic Peace 
| align="right" | 29
|-
| height="20" | IFs Calculations
| align="right" | 66
|-
| height="20" | Freedom House
| align="right" | 2
|-
| height="20" | Global Footprint Network
| align="right" | 10
|-
| height="20" | IHME
| align="right" | 69
|-
| height="20" | International Labor Organization (ILO) 
| align="right" | 8
|-
| height="20" | ITU 
| align="right" | 6
|-
| height="20" | OECD
| align="right" | 2
|-
| height="20" | Polity Project
| align="right" | 4
|-
| height="20" | PovCalNet
| align="right" | 7
|-
| height="20" | Stockholm International Peace Research Institute (SIPRI) 
| align="right" | 1
|-
| height="20" | Transparency International 
| align="right" | 1
|-
| height="20" | UNESCO Institute of Statistics (UIS) 
| align="right" | 31
|-
| height="20" | United Nations Program on HIV/AIDS (UNAIDS) 
| align="right" | 9 
|-
| height="20" | UNICEF/WHO/WBG 
| align="right" | 1
|-
| height="20" | United Nations Office of Drug and Crime (UNODC) 
| align="right" | 9
|-
| height="20" | US Energy Information Administration (EIA) 
| align="right" | 1
|-
| height="20" | World Development Indicators (WDI) 
| align="right" | 266
|-
| height="20" | World Bank Doing Business 
| align="right" | 14
|-
| height="20" | World Health Organization (WHO) 
| align="right" | 6 
|-
| height="20" | Worldwide Governance Indicators 
| align="right" | 6
|}

= Bug fixes/updates since 7.36 (internal version of IFs)  =

Version notes 7.41 (June 2019)

2019-08-09T17:29:38Z

JakeDubbert:

= Recent model updates =

= Recent data updates (since January 2018) =

{| cellpadding="0" cellspacing="0" border="1" style="color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;" width="471"
|-
| width="347" height="20" | '''Source'''
| width="124" | '''Number of series '''
|-
| height="20" | BGR
| align="right" | 15
|-
| height="20" | BP’s Statistical Review of World Energy 
| align="right" | 17
|-
| height="20" | Center for Systemic Peace 
| align="right" | 29
|-
| height="20" | IFs Calculations
| align="right" | 66
|-
| height="20" | Freedom House
| align="right" | 2
|-
| height="20" | Global Footprint Network
| align="right" | 10
|-
| height="20" | IHME
| align="right" | 69
|-
| height="20" | International Labor Organization (ILO) 
| align="right" | 8
|-
| height="20" | ITU 
| align="right" | 6
|-
| height="20" | OECD
| align="right" | 2
|-
| height="20" | Polity Project
| align="right" | 4
|-
| height="20" | PovCalNet
| align="right" | 7
|-
| height="20" | Stockholm International Peace Research Institute (SIPRI) 
| align="right" | 1
|-
| height="20" | Transparency International 
| align="right" | 1
|-
| height="20" | UNESCO Institute of Statistics (UIS) 
| align="right" | 31
|-
| height="20" | United Nations Program on HIV/AIDS (UNAIDS) 
| align="right" | 9 
|-
| height="20" | UNICEF/WHO/WBG 
| align="right" | 1
|-
| height="20" | United Nations Office of Drug and Crime (UNODC) 
| align="right" | 9
|-
| height="20" | US Energy Information Administration (EIA) 
| align="right" | 1
|-
| height="20" | World Development Indicators (WDI) 
| align="right" | 266
|-
| height="20" | World Bank Doing Business 
| align="right" | 14
|-
| height="20" | World Health Organization (WHO) 
| align="right" | 6 
|-
| height="20" | Worldwide Governance Indicators 
| align="right" | 6
|}

= Bug fixes/updates since 7.36 (internal version of IFs)  =

Version notes 7.41 (June 2019)

2019-08-09T17:04:05Z

JakeDubbert: Created page with "= <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"..."

= <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 26px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 1.2; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">Recent model updates =

= Recent data updates (since January 2018) =

{| cellpadding="0" cellspacing="0" border="1" style="color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;" width="471"
|-
| width="347" height="20" | '''Source'''
| width="124" | '''Number of series '''
|-
| height="20" | AQU (AQUASTAT) BATCH PULL
| align="right" | 51
|-
| height="20" | Barro-Lee
| align="right" | 22
|-
| height="20" | BP’s Statistical Review of World Energy 2016
| align="right" | 6
|-
| height="20" | Carbon Dioxide Information Analysis Center
| align="right" | 1
|-
| height="20" | FAO
| align="right" | 38
|-
| height="20" | Freedom House
| align="right" | 1
|-
| height="20" | IFs calculations (drugs, education quality, Minerva)
| align="right" | 6
|-
| height="20" | IHME
| align="right" | 51
|-
| height="20" | IMF GFS
| align="right" | 8
|-
| height="20" | IMF World Economic Outlook 2017
| align="right" | 2
|-
| height="20" | JMP
| align="right" | 5
|-
| height="20" | PovCalNet
| align="right" | 1
|-
| height="20" | UNAIDS
| align="right" | 6
|-
| height="20" | UNESCO Institute for Statistics (UIS)
| align="right" | 97
|-
| height="20" | UNODC
| align="right" | 4
|-
| height="20" | UNPD
| align="right" | 3 
|-
| height="20" | WDI
| align="right" | 392 
|}

<span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 26px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 1.2; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">Bug fixes/updates since 7.36 (internal version of IFs)

Version notes 7.42 (August 2019)

2019-08-09T15:31:12Z

JakeDubbert:

= Recent Model Updates =

= Recent Data Updates (since version 7.41) =

{| cellpadding="1" cellspacing="1" border="1" style="width: 300px;"
|-
| style="width: 177px; text-align: center;" | Source 
| style="width: 110px; text-align: center;" | Number of Series 
|-
| style="width: 177px; text-align: center;" | IMF WEO 
| style="width: 110px; text-align: center;" | 1 
|-
| style="width: 177px; text-align: center;" | IMF GFS 
| style="width: 110px; text-align: center;" | 8
|-
| style="width: 177px; text-align: center;" | WDI
| style="width: 110px; text-align: center;" | 2 
|-
| style="width: 177px; text-align: center;" | UNPD 
| style="width: 110px; text-align: center;" | 1
|-
| style="width: 177px; text-align: center;" | ITU
| style="width: 110px; text-align: center;" | 1 
|-
| style="width: 177px; text-align: center;" | UN SDGs
| style="width: 110px; text-align: center;" | 127 
|-
| style="width: 177px; text-align: center;" | World Bank Doing Business 
| style="width: 110px; text-align: center;" | 22 
|-
| style="width: 177px; text-align: center;" | IHME
| style="width: 110px; text-align: center;" | 45
|-
| style="width: 177px; text-align: center;" | EDGAR
| style="width: 110px; text-align: center;" | 6 
|-
| style="width: 177px; text-align: center;" | Dellink 
| style="width: 110px; text-align: center;" | 15 
|}

== '''Preprocessor Updates''' ==

A total of 13 preprocessor series were included in this update. 

The main series updated are: GDP from the IMF (GDP2011), GDP per capita at PPP from WDI (GDP2011PCPPP), Population from UNPD, IMF government expenditures, revenue and tax data (GovtCalcExpendCen%GDP, GovtCalcRevCen%GDP, GovtCalcEdCen%GDP, GovtCalcEdTot%GDP, GovtCalcExpendTot%GDP, GovtCalcHealthTot%GDP, GovtCalcRevTot%GDP, TotRev%GDPIMF, TaxSocSec%CurRev), and Mobile broadband cellular subscriptions per 100 inhabitants from the ITU (ICTBroadbandMobileSubsPer100).

== '''Non-preprocessor Updates''' ==

<span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(37, 37, 37); font-family: sans-serif; font-size: 13.93px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; line-height: inherit; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">A total of 219 non pre-processor series have been updated in this data collection cycle. The majority of these series updated are SDG series from the UN SDG Indicators Global Database. Additionally, new series from IHME on deaths by disease by sex and methane emissions series from EDGAR were added to the model. Finally, ForecastGDPP2011OECDSSP2, TFP data from Dellink, and World Bank Doing business series were included in this update.  

== '''New Series Added''' ==

=== IHME Data ===

IHME now provides deaths by disease broken out by sex which have been included in this update. There were 30 new series added to IFs which are broken out by Male and Female.

=== '''TFP Data from Dellink''' ===

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

=== '''Methane Emissions Data from EDGAR''' ===

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- <a href="[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]" alt="[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]" title="[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]">[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]</a>

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== '''Bug fixes/updates since 7.41''' ==

'''TaxSocSec%CurRev'''

There were abnormal values for Australia from 2000 and on that were over 1000% in the TaxSocSec%CurRev. The series was repulled and the issued was fixed.

'''SeriesEdSecLowerGradRateAllMal, SeriesEdSecLowerGradRateAllFem, SeriesEdSecLowerGradRateAllTot'''

Each of these education series from the UIS had very low values for South Africa (1-4%) in 2016 and 2017 that seemed to be inconsistent with the data. These values were deleted.

'''SeriesForecastGDPP2011OECDSSP2'''

An update to SeriesForecastGDPP2011OECDSSP2 was included in the consolidation to fix the transient values seen between 2009 and 2010. There is still a transient between 2009 and 2010 for some countries. But that is because we are using GDP values from the WDI till 2009. We need the older values so that the aggregation rule we created for the TFP will work correctly. (From Kanishka).

== ==

Version notes 7.42 (August 2019)

2019-08-09T15:30:31Z

JakeDubbert:

= Recent Model Updates =

= Recent Data Updates (since version 7.41) =

{| cellpadding="1" cellspacing="1" border="1" style="width: 300px;"
|-
| style="width: 177px; text-align: center;" | Source 
| style="width: 110px; text-align: center;" | Number of Series 
|-
| style="width: 177px; text-align: center;" | IMF WEO 
| style="width: 110px; text-align: center;" | 1 
|-
| style="width: 177px; text-align: center;" | IMF GFS 
| style="width: 110px; text-align: center;" | 8
|-
| style="width: 177px; text-align: center;" | WDI
| style="width: 110px; text-align: center;" | 2 
|-
| style="width: 177px; text-align: center;" | UNPD 
| style="width: 110px; text-align: center;" | 1
|-
| style="width: 177px; text-align: center;" | ITU
| style="width: 110px; text-align: center;" | 1 
|-
| style="width: 177px; text-align: center;" | UN SDGs
| style="width: 110px; text-align: center;" | 127 
|-
| style="width: 177px; text-align: center;" | World Bank Doing Business 
| style="width: 110px; text-align: center;" | 22 
|-
| style="width: 177px; text-align: center;" | IHME
| style="width: 110px; text-align: center;" | 
|-
| style="width: 177px; text-align: center;" | EDGAR
| style="width: 110px; text-align: center;" | 6 
|-
| style="width: 177px; text-align: center;" | Dellink 
| style="width: 110px; text-align: center;" | 15 
|}

== '''Preprocessor Updates''' ==

A total of 13 preprocessor series were included in this update. 

The main series updated are: GDP from the IMF (GDP2011), GDP per capita at PPP from WDI (GDP2011PCPPP), Population from UNPD, IMF government expenditures, revenue and tax data (GovtCalcExpendCen%GDP, GovtCalcRevCen%GDP, GovtCalcEdCen%GDP, GovtCalcEdTot%GDP, GovtCalcExpendTot%GDP, GovtCalcHealthTot%GDP, GovtCalcRevTot%GDP, TotRev%GDPIMF, TaxSocSec%CurRev), and Mobile broadband cellular subscriptions per 100 inhabitants from the ITU (ICTBroadbandMobileSubsPer100).

== '''Non-preprocessor Updates''' ==

<span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(37, 37, 37); font-family: sans-serif; font-size: 13.93px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; line-height: inherit; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">A total of 219 non pre-processor series have been updated in this data collection cycle. The majority of these series updated are SDG series from the UN SDG Indicators Global Database. Additionally, new series from IHME on deaths by disease by sex and methane emissions series from EDGAR were added to the model. Finally, ForecastGDPP2011OECDSSP2, TFP data from Dellink, and World Bank Doing business series were included in this update.   

== '''New Series Added''' ==

=== IHME Data ===

IHME now provides deaths by disease broken out by sex which have been included in this update. There were 30 new series added to IFs which are broken out by Male and Female.

=== '''TFP Data from Dellink''' ===

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

=== '''Methane Emissions Data from EDGAR''' ===

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- <a href="[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]" alt="[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]" title="[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]">[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]</a>

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== '''Bug fixes/updates since 7.41''' ==

'''TaxSocSec%CurRev'''

There were abnormal values for Australia from 2000 and on that were over 1000% in the TaxSocSec%CurRev. The series was repulled and the issued was fixed.

'''SeriesEdSecLowerGradRateAllMal, SeriesEdSecLowerGradRateAllFem, SeriesEdSecLowerGradRateAllTot'''

Each of these education series from the UIS had very low values for South Africa (1-4%) in 2016 and 2017 that seemed to be inconsistent with the data. These values were deleted.

'''SeriesForecastGDPP2011OECDSSP2'''

An update to SeriesForecastGDPP2011OECDSSP2 was included in the consolidation to fix the transient values seen between 2009 and 2010. There is still a transient between 2009 and 2010 for some countries. But that is because we are using GDP values from the WDI till 2009. We need the older values so that the aggregation rule we created for the TFP will work correctly. (From Kanishka).

== ==

August 2019 consolidation

2019-08-09T15:29:01Z

JakeDubbert:

232 series were updated/pulled into the IFs database for consolidation in August 2019 for Version 7.42. There were 13 preprocessor series and 174 non-preporcessor series.

= '''Data Updates''' =

A total of 13 pre-processor series have been updated in this data collection cycle. The main series updated are: GDP from IMF WEO, GDP per capita at PPP from WDI, Population from UNPD, IMF government expenditures, revenue and tax data, and Mobile cellular subscriptions with access to data communication at broadband speed per 100 inhabitants from the ITU.

{| cellpadding="1" cellspacing="1" border="1" style="width: 500px;"
|-
| Variable
| Source
|-
| GDP2011 
| IMF WEO
|-
| GDP2011PCPP 
| WDI
|-
| Population 
| UNPD
|-
| GovtCalcExpendCen%GDP 
| IMF GFS 
|-
| GovtCalcRevCen%GDP 
| IMF GFS 
|-
| GovtCalcEdCen%GDP 
| IMF GFS 
|-
| GovtCalcEdTot%GDP 
| IMF GFS 
|-
| GovtCalcExpendTot%GDP 
| IMF GFS 
|-
| GovtCalcHealthTot%GDP 
| IMF GFS 
|-
| GovtCalcRevTot%GDP 
| IMF GFS 
|-
| TotRev%GDPIMF 
| IMF GFS 
|-
| TaxSocSec%CurRev 
| IMF GFS 
|-
| ICTBroadbandMobileSubsPer100 
| ITU
|}

A total of 219 non pre-processor series have been updated in this data collection cycle. The majority of these series updated are SDG series from the UN SDG Indicators Global Database. Additionally, new series from IHME on deaths by disease by sex and methane emissions series from EDGAR were added to the model. Finally, ForecastGDPP2011OECDSSP2, TFP data from Dellink, and World Bank Doing business series were included in this update.  

== [[TaxSocSec%CurRev|TaxSocSec%CurRev]] ==

There were abnormal values for Australia from 2000 and on that were over 1000% in the TaxSocSec%CurRev. The series was repulled and the issued was fixed.

== [[SeriesEdSecLowerGradRateAllMal,_SeriesEdSecLowerGradRateAllFem,_SeriesEdSecLowerGradRateAllTot|SeriesEdSecLowerGradRateAllMal, SeriesEdSecLowerGradRateAllFem, SeriesEdSecLowerGradRateAllTot]] ==

Each of these education series from the UIS had very low values for South Africa (1-4%) in 2016 and 2017 that seemed to be inconsistent with the data. These values were deleted.

== [[SeriesForecastGDPP2011OECDSSP2|SeriesForecastGDPP2011OECDSSP2]] ==

An update to SeriesForecastGDPP2011OECDSSP2 was included in the consolidation to fix the transient values seen between 2009 and 2010. There is still a transient between 2009 and 2010 for some countries. But that is because we are using GDP values from the WDI till 2009. We need the older values so that the aggregation rule we created for the TFP will work correctly. (From Kanishka).

== [[TFP_Data_from_Dellink|TFP Data from Dellink]] ==

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

== [[Methane_Emissions_Data_from_EDGAR|Methane Emissions Data from EDGAR]] ==

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- [https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== [[IHME|IHME]] ==

IHME now provides deaths by disease broken out by sex which have been included in this update. 30 new series were added.

= '''Affected Modules''' =

== '''GDP''' ==

== '''Government Expenditures''' ==

== '''Government Revenue''' ==

== '''Government Debt''' ==

== '''ITU Index - ICT Development Index''' ==

= '''Bugs and Issues''' =

August 2019 consolidation

2019-08-09T15:27:15Z

JakeDubbert:

232 series were updated/pulled into the IFs database for consolidation in August 2019 for Version 7.42. There were 13 preprocessor series and 174 non-preporcessor series.

= '''Data Updates''' =

A total of 13 pre-processor series have been updated in this data collection cycle. The main series updated are: GDP from IMF WEO, GDP per capita at PPP from WDI, Population from UNPD, IMF government expenditures, revenue and tax data, and Mobile cellular subscriptions with access to data communication at broadband speed per 100 inhabitants from the ITU.

{| cellpadding="1" cellspacing="1" border="1" style="width: 500px;"
|-
| Variable
| Source
|-
| GDP2011 
| IMF WEO
|-
| GDP2011PCPP 
| WDI
|-
| Population 
| UNPD
|-
| GovtCalcExpendCen%GDP 
| IMF GFS 
|-
| GovtCalcRevCen%GDP 
| IMF GFS 
|-
| GovtCalcEdCen%GDP 
| IMF GFS 
|-
| GovtCalcEdTot%GDP 
| IMF GFS 
|-
| GovtCalcExpendTot%GDP 
| IMF GFS 
|-
| GovtCalcHealthTot%GDP 
| IMF GFS 
|-
| GovtCalcRevTot%GDP 
| IMF GFS 
|-
| TotRev%GDPIMF 
| IMF GFS 
|-
| TaxSocSec%CurRev 
| IMF GFS 
|-
| ICTBroadbandMobileSubsPer100 
| ITU
|}

A total of 219 non pre-processor series have been updated in this data collection cycle. The majority of these series updated are SDG series from the UN SDG Indicators Global Database. Additionally, new series from IHME on deaths by disease by sex and methane emissions series from EDGAR were added to the model. Finally, ForecastGDPP2011OECDSSP2, TFP data from Dellink, and World Bank Doing business series were included in this update.  

== [[TaxSocSec%CurRev|TaxSocSec%CurRev]] ==

There were abnormal values for Australia from 2000 and on that were over 1000% in the TaxSocSec%CurRev. The series was repulled and the issued was fixed.

== [[SeriesEdSecLowerGradRateAllMal,_SeriesEdSecLowerGradRateAllFem,_SeriesEdSecLowerGradRateAllTot|SeriesEdSecLowerGradRateAllMal, SeriesEdSecLowerGradRateAllFem, SeriesEdSecLowerGradRateAllTot]] ==

Each of these education series from the UIS had very low values for South Africa (1-4%) in 2016 and 2017 that seemed to be inconsistent with the data. These values were deleted.

== [[SeriesForecastGDPP2011OECDSSP2|SeriesForecastGDPP2011OECDSSP2]] ==

An update to SeriesForecastGDPP2011OECDSSP2 was included in the consolidation to fix the transient values seen between 2009 and 2010. There is still a transient between 2009 and 2010 for some countries. But that is because we are using GDP values from the WDI till 2009. We need the older values so that the aggregation rule we created for the TFP will work correctly. (From Kanishka).

== [[TFP_Data_from_Dellink|TFP Data from Dellink]] ==

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

== [[Methane_Emissions_Data_from_EDGAR|Methane Emissions Data from EDGAR]] ==

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- [https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== [[IHME|IHME]] ==

IHME now provides deaths by disease broken out by sex which have been included in this update. 15 new series were added.

= '''Affected Modules''' =

== '''GDP''' ==

== '''Government Expenditures''' ==

== '''Government Revenue''' ==

== '''Government Debt''' ==

== '''ITU Index - ICT Development Index''' ==

= '''Bugs and Issues''' =

Version notes 7.42 (August 2019)

2019-08-09T02:20:19Z

JakeDubbert:

= Recent Model Updates =

= Recent Data Updates (since version 7.41) =

{| cellpadding="1" cellspacing="1" border="1" style="width: 300px;"
|-
| style="width: 177px; text-align: center;" | Source 
| style="width: 110px; text-align: center;" | Number of Series 
|-
| style="width: 177px; text-align: center;" | IMF WEO 
| style="width: 110px; text-align: center;" | 1 
|-
| style="width: 177px; text-align: center;" | IMF GFS 
| style="width: 110px; text-align: center;" | 8
|-
| style="width: 177px; text-align: center;" | WDI
| style="width: 110px; text-align: center;" | 2 
|-
| style="width: 177px; text-align: center;" | UNPD 
| style="width: 110px; text-align: center;" | 1
|-
| style="width: 177px; text-align: center;" | ITU
| style="width: 110px; text-align: center;" | 1 
|-
| style="width: 177px; text-align: center;" | UN SDGs
| style="width: 110px; text-align: center;" | 127 
|-
| style="width: 177px; text-align: center;" | World Bank Doing Business 
| style="width: 110px; text-align: center;" | 22 
|-
| style="width: 177px; text-align: center;" | IHME
| style="width: 110px; text-align: center;" | 
|-
| style="width: 177px; text-align: center;" | EDGAR
| style="width: 110px; text-align: center;" | 6 
|-
| style="width: 177px; text-align: center;" | Dellink 
| style="width: 110px; text-align: center;" | 15 
|}

== '''Pre-Processor Updates''' ==

A total of 13 preprocessor series were included in this update. 

The main series updated are: GDP from the IMF (GDP2011), GDP per capita at PPP from WDI (GDP2011PCPPP), Population from UNPD, IMF government expenditures, revenue and tax data (GovtCalcExpendCen%GDP, GovtCalcRevCen%GDP, GovtCalcEdCen%GDP, GovtCalcEdTot%GDP, GovtCalcExpendTot%GDP, GovtCalcHealthTot%GDP, GovtCalcRevTot%GDP, TotRev%GDPIMF, TaxSocSec%CurRev), and Mobile broadband cellular subscriptions per 100 inhabitants from the ITU (ICTBroadbandMobileSubsPer100).

== '''New Series Added''' ==

=== IHME Data ===

IHME now provides deaths by disease broken out by sex which have been included in this update.

=== '''TFP Data from Dellink ''' ===

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

=== '''Methane Emissions Data from EDGAR ''' ===

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- <a href="[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]" alt="[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]" title="[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]">[https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]</a>

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== '''Bug fixes/updates since 7.41''' ==

'''TaxSocSec%CurRev '''

There were abnormal values for Australia from 2000 and on that were over 1000% in the TaxSocSec%CurRev. The series was repulled and the issued was fixed.

'''SeriesEdSecLowerGradRateAllMal, SeriesEdSecLowerGradRateAllFem, SeriesEdSecLowerGradRateAllTot '''

Each of these education series from the UIS had very low values for South Africa (1-4%) in 2016 and 2017 that seemed to be inconsistent with the data. These values were deleted.

'''SeriesForecastGDPP2011OECDSSP2'''

An update to SeriesForecastGDPP2011OECDSSP2 was included in the consolidation to fix the transient values seen between 2009 and 2010. There is still a transient between 2009 and 2010 for some countries. But that is because we are using GDP values from the WDI till 2009. We need the older values so that the aggregation rule we created for the TFP will work correctly. (From Kanishka).

== ==

Version notes 7.42 (August 2019)

2019-08-09T02:14:31Z

JakeDubbert:

<h1> Recent Model Updates </h1>
<h1> Recent Data Updates (since version 7.41) </h1>
<table cellpadding="1" cellspacing="1" border="1" style="width: 300px;">

<tr>
<td style="width: 177px; text-align: center;"> Source 
</td>
<td style="width: 110px; text-align: center;"> Number of Series 
</td></tr>
<tr>
<td style="width: 177px; text-align: center;"> IMF WEO 
</td>
<td style="width: 110px; text-align: center;"> 1 
</td></tr>
<tr>
<td style="width: 177px; text-align: center;"> IMF GFS 
</td>
<td style="width: 110px; text-align: center;"> 8
</td></tr>
<tr>
<td style="width: 177px; text-align: center;"> WDI
</td>
<td style="width: 110px; text-align: center;"> 2 
</td></tr>
<tr>
<td style="width: 177px; text-align: center;"> UNPD 
</td>
<td style="width: 110px; text-align: center;"> 1
</td></tr>
<tr>
<td style="width: 177px; text-align: center;"> ITU
</td>
<td style="width: 110px; text-align: center;"> 1 
</td></tr>
<tr>
<td style="width: 177px; text-align: center;"> UN SDGs
</td>
<td style="width: 110px; text-align: center;"> 127 
</td></tr>
<tr>
<td style="width: 177px; text-align: center;"> World Bank Doing Business 
</td>
<td style="width: 110px; text-align: center;"> 22 
</td></tr>
<tr>
<td style="width: 177px; text-align: center;"> IHME
</td>
<td style="width: 110px; text-align: center;"> 
</td></tr>
<tr>
<td style="width: 177px; text-align: center;"> EDGAR
</td>
<td style="width: 110px; text-align: center;"> 6 
</td></tr>
<tr>
<td style="width: 177px; text-align: center;"> Dellink 
</td>
<td style="width: 110px; text-align: center;"> 15 
</td></tr></table>
<h2> Pre-Processor Updates </h2>
A total of 13 preprocessor series were included in this update. 
The main series updated are: GDP from the IMF (GDP2011), GDP per capita at PPP from WDI (GDP2011PCPPP), Population from UNPD, IMF government expenditures, revenue and tax data (GovtCalcExpendCen%GDP, GovtCalcRevCen%GDP, GovtCalcEdCen%GDP, GovtCalcEdTot%GDP, GovtCalcExpendTot%GDP, GovtCalcHealthTot%GDP, GovtCalcRevTot%GDP, TotRev%GDPIMF, TaxSocSec%CurRev), and Mobile broadband cellular subscriptions per 100 inhabitants from the ITU (ICTBroadbandMobileSubsPer100).

<h2> New Series Added </h2>
<h3> IHME Data </h3>
IHME now provides deaths by disease broken out by sex which have been included in this update.

<h3> <a href="TFP%20Data%20from%20Dellink%20%28August%202019%29">TFP Data from Dellink</a> </h3>
We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

<h3> <a href="Methane%20Emissions%20Data%20from%20EDGAR%20%28August%202019%29">Methane Emissions Data from EDGAR</a> </h3>
We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- <a href="https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG" alt="https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG" title="https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG">https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG</a>
Data is available for 181 countries from 1970-2012. Categories:
1.      Methane emissions from fermentation
2.      Methane emissions from solid fuel production
3.      Methane emissions from oil and gas production
4.      Methane emissions from wastewater handling
5.      Methane emissions from rice cultivation
6.      Methane emissions from other sources

<h2> Bug fixes/updates since 7.41 </h2>


<h2> <a href="https://pardee.du.edu/w/TaxSocSec%CurRev" alt="https://pardee.du.edu/w/TaxSocSec%CurRev" title="https://pardee.du.edu/w/TaxSocSec%CurRev">TaxSocSec%CurRev</a> </h2>
There were abnormal values for Australia from 2000 and on that were over 1000% in the TaxSocSec%CurRev. The series was repulled and the issued was fixed.

<h2> <a href="https://pardee.du.edu/w/SeriesEdSecLowerGradRateAllMal,%20SeriesEdSecLowerGradRateAllFem,%20SeriesEdSecLowerGradRateAllTot" alt="https://pardee.du.edu/w/SeriesEdSecLowerGradRateAllMal,%20SeriesEdSecLowerGradRateAllFem,%20SeriesEdSecLowerGradRateAllTot" title="https://pardee.du.edu/w/SeriesEdSecLowerGradRateAllMal,%20SeriesEdSecLowerGradRateAllFem,%20SeriesEdSecLowerGradRateAllTot">SeriesEdSecLowerGradRateAllMal, SeriesEdSecLowerGradRateAllFem, SeriesEdSecLowerGradRateAllTot</a> </h2>
Each of these education series from the UIS had very low values for South Africa (1-4%) in 2016 and 2017 that seemed to be inconsistent with the data. These values were deleted.

<h2> <a href="https://pardee.du.edu/w/SeriesForecastGDPP2011OECDSSP2" alt="https://pardee.du.edu/w/SeriesForecastGDPP2011OECDSSP2" title="https://pardee.du.edu/w/SeriesForecastGDPP2011OECDSSP2">SeriesForecastGDPP2011OECDSSP2</a> </h2>
An update to SeriesForecastGDPP2011OECDSSP2 was included in the consolidation to fix the transient values seen between 2009 and 2010. There is still a transient between 2009 and 2010 for some countries. But that is because we are using GDP values from the WDI till 2009. We need the older values so that the aggregation rule we created for the TFP will work correctly. (From Kanishka).


<h2> </h2>

August 2019 consolidation

2019-08-09T00:46:40Z

JakeDubbert:

187 series were updated/pulled into the IFs database for consolidation in August 2019 for Version 7.42. There were 13 preprocessor series and 174 non-preporcessor series.

= '''Data Updates''' =

A total of 13 pre-processor series have been updated in this data collection cycle. The main series updated are: GDP from IMF WEO, GDP per capita at PPP from WDI, Population from UNPD, IMF government expenditures, revenue and tax data, and Mobile cellular subscriptions with access to data communication at broadband speed per 100 inhabitants from the ITU.

{| cellpadding="1" cellspacing="1" border="1" style="width: 500px;"
|-
| Variable
| Source
|-
| GDP2011 
| IMF WEO
|-
| GDP2011PCPP 
| WDI
|-
| Population 
| UNPD
|-
| GovtCalcExpendCen%GDP 
| IMF GFS 
|-
| GovtCalcRevCen%GDP 
| IMF GFS 
|-
| GovtCalcEdCen%GDP 
| IMF GFS 
|-
| GovtCalcEdTot%GDP 
| IMF GFS 
|-
| GovtCalcExpendTot%GDP 
| IMF GFS 
|-
| GovtCalcHealthTot%GDP 
| IMF GFS 
|-
| GovtCalcRevTot%GDP 
| IMF GFS 
|-
| TotRev%GDPIMF 
| IMF GFS 
|-
| TaxSocSec%CurRev 
| IMF GFS 
|-
| ICTBroadbandMobileSubsPer100 
| ITU
|}

A total of 174 non pre-processor series have been updated in this data collection cycle. The majority of these series updated are SDG series from the UN SDG Indicators Global Database. Additionally, new series from IHME on deaths by disease by sex and methane emissions series from EDGAR were added to the model. Finally, ForecastGDPP2011OECDSSP2, TFP data from Dellink, and World Bank Doing business series were included in this update.  

== [[TaxSocSec%CurRev|TaxSocSec%CurRev]] ==

There were abnormal values for Australia from 2000 and on that were over 1000% in the TaxSocSec%CurRev. The series was repulled and the issued was fixed.

== [[SeriesEdSecLowerGradRateAllMal,_SeriesEdSecLowerGradRateAllFem,_SeriesEdSecLowerGradRateAllTot|SeriesEdSecLowerGradRateAllMal, SeriesEdSecLowerGradRateAllFem, SeriesEdSecLowerGradRateAllTot]] ==

Each of these education series from the UIS had very low values for South Africa (1-4%) in 2016 and 2017 that seemed to be inconsistent with the data. These values were deleted.

== [[SeriesForecastGDPP2011OECDSSP2|SeriesForecastGDPP2011OECDSSP2]] ==

An update to SeriesForecastGDPP2011OECDSSP2 was included in the consolidation to fix the transient values seen between 2009 and 2010. There is still a transient between 2009 and 2010 for some countries. But that is because we are using GDP values from the WDI till 2009. We need the older values so that the aggregation rule we created for the TFP will work correctly. (From Kanishka).

== [[TFP_Data_from_Dellink|TFP Data from Dellink]] ==

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

== [[Methane_Emissions_Data_from_EDGAR|Methane Emissions Data from EDGAR]] ==

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- [https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== [[IHME|IHME]] ==

IHME now provides deaths by disease broken out by sex which have been included in this update.

= '''Affected Modules''' =

== '''GDP''' ==

== '''Government Expenditures''' ==

== '''Government Revenue''' ==

== '''Government Debt''' ==

== '''ITU Index - ICT Development Index''' ==

= '''Bugs and Issues''' =

Version notes 7.42 (August 2019)

2019-08-09T00:45:35Z

JakeDubbert:

= Recent Model Updates =

= Recent Data Updates (since version 7.41) =

{| cellpadding="1" cellspacing="1" border="1" style="width: 300px;"
|-
| style="width: 177px; text-align: center;" | Source 
| style="width: 110px; text-align: center;" | Number of Series 
|-
| style="width: 177px; text-align: center;" | IMF WEO 
| style="width: 110px; text-align: center;" | 1 
|-
| style="width: 177px; text-align: center;" | IMF GFS 
| style="width: 110px; text-align: center;" | 8
|-
| style="width: 177px; text-align: center;" | WDI
| style="width: 110px; text-align: center;" | 2 
|-
| style="width: 177px; text-align: center;" | UNPD 
| style="width: 110px; text-align: center;" | 1
|-
| style="width: 177px; text-align: center;" | ITU
| style="width: 110px; text-align: center;" | 1 
|-
| style="width: 177px; text-align: center;" | UN SDGs
| style="width: 110px; text-align: center;" | 127 
|-
| style="width: 177px; text-align: center;" | World Bank Doing Business 
| style="width: 110px; text-align: center;" | 22 
|-
| style="width: 177px; text-align: center;" | IHME
| style="width: 110px; text-align: center;" | 
|-
| style="width: 177px; text-align: center;" | EDGAR
| style="width: 110px; text-align: center;" | 6 
|-
| style="width: 177px; text-align: center;" | Dellink 
| style="width: 110px; text-align: center;" | 15 
|}

== '''Pre-Processor Updates''' ==

A total of 13 preprocessor series were included in this update. 

The main series updated are: GDP from the IMF (GDP2011), GDP per capita at PPP from WDI (GDP2011PCPPP), Population from UNPD, IMF government expenditures, revenue and tax data (GovtCalcExpendCen%GDP, GovtCalcRevCen%GDP, GovtCalcEdCen%GDP, GovtCalcEdTot%GDP, GovtCalcExpendTot%GDP, GovtCalcHealthTot%GDP, GovtCalcRevTot%GDP, TotRev%GDPIMF, TaxSocSec%CurRev), and Mobile broadband cellular subscriptions per 100 inhabitants from the ITU (ICTBroadbandMobileSubsPer100).

== '''New Series Added''' ==

=== IHME Data ===

IHME now provides deaths by disease broken out by sex which have been included in this update.

=== [[TFP_Data_from_Dellink_(August_2019)|TFP Data from Dellink]] ===

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

=== [[Methane_Emissions_Data_from_EDGAR_(August_2019)|Methane Emissions Data from EDGAR]] ===

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- [https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== '''Bug fixes/updates since 7.41''' ==

== [https://pardee.du.edu/w/TaxSocSec%CurRev TaxSocSec%CurRev] ==

There were abnormal values for Australia from 2000 and on that were over 1000% in the TaxSocSec%CurRev. The series was repulled and the issued was fixed.

== [https://pardee.du.edu/w/SeriesEdSecLowerGradRateAllMal,%20SeriesEdSecLowerGradRateAllFem,%20SeriesEdSecLowerGradRateAllTot SeriesEdSecLowerGradRateAllMal, SeriesEdSecLowerGradRateAllFem, SeriesEdSecLowerGradRateAllTot] ==

Each of these education series from the UIS had very low values for South Africa (1-4%) in 2016 and 2017 that seemed to be inconsistent with the data. These values were deleted.

== [https://pardee.du.edu/w/SeriesForecastGDPP2011OECDSSP2 SeriesForecastGDPP2011OECDSSP2] ==

An update to SeriesForecastGDPP2011OECDSSP2 was included in the consolidation to fix the transient values seen between 2009 and 2010. There is still a transient between 2009 and 2010 for some countries. But that is because we are using GDP values from the WDI till 2009. We need the older values so that the aggregation rule we created for the TFP will work correctly. (From Kanishka).

== ==

Version notes 7.42 (August 2019)

2019-08-08T22:18:52Z

JakeDubbert:

Version notes 7.42 (August 2019)

2019-08-08T22:18:24Z

JakeDubbert:

Version notes 7.42 (August 2019)

2019-08-08T22:17:31Z

JakeDubbert:

Consolidation

2019-08-08T22:01:27Z

JakeDubbert:

[[April_consolidation|April 2016 consolidation]]

[[May_2016_consolidation|May 2016 consolidation]]

[[June_2016_consolidation|June 2016 consolidation]]

[[December_2016_version_notes|December 2016 version notes (v 7.27)]]

*[[December_2016_vetting|December 2016 vetting (v 7.27)]]

[[China_Sub-Regional_June_2017_Consolidation|China Sub-Regional June 2017 Consolidation]]

[[India_2017_Consolidation|India Sub-Regional June 2017 Consolidation]]

[[August_2019_consolidation|August 2019 Consolidation]]

August 2019 consolidation

2019-08-08T22:00:47Z

JakeDubbert:

187 series were updated/pulled into the IFs database for consolidation in August 2019 for Version 7.42. There were 13 preprocessor series and 174 non-preporcessor series.

= '''Data Updates''' =

A total of 13 pre-processor series have been updated in this data collection cycle. The main series updated are: GDP from IMF WEO, GDP per capita at PPP from WDI, Population from UNPD, IMF government expenditures, revenue and tax data, and Mobile cellular subscriptions with access to data communication at broadband speed per 100 inhabitants from the ITU.

{| cellpadding="1" cellspacing="1" border="1" style="width: 500px;"
|-
| Variable
| Source
|-
| GDP2011 
| IMF WEO
|-
| GDP2011PCPP 
| WDI
|-
| Population 
| UNPD
|-
| GovtCalcExpendCen%GDP 
| IMF GFS 
|-
| GovtCalcRevCen%GDP 
| IMF GFS 
|-
| GovtCalcEdCen%GDP 
| IMF GFS 
|-
| GovtCalcEdTot%GDP 
| IMF GFS 
|-
| GovtCalcExpendTot%GDP 
| IMF GFS 
|-
| GovtCalcHealthTot%GDP 
| IMF GFS 
|-
| GovtCalcRevTot%GDP 
| IMF GFS 
|-
| TotRev%GDPIMF 
| IMF GFS 
|-
| TaxSocSec%CurRev 
| IMF GFS 
|-
| ICTBroadbandMobileSubsPer100 
| ITU
|}

A total of 174 non pre-processor series have been updated in this data collection cycle. The majority of these series updated are SDG series from the UN SDG Indicators Global Database. Additionally, new series from IHME on deaths by disease by sex and methane emissions series from EDGAR were added to the model. Finally, ForecastGDPP2011OECDSSP2, TFP data from Dellink, and World Bank Doing business series were included in this update.  

== [[TaxSocSec%CurRev|TaxSocSec%CurRev]] ==

There were abnormal values for Australia from 2000 and on that were over 1000% in the TaxSocSec%CurRev. The series was repulled and the issued was fixed.

== [[SeriesEdSecLowerGradRateAllMal,_SeriesEdSecLowerGradRateAllFem,_SeriesEdSecLowerGradRateAllTot|SeriesEdSecLowerGradRateAllMal, SeriesEdSecLowerGradRateAllFem, SeriesEdSecLowerGradRateAllTot]] ==

Each of these education series from the UIS had very low values for South Africa (1-4%) in 2016 and 2017 that seemed to be inconsistent with the data. These values were deleted.

== [[SeriesForecastGDPP2011OECDSSP2|SeriesForecastGDPP2011OECDSSP2]] ==

An update to SeriesForecastGDPP2011OECDSSP2 was included in the consolidation to fix the transient values seen between 2009 and 2010. There is still a transient between 2009 and 2010 for some countries. But that is because we are using GDP values from the WDI till 2009. We need the older values so that the aggregation rule we created for the TFP will work correctly. (From Kanishka).

== [[TFP_Data_from_Dellink|TFP Data from Dellink]] ==

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

== [[Methane_Emissions_Data_from_EDGAR|Methane Emissions Data from EDGAR]] ==

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- [https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== [[IHME|IHME]] ==

IHME now provides deaths by disease broken out by sex which have been included in this update.

= '''Affected Modules''' =

== '''GDP''' ==

=== '''Government Expenditures''' ===

=== '''Government Revenue''' ===

=== '''Government Debt''' ===

== '''ITU Index - ICT Development Index''' ==

= '''Bugs and Issues''' =

August 2019 consolidation

2019-08-08T21:59:42Z

JakeDubbert:

187 series were updated/pulled into the IFs database for consolidation in August 2019 for Version 7.42. There were 13 preprocessor series and 174 non-preporcessor series.

= '''Data Updates''' =

A total of 13 pre-processor series have been updated in this data collection cycle. The main series updated are: GDP from IMF WEO, GDP per capita at PPP from WDI, Population from UNPD, IMF government expenditures, revenue and tax data, and Mobile cellular subscriptions with access to data communication at broadband speed per 100 inhabitants from the ITU.

{| cellpadding="1" cellspacing="1" border="1" style="width: 500px;"
|-
| Variable
| Source
|-
| GDP2011 
| IMF WEO
|-
| GDP2011PCPP 
| WDI
|-
| Population 
| UNPD
|-
| GovtCalcExpendCen%GDP 
| IMF GFS 
|-
| GovtCalcRevCen%GDP 
| IMF GFS 
|-
| GovtCalcEdCen%GDP 
| IMF GFS 
|-
| GovtCalcEdTot%GDP 
| IMF GFS 
|-
| GovtCalcExpendTot%GDP 
| IMF GFS 
|-
| GovtCalcHealthTot%GDP 
| IMF GFS 
|-
| GovtCalcRevTot%GDP 
| IMF GFS 
|-
| TotRev%GDPIMF 
| IMF GFS 
|-
| TaxSocSec%CurRev 
| IMF GFS 
|-
| ICTBroadbandMobileSubsPer100 
| ITU
|}

A total of 174 non pre-processor series have been updated in this data collection cycle. The majority of these series updated are SDG series from the UN SDG Indicators Global Database. Additionally, new series from IHME on deaths by disease by sex and methane emissions series from EDGAR were added to the model. Finally, ForecastGDPP2011OECDSSP2, TFP data from Dellink, and World Bank Doing business series were included in this update.  

== [[TaxSocSec%CurRev]] ==

There were abnormal values for Australia from 2000 and on that were over 1000% in the TaxSocSec%CurRev. The series was repulled and the issued was fixed.

== [[SeriesEdSecLowerGradRateAllMal, SeriesEdSecLowerGradRateAllFem, SeriesEdSecLowerGradRateAllTot]] ==

Each of these education series from the UIS had very low values for South Africa (1-4%) in 2016 and 2017 that seemed to be inconsistent with the data. These values were deleted.

== [[SeriesForecastGDPP2011OECDSSP2]] ==

An update to SeriesForecastGDPP2011OECDSSP2 was included in the consolidation to fix the transient values seen between 2009 and 2010. There is still a transient between 2009 and 2010 for some countries. But that is because we are using GDP values from the WDI till 2009. We need the older values so that the aggregation rule we created for the TFP will work correctly. (From Kanishka).

== [[TFP Data from Dellink]] ==

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

== [[Methane Emissions Data from EDGAR]] ==

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- [https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== [[IHME]] ==

IHME now provides deaths by disease broken out by sex which have been included in this update.

= '''Affected Modules''' =

== '''GDP''' ==

=== '''Government Expenditures''' ===

=== '''Government Revenue''' ===

=== '''Government Debt''' ===

== '''ITU Index - ICT Development Index''' ==

= '''Bugs and Issues''' =

August 2019 consolidation

2019-08-08T21:58:43Z

JakeDubbert:

August 2019 consolidation

2019-08-08T21:57:24Z

JakeDubbert:

August 2019 consolidation

2019-08-08T21:56:30Z

JakeDubbert:

August 2019 consolidation

2019-08-08T21:53:18Z

JakeDubbert: Created page with "187 series were updated/pulled into the IFs database for consolidation in August 2019 for Version 7.42. There were 13 preprocessor series and 174 non-preporcessor series. A..."

187 series were updated/pulled into the IFs database for consolidation in August 2019 for Version 7.42. There were 13 preprocessor series and 174 non-preporcessor series.

A total of 13 pre-processor series have been updated in this data collection cycle. The main series updated are: GDP from IMF WEO, GDP per capita at PPP from WDI, Population from UNPD, IMF government expenditures, revenue and tax data, and Mobile cellular subscriptions with access to data communication at broadband speed per 100 inhabitants from the ITU.

{| cellpadding="1" cellspacing="1" border="1" style="width: 500px;"
|-
| Variable
| Source
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">GDP2011 
| IMF WEO
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">GDP2011PCPP 
| WDI
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">Population 
| UNPD
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">GovtCalcExpendCen%GDP 
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">IMF GFS 
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">GovtCalcRevCen%GDP 
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">IMF GFS 
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">GovtCalcEdCen%GDP 
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">IMF GFS 
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">GovtCalcEdTot%GDP 
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">IMF GFS 
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">GovtCalcExpendTot%GDP 
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">IMF GFS 
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">GovtCalcHealthTot%GDP 
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">IMF GFS 
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">GovtCalcRevTot%GDP 
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">IMF GFS 
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">TotRev%GDPIMF 
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">IMF GFS 
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">TaxSocSec%CurRev 
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">IMF GFS 
|-
| <span style="display: inline !important; float: none; background-color: rgb(255, 255, 255); color: rgb(51, 51, 51); font-family: sans-serif,arial,verdana,"trebuchet ms"; font-size: 13px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">ICTBroadbandMobileSubsPer100 
| ITU
|}

A total of 174 non pre-processor series have been updated in this data collection cycle. The majority of these series updated are SDG series from the UN SDG Indicators Global Database. Additionally, new series from IHME on deaths by disease by sex and methane emissions series from EDGAR were added to the model. Finally, ForecastGDPP2011OECDSSP2, TFP data from Dellink, and World Bank Doing business series were included in this update.

Additional resources

2019-08-08T20:52:12Z

JakeDubbert:

[[Data|Data]]

[[SubRegionalization_Handbook|Sub-Regionalization Handbook]]

[[Version_notes|Version notes]]

[[IFs_Bibliography|IFs Bibliography]]

[[Development_Mode_Features|Development Mode Features]]

[[Publications_on_IFs|Publications on IFs]]

[[Instructional_Use_of_IFs_(For_Classrooms)|IFs for the Classroom]]

[[Acknowledgements|Acknowledgements]]

=== In progress[[http://pardee.du.edu/wiki/index.php?title=International_Futures_(IFs)&action=edit&section=1 edit]] ===

[[Sandbox|Sandbox]]

Version notes 7.42 (August 2019)

2019-08-08T20:50:39Z

JakeDubbert:

Version notes 7.42 (August 2019)

2019-08-08T20:49:20Z

JakeDubbert:

= Recent Model Updates =

= Recent Data Updates (since version 7.41) =

{| cellpadding="1" cellspacing="1" border="1" style="width: 500px;"
|-
| Source
| Number of Series
|-
| IMF WEO 
| 1 
|-
| IMF GFS 
| 8
|-
| WDI
| 2 
|-
| UNPD 
| 1
|-
| ITU
| 1 
|-
| UN SDGs
| 127 
|-
| World Bank Doing Business 
| 22 
|-
| IHME
| 
|-
| EDGAR
| 6 
|-
| Dellink 
| 15 
|}

== '''Pre-Processor Updates''' ==

A total of 13 preprocessor series were included in this update. 

The main series updated are: GDP from the IMF (GDP2011), GDP per capita at PPP from WDI (GDP2011PCPPP), Population from UNPD, IMF government expenditures, revenue and tax data (GovtCalcExpendCen%GDP, GovtCalcRevCen%GDP, GovtCalcEdCen%GDP, GovtCalcEdTot%GDP, GovtCalcExpendTot%GDP, GovtCalcHealthTot%GDP, GovtCalcRevTot%GDP, TotRev%GDPIMF, TaxSocSec%CurRev), and Mobile broadband cellular subscriptions per 100 inhabitants from the ITU (ICTBroadbandMobileSubsPer100).

== '''New Series Added''' ==

=== [[TFP_Data_from_Dellink_(August_2019)|TFP Data from Dellink]] ===

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

=== [[Methane_Emissions_Data_from_EDGAR_(August_2019)|Methane Emissions Data from EDGAR]] ===

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- [https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== '''Bug fixes/updates since 7.41''' ==

== ==

Version notes 7.42 (August 2019)

2019-08-08T20:46:14Z

JakeDubbert:

= Recent Model Updates =

= Recent Data Updates (since version 7.41) =

{| cellpadding="1" cellspacing="1" border="1" style="width: 500px;"
|-
| Source
| Number of Series
|-
| IMF WEO 
| 1 
|-
| IMF GFS 
| 8
|-
| WDI
| 2 
|-
| UNPD 
| 1
|-
| ITU
| 1 
|-
| UN SDGs
| 127 
|-
| World Bank Doing Business 
| 22 
|-
| IHME
| 
|-
| EDGAR
| 6 
|-
| Dellink 
| 15 
|}

== '''Important Pre-Processor Updates''' ==

A total of 13 preprocessor series were included in this update. 

The main series updated are: GDP from the IMF (GDP2011), GDP per capita at PPP from WDI (GDP2011PCPPP), Population from UNPD, IMF government expenditures, revenue and tax data (GovtCalcExpendCen%GDP, GovtCalcRevCen%GDP, GovtCalcEdCen%GDP, GovtCalcEdTot%GDP, GovtCalcExpendTot%GDP, GovtCalcHealthTot%GDP, GovtCalcRevTot%GDP, TotRev%GDPIMF, TaxSocSec%CurRev), and Mobile broadband cellular subscriptions per 100 inhabitants from the ITU (ICTBroadbandMobileSubsPer100).

== '''New Series Added''' ==

=== [[TFP_Data_from_Dellink_(August_2019)|TFP Data from Dellink]] ===

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

=== [[Methane_Emissions_Data_from_EDGAR_(August_2019)|Methane Emissions Data from EDGAR]] ===

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- [https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== '''Bug fixes/updates since 7.41''' ==

== ==

Version notes 7.42 (August 2019)

2019-08-08T20:45:23Z

JakeDubbert:

= Recent Model Updates =

= Recent Data Updates (since version 7.41) =

{| style="width: 500px;" cellpadding="1" cellspacing="1" border="1"
|-
| Source
| Number of Series
|-
| IMF WEO 
| 1 
|-
| IMF GFS 
| 8
|-
| WDI
| 2 
|-
| UNPD 
| 1
|-
| ITU
| 1 
|-
| UN SDGs
| 127 
|-
| World Bank Doing Business 
| 22 
|-
| IHME
| 
|-
| EDGAR
| 6 
|-
| Dellink 
| 15 
|}

== '''Important Pre-Processor Updates''' ==

A total of 13 preprocessor series were included in this update. 

The main series updated are: GDP from the IMF (GDP2011), GDP per capita at PPP from WDI (GDP2011PCPPP), Population from UNPD, IMF government expenditures, revenue and tax data (GovtCalcExpendCen%GDP, GovtCalcRevCen%GDP, GovtCalcEdCen%GDP, GovtCalcEdTot%GDP, GovtCalcExpendTot%GDP, GovtCalcHealthTot%GDP, GovtCalcRevTot%GDP, TotRev%GDPIMF, TaxSocSec%CurRev), and Mobile broadband cellular subscriptions per 100 inhabitants from the ITU (ICTBroadbandMobileSubsPer100).

== '''New Series Added''' ==

=== [[TFP_Data_from_Dellink_(August_2019)|TFP Data from Dellink]] ===

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

=== [[Methane_Emissions_Data_from_EDGAR_(August_2019)|Methane Emissions Data from EDGAR]] ===

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- [https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== '''Bug fixes/updates since 7.41''' ==

== ==

Version notes 7.42 (August 2019)

2019-08-08T20:44:25Z

JakeDubbert:

= Recent Model Updates =

= Recent Data Updates (since version 7.42) =

{| style="width: 500px;" cellpadding="1" cellspacing="1" border="1"
|-
| Source
| Number of Series
|-
| IMF WEO 
| 1 
|-
| IMF GFS 
| 8
|-
| WDI
| 2 
|-
| UNPD 
| 1
|-
| ITU
| 1 
|-
| UN SDGs
| 127 
|-
| World Bank Doing Business 
| 22 
|-
| IHME
| 
|-
| EDGAR
| 6 
|-
| Dellink 
| 15 
|}

== '''Important Pre-Processor Updates''' ==

A total of 13 preprocessor series were included in this update. 

The main series updated are: GDP from the IMF (GDP2011), GDP per capita at PPP from WDI (GDP2011PCPPP), Population from UNPD, IMF government expenditures, revenue and tax data (GovtCalcExpendCen%GDP, GovtCalcRevCen%GDP, GovtCalcEdCen%GDP, GovtCalcEdTot%GDP, GovtCalcExpendTot%GDP, GovtCalcHealthTot%GDP, GovtCalcRevTot%GDP, TotRev%GDPIMF, TaxSocSec%CurRev), and Mobile broadband cellular subscriptions per 100 inhabitants from the ITU (ICTBroadbandMobileSubsPer100).

== '''New Series Added''' ==

=== [[TFP Data from Dellink (August 2019)|TFP Data from Dellink]] ===

We have added 17 new TFP forecast series from Dellink. These include SSP1-SSP5 data and percent change from 2005-2100, as well as shortened versions that start in 2015. Additionally, history and historical percent change were added which start in 1960.

=== [[Methane Emissions Data from EDGAR (August 2019)|Methane Emissions Data from EDGAR]] ===

We now have data on 6 broad categories pulled in from EDGAR. Data for methane emissions has been collected from the EDGAR database maintained by PNNL. The database can be found here- [https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG https://edgar.jrc.ec.europa.eu/overview.php?v=432_GHG]

Data is available for 181 countries from 1970-2012. Categories:

1.      Methane emissions from fermentation

2.      Methane emissions from solid fuel production

3.      Methane emissions from oil and gas production

4.      Methane emissions from wastewater handling

5.      Methane emissions from rice cultivation

6.      Methane emissions from other sources

== '''Bug fixes/updates since 7.41''' ==

== ==

Version notes 7.42 (August 2019)

2019-08-08T20:42:00Z

JakeDubbert:

= Recent Data Updates (since version 7.42) =

{| style="width: 500px;" cellpadding="1" cellspacing="1" border="1"
|-
| Source
| Number of Series
|-
| IMF WEO 
| 1 
|-
| IMF GFS 
| 8
|-
| WDI
| 2 
|-
| UNPD 
| 1
|-
| ITU
| 1 
|-
| UN SDGs
| 127 
|-
| World Bank Doing Business 
| 22 
|-
| IHME
| 
|-
| EDGAR
| 6 
|-
| Dellink 
| 15 
|}

== '''Important Pre-Processor Updates''' ==

A total of 13 preprocessor series were included in this update.  

The main series updated are: GDP from the IMF (GDP2011), GDP per capita at PPP from WDI (GDP2011PCPPP), Population from UNPD, IMF government expenditures, revenue and tax data (GovtCalcExpendCen%GDP, GovtCalcRevCen%GDP, GovtCalcEdCen%GDP, GovtCalcEdTot%GDP, GovtCalcExpendTot%GDP, GovtCalcHealthTot%GDP, GovtCalcRevTot%GDP, TotRev%GDPIMF, TaxSocSec%CurRev), and Mobile broadband cellular subscriptions per 100 inhabitants from the ITU (ICTBroadbandMobileSubsPer100).

== '''New Series Added''' ==

=== '''<span style="text-align: left; color: rgb(0, 0, 0); text-transform: none; text-indent: 0px; letter-spacing: normal; font-family: sans-serif; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; word-spacing: 0px; display: inline !important; white-space: normal; orphans: 2; float: none; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">Bug fixes/updates since 7.36 (internal version of IFs)''' ===

Version notes 7.42 (August 2019)

2019-08-08T20:40:27Z

JakeDubbert:

== Recent Data Updates (since version 7.42) ==

{| style="width: 500px;" cellpadding="1" cellspacing="1" border="1"
|-
| Source
| Number of Series
|-
| IMF WEO 
| 1
|-
| IMF GFS 
| 8
|-
| WDI
| 2 
|-
| UNPD 
| 1
|-
| ITU
| 1 
|-
| UN SDGs
| 127 
|-
| World Bank Doing Business 
| 22 
|-
| IHME
| 
|-
| EDGAR
| 6 
|-
| Dellink 
| 15 
|}

=== '''Important Pre-Processor Updates''' ===

A total of 13 preprocessor series were included in this update. 

=== The main series updated are: GDP from the IMF (GDP2011), GDP per capita at PPP from WDI (GDP2011PCPPP), Population from UNPD, IMF government expenditures, revenue and tax data (GovtCalcExpendCen%GDP, GovtCalcRevCen%GDP, GovtCalcEdCen%GDP, GovtCalcEdTot%GDP, GovtCalcExpendTot%GDP, GovtCalcHealthTot%GDP, GovtCalcRevTot%GDP, TotRev%GDPIMF, TaxSocSec%CurRev), and Mobile broadband cellular subscriptions per 100 inhabitants from the ITU (ICTBroadbandMobileSubsPer100).''' ''' ===

=== '''New Series Added ''' ===

=== '''<span style="text-align: left; color: rgb(0, 0, 0); text-transform: none; text-indent: 0px; letter-spacing: normal; font-family: sans-serif; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; word-spacing: 0px; display: inline !important; white-space: normal; orphans: 2; float: none; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">Bug fixes/updates since 7.36 (internal version of IFs)''' ===

Version notes 7.42 (August 2019)

2019-08-08T20:38:34Z

JakeDubbert:

Version notes 7.42 (August 2019)

2019-08-08T20:38:04Z

JakeDubbert: Created page with "== Recent Data Updates (since version 7.42) == {| style="width: 500px;" cellpadding="1" cellspacing="1" border="1" |- | Source | Number of Series |- | IMF WEO | 1 |- | I..."

Index.php

2019-08-08T20:21:51Z

JakeDubbert: Version Notes 7.42 (August 2019)

Vetting data

2019-07-25T16:51:33Z

JakeDubbert:

'''Vetting Data'''

Vetting is the process of checking to assure the quality of the data we import into IFs. Even when data is imported through an automated application there could be errors or missing data in the imported series. Vetting the new data you are bringing into IFs requires you to compare the new series to the existing historical series in IFs.

There are a number of reasons for vetting data. One reason is that a country (or a number of countries) could be missed when importing due to not properly concording countries with the proper country concordance table. Make sure the IFs country concordance list (i.e., the relevant column in the '''Country '''Translation table in IFs.mdb) is correct and up to date and you are using the correct concordance list as listed in the DataDict. Additionally, make sure there are no missing years in the new data. Sometimes the year column is missed because the data is formatted as text and not as numbers, or the source simply did not provide an update for that year.  If this is the case, be sure to blend in the missing years in the IFs vetting tool. One more general issue is around the units. Make sure you perform any unit conversion required before you import the data. Comparing the new data with the current IFs series will give you an idea on any unit conversion that might be needed.

 

'''Vetting Data''' '''Checklist:'''

*Data in Access file and source Excel file match
*Use IFs vetting tool to compare new data to historical data
*Large discrepancies between new and old data are documented
*Check to see if there are big spikes in the new, imported data
*Blend countries/years with missing data in IFs vetting tool
*Zeroes in Access file are actually zeroes and not null values
*No missing countries or years
*DataDict contains no errors, Original Source has website name and Name in Source contains variable source
*Initials added to both Access and Excel file names, as well as in the DataDict notes
*Send Word file with vetting notes, along with original files to puller and project leads

'''Process:'''

#After the data series are imported into IFs, the IFsDataImport.mdb file is passed to another RA (assigned as a vetter) who brings the data from the IFsDataImport.mdb file into the model via the "Vet Imported Data" feature in the Extended Features menu
#The vetter uses the vetting tool to determine if there are any remarkable inconsistencies between the old and new data. This is a bit subjective, but necessarily so as the threshold for concern will be different depending on which series from which source is being vetted
#If significant errors are found that need to be corrected, and these errors had occurred during the import process, these are documented in the RA's vetting notes and the files are passed back to the original data puller to correct and re-send. 
#If no errors are found, the vetter blends columns for new years and preserves historical data points as appropriate, and saves the changes in the IFsDataImport file. 
#The vetter passes the IFsDataImport file (renamed to reflect series/batch update name and completion date) back to the project lead and data team supervisor
#Data team supervisor stores data import file for consolidation process

 

'''You can ensure data quality by following certain procedures''':

#For an updated table, open the existing and updated table side by side. Compare data points for all years and all countries to make sure there are no, or very small, differences.
#For all series, be sure to check large and important countries like USA, China or Germany.
#Check countries with similar name (like the two Congos and two Koreas), as these can sometimes get mixed up.  
#Check for zeros and make sure they are actually zeros in the source data. We take no data as an empty cell. If we have zeros that need to be data and must be a feasible value (e.g., GDP cannot be 0).
#Check the variable definition and make sure it makes sense and is in fact the right definition.
#Make sure percentages are below 100 (there are cases when percentages can be above 100, e.g., gross enrollment rate).
#Check values: GDP growth rate of more than 10% should raise a flag. For instances like this, check against the source data.
#Creating line graphs for the countries in a series is a great way to quickly check for transients. This can easily be done in Excel or Tableau.

 

'''Using the IFs Vetting Tool'''

IFs has a feature to do some basic initial vetting and is a tool that should be used in every vetting process. The vetting tool can be found by the following:

'''IFs -> MainMenu->Extended Features -> Manage Country Data->Historic Data File Processing-> Vet Imported Data'''.

 [[File:2019-07-25 (1).png|900x500px]]

The Compare Imported Table With Existing Table screen is then opened.

[[File:2019-07-25 (2).png|900x500px]]

Here, go to '''File>Open Database with Imported Tables, '''and open the Access file with the data series that you want to import into IFs. This Access file contains data series tables and a dictionary table with a list of new and/or updated data variables. This will then populate the top grid with every series within that Access file.

[[File:2019-07-25 (3).png|900x500px]]

Next, double click on one of the series (rows) in the top grid to display the data. This will then populate the new data (the data you are importing) and the old, historical data that is in IFs. The old data is in the first grid and the new data is in the second grid, as shown above.

Here you have a lot of different options to compare the new data with the old. The vetting tool will automatically mark any zeros in the table. You will want to double check to see if, in fact, these are suppose to be zeros or if they are just null values. If they are just null values, you can click the button, '''Delete Zeros.'''

Next, you need to click the button, '''Mark Differences between New and Old Data for same country-year. '''This will allow you to easily observe any differences, as the vetting tool will highlight them. You can also change the threshold that it will mark. The default is that it will mark any more than 10% difference. As you can see in the screenshot below, the values for Afghanistan in 2014 and 2015 are marked. You will want to go through all of the countries in each series you are importing to look at these marked differences and write down any that are very large.

[[File:2019-07-25 (4).png|900x500px]]

You can also click the button '''Mark''' '''Year to Year Jumps in New Data, '''which is helpful to observe any large changes year-to-year.

'''Blending'''

This vetting tool is also used to merge country and year columns between updated and existing Access tables. This is a necessity when blending to preserve the existing historical IFs values when there is no corresponding value in the new data series. For example, often times there will be an update from a source but it is missing multiple countries we already have data for. In this case, you can use the blending tool to blend in any historical country values that are not in the new data. The same can be done for missing years in the new data.  

Once you have thoroughly vetted a data series, you will want to blend the new and historical series together. This ensures that we do not lose any values from the historical series. To do this for the Country values, you will want to check the '''Select All '''box in the bottom right hand corner of the screen. This will automatically check all of the countries, as seen below. Once you have check this box, press the '''Blend Data Points '''button, and click '''Yes. '''This will update the new Access file with missing Country observations.

You will also want to do the same process for the '''Blend Columns (Years) '''button, if there are any missing years. This ensures that we retain the historical values.

[[File:2019-07-25 (5).png|900x500px]]

 

File:2019-07-25 (5).png

2019-07-25T16:47:30Z

JakeDubbert:

File:2019-07-25 (4).png

2019-07-25T16:47:17Z

JakeDubbert:

File:2019-07-25 (3).png

2019-07-25T16:47:04Z

JakeDubbert:

File:2019-07-25 (2).png

2019-07-25T16:46:50Z

JakeDubbert:

Vetting data

2019-07-25T16:46:25Z

JakeDubbert:

'''Vetting Data'''

Vetting is the process of checking to assure the quality of the data we import into IFs. Even when data is imported through an automated application there could be errors or missing data in the imported series. Vetting the new data you are bringing into IFs requires you to compare the new series to the existing historical series in IFs.

There are a number of reasons for vetting data. One reason is that a country (or a number of countries) could be missed when importing due to not properly concording countries with the proper country concordance table. Make sure the IFs country concordance list (i.e., the relevant column in the '''Country '''Translation table in IFs.mdb) is correct and up to date and you are using the correct concordance list as listed in the DataDict. Additionally, make sure there are no missing years in the new data. Sometimes the year column is missed because the data is formatted as text and not as numbers, or the source simply did not provide an update for that year.  If this is the case, be sure to blend in the missing years in the IFs vetting tool. One more general issue is around the units. Make sure you perform any unit conversion required before you import the data. Comparing the new data with the current IFs series will give you an idea on any unit conversion that might be needed.

 

'''Vetting Data''' '''Checklist:'''

*Data in Access file and source Excel file match
*Use IFs vetting tool to compare new data to historical data
*Large discrepancies between new and old data are documented
*Check to see if there are big spikes in the new, imported data
*Blend countries/years with missing data in IFs vetting tool
*Zeroes in Access file are actually zeroes and not null values
*No missing countries or years
*DataDict contains no errors, Original Source has website name and Name in Source contains variable source
*Initials added to both Access and Excel file names, as well as in the DataDict notes
*Send Word file with vetting notes, along with original files to puller and project leads

'''Process:'''

#After the data series are imported into IFs, the IFsDataImport.mdb file is passed to another RA (assigned as a vetter) who brings the data from the IFsDataImport.mdb file into the model via the "Vet Imported Data" feature in the Extended Features menu
#The vetter uses the vetting tool to determine if there are any remarkable inconsistencies between the old and new data. This is a bit subjective, but necessarily so as the threshold for concern will be different depending on which series from which source is being vetted
#If significant errors are found that need to be corrected, and these errors had occurred during the import process, these are documented in the RA's vetting notes and the files are passed back to the original data puller to correct and re-send. 
#If no errors are found, the vetter blends columns for new years and preserves historical data points as appropriate, and saves the changes in the IFsDataImport file. 
#The vetter passes the IFsDataImport file (renamed to reflect series/batch update name and completion date) back to the project lead and data team supervisor
#Data team supervisor stores data import file for consolidation process

 

'''You can ensure data quality by following certain procedures''':

#For an updated table, open the existing and updated table side by side. Compare data points for all years and all countries to make sure there are no, or very small, differences.
#For all series, be sure to check large and important countries like USA, China or Germany.
#Check countries with similar name (like the two Congos and two Koreas), as these can sometimes get mixed up.  
#Check for zeros and make sure they are actually zeros in the source data. We take no data as an empty cell. If we have zeros that need to be data and must be a feasible value (e.g., GDP cannot be 0).
#Check the variable definition and make sure it makes sense and is in fact the right definition.
#Make sure percentages are below 100 (there are cases when percentages can be above 100, e.g., gross enrollment rate).
#Check values: GDP growth rate of more than 10% should raise a flag. For instances like this, check against the source data.
#Creating line graphs for the countries in a series is a great way to quickly check for transients. This can easily be done in Excel or Tableau.

 

'''Using the IFs Vetting Tool'''

IFs has a feature to do some basic initial vetting and is a tool that should be used in every vetting process. The vetting tool can be found by the following:

'''IFs -> MainMenu->Extended Features -> Manage Country Data->Historic Data File Processing-> Vet Imported Data'''.

 [[File:2019-07-25 (1).png]]

The Compare Imported Table With Existing Table screen is then opened.

[[File:|624x351px|A screenshot of a social media post

Description automatically generated]]

Here, go to '''File>Open Database with Imported Tables, '''and open the Access file with the data series that you want to import into IFs. This Access file contains data series tables and a dictionary table with a list of new and/or updated data variables. This will then populate the top grid with every series within that Access file.

[[File:|624x351px|A screenshot of a cell phone

Description automatically generated]]

Next, double click on one of the series (rows) in the top grid to display the data. This will then populate the new data (the data you are importing) and the old, historical data that is in IFs. The old data is in the first grid and the new data is in the second grid, as shown above.

Here you have a lot of different options to compare the new data with the old. The vetting tool will automatically mark any zeros in the table. You will want to double check to see if, in fact, these are suppose to be zeros or if they are just null values. If they are just null values, you can click the button, '''Delete Zeros.'''

Next, you need to click the button, '''Mark Differences between New and Old Data for same country-year. '''This will allow you to easily observe any differences, as the vetting tool will highlight them. You can also change the threshold that it will mark. The default is that it will mark any more than 10% difference. As you can see in the screenshot below, the values for Afghanistan in 2014 and 2015 are marked. You will want to go through all of the countries in each series you are importing to look at these marked differences and write down any that are very large.

[[File:|624x351px|A screenshot of a social media post

Description automatically generated]]

You can also click the button '''Mark''' '''Year to Year Jumps in New Data, '''which is helpful to observe any large changes year-to-year.

'''Blending'''

This vetting tool is also used to merge country and year columns between updated and existing Access tables. This is a necessity when blending to preserve the existing historical IFs values when there is no corresponding value in the new data series. For example, often times there will be an update from a source but it is missing multiple countries we already have data for. In this case, you can use the blending tool to blend in any historical country values that are not in the new data. The same can be done for missing years in the new data.  

Once you have thoroughly vetted a data series, you will want to blend the new and historical series together. This ensures that we do not lose any values from the historical series. To do this for the Country values, you will want to check the '''Select All '''box in the bottom right hand corner of the screen. This will automatically check all of the countries, as seen below. Once you have check this box, press the '''Blend Data Points '''button, and click '''Yes. '''This will update the new Access file with missing Country observations.

You will also want to do the same process for the '''Blend Columns (Years) '''button, if there are any missing years. This ensures that we retain the historical values.

[[File:|624x351px|A screenshot of a social media post

Description automatically generated]]

 

File:2019-07-25 (1).png

2019-07-25T16:45:33Z

JakeDubbert:

Vetting data

2019-07-25T16:44:43Z

JakeDubbert:

'''Vetting Data'''

Vetting is the process of checking to assure the quality of the data we import into IFs. Even when data is imported through an automated application there could be errors or missing data in the imported series. Vetting the new data you are bringing into IFs requires you to compare the new series to the existing historical series in IFs.

There are a number of reasons for vetting data. One reason is that a country (or a number of countries) could be missed when importing due to not properly concording countries with the proper country concordance table. Make sure the IFs country concordance list (i.e., the relevant column in the '''Country '''Translation table in IFs.mdb) is correct and up to date and you are using the correct concordance list as listed in the DataDict. Additionally, make sure there are no missing years in the new data. Sometimes the year column is missed because the data is formatted as text and not as numbers, or the source simply did not provide an update for that year.  If this is the case, be sure to blend in the missing years in the IFs vetting tool. One more general issue is around the units. Make sure you perform any unit conversion required before you import the data. Comparing the new data with the current IFs series will give you an idea on any unit conversion that might be needed.

 

'''Vetting Data''' '''Checklist:'''

*Data in Access file and source Excel file match
*Use IFs vetting tool to compare new data to historical data
*Large discrepancies between new and old data are documented
*Check to see if there are big spikes in the new, imported data
*Blend countries/years with missing data in IFs vetting tool
*Zeroes in Access file are actually zeroes and not null values
*No missing countries or years
*DataDict contains no errors, Original Source has website name and Name in Source contains variable source
*Initials added to both Access and Excel file names, as well as in the DataDict notes
*Send Word file with vetting notes, along with original files to puller and project leads

'''Process:'''

#After the data series are imported into IFs, the IFsDataImport.mdb file is passed to another RA (assigned as a vetter) who brings the data from the IFsDataImport.mdb file into the model via the "Vet Imported Data" feature in the Extended Features menu
#The vetter uses the vetting tool to determine if there are any remarkable inconsistencies between the old and new data. This is a bit subjective, but necessarily so as the threshold for concern will be different depending on which series from which source is being vetted
#If significant errors are found that need to be corrected, and these errors had occurred during the import process, these are documented in the RA's vetting notes and the files are passed back to the original data puller to correct and re-send. 
#If no errors are found, the vetter blends columns for new years and preserves historical data points as appropriate, and saves the changes in the IFsDataImport file. 
#The vetter passes the IFsDataImport file (renamed to reflect series/batch update name and completion date) back to the project lead and data team supervisor
#Data team supervisor stores data import file for consolidation process

 

'''You can ensure data quality by following certain procedures''':

#For an updated table, open the existing and updated table side by side. Compare data points for all years and all countries to make sure there are no, or very small, differences.
#For all series, be sure to check large and important countries like USA, China or Germany.
#Check countries with similar name (like the two Congos and two Koreas), as these can sometimes get mixed up.  
#Check for zeros and make sure they are actually zeros in the source data. We take no data as an empty cell. If we have zeros that need to be data and must be a feasible value (e.g., GDP cannot be 0).
#Check the variable definition and make sure it makes sense and is in fact the right definition.
#Make sure percentages are below 100 (there are cases when percentages can be above 100, e.g., gross enrollment rate).
#Check values: GDP growth rate of more than 10% should raise a flag. For instances like this, check against the source data.
#Creating line graphs for the countries in a series is a great way to quickly check for transients. This can easily be done in Excel or Tableau.

 

'''Using the IFs Vetting Tool'''

IFs has a feature to do some basic initial vetting and is a tool that should be used in every vetting process. The vetting tool can be found by the following:

'''IFs -> MainMenu->Extended Features -> Manage Country Data->Historic Data File Processing-> Vet Imported Data'''.

 

The Compare Imported Table With Existing Table screen is then opened.

[[File:|624x351px|A screenshot of a social media post

Description automatically generated]]

Here, go to '''File>Open Database with Imported Tables, '''and open the Access file with the data series that you want to import into IFs. This Access file contains data series tables and a dictionary table with a list of new and/or updated data variables. This will then populate the top grid with every series within that Access file.

[[File:|624x351px|A screenshot of a cell phone

Description automatically generated]]

Next, double click on one of the series (rows) in the top grid to display the data. This will then populate the new data (the data you are importing) and the old, historical data that is in IFs. The old data is in the first grid and the new data is in the second grid, as shown above.

Here you have a lot of different options to compare the new data with the old. The vetting tool will automatically mark any zeros in the table. You will want to double check to see if, in fact, these are suppose to be zeros or if they are just null values. If they are just null values, you can click the button, '''Delete Zeros.'''

Next, you need to click the button, '''Mark Differences between New and Old Data for same country-year. '''This will allow you to easily observe any differences, as the vetting tool will highlight them. You can also change the threshold that it will mark. The default is that it will mark any more than 10% difference. As you can see in the screenshot below, the values for Afghanistan in 2014 and 2015 are marked. You will want to go through all of the countries in each series you are importing to look at these marked differences and write down any that are very large.

[[File:|624x351px|A screenshot of a social media post

Description automatically generated]]

You can also click the button '''Mark''' '''Year to Year Jumps in New Data, '''which is helpful to observe any large changes year-to-year.

'''Blending'''

This vetting tool is also used to merge country and year columns between updated and existing Access tables. This is a necessity when blending to preserve the existing historical IFs values when there is no corresponding value in the new data series. For example, often times there will be an update from a source but it is missing multiple countries we already have data for. In this case, you can use the blending tool to blend in any historical country values that are not in the new data. The same can be done for missing years in the new data.  

Once you have thoroughly vetted a data series, you will want to blend the new and historical series together. This ensures that we do not lose any values from the historical series. To do this for the Country values, you will want to check the '''Select All '''box in the bottom right hand corner of the screen. This will automatically check all of the countries, as seen below. Once you have check this box, press the '''Blend Data Points '''button, and click '''Yes. '''This will update the new Access file with missing Country observations.

You will also want to do the same process for the '''Blend Columns (Years) '''button, if there are any missing years. This ensures that we retain the historical values.

[[File:|624x351px|A screenshot of a social media post

Description automatically generated]]

 