May 2016 consolidation: Difference between revisions

From Pardee Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(25 intermediate revisions by the same user not shown)
Line 147: Line 147:
| height="20" | SeriesWaterWithdIndustrial
| height="20" | SeriesWaterWithdIndustrial
|-
|-
| height="20" | SeriesWaterWithdMunicipal
| height="20" | SeriesWaterWithdMunicipal<br/>
|}
|}
= Code changes (from Steve) =
Both MALNPOPP and MALNCHP are initialized in DataAgri and then saved to memory:
<div>
<span style="color: rgb(84, 130, 53);"><font size="3" face="calibri">'dsr 2015/11/06 Deal with calories, protein, and undernutrition in memory</font></span>
<span style="color: rgb(84, 130, 53);"><font size="3" face="calibri">' could do for all other variables</font></span>
<span style="color: black;"><font face="calibri"><span><font size="3">&nbsp;&nbsp;&nbsp;</font></span><font size="3">Call RegAvgOneDimInMem(CMALNCHP, CPop(), MALNCHP(), "CMALNCHP")</font></font></span>
<span style="color: black;"><font face="calibri"><span><font size="3">&nbsp;&nbsp;&nbsp;</font></span><font size="3">Call RegAvgOneDimInMem(CMALNPOPP, CPop(), MALNPOPP(), "CMALNPOPP")</font></font></span>
</div><div><br/></div><div>In the RebuildBase form, MALNCHP is then put into PopulationBase (in IFs.mdb) from memory:</div><div>&nbsp;&nbsp;&nbsp; Call BasePutOneDimFromMemoryGrl("MALNCHP", RCount, MALNCHP)</div><div><br/></div><div>But MALNPOPP is not put in from memory:</div><div>&nbsp;&nbsp;&nbsp; Call BasePutOneDim("MALNPOPP", RCount)</div><div><br/></div><div>So then the model reads from PopulationBase whatever numbers are there for MALNPOPP, but they are not the ones read into memory in DataAgri.bas.</div><div><br/></div><div>I changed the line in RebuildBase for MALNPOPP to:</div><div>Call BasePutOneDimFromMemoryGrl("MALNPOPP", RCount, MALNPOPP)</div><div><br/></div><div>So that it reads it FROM memory. That seems to work and the values are initialized correctly.<br/></div><div><br/></div>
= Changes to DataDict =
We have added a new field in the datadict titled, "UsedInPreprocessorFileName". If the series is used in the preprocessor, this field is filled (through Jose's code) with the filename of where it is read.


= Affected modules =
= Affected modules =
== Hunger ==
Global hunger is now initialized at 867.4491 (million people) in 2014. It was initializing at 946.7728. The countries with the largest reductions in hunger are China, Indonesia, India, DRC, and Ethiopia. The country with the largest increase in hunger is Nigeria.


== Infrastructure ==
== Infrastructure ==
Line 162: Line 182:
Global access to electricity is now initialized at 85.3345%&nbsp;in 2015 rather than 84.2006%. The countries with the largest relative increase in access to electricity are Comoros, Somalia, Sierra Leone, Guinea-Bissau, and Rwanda. This is because there was previously either very old data for these countries or no data at all. The largest relative decreases occur in South Sudan, Seychelles, and St. Vincent and the Grenadines, for the same reason.
Global access to electricity is now initialized at 85.3345%&nbsp;in 2015 rather than 84.2006%. The countries with the largest relative increase in access to electricity are Comoros, Somalia, Sierra Leone, Guinea-Bissau, and Rwanda. This is because there was previously either very old data for these countries or no data at all. The largest relative decreases occur in South Sudan, Seychelles, and St. Vincent and the Grenadines, for the same reason.


''<span>'''Note: '''This variable is used to fill holes for the portion of the population using solid fuels (CENSOLFUEL in DataInfra.bas and DataEnv.bas). That equation: '''"Percent of People using Primarily Solid Fuels (Full Model 2010)"''' was last updated on 12/18/2013 at 7:26:28 PM and should be reestimated with this new data. It also uses GDPPCP and PopUrban% as independent variables.</span>''
'''''Note:'''This variable is used to fill holes for the portion of the population using solid fuels (CENSOLFUEL in DataInfra.bas and DataEnv.bas). That equation: '''"Percent of People using Primarily Solid Fuels (Full Model 2010)"'''was last updated on 12/18/2013 at 7:26:28 PM and should be reestimated with this new data. It also uses GDPPCP and PopUrban% as independent variables.''


== <span>Health</span> ==
== <span>Health</span> ==


=== <span></span>Diabetes ===
=== Diabetes ===
 
The&nbsp; countries with the biggest absolute increases in DALYs from diabetes are: India, South Africa, Kenya, Tanzania, and Uganda. The countries with the largest absolute decreases in DALYs are Indonesia, China, and South Sudan. The countries with the largest relative increases are Lesotho, Zambia, and Kenya. The countries with the largest relative decreases are South Sudan, Sao Tome and Principe and the Bahamas.
 
'''''Note:'''If&nbsp;impaired glucose intolerance (HelathIGTPrev%) is null it is estimated using the following equation: '''"GDP/Capita (PPP 2000) Versus Impaired Glucose Tolerance (2003) Log"'''. This needs to be updated - unknown when it was last updated.''
 
''The same goes for filling holes in diabetes prevalence in the preprocessor. Holes are filled using: '''"GDP/Capita (PPP 2000) Versus Diabetes Prevalence (2003) Log"'''. Unknown when this was last updated.''
 
''Also, these are initialized as CHLDIABIGT(ICount%) and CHLDIABPREV(ICount%) but I do not believe the data is for children. This needs to be investigated.''
 
=== HIV ===
 
Global HIV prevalence is initialized higher after this data update. Global HIV prevalence is initialized at .472% in 2014 - it was initialized at .458% in 2014 before this data update. HIV prevalence in Africa is now initialized (2014) at 2.226% rather than 1.987% before this update.
 
The largest increases are in Botswana, Namibia, Zimbabwe, Uganda, and Swaziland. The largest decreases are in Kenya, CAR, and Sudan.
 
'''''Note:''' HIV prevalence is heavily influenced by peak year, peak prevalence,&nbsp;and an initial growth rate (CHIVINCR(ICount%)). This growth rate is initialized as&nbsp;the&nbsp;square&nbsp;root of the&nbsp;ratio&nbsp;of 2006 data to 2004 data: CHIVINCR(ICount%) = (CHIVRATE06 / CHIVRATE04) ^ (1 / 2) - 1. We can update this using more recent data.''
 
''Also, the peak year and peak prevalence data are read from SeriesHIVPeaks which was updated in 2013. This table has both the year of peak and the prevalence of HIV in that year. BUT, the code is reading the columns "Peak Year" and "Peak Prevalence", which do not necessarily align with the most recent data.''
 
== Water ==
 
=== Municipal demand ===
 
Global municipal water demand is decreased from 477.54 km3&nbsp;to 462.45 km3 in 2014. The largest increases are in Egypt, Algeria, Brazil, Australia, and Kenya.
 
=== Industrial demand ===
 
Global industrial water demand increases from 742.3 km3 to 770.1 km3 in 2014. The largest increase is in the US where industrial water demand is now initialized at 248.4 km3 and previously it was 220.6 km3.
 
=== Agricultural demand ===


The&nbsp; countries with the biggest absolute increases in DALYs from diabetes are: India, South Africa, Kenya, Tanzania, and Uganda. The countries with the largest absolute decreases in DALYs are Indonesia, China, and South Sudan. The countries with the largest relative increases
Global ag water demand increased from 2,706 km3 to 2,729 km3 in 2014. The largest increases are in Brazil, Egypt, Chile, Argentina, and Australia. These countries have all been increasing agricultural water withdrawals, according to the AQUASTAT data.


= Bugs and issues =
= Bugs and issues =


*The "infrastructure overview" category produces a "Run -ime error 3021: No current record" error when clicked
== Previous Issues ==
 
*All fish dimensions seem to be missing formulas when trying to view "history and forecast"
**Still seems to be a problem for "Agricultural production, history and forecast - million metric tons"
*Many agricultural series have a gap in the data because the food balance sheets end in 2011/12&nbsp;and the new base year is 2014.
*There is no historical analog (for Indonesia) for "population per hectare of crop land (urban)"
**Still an issue
*Transient in 1998 in GDP (both MER and PPP) for Indonesia
*No historical analog for VADD of energy and materials for Indonesia
*VADD for Indonesia for all sectors jumps around
*Transient in initial year of HDI(new) for Indonesia
*Transient in initial year of MALNPOPP for Indonesia
**Addressed above
*Transient in initial year of MalnPop for Indonesia
**Addressed above
*Transient in initial year of GINIDOM&nbsp;for Indonesia
*No historical analog for government consumption by destination as a percent of GDP for InfraOther for Indonesia
*No historical analog for GovRev%GDP for Indonesia
*Transient in initial year of INCOMELT200LN2005 for Indonesia
*Transients in inital year of primary intake in Indonesia
*Gaps in ENDEM because data not up-to-date
*No historical analog for reserve to production ratio for Indonesia for hydro
*ICT mobile broadband initalized at zero
**<span><span style="background-color: rgb(255, 255, 255);">Seems to be fixed</span></span>
*No historical analog for ICTCONSURPLUS for Indonesia
*Transients for intialized values of HIVRATE, AIDS death rate, and AIDSDTHS for Indonesia
*"Invalid dimension 2 parameters" for "HD Multivariate Report with Land" for Indonesia
 
== New Issues ==
 
*The "infrastructure overview" category produces a "Run-time error 3021: No current record" error when clicked
*Equation used to estimate solid fuel use should be updated with new electricity access data (see above)
*Diabetes and IGT hole filling equations should be updated with new data (see above)
*HIV issues raised above
 
=== Issues raised during Indonesia training (from Dave) ===
 
*All source names in DataDict need to be more explicit
*Some links in Block Diagram are broken
*Update corruption to use new CPI (chain and/or re-specify)
*Selecting Display Categories: Infrastructure Assumptions causes failure.
*GOVSEC has some counter-intuitive initializations (probably estimation issue for missing data)
*Radial Graph does not advance and does not properly show more than 4 counties (was using security, capacity, inclusions) — changing number of variables did not help.
*Agricultural Investment doesn’t impact Crop Land
*ENP only seems to initialize with EnProdGeothermalIEA
*Indonesia less than $2 per day is higher than $3.10 historically (IncBelow2DWDI2011 and IncBelow3DWDI). Large transient with INCLT200LN2005
*HDI transient in 2014
*Malnourished population transient in 2014
*Category Infrastructure Overview and Infrastructure Something break the model if you select display lists within them (maybe just certain display lists?)
*Takes hours to break out into subregions
*Window’s regionalization (not just language) must be set to USA or else the model encounters a number of issues with the GUI (I believe this is related to how the computer reads and displays dates.
*GDPGR transients after it goes endogenous
*Drop the number 2 in the Severe Acute Malnutrition Display Category
*WATSAFE Piped and OthImproved forecast trend is quite different from data (for Indonesia)
*Display variable "FDI Inflows as Percent of GDP, Hist & Fore” doesn't show a historical analog&nbsp;
*"Health&nbsp;costs" in specialized displays spelt incorrectly
*WORKAGERETIREND is initalized at 70 rather than 65

Latest revision as of 23:27, 3 June 2016

New data

SeriesEnElecTotalCapacityEIA
SeriesEnElecAccess%National
SeriesEnElecAccess%Rural
SeriesEnElecAccess%Urban
SeriesHealthDiabetesPrev%
SeriesHealthIGTPrev%
  • Health - HIV - UNAIDS - 6 series, all preprocessor - pulled by Joel, vetted by Brandon
SeriesHealthUNAIDSDeathsHighEst
SeriesHealthUNAIDSDeathsLowEst
SeriesHealthUNAIDSDeathsMidEst
SeriesHealthUNAIDSTotalHIVHighEst
SeriesHealthUNAIDSTotalHIVLowEst
SeriesHealthUNAIDSTotalHIVMidEst
  • Water - AQUASTAT - 50 series, many preprocessor - pulled by Kristen, vetted by Joel
SeriesDesalinatedWater
SeriesIrrigatedCropIntensity
SeriesIrWaterReq
SeriesIrWaterWith
SeriesLandCultivatedArea
SeriesLandEquipIrActual
SeriesLandEquipIrFullControl
SeriesLandEquipIrFullControlActual
SeriesLandIr%Grain
SeriesLandIrAreaSalinized
SeriesLandIrEquipDrained
SeriesLandIrEquipGround
SeriesLandIrHarvestedCropArea
SeriesLandIrWaterLogged
SeriesTotalDamCapacity
SeriesWasterwaterTreated
SeriesWasteWaterDirectNotTreated
SeriesWastewaterIrDirectTreated
SeriesWasteWaterLandEquipDirectNotTreated
SeriesWasteWaterLandEquipDirectTreated
SeriesWastewaterProduced
SeriesWastewaterTreatedReused
SeriesWaterDependencyRatio
SeriesWaterDesalinated
SeriesWaterGroundEntering
SeriesWaterGroundLeaving
SeriesWaterGroundProdInternal
SeriesWaterGroundTotal
SeriesWaterGroundWithD
SeriesWaterResExploitGround
SeriesWaterResExploitSurface
SeriesWaterResOverlap
SeriesWaterResTotalExploit
SeriesWaterResTotalRenew
SeriesWaterResTotalRenewGround
SeriesWaterResTotalRenewSurface
SeriesWaterSurfaceWithD
SeriesWaterTotalRenewPC
SeriesWaterTotalWithd
SeriesWaterTotalWithdPC
SeriesWaterTotalWithdSector
SeriesWaterTotalWithdSources
SeriesWaterWith%Agric
SeriesWaterWith%Fresh
SeriesWaterWith%Household
SeriesWaterWith%Ind
SeriesWaterWithAgr%FreshAquastat
SeriesWaterWithdAgriculture
SeriesWaterWithdIndustrial
SeriesWaterWithdMunicipal

Code changes (from Steve)

Both MALNPOPP and MALNCHP are initialized in DataAgri and then saved to memory:

'dsr 2015/11/06 Deal with calories, protein, and undernutrition in memory

' could do for all other variables

   Call RegAvgOneDimInMem(CMALNCHP, CPop(), MALNCHP(), "CMALNCHP")

   Call RegAvgOneDimInMem(CMALNPOPP, CPop(), MALNPOPP(), "CMALNPOPP")


In the RebuildBase form, MALNCHP is then put into PopulationBase (in IFs.mdb) from memory:
    Call BasePutOneDimFromMemoryGrl("MALNCHP", RCount, MALNCHP)

But MALNPOPP is not put in from memory:
    Call BasePutOneDim("MALNPOPP", RCount)

So then the model reads from PopulationBase whatever numbers are there for MALNPOPP, but they are not the ones read into memory in DataAgri.bas.

I changed the line in RebuildBase for MALNPOPP to:
Call BasePutOneDimFromMemoryGrl("MALNPOPP", RCount, MALNPOPP)

So that it reads it FROM memory. That seems to work and the values are initialized correctly.

Changes to DataDict

We have added a new field in the datadict titled, "UsedInPreprocessorFileName". If the series is used in the preprocessor, this field is filled (through Jose's code) with the filename of where it is read.

Affected modules

Hunger

Global hunger is now initialized at 867.4491 (million people) in 2014. It was initializing at 946.7728. The countries with the largest reductions in hunger are China, Indonesia, India, DRC, and Ethiopia. The country with the largest increase in hunger is Nigeria.

Infrastructure

Electricity capacity

The countries with the largest relative changes were mostly small island countries because previously we had little or no data for these countries. The countries with the largest relative increases in electrical capacity are Equatorial Guinea, Comoros, and Benin. The largest relative decreases occur in Timor-Leste and Seychelles.

Electricity access

Global access to electricity is now initialized at 85.3345% in 2015 rather than 84.2006%. The countries with the largest relative increase in access to electricity are Comoros, Somalia, Sierra Leone, Guinea-Bissau, and Rwanda. This is because there was previously either very old data for these countries or no data at all. The largest relative decreases occur in South Sudan, Seychelles, and St. Vincent and the Grenadines, for the same reason.

Note:This variable is used to fill holes for the portion of the population using solid fuels (CENSOLFUEL in DataInfra.bas and DataEnv.bas). That equation: "Percent of People using Primarily Solid Fuels (Full Model 2010)"was last updated on 12/18/2013 at 7:26:28 PM and should be reestimated with this new data. It also uses GDPPCP and PopUrban% as independent variables.

Health

Diabetes

The  countries with the biggest absolute increases in DALYs from diabetes are: India, South Africa, Kenya, Tanzania, and Uganda. The countries with the largest absolute decreases in DALYs are Indonesia, China, and South Sudan. The countries with the largest relative increases are Lesotho, Zambia, and Kenya. The countries with the largest relative decreases are South Sudan, Sao Tome and Principe and the Bahamas.

Note:If impaired glucose intolerance (HelathIGTPrev%) is null it is estimated using the following equation: "GDP/Capita (PPP 2000) Versus Impaired Glucose Tolerance (2003) Log". This needs to be updated - unknown when it was last updated.

The same goes for filling holes in diabetes prevalence in the preprocessor. Holes are filled using: "GDP/Capita (PPP 2000) Versus Diabetes Prevalence (2003) Log". Unknown when this was last updated.

Also, these are initialized as CHLDIABIGT(ICount%) and CHLDIABPREV(ICount%) but I do not believe the data is for children. This needs to be investigated.

HIV

Global HIV prevalence is initialized higher after this data update. Global HIV prevalence is initialized at .472% in 2014 - it was initialized at .458% in 2014 before this data update. HIV prevalence in Africa is now initialized (2014) at 2.226% rather than 1.987% before this update.

The largest increases are in Botswana, Namibia, Zimbabwe, Uganda, and Swaziland. The largest decreases are in Kenya, CAR, and Sudan.

Note: HIV prevalence is heavily influenced by peak year, peak prevalence, and an initial growth rate (CHIVINCR(ICount%)). This growth rate is initialized as the square root of the ratio of 2006 data to 2004 data: CHIVINCR(ICount%) = (CHIVRATE06 / CHIVRATE04) ^ (1 / 2) - 1. We can update this using more recent data.

Also, the peak year and peak prevalence data are read from SeriesHIVPeaks which was updated in 2013. This table has both the year of peak and the prevalence of HIV in that year. BUT, the code is reading the columns "Peak Year" and "Peak Prevalence", which do not necessarily align with the most recent data.

Water

Municipal demand

Global municipal water demand is decreased from 477.54 km3 to 462.45 km3 in 2014. The largest increases are in Egypt, Algeria, Brazil, Australia, and Kenya.

Industrial demand

Global industrial water demand increases from 742.3 km3 to 770.1 km3 in 2014. The largest increase is in the US where industrial water demand is now initialized at 248.4 km3 and previously it was 220.6 km3.

Agricultural demand

Global ag water demand increased from 2,706 km3 to 2,729 km3 in 2014. The largest increases are in Brazil, Egypt, Chile, Argentina, and Australia. These countries have all been increasing agricultural water withdrawals, according to the AQUASTAT data.

Bugs and issues

Previous Issues

  • All fish dimensions seem to be missing formulas when trying to view "history and forecast"
    • Still seems to be a problem for "Agricultural production, history and forecast - million metric tons"
  • Many agricultural series have a gap in the data because the food balance sheets end in 2011/12 and the new base year is 2014.
  • There is no historical analog (for Indonesia) for "population per hectare of crop land (urban)"
    • Still an issue
  • Transient in 1998 in GDP (both MER and PPP) for Indonesia
  • No historical analog for VADD of energy and materials for Indonesia
  • VADD for Indonesia for all sectors jumps around
  • Transient in initial year of HDI(new) for Indonesia
  • Transient in initial year of MALNPOPP for Indonesia
    • Addressed above
  • Transient in initial year of MalnPop for Indonesia
    • Addressed above
  • Transient in initial year of GINIDOM for Indonesia
  • No historical analog for government consumption by destination as a percent of GDP for InfraOther for Indonesia
  • No historical analog for GovRev%GDP for Indonesia
  • Transient in initial year of INCOMELT200LN2005 for Indonesia
  • Transients in inital year of primary intake in Indonesia
  • Gaps in ENDEM because data not up-to-date
  • No historical analog for reserve to production ratio for Indonesia for hydro
  • ICT mobile broadband initalized at zero
    • Seems to be fixed
  • No historical analog for ICTCONSURPLUS for Indonesia
  • Transients for intialized values of HIVRATE, AIDS death rate, and AIDSDTHS for Indonesia
  • "Invalid dimension 2 parameters" for "HD Multivariate Report with Land" for Indonesia

New Issues

  • The "infrastructure overview" category produces a "Run-time error 3021: No current record" error when clicked
  • Equation used to estimate solid fuel use should be updated with new electricity access data (see above)
  • Diabetes and IGT hole filling equations should be updated with new data (see above)
  • HIV issues raised above

Issues raised during Indonesia training (from Dave)

  • All source names in DataDict need to be more explicit
  • Some links in Block Diagram are broken
  • Update corruption to use new CPI (chain and/or re-specify)
  • Selecting Display Categories: Infrastructure Assumptions causes failure.
  • GOVSEC has some counter-intuitive initializations (probably estimation issue for missing data)
  • Radial Graph does not advance and does not properly show more than 4 counties (was using security, capacity, inclusions) — changing number of variables did not help.
  • Agricultural Investment doesn’t impact Crop Land
  • ENP only seems to initialize with EnProdGeothermalIEA
  • Indonesia less than $2 per day is higher than $3.10 historically (IncBelow2DWDI2011 and IncBelow3DWDI). Large transient with INCLT200LN2005
  • HDI transient in 2014
  • Malnourished population transient in 2014
  • Category Infrastructure Overview and Infrastructure Something break the model if you select display lists within them (maybe just certain display lists?)
  • Takes hours to break out into subregions
  • Window’s regionalization (not just language) must be set to USA or else the model encounters a number of issues with the GUI (I believe this is related to how the computer reads and displays dates.
  • GDPGR transients after it goes endogenous
  • Drop the number 2 in the Severe Acute Malnutrition Display Category
  • WATSAFE Piped and OthImproved forecast trend is quite different from data (for Indonesia)
  • Display variable "FDI Inflows as Percent of GDP, Hist & Fore” doesn't show a historical analog 
  • "Health costs" in specialized displays spelt incorrectly
  • WORKAGERETIREND is initalized at 70 rather than 65