IFs Base comparison: Difference between revisions

From Pardee Wiki
Jump to navigation Jump to search
No edit summary
m (Ifsdatateam moved page IFs Base comparson to IFs Base comparison)
 
(9 intermediate revisions by the same user not shown)
Line 15: Line 15:
'''IFs Base Comparison'''
'''IFs Base Comparison'''


#'''Things to do for IFs base comparison'''<ol style="list-style-type:lower-alpha;">
'''Things to do for IFs base comparison'''
<li>See [https://pardee.du.edu/wiki/Data_listed_2015_and_2100_08142020.xlsx Data listed 2015 and 2100]&nbsp;as an example.<ol style="list-style-type:lower-roman;">
<li>In that excel file you can see tabs for variables and the world.
##For all variable world data is generated for 2015 and 2100. The measure used is ratio measure to compare old and new base. This allows us to see which variables deviate most in 2015 and 2100 relative to old base.
##Then you can see few examples of&nbsp;important variables (this is done for all important variables). Each variable has all countries and 2015 & 2100 raw data. Then we can calculate different ratio from here.
##The final sheet is cumulation of all other individual variable sheets.</li>
</li>


##Check all series and their differences between old and new data. This is very important to see impacts during initialization data.
#See [[Example_on_a_report_for_initial_and_last_year|Data listed 2015 and 2100]]&nbsp;an example.
##Start with important variables such as: GDP, POP, EDYRSAG15. See long term model impacts in these variables.
##In that excel file you can see tabs for variables and the world.
##It is also very important to see differences in initialization year. This might explain the long-term impact. In the end, we are trying to find what changes in data causing such model behavior.<ol style="list-style-type:lower-roman;">
###For all variable world data is generated for 2015 and 2100. The measure used is ratio measure to compare old and new base. This allows us to see which variables deviate most in 2015 and 2100 relative to old base.
<li>If data are vastly different during initialization year, check the data source and find the cause of those data difference.</li>
###&nbsp;Then you can see few examples of&nbsp;important variables (this is done for all important variables). Each variable has all countries and 2015 & 2100 raw data. Then we can calculate different ratio from here.
</ol><li>Check drivers. Drivers for a variable can be accessed in IFs. We have built a driver list for MFP and some other variables. Future RAs are expected to build more on this as they work. See “Drivers” folder to see more examples.</li>
###The final sheet is cumulation of all other individual variable sheets.
<li>See example of base comparison analysis.<ol style="list-style-type:lower-roman;">
#Check all series and their differences between old and new data. This is very important to see impacts during initialization data.
<li>&nbsp;See [https://pardee.du.edu/wiki/Base_comparison_analysis_write_up_examples Base comparison analysis write up examples]</li>
#Start with important variables such as: GDP, POP, EDYRSAG15. See long term model impacts in these variables.
<li>See Model Vetting Note Link examples
#It is also very important to see differences in initialization year. This might explain the long-term impact. In the end, we are trying to find what changes in data causing such model behavior.
##
##If data are vastly different during initialization year, check the data source and find the cause of those data difference.
[https://mailtrack.io/trace/link/8ecdf57be2914554303e305fc2167923555e17b8?url=https://pardee.du.edu/wiki/December_2016_vetting&userId=1065818&signature=2784652d1063774b https://pardee.du.edu/wiki/December_2016_vetting]
#Check drivers. Drivers for a variable can be accessed in IFs. We have built a driver list for MFP and some other variables. Future RAs are expected to build more on this as they work. See “Drivers” folder to see more examples.
 
#See example of base comparison analysis.
##
##&nbsp;See [https://pardee.du.edu/wiki/Base_comparison_analysis_write_up_examples Base comparison analysis write up examples]
&nbsp;[https://mailtrack.io/trace/link/f572d0971548a5acef2eda8d8296070ec0b39a71?url=https://pardee.du.edu/wiki/April_2016_consolidation&userId=1065818&signature=af961658468f2a28 https://pardee.du.edu/wiki/April_2016_consolidation]&nbsp;
##See Model Vetting Note Link examples​
 
###[https://pardee.du.edu/wiki/December_2016_vetting https://pardee.du.edu/wiki/December_2016_vetting]
##
###[https://pardee.du.edu/wiki/April_2016_consolidation https://pardee.du.edu/wiki/April_2016_consolidation]
&nbsp;[https://mailtrack.io/trace/link/cbadaac9ef9abf3b74b284276430d1704b96860a?url=https://pardee.du.edu/wiki/May_2016_consolidation&userId=1065818&signature=4f8ee6a232a4d2b1 https://pardee.du.edu/wiki/May_2016_consolidation]&nbsp;
###&nbsp;[https://pardee.du.edu/wiki/May_2016_consolidation https://pardee.du.edu/wiki/May_2016_consolidation]
 
###&nbsp;[https://pardee.du.edu/wiki/June_2016_consolidation https://pardee.du.edu/wiki/June_2016_consolidation]
##
#Some other tips for base comparison.
&nbsp;[https://mailtrack.io/trace/link/808e14edc4f1a53674dc263da3a81cfc422fb66a?url=https://pardee.du.edu/wiki/June_2016_consolidation&userId=1065818&signature=bb49e5031a19ae27 https://pardee.du.edu/wiki/June_2016_consolidation]
##On variables that go near 0 or change signs, such as MFP, ratios are not really useful, but differences are.&nbsp;So, check the raw values before doing ratio comparison.
</li>
##As you work on this make a list of other tips that you think is beneficial for future RAs. You can add those here once you are done with this work.
</li>
 
##Some other tips for base comparison.<ol style="list-style-type:lower-roman;">
<li>On variables that go near 0 or change signs, such as MFP, ratios are not really useful, but differences are.&nbsp;So, check the raw values before doing ratio comparison.</li>
<li>As you work on this make a list of other tips that you think is beneficial for future RAs. You can add those here once you are done with this work.</li>
</ol></ol></ol>
 
&nbsp;


Note: This guide is supposed to be a help book for RAs. By any means, it is not a comprehensive list of things to do as different base comparisons have their own uniqueness and should be consulted with your supervisor about what direction should you embrace.
Note: This guide is supposed to be a help book for RAs. By any means, it is not a comprehensive list of things to do as different base comparisons have their own uniqueness and should be consulted with your supervisor about what direction should you embrace.

Latest revision as of 19:21, 25 August 2020

This IFs Base Comparison process relates to the consolidation of big data update. It is usually conducted once a year during the big data update.

Top-down approach:
Top down approach implies using all new updated data to build a new base and comparing with old/existing base. After few observations, we can replace few new series by old series to see the impact of addition of those new series. This page follows this approach.

Bottom-up approach:
It is the opposite of top-down approach. This allows us to add one new series at a time and observe its impact.

Things to do first before IFs Base Comparison:

  1. Rebuilding Base
    1. See Rebuild Base Notes for this specific task
    2. See more on rebuilding the base here
  2. Variable names
    1. Variable names to compare model are in page Variable names

IFs Base Comparison

Things to do for IFs base comparison

  1. See Data listed 2015 and 2100 an example.
    1. In that excel file you can see tabs for variables and the world.
      1. For all variable world data is generated for 2015 and 2100. The measure used is ratio measure to compare old and new base. This allows us to see which variables deviate most in 2015 and 2100 relative to old base.
      2.  Then you can see few examples of important variables (this is done for all important variables). Each variable has all countries and 2015 & 2100 raw data. Then we can calculate different ratio from here.
      3. The final sheet is cumulation of all other individual variable sheets.
  2. Check all series and their differences between old and new data. This is very important to see impacts during initialization data.
  3. Start with important variables such as: GDP, POP, EDYRSAG15. See long term model impacts in these variables.
  4. It is also very important to see differences in initialization year. This might explain the long-term impact. In the end, we are trying to find what changes in data causing such model behavior.
    1. If data are vastly different during initialization year, check the data source and find the cause of those data difference.
  5. Check drivers. Drivers for a variable can be accessed in IFs. We have built a driver list for MFP and some other variables. Future RAs are expected to build more on this as they work. See “Drivers” folder to see more examples.
  6. See example of base comparison analysis.
    1.  See Base comparison analysis write up examples
    2. See Model Vetting Note Link examples​
      1. https://pardee.du.edu/wiki/December_2016_vetting
      2. https://pardee.du.edu/wiki/April_2016_consolidation
      3.  https://pardee.du.edu/wiki/May_2016_consolidation
      4.  https://pardee.du.edu/wiki/June_2016_consolidation
  7. Some other tips for base comparison.
    1. On variables that go near 0 or change signs, such as MFP, ratios are not really useful, but differences are. So, check the raw values before doing ratio comparison.
    2. As you work on this make a list of other tips that you think is beneficial for future RAs. You can add those here once you are done with this work.

Note: This guide is supposed to be a help book for RAs. By any means, it is not a comprehensive list of things to do as different base comparisons have their own uniqueness and should be consulted with your supervisor about what direction should you embrace.



This page was updated on 08/24/2020.