Data (New): Difference between revisions

From Pardee Wiki
Jump to navigation Jump to search
mNo edit summary
m (Yutang.Xiong moved page Date (New) to Data (New) without leaving a redirect)
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
== '''Database Files in IFs''' ==
== Database Files in IFs ==


=== Historical Databases ===
==== '''Historical Databases''' ====
The International Futures (IFs) modeling system relies on two major large historical databases: '''IFsHistSeries''' and '''IFsHistSeriesDyadic'''.
The International Futures (IFs) modeling system relies on two major large historical databases: ''IFsHistSeries.db'' and ''IFsHistSeriesDyadic.db''.


* '''IFsHistSeries''' contains data for individual countries and regions (''monadic data'').
* ''IFsHistSeries'' contains data for individual countries and regions (''monadic data'').
* '''IFsHistSeriesDyadic''' holds data on relationships between country pairs (''bilateral data'').
* ''IFsHistSeriesDyadic'' holds data on relationships between country pairs (''bilateral data'').


Both databases store a vast collection of time-series data tables covering areas like population, education, health, agriculture, energy, economics, the environment, infrastructure, governance, and international politics. Hence, each data table is also referred to as a data series. Currently, these data series include '''188 entities (countries/regions)''', with most data series going back to '''1960'''.
Both databases store a vast collection of time-series data tables covering areas like population, education, health, agriculture, energy, economics, the environment, infrastructure, governance, and international politics. Hence, each data table is also referred to as a data series. Currently, these data series include 188 entities (countries/regions), with most data series going back to 1960.


The IFs model uses these data series in several ways:
The IFs model uses these data series in several ways:


# '''Initializing model variables'''—these series act as ''preprocessors'' that set the base-year values for the model.
# Initializing model variables—these series act as ''preprocessors'' that set the base-year values for the model.
# '''Building relationships between variables'''—helping IFs analyze connections between different variables, e.g., through a statistical model. These tables serve as ''Analytical Function'' Series.
# Building relationships between variables—helping IFs analyze connections between different variables, e.g., through a statistical model. These tables serve as ''Analytical Function'' Series.
# '''Displaying historical trends'''—providing users with visual insights into past patterns. These are also referred to as series for ''Historical Analog''.
# Displaying historical trends—providing users with visual insights into past patterns. These are also referred to as series for ''Historical Analog''.
# '''Incorporating external projections'''—allowing comparisons with forecasts from other organizations. Furthermore, some of these series can be made exogenous in IFs to change model projections.
# Incorporating external projections—allowing comparisons with forecasts from other organizations. Furthermore, some of these series can be made exogenous in IFs to change model projections.
# '''Helping users explore patterns and relationships'''—supporting research, analysis, and parameter estimation.
# Helping users explore patterns and relationships—supporting research, analysis, and parameter estimation.


For a deeper dive into the database structure, check out [a wiki link to IFsHistSeries & IFsHistSeriesDyadic].
For a deeper dive into the database structure, check out [a wiki link to IFsHistSeries & IFsHistSeriesDyadic].


=== Meta Data ===
==== Metadata ====
All meta data (e.g., definitions, units, data source) are stored separately under the data dictionary files (DataDict.db or DataDictDyadic.db). For each data series in the historical databases, we have a reference (one row) in the data dictionary. IFs users can access this information directly through the database files, or through the IFs interface by clicking the Data Information tab when available.
All metadata—such as definitions, units, and data sources—are stored separately in data dictionary files (''DataDict.db'' for monadic data and ''DataDictDyadic.db'' for dyadic data). Each data series in the historical databases is referenced by a corresponding entry in the data dictionary. Users can access metadata directly through the database files or via the ''Data Information'' tab in the IFs interface when available.


Detailed description on the structure of data dictionary files can be found here.
For more details on the data dictionary structure, click [a wiki link to DataDict].


=== Sing-Year Data and Parameters ===
==== Sing-Year Data and Parameters ====
In addition to historical databases. IFs sometimes relies on single-year data in its model initialization and projection. Single-year data are normally stored under the file IFs.db. Some model parameters and settings, such as country concordance tables, can also be found under the same file. You can find more information about tables in the IFs.db here.
Beyond historical databases, IFs also uses single-year data for model initialization and projections. These datasets are typically stored in ''IFs.db'', along with various model parameters and settings, including C''ountry Concordance'' tables.


== '''Data Collection & Update''' ==
More information on ''IFs.db'' tables can be found [here].
IFs databases are compiled, in most cases, from publicly available data sources, e.g., World Development Indicators, UNESCO Institute for Education database on education, FAO database on agricultural production, Correlates of War data on conflicts etc. When needed, new series will be added to the databases, while existing ones are updated by data sources on a regular basis.


Introductions on data collection can be traced using the list of data sources here.
== Data Collection & Update ==
IFs databases are compiled, in most cases, from publicly available data sources, e.g., World Development Indicators, UNESCO Institute for Education database on education, FAO database on agricultural production, Correlates of War data on conflicts etc. New data series are added as needed, and existing ones are regularly updated based on source revisions.
 
Details on data collection methods are available on each source's introduction page. Here is [[Data Sources|'''the list of data sources''']].

Latest revision as of 21:07, 20 February 2025

Database Files in IFs

Historical Databases

The International Futures (IFs) modeling system relies on two major large historical databases: IFsHistSeries.db and IFsHistSeriesDyadic.db.

  • IFsHistSeries contains data for individual countries and regions (monadic data).
  • IFsHistSeriesDyadic holds data on relationships between country pairs (bilateral data).

Both databases store a vast collection of time-series data tables covering areas like population, education, health, agriculture, energy, economics, the environment, infrastructure, governance, and international politics. Hence, each data table is also referred to as a data series. Currently, these data series include 188 entities (countries/regions), with most data series going back to 1960.

The IFs model uses these data series in several ways:

  1. Initializing model variables—these series act as preprocessors that set the base-year values for the model.
  2. Building relationships between variables—helping IFs analyze connections between different variables, e.g., through a statistical model. These tables serve as Analytical Function Series.
  3. Displaying historical trends—providing users with visual insights into past patterns. These are also referred to as series for Historical Analog.
  4. Incorporating external projections—allowing comparisons with forecasts from other organizations. Furthermore, some of these series can be made exogenous in IFs to change model projections.
  5. Helping users explore patterns and relationships—supporting research, analysis, and parameter estimation.

For a deeper dive into the database structure, check out [a wiki link to IFsHistSeries & IFsHistSeriesDyadic].

Metadata

All metadata—such as definitions, units, and data sources—are stored separately in data dictionary files (DataDict.db for monadic data and DataDictDyadic.db for dyadic data). Each data series in the historical databases is referenced by a corresponding entry in the data dictionary. Users can access metadata directly through the database files or via the Data Information tab in the IFs interface when available.

For more details on the data dictionary structure, click [a wiki link to DataDict].

Sing-Year Data and Parameters

Beyond historical databases, IFs also uses single-year data for model initialization and projections. These datasets are typically stored in IFs.db, along with various model parameters and settings, including Country Concordance tables.

More information on IFs.db tables can be found [here].

Data Collection & Update

IFs databases are compiled, in most cases, from publicly available data sources, e.g., World Development Indicators, UNESCO Institute for Education database on education, FAO database on agricultural production, Correlates of War data on conflicts etc. New data series are added as needed, and existing ones are regularly updated based on source revisions.

Details on data collection methods are available on each source's introduction page. Here is the list of data sources.

Retrieved from "https://pardeewiki.du.edu//index.php?title=Data_(New)&oldid=12235"