IFs Dyadic Trade Data - Revision history

Yutang.Xiong at 00:30, 7 February 2025

2025-02-07T00:30:59Z

Yutang.Xiong: /* Data Processing */

2025-02-07T00:24:04Z

Data Processing

← Older revision		Revision as of 00:24, 7 February 2025
Line 43:		Line 43:
	=== Data Processing ===		=== Data Processing ===

	We process all trade related tables through Python scripts, which can be found on our GitHub repository. Here, we only describe the general process. Users are encouraged to read the GitHub repo if you want to understand, replicate, or even improve our data processing procedure. As stated at the beginning of this page, IFs provide four sets of trade related dyadic data tables.		We process all trade related tables through Python scripts, which can be found on our [https://github.com/PardeeCenterDU/IFs-Dyadic-Trade GitHub repository]. Here, we only describe the general process. Users are encouraged to read the GitHub repo if you want to understand, replicate, or even improve our data processing procedure. As stated at the beginning of this page, IFs provide four sets of trade related dyadic data tables.

	====Exports & Imports====		====Exports & Imports====

Yutang.Xiong: /* Tariff Revenues */

2025-02-07T00:23:16Z

Tariff Revenues

← Older revision		Revision as of 00:23, 7 February 2025
Line 63:		Line 63:

	Since we have trade values down to the commoditiy level, we calculate the TCI indices for all the sub-sectors under Goods (agriculture, energy, ICT, materials, and manufacturing), as well as the TCI for overall Goods. Hence, there are 6 TCI tables in total in the IFs dyadic database. Note that IFs do not have TCI for the services sector because we currenty do not have sub-sectoral data for services. As you can see from the example above, calculating TCI can take a substantial amount of time, depending on the processing power and RAM capacity your device has. In the calculation of TCI, you will still need to separate trade values for those economic organizations, but there is no need for currency conversion.		Since we have trade values down to the commoditiy level, we calculate the TCI indices for all the sub-sectors under Goods (agriculture, energy, ICT, materials, and manufacturing), as well as the TCI for overall Goods. Hence, there are 6 TCI tables in total in the IFs dyadic database. Note that IFs do not have TCI for the services sector because we currenty do not have sub-sectoral data for services. As you can see from the example above, calculating TCI can take a substantial amount of time, depending on the processing power and RAM capacity your device has. In the calculation of TCI, you will still need to separate trade values for those economic organizations, but there is no need for currency conversion.
	====Tariff Revenues====		====Tariff Revenues & Weighted Tariff====
	~~TCI measures~~		There are 12 bilateral tables for tariff revenues and weighted tariffs, with 2 tables for each goods sector (goods, agriculture, energy, ICT, materials, and manufacturing). First, we calculate tariff revenues for each commodity (at 6-digit HS code level) after merging trade data from BACI and tariff data from WITS. Then, we aggregate tariff revenues for each country pair for each sector, and divide them by their corresponding trade values. Again, trade values for each economic organization are proportionally broken down to country level before this process, and you do need currency conversion because trade revenues need to match IFs' unit, which is now in constant 2017 USD.

			For tariff data from WITS, you also need to deal with economic organiations, such as the European Union. However, unlike trade values, we simply assign those unions' rates to their member states. During this process, you need to be aware of membership changes throughout the years. Missing values in WITS also need to be imputed. For a commodity traded between a country pair that is recorded under BACI but not presented by WITS. We look for the average effective applied tariff rate the same importer charged to the rest of world for that commodity. If the first imputation method fails, we look for the average effective applied tariff rate charged by the rest of world for that commodity. If both imputation methods fail, we implement 0 tariff for that commodity.

Yutang.Xiong: /* Trade Complementarity Index */

2025-02-07T00:04:35Z

Trade Complementarity Index

← Older revision		Revision as of 00:04, 7 February 2025
Line 58:		Line 58:

	====Trade Complementarity Index====		====Trade Complementarity Index====
	TCI measures how well a country's exports match another country's imports. Its definition is ''The sum of the absolute value of the difference between the import category shares and the export shares of the countries under study, divided by two''. The index is converted to percentage form. Its range of values takes a value between 0 and 100, with zero indicating no overlap and 100 indicating a perfect match in the import/export pattern. ~~To better illustrate~~ the ~~idea~~, ~~please~~ see the ~~graph below~~ for ~~an example~~,		TCI measures how well a country's exports match another country's imports. Its definition is ''The sum of the absolute value of the difference between the import category shares and the export shares of the countries under study, divided by two''. The index is converted to percentage form. Its range of values takes a value between 0 and 100, with zero indicating no overlap and 100 indicating a perfect match in the import/export pattern. The graph below shows an example of TCI calculation,

			[[File:Example of TCI Calculation.png\|border\|807x807px]]

			Since we have trade values down to the commoditiy level, we calculate the TCI indices for all the sub-sectors under Goods (agriculture, energy, ICT, materials, and manufacturing), as well as the TCI for overall Goods. Hence, there are 6 TCI tables in total in the IFs dyadic database. Note that IFs do not have TCI for the services sector because we currenty do not have sub-sectoral data for services. As you can see from the example above, calculating TCI can take a substantial amount of time, depending on the processing power and RAM capacity your device has. In the calculation of TCI, you will still need to separate trade values for those economic organizations, but there is no need for currency conversion.
			====Tariff Revenues====
			TCI measures

Yutang.Xiong: /* Exports & Imports */

2025-02-06T23:51:13Z

Exports & Imports

← Older revision		Revision as of 23:51, 6 February 2025
Line 46:		Line 46:

	====Exports & Imports====		====Exports & Imports====
	There are 16 tables in total, with 8 tables for each trade flow. Within each trade flow (exports or imports), trade values are separated into ~~following~~ sectors-		There are 16 tables in total, with 8 tables for each trade flow. Within each trade flow (exports or imports), trade values are separated into sub-sectors,

	<math> Total = Goods + Services </math>, where		<math> Total = Goods + Services </math>, where
	<math> Goods = Agriculture + Energy + ICT + Materials + Manufacturing </math>
			<math> Goods = Agriculture + Energy + ICT + Materials + Manufacturing </math>.

			Trade values from BACI are recorded at 6-digit HS code level. To aggregate trade values into IFs sectors, simply use the concordance table provided in the GitHub repo. Note that the mapping table from HS code to GTAP is directly provided by WITS, but the mapping table between GTAP to IFs can be altered if needed. In addition to aggregation, following data manipulation steps are also done,

			* Trade values in BACI are in current USD. We use [https://data.worldbank.org/indicator/NY.GDP.DEFL.ZS?locations=US USD deflator of the United States provided by the World Bank] to convert all trade values to constant 2017 USD. It is debatable whether this is the best or even the correct approach, because GDP conversion is itself a complicated process.
			* Some historical data were reported by economic organizations instead of individual countries, e.g., South African Customs Union (SACU), we use each member's economic size, i.e., GDP, to proportionally distribute their trade values.

			====Trade Complementarity Index====
			TCI measures how well a country's exports match another country's imports. Its definition is ''The sum of the absolute value of the difference between the import category shares and the export shares of the countries under study, divided by two''. The index is converted to percentage form. Its range of values takes a value between 0 and 100, with zero indicating no overlap and 100 indicating a perfect match in the import/export pattern. To better illustrate the idea, please see the graph below for an example,

Yutang.Xiong: /* Exports & Imports */

2025-02-06T23:35:05Z

Exports & Imports

← Older revision		Revision as of 23:35, 6 February 2025
Line 48:		Line 48:
	There are 16 tables in total, with 8 tables for each trade flow. Within each trade flow (exports or imports), trade values are separated into following sectors-		There are 16 tables in total, with 8 tables for each trade flow. Within each trade flow (exports or imports), trade values are separated into following sectors-

	Total = Goods + Services, where Goods = Agriculture + Energy + ICT + Materials + Manufacturing		<math> Total = Goods + Services </math>, where
			<math> Goods = Agriculture + Energy + ICT + Materials + Manufacturing </math>

Yutang.Xiong: /* Data Processing */

2025-02-06T23:32:51Z

Data Processing

← Older revision		Revision as of 23:32, 6 February 2025
Line 46:		Line 46:

	====Exports & Imports====		====Exports & Imports====
	~~Data from BACI can be freely accessed from their official website~~. ~~All years of data needed for IFs~~ (~~1995 - now~~) ~~can be downloaded under HS92 system. You will receive a zipped file with~~ trade ~~data by year, country and product codes~~, ~~as well as a text note explaining the structure of data. Trade data by year are~~		There are 16 tables in total, with 8 tables for each trade flow. Within each trade flow (exports or imports), trade values are separated into following sectors-

			Total = Goods + Services, where Goods = Agriculture + Energy + ICT + Materials + Manufacturing

Yutang.Xiong at 22:22, 6 February 2025

2025-02-06T22:22:54Z

← Older revision		Revision as of 22:22, 6 February 2025
Line 13:		Line 13:

	In IFs, we only present economic activities under IFs sectors (agriculture, energy, manufacturing, materials, ICT, and services) as well as GTAP sectors. Hence, during the data processing steps, 6-digit commodity level values are aggregated into GTAP sectors first, and then IFs sectors. The concordance table between HS codes and GTAP sectors can be found under WITS' product concordance page. The mapping table between GTAP sectors and IFs sectors can be found in the IFs system, under the table GTAPSectorConversion in the DATA/IFs.db file.		In IFs, we only present economic activities under IFs sectors (agriculture, energy, manufacturing, materials, ICT, and services) as well as GTAP sectors. Hence, during the data processing steps, 6-digit commodity level values are aggregated into GTAP sectors first, and then IFs sectors. The concordance table between HS codes and GTAP sectors can be found under WITS' product concordance page. The mapping table between GTAP sectors and IFs sectors can be found in the IFs system, under the table GTAPSectorConversion in the DATA/IFs.db file.
			----
	===Data Sources===		===Data Sources===

Line 40:		Line 40:
	* It is highly recommended that you download one year of data at a time, because WITS sets an internal upper limit for the total number of rows a resultant data query can carry (49,999,999 rows).		* It is highly recommended that you download one year of data at a time, because WITS sets an internal upper limit for the total number of rows a resultant data query can carry (49,999,999 rows).
	Each data query will take WITS some time to process, and you can download each data table in your desired format, as in .csv files. In the data processing steps, you will need to merge tariff data from WITS and trade data from BACI to calculate tariff revenue and weighted tariff.		Each data query will take WITS some time to process, and you can download each data table in your desired format, as in .csv files. In the data processing steps, you will need to merge tariff data from WITS and trade data from BACI to calculate tariff revenue and weighted tariff.
			----
	=== Data Processing ===		=== Data Processing ===

Yutang.Xiong at 22:21, 6 February 2025

2025-02-06T22:21:32Z

← Older revision		Revision as of 22:21, 6 February 2025
Line 40:		Line 40:
	* It is highly recommended that you download one year of data at a time, because WITS sets an internal upper limit for the total number of rows a resultant data query can carry (49,999,999 rows).		* It is highly recommended that you download one year of data at a time, because WITS sets an internal upper limit for the total number of rows a resultant data query can carry (49,999,999 rows).
	Each data query will take WITS some time to process, and you can download each data table in your desired format, as in .csv files. In the data processing steps, you will need to merge tariff data from WITS and trade data from BACI to calculate tariff revenue and weighted tariff.		Each data query will take WITS some time to process, and you can download each data table in your desired format, as in .csv files. In the data processing steps, you will need to merge tariff data from WITS and trade data from BACI to calculate tariff revenue and weighted tariff.
			=== Data Processing ===

			We process all trade related tables through Python scripts, which can be found on our GitHub repository. Here, we only describe the general process. Users are encouraged to read the GitHub repo if you want to understand, replicate, or even improve our data processing procedure. As stated at the beginning of this page, IFs provide four sets of trade related dyadic data tables.

			====Exports & Imports====
			Data from BACI can be freely accessed from their official website. All years of data needed for IFs (1995 - now) can be downloaded under HS92 system. You will receive a zipped file with trade data by year, country and product codes, as well as a text note explaining the structure of data. Trade data by year are

Yutang.Xiong at 22:16, 6 February 2025

2025-02-06T22:16:22Z

← Older revision		Revision as of 22:16, 6 February 2025
Line 27:		Line 27:

	====WITS====		====WITS====
	[https://wits.worldbank.org/ WITS] is a powerful tool that provides access to international merchandise trade, tariff and non-tariff measures (NTM) data. As of now, we rely on WITS for collecting tariff data. WITS only provides three types of tariffs- the bond rate (highest), MFN, and preferential rate (lowest). See the graph illustrated below, [https://wits.worldbank.org/wits/wits/witshelp/content/data_retrieval/p/intro/c2.types_of_tariffs.htm#:~:text=The%203%20types%20of%20tariffs,other%20two%20as%20illustrated%20below. according to the data manual of WITS]. In addition, it provides the so called “effective applied tariff”, which is being used for our dyadic data. ~~It’s~~ definition is “If a preferential tariff exists, it will be used as the effectively applied tariff. Otherwise, the MFN applied tariff will be used.”		[https://wits.worldbank.org/ WITS] is a powerful tool that provides access to international merchandise trade, tariff and non-tariff measures (NTM) data. As of now, we rely on WITS for collecting tariff data. WITS only provides three types of tariffs- the bond rate (highest), MFN, and preferential rate (lowest). See the graph illustrated below, [https://wits.worldbank.org/wits/wits/witshelp/content/data_retrieval/p/intro/c2.types_of_tariffs.htm#:~:text=The%203%20types%20of%20tariffs,other%20two%20as%20illustrated%20below. according to the data manual of WITS]. In addition, it provides the so called “effective applied tariff”, which is being used for our dyadic data. It's definition is “If a preferential tariff exists, it will be used as the effectively applied tariff. Otherwise, the MFN applied tariff will be used.”

			There are several limitations worth noting for the WITS data. First, its trade data are directly from UN Comtrade, which lacks the harmonization process implemented by BACI. Hence, currently, IFs solely rely on BACI for trade information (imports and exports). Secondly, there are imperfections in tariff data from WITS where certain categories of commodities have unrealistic tariff values. In addition, there are missing values (unreported values) in tariffs. Lastly, any special tariff barriers, such as additional tariffs under US-China trade war, are not recorded. Adding these exceptional tariff rates into our IFs database will require additional manual labor unless a global data source appears. It is suggested that users can use scenarios to simulate such tariffs.
	[[File:3 types of tariff for a given product, WITS.png\|none\|thumb]]To query and download tariff data from WITS, a registration is required. Once logged in, you will be able to access tariff data for free. Note that a paid subscription from UN Comtrade is required to use the bulk download option, hence, you will have to query the data multiple times in order to get the full coverage. In order to query the data, select ADVANCED QUERY from the drop-down menu and click on Tariff and Trade Analysis. You will be directed to the Tariff and Trade Analysis interface where you can create data queries on your own. Make sure you pick TRAINS as your data source.		[[File:3 types of tariff for a given product, WITS.png\|none\|thumb]]To query and download tariff data from WITS, a registration is required. Once logged in, you will be able to access tariff data for free. Note that a paid subscription from UN Comtrade is required to use the bulk download option, hence, you will have to query the data multiple times in order to get the full coverage. In order to query the data, select ADVANCED QUERY from the drop-down menu and click on Tariff and Trade Analysis. You will be directed to the Tariff and Trade Analysis interface where you can create data queries on your own. Make sure you pick TRAINS as your data source.
	[[File:Interface of the WITS Data Query Tool.png\|border\|744x744px]]		[[File:Interface of the WITS Data Query Tool.png\|border\|744x744px]]
Line 37:		Line 39:
	* ''Include Effective Applied rates'' as the duty type, with ''UNCTAD'' as the computing method for average;		* ''Include Effective Applied rates'' as the duty type, with ''UNCTAD'' as the computing method for average;
	* It is highly recommended that you download one year of data at a time, because WITS sets an internal upper limit for the total number of rows a resultant data query can carry (49,999,999 rows).		* It is highly recommended that you download one year of data at a time, because WITS sets an internal upper limit for the total number of rows a resultant data query can carry (49,999,999 rows).
			Each data query will take WITS some time to process, and you can download each data table in your desired format, as in .csv files. In the data processing steps, you will need to merge tariff data from WITS and trade data from BACI to calculate tariff revenue and weighted tariff.