Artificial Intelligence Methods for Evaluating Global Trade Flows

Board of Governors of the Federal Reserve System International Finance Discussion Papers Number 1296 August 2020 Artificial Intelligence Methods for Evaluating Global Trade Flows Feras A. Batarseh, Munisamy Gopinath, and Anderson Monken Please cite this paper as: Batarseh, Feras A., Munisamy Gopinath, and Anderson Monken (2020). “Artificial Intelligence Methods for Evaluating Global Trade Flows,” International Finance Discussion Papers 1296. Washington: Board of Governors of the Federal Reserve System, https://doi.org/10.17016/IFDP.2020.1296. NOTE: International Finance Discussion Papers (IFDPs) are preliminary materials circulated to stimulate discussion and critical comment. The analysis and conclusions set forth are those of the authors and do not indicate concurrence by other members of the research staff or the Board of Governors. References in publications to the International Finance Discussion Papers Series (other than acknowledgement) should be cleared with the author(s) to protect the tentative character of these papers. Recent IFDPs are available on the Web at www.federalreserve.gov/pubs/ifdp/. This paper can be downloaded without charge from the Social Science Research Network electronic library at www.ssrn.com.

Artificial Intelligence Methods for Evaluating Global Trade Flows* Feras A. Batarseh† Munisamy Gopinath‡ Anderson Monken§ Abstract International trade policies remain in the spotlight given the recent rethink on the benefits of globalization by major economies. Since trade critically affects employment, production, prices and wages, understanding and predicting future patterns of trade is a high-priority for decision making within and across countries. While traditional economic models aim to be reliable predictors, we consider the possibility that Artificial Intelligence (AI) techniques allow for better predictions and associations to inform policy decisions. Moreover, we outline contextual AI methods to decipher trade patterns affected by outlier events such as trade wars and pandemics. Open-government data are essential to providing the fuel to the algorithms that can forecast, recommend, and classify policies. Data collected for this study describe international trade transactions and commonly associated economic factors. Models deployed include Association Rules for grouping commodity pairs; and ARIMA, GBoosting, XGBoosting, and LightGBM for predicting future trade patterns. Models and their results are introduced and evaluated for prediction and association quality with example policy implications. Keywords: AI, international trade, boosting, prediction, data mining, imports and exports, outlier events JEL codes: F13, F17, C55, C8 * The views in this paper are solely the responsibility of the authors and should not be interpreted as reflecting the views of the Board of Governors of the Federal Reserve System or of any other person associated with the Federal Reserve System. We thank Rob Vigfusson for his valuable comments. † Batarseh: College of Science, George Mason University. Fairfax, VA, USA. Email: fbatarse@gmu.edu ‡ Gopinath: Department of Agricultural and Applied Economics, University of Georgia. Athens, Georgia, USA. Email: m.gopinath@uga.edu § Monken: Division of International Finance, Federal Reserve Board. Washington, DC, USA. Email: anderson.e.monken@frb.gov 1

1. Background and Motivation In recent years, many countries are concerned about rising trade deficits (value of exports less imports) and their implications for employment, production, and wages. For instance, the United States’ goods and services trade deficit with China was $378.8 billion in 2018. Such numbers are forcing countries to either exit trade agreements or enforce tariffs, (e.g. Brexit, U.S. tariffs on Chinese goods). These shocks to global trade in commodities pose challenges to predict future trading patterns. International economics has a long history of improving our understanding of factors causing trade and the consequences of free flow of goods and services across countries. Nonetheless, the recent shocks to the free-trade regime raise questions on the quality of earlier predictions and their applicability in the context of large trade disputes. To address these challenges, this article, identifies AI techniques appropriate for the international trade setting and tests their validity in making high quality projections (Batarseh and Yang, 2018). Recent technological advancements in Artificial Intelligence (AI) as well as data democratization (Batarseh and Yang, 2020) have helped increase transparency, which is critical in the context of public decision-making. Given the Open Data and Big Data initiatives presented in 2008 and 2012 (White House, 2008), federal agencies are forced to share their data on public repositories such as www.data.gov, as well as many agency-specific repositories. The combination of data availability, AI advances and a contemporary context, i.e. trade wars, offer a unique opportunity to explore the applicability of AI techniques for nimble and improved trade projections to aid in decision-making. The primary objective of the paper is to identify techniques most appropriate for economic forecasts, especially in the context of international trade, test their relevance using historical data on trade patterns and wherever appropriate, make quantitative and qualitative comparisons to current approaches. 2. AI for Economics and International Trade Based on a recent study by the National Bureau of Economic Research (NBER), AI is only recently being applied to address economic issues. AI has been applied across multiple domains; it has been employed in addressing challenges in healthcare (Reddy and Aggarwal, 2015) (Batarseh and Latif, 2015), education (Niemi et al., 2018), and sports (Alamar, 2013). Athey and Imbens (2019) provide a detailed overview of AI techniques that economists should know about, but to date, AI applications to understand or predict patterns of international trade are limited (Gopinath, Batarseh and Beckman 2020). 2

The few economic applications include Gevel et al. (2013) on the nexus between Artificial Intelligence and Economics; Feng et al. (2014) on economic growth in the Chinese province of Zhejiang; Abadie et al. (2010) to the rising economics of tobacco in California; Milacic et al. (2016) and Kordanuli et al. (2016) for GDP growth; and Falat et al. (2015) for economic patterns. Within AI, Machine learning (ML), Deep Learning (DL), and Reinforcement Learning (RL) are 3 major pillars that have showed success in applications to various disciplines. ML for instance, is commonly understood as a number of computational algorithms that extract hidden insights from large sets of data. DL is a set of bio-inspired methods that deploy neural networks to classify variable outputs, and allocate patterns that require extensive model training and hierarchical feature learning. RL however is a set of models that influence software agents to take action(s) in an environment in order to maximize the notion of cumulative reward and avoid punishment (using value functions). In this study, multiple AI methods are applied to a big data set of international trade (imports/exports) with a focus on improved predictions. Experimental work presented in this paper utilizes AI methods in an optimized manner to provide predictions and associations regarding trade of specific commodities and countries (regressions, classifiers, clusters, associations and multiple other actionable outcomes). Given the dimensions of the data, and the high number of variables involved, several models are developed; few are compared to explain international trade patterns. Figure 1 illustrates top AI methods considered for this study. Furthermore, the paper outlines approaches that are appropriate for the uncertain global trade environment in recent years. Trade wars between major economies, de-globalization trends, as well as the Covid- 19 pandemic have seriously disrupted flows of goods and services within and across countries. Contextual AI, appropriate for this setting, is also discussed as a possible application to the trade setting. 3

Figure 1: A Subset of the Most Commonplace AI Methods 3. Data Collection and Pre-Processing For predictions using AI methods, data has been collected on several specific products – all the Harmonized System 4-digit level from USDA’s Foreign Agricultural Services’ Global Agricultural Trade System (FAS - GATS) (USDA 2019). GATS is a system published by the United States Department of Agriculture. To apply supervised methods, additional economic data is collected from the World Bank’s World Integrated Trade Solution (WITS 2019) and U.S. ITC’s Gravity Portal (2019). The trade data cover seven major commodities with a long history of trade data (starting in the 1960s): Wheat, Milk, Rice, Corn, Beef, Soy, and Sugar. They are merged with 30+ economic variables, such as: Population, Currency, Island or Not, GDP of Origin, GDP of Destination, Distance, Landlocked or not, WTO Member, Hostility, EU Member, and other ones (U.S. ITC Gravity Portal). Afterwards, the economic and commodity data are merged into a SQL database. An R code is used to merge on country-to-country trade transactions, as well as year of economic variables. The data are merged using a SQL Inner Join. The 30+ economic variables’ correlations are studied; results for the 4

correlations (done in R) are plotted in Figure 2. Highest economic correlations, for example, are found between: population and whether the country is an island, also, between currency and GDP, and between WTO membership and Free Trade Agreements; amongst other existing factors. For Association Rules (AR), data come from the World Trade Organization (WTO) Bilateral Imports dataset, which comprises annual country to country trade data from 1996 to 2018. Data come from countries reporting imports from trading partners around the world. Only countries that are part of the WTO report imports (the sample of countries is ~190). The AR application focused on the chapter level of the system: the 2-digit codes (HS-2). Data on the 96-chapter level trade products are downloaded from the WTO developer portal (https://apiportal.wto.org/). Using Pandas and Numpy libraries in Python, data are loaded into a PostgreSQL database for ease of analysis. The next two sections present the methods and the results of 1) Predictions and 2) Associations. Figure 2: Correlations of 30+ Economic Variables 5

4. Predictions: Methods and Results Supervised and unsupervised methods have been explored: Linear Regression, K-means clustering, Pearson correlations, Boosting, and Time Series such as Autoregressive Integrated Moving Average (ARIMA). Simple linear regression modeling is applied to the seven major commodities mentioned; the aim is to predict exports or imports of a specific commodity. For example, after importing required columns into a python environment, linear regression is deployed using the python libraries: sklearn.linear_model and pandas (Python 2019). The results below focus on one commodity: beef. Top countries exporting beef are: Australia, Germany, Netherlands, France, and United States. Although data for beef trade are available from 1960, data from 1989 to 2018 is used because of missing information on tariffs before 1988; years 2019-2021 are predicted (red line in Figure 3). As the figure illustrates, trade between nations is variant, and can change drastically over time; even for one commodity. Therefore, due to the high variance in the data, a simple regression model, although supervised, provides straight-line pointers to the future of beef trade (implying growth remains constant). Consequently, as an experimental model, an unsupervised K-means clustering model is developed to group countries into clusters (using sklearn’s cluster and K-Means libraries in python). Besides trade values, other economic variables are incrementally added to the modeling process. When all the economic variables are added, the aim is to identify which variables have the highest influence on trade predictions, and which ones could be controlled and tuned to change the forecasts. Different commodities had different rankings of economic variables, however, distance (between the 2 countries undergoing trade), population of the exporter, and GDP of both countries had the highest impact on whether two countries would trade one of the seven major commodities or not. Feature importance (Gain of top economic variables) is illustrated in Figure 4. Consequently, ARIMA is applied to beef trade, the advantage of ARIMA is that it provides univariate predictions that improve the output. ARIMA results are presented in Table 1 and Figure 5; they illustrate the high and low confidence intervals of the model. 6

Figure 3: Australia’s Beef Exports 1988-2021 Afterwards, boosting has been applied to elevate the quality of the models. Three different boosting models are deployed: Gradient Boost (GBoost), Extreme Gradient Boosting (XGBoost), and Light Gradient Boosting Decision Tree (LightGBM). Python libraries are used to deploy the models (GBoost, XGBoost, and LightGBM) (Ke et al. 2017). After multiple iterations and hyper-parameters’ tuning, LightGBM performed best for most commodities. A boosting algorithm is an algorithm that converts weak learners to strong learners. It is a method that improves predictions’ quality of a model. Boosting trains weak learners sequentially, and in every cycle, each trying to correct its predecessor. Figure 4: Strength of Variables in Predicting Trade Trends 7

Table 1: ARIMA Forecasting of Beef Trade Trends Year Actual Forecast Low 80 High 80 Low 95 High 95 2014 6939233 5594243 5208462 5980024 5004242 6184244 2015 7357932 5698666 5153090 6244242 4864279 6533053 2016 5921218 5803089 5134898 6471281 4781178 6825000 2017 5843209 5907513 5135951 6679074 4727511 7087514 2018 X 6011936 5149304 6874567 4692654 7331217 2019 X 6116359 5171393 7061325 4671159 7561559 2020 X 6220782 5200102 7241462 4659787 7781777 2021 X 6325205 5234053 7416358 4656432 7993979 Figure 5: Australia’s Beef Exports Predictions (ARIMA) Results for trade predictions through the XGBoost Model scored predictions’ quality = 69%, and through LightGBM scored a quality of 88% (in contrast, GBoost scored the lowest of the three approaches). Parameter tuning for boosting models include: Number of leaves, Maximum Depth of the tree, Learning Rate, and Feature fraction. Small learning rates are optimal (0.01), with large tree depths. Additionally, to speed up training and avoid over-fitting, feature fraction is set to 0.6; that is, selecting 60% of the features before training each tree. Early stopping round is set to 500; that allowed the model to train until the validation score stops improving. Maximum tree depth is set to 8. Those tunings led to the best output through LightGBM. Sugar for instance had an R2 score of 0.73, 0.88 for Beef, and 0.66 for Corn. Additionally, such tunings allowed for the extractions of the best economic variables that would affect 8

trade of specific commodities. As mentioned for beef for example, distance had the highest effect (i.e. the US is better off trading beef to Canada and Mexico, its two closest neighbors). While Australia, being an island, has to focus its policies for beef exports on GDP measures, and the population of the importer. Feature Importance for all economic variables are (name: split, gain.): Distance: 1469, 6.38. GDP of Exporter: 1431, 6.22. Year: 993, 4.318. Population of Exporter: 882, 3.83. Population of Importer: 847, 3.68. Currency of Importer: 801, 3.48. Figure 6 shows predictions for aggregate Australian exports of beef as well as its exports to the major trade partner: Japan. The clustering model yielded very expected results: China and the US ended up in Cluster 1, as the biggest exporters and importers. Cluster 2 has other major exporters and importers: Japan, Germany, Canada, UK, India, and France. Cluster 3 has less important importer countries, such as most third-world countries. Beef: Australia-Japan Australia -Beef Exports 2.0 8000 1.5 6000 1.0 4000 0.5 2000 0.0 0 2011 2012 2013 2014 2015 2016 2014 2015 2016 2017 2018 2019 2020 Actual Bil $ Predicted Bil $ Actual mil $ Predicted mil $ Figure 6(a): Supervised Model Projections Figure 6(b): Unsupervised Model Projections 5. Associations: Methods and Results Identifying commodities that are traded in association with each other point to substitutability and complementarities and has a direct implication on decision making when one commodity is targeted, e.g. U.S. soybeans or Chinese steel. AR is a popular method for discovering hidden relations between variables in big datasets. Piatetsky and Shapiro (1991) describe analyzing strong rules discovered in datasets. Based on the notion of strong rules, Agrawal et al. (1993) introduced the problem of mining association rules from transaction data. The idea of AR is as follows: Let X = {x , x , ..., x } be a set of n commodities. Let T = {t , t , ..., t } 1 2 n 1 2 n be a set of transactions (where Tt is the total number of transactions). Each transaction in T has a unique 9

ID and contains a subset of the commodities (X) that are traded. A rule is defined as an implication of the form X ⇒ X where X ,X  X and X ∩ X = . a b a b a b The sets of commodities X is called antecedents (left-hand-side or LHS) and the set of commodities X a b is called consequents (right-hand-side or RHS) of the rule. Besides antecedent-consequent rules, the quality of the associations is measured through the following three metrics: 1. Support = 𝑋𝑎+ 𝑋 𝑏 𝑇𝑡 2. Confidence = 𝑋𝑎+ 𝑋 𝑏 𝑋𝑎 3. Lift = (𝑋𝑎+ 𝑋 𝑏 )/ 𝑋𝑎 (𝑋 /𝑇𝑡) 𝑏 Support indicates that for example 67% of customers purchased beer and diapers together. Confidence is that 90% of the customers who bought beer also bought diapers (confidence is the best indicator of AR). While lift represents the 28% increase in expectation that someone will buy diapers, when we know that they bought beer (i.e. lift is the conditional probability). In our study, AR mining is performed using the arules library in R. Data are pulled and processed from PostgreSQL. Transactions in the data represent each country-country pair’s trade for a given year. The goods in the data are the 96 commodity code trade dummies, which are boolean values depending on whether trade occurred for a specific country-country pair for a given year. Apriori association rules are collected with a minimum support of 0.35 and a maximum number of antecedents set to 3. Results are visualized in plotly and exported using R. The top 4 million+ rules are pulled out of the models, and migrated into a structured relational SQL database (called: AR-Trade). The rules are pulled for U.S. trade for top trade countries in Asia (China, Korea, and Japan); as well as the top trade countries in Europe (UK, Spain, France, and Germany). Relational SQL tables include the following columns: Lhs (antecedent), Rhs (consequent), Lhs name, Rhs name, Support, Confidence, Lift, Count, Country_O, Country_D. AR results are plotted using an R-Shiny dashboard, as well as R plots using the arulesViz library such as through this script: isS4(AR-Trade) AR-Trade@lhs plot(AR-Trade) Results are recorded and analyzed. 10

AR is applied to HS2 commodity codes, and so for instance, if beer leads to diapers at the grocery store, then oil seeds lead to cotton in international trade. In future work, we aim to deploy HS6 AR analysis to directly influence production decisions in the U.S., provide tariff insights, and other potential trade policies. Table 2 presents example top commodity pairs as antecedents and consequents. Table 2: A Sample of Top AR Associations ID Antecedent Consequent Sum of Confidence 1 Products of the milling industry; malt; Knitted or crocheted fabrics 935 starches; inulin; wheat gluten 2 Animal or vegetable fats and oils and Cereals 902 their cleavage products; prepared edible fats; animal or vegetable waxes 3 Animal or vegetable fats and oils and Clocks and watches and parts thereof 891 their cleavage products; prepared edible fats; animal or vegetable waxes 4 Animal or vegetable fats and oils and Cocoa and cocoa preparations 883.71 their cleavage products; prepared edible fats; animal or vegetable waxes 5 Animal or vegetable fats and oils and Copper and articles thereof 891 their cleavage products; prepared edible fats; animal or vegetable waxes 6 Animal or vegetable fats and oils and Cork and articles of cork 879.71 their cleavage products; prepared edible fats; animal or vegetable waxes 7 Animal or vegetable fats and oils and Cotton 891 their cleavage products; prepared edible fats; animal or vegetable waxes 8 Animal or vegetable fats and oils and Electrical machinery and equipment and 891 their cleavage products; prepared edi- parts thereof; sound recorders and reproble fats; animal or vegetable waxes ducers, television image and sound recorders and reproducers, and parts and accessories of such articles 9 Animal or vegetable fats and oils and Essential oils and resinoids; perfumery, 891 their cleavage products; prepared edi- cosmetic or toilet preparations ble fats; animal or vegetable waxes 10 Animal or vegetable fats and oils and Explosives; pyrotechnic products; 891 their cleavage products; prepared edi- matches; pyrophoric alloys; certain comble fats; animal or vegetable waxes bustible preparations The next section digs deeper into AR, and presents examples on bilateral trade between China and Australia, USA and top world traders, as well as China’s top food and agriculture commodities’ associations. 11

6. Bilateral Association Results As discussed prior, ARs provide means to determine goods that are often traded together. This cause and effect analysis is especially important when countries choose to enact trade restrictions and tariffs against other nations. The effect of losing the trade of one commodity between two countries depends on that commodity’s relative impact on other commodities’ trade - how many other commodities get traded alongside it. Table 3 is an example that describes how AR can be used to analyze the effects of recent Chinese trade restrictions. Tariffs imposed between countries can both have international effects while also resulting in domestic economic consequences. In May of 2020, China placed tariffs against Australian Barley and restricted import of Australian Beef in apparent retaliation for Australia’s government echoing the calls of the international community to investigate Beijing’s response to Covid-19. Using AR confirms the choice of commodities to target were statistically sound for China’s economy, because beef and barley are not in the top commodities that most impact the Australian-Sino commodity trade flows, AR analysis is performed for imports of Chinese goods to Australia and vice-versa for 1996 to 2019 across all available HS-6 level commodities (count = 4615). Rule building confidence threshold is set to 0.8, support threshold at 0.35 and up to three antecedents are allowed per rule. A total of 5,676,885 rules resulted, and antecedents across these rules were summed to determine commodities that have the most impact on the Sino-Australian trade relationship. Table 3 shows the top 25 commodities and the appearance count of AR. More results are available in a Github public repository: https://github.com/fbatarsegmu/TradeAI. Table 3: China-Australia Top Trade Associations (HS-6) Rule # Antecedent commodity HS-6 Code Count of Rules 1 Cereals; rice, semi-milled or wholly milled, whether or not 100630 46990 polished or glazed 2 Peel; of citrus fruit or melons (including watermelons), 081400 46927 fresh, frozen, dried or provisionally preserved in brine, in sulphur water and other preservative solutions 3 Vegetables, leguminous; beans (vigna spp., phaseolus spp.), 071022 46920 shelled or unshelled, uncooked or cooked by steaming or boiling in water, frozen 12

4 Vegetables, leguminous; peas (pisum sativum), shelled or 070810 46920 unshelled, fresh or chilled 5 Fish; dried (whether or not salted but not smoked), n.e.s. in 030559 46875 item no. 0305.51 6 Bamboo used primarily for plaiting 140110 46832 7 Vegetables, leguminous; n.e.s. in heading no. 0713, shelled, 071390 46781 whether or not skinned or split, dried 8 Fish preparations; fish prepared or preserved, whole or in 160419 46764 pieces (but not minced), n.e.s. in heading no. 1604 9 Vegetable preparations; mushrooms, prepared or preserved 200310 46710 otherwise than by vinegar or acetic acid 10 Food preparations; tapioca and substitutes thereof, prepared 190300 46677 from starch in the form of flakes, grains, pearls, siftings or similar 11 Flours and meals of oil seeds or oleaginous fruits; excluding 120890 46590 soya beans and mustard seeds 12 Vegetables and mixed vegetables; n.e.s. in heading no. 0711, 071190 46545 provisionally preserved but unsuitable in that state for immediate consumption 13 Fruit, edible; dates, fresh or dried 080410 46512 14 Vegetables, leguminous; (other than peas or beans), shelled 071029 46482 or unshelled, uncooked or cooked by steaming or boiling in water, frozen 15 Vegetable roots and tubers; sweet potatoes, with high starch 071420 46467 or inulin content, whether or not sliced or in the form of pellets, fresh or dried 16 Fruit, edible; figs, fresh or dried 080420 46443 17 Meat preparations; of swine, meat or meat offal (including 160249 46410 mixtures), prepared or preserved, n.e.s. in heading no. 1602 18 Vegetable preparations; vegetables and mixtures of vegeta- 200490 46405 bles (excluding potatoes), prepared or preserved otherwise than by vinegar or acetic acid, frozen 19 Vegetables, leguminous; n.e.s. in item no. 0713.30, dried, 071339 46324 shelled, whether or not skinned or split 20 Vegetables; uncooked or cooked by steaming or boiling in 071080 46279 water, frozen, n.e.s. in chapter 7 21 Vegetables, leguminous; peas (pisum sativum), shelled or 071021 46251 unshelled, uncooked or cooked by steaming or boiling in water, frozen 22 Fish; salted or in brine, but not dried or smoked, n.e.s. in 030569 46113 item no. 0305.6 23 Vegetables, alliaceous; garlic, fresh or chilled 070320 46113 24 Vegetable oils; ground-nut oil and its fractions, other than 150890 46069 crude, whether or not refined, but not chemically modified 13

25 Vegetable preparations; beans, (not shelled), prepared or 200559 46025 preserved otherwise than by vinegar or acetic acid, not frozen Additionally, top HS-2 ARs from USA’s bilateral trade with China, Japan, France, and Germany are presented in Table 4. Table 5 presents the top five results for China’s food and agricultural commodities trade. Table 4: USA Trade Associations with other Countries (HS-2) Antecedent 1 Antecedent 2 Consequent Coun- Country_O try_D Ores, slag and ash Works of art, collectors' Mineral fuels, mineral USA China pieces and antiques oils and products of their distillation; bituminous substances; mineral waxes Mineral fuels, min- Works of art, collectors' Ores, slag and ash USA China eral oils and prod- pieces and antiques ucts of their distillation; bituminous substances; mineral waxes Ores, slag and ash Mineral fuels, mineral Works of art, collectors' USA China oils and products of their pieces and antiques distillation; bituminous substances; mineral waxes Ores, slag and ash Works of art, collectors' Inorganic chemicals; USA China pieces and antiques organic or inorganic compounds of precious metals, of rare earth metals, of radioactive elements or of isotopes Inorganic chemicals; Works of art, collectors' Ores, slag and ash USA China organic or inorganic pieces and antiques compounds of precious metals, of rare earth metals, of radioactive elements or of isotopes Ores, slag and ash Inorganic chemicals; or- Works of art, collectors' USA China ganic or inorganic com- pieces and antiques pounds of precious metals, of rare earth metals, 14

of radioactive elements or of isotopes Mineral fuels, min- Works of art, collectors' Inorganic chemicals; USA China eral oils and prod- pieces and antiques organic or inorganic ucts of their distilla- compounds of precious tion; bituminous metals, of rare earth substances; mineral metals, of radioactive waxes elements or of isotopes Inorganic chemicals; Works of art, collectors' Mineral fuels, mineral USA China organic or inorganic pieces and antiques oils and products of compounds of pre- their distillation; bitucious metals, of rare minous substances; earth metals, of ra- mineral waxes dioactive elements or of isotopes Mineral fuels, min- Inorganic chemicals; or- Works of art, collectors' USA China eral oils and prod- ganic or inorganic com- pieces and antiques ucts of their distilla- pounds of precious mettion; bituminous als, of rare earth metals, substances; mineral of radioactive elements waxes or of isotopes Ores, slag and ash Mineral fuels, mineral Inorganic chemicals; USA China oils and products of their organic or inorganic distillation; bituminous compounds of precious substances; mineral metals, of rare earth waxes metals, of radioactive elements or of isotopes Ores, slag and ash Inorganic chemicals; or- Mineral fuels, mineral USA China ganic or inorganic com- oils and products of pounds of precious met- their distillation; bituals, of rare earth metals, minous substances; of radioactive elements mineral waxes or of isotopes Mineral fuels, min- Inorganic chemicals; or- Ores, slag and ash USA China eral oils and prod- ganic or inorganic comucts of their distilla- pounds of precious mettion; bituminous als, of rare earth metals, substances; mineral of radioactive elements waxes or of isotopes Ores, slag and ash Works of art, collectors' Organic chemicals USA China pieces and antiques Organic chemicals Works of art, collectors' Ores, slag and ash USA China pieces and antiques Inorganic chemicals; Works of art, collectors' Mineral fuels, mineral USA France organic or inorganic pieces and antiques oils and products of 15

compounds of pre- their distillation; bitucious metals, of rare minous substances; earth metals, of ra- mineral waxes dioactive elements or of isotopes Mineral fuels, min- Inorganic chemicals; or- Works of art, collectors' USA France eral oils and prod- ganic or inorganic com- pieces and antiques ucts of their distilla- pounds of precious mettion; bituminous als, of rare earth metals, substances; mineral of radioactive elements waxes or of isotopes Mineral fuels, min- Works of art, collectors' Organic chemicals USA France eral oils and prod- pieces and antiques ucts of their distillation; bituminous substances; mineral waxes Organic chemicals Works of art, collectors' Mineral fuels, mineral USA France pieces and antiques oils and products of their distillation; bituminous substances; mineral waxes Mineral fuels, min- Organic chemicals Works of art, collectors' USA France eral oils and prod- pieces and antiques ucts of their distillation; bituminous substances; mineral waxes Inorganic chemicals; Works of art, collectors' Organic chemicals USA France organic or inorganic pieces and antiques compounds of precious metals, of rare earth metals, of radioactive elements or of isotopes Organic chemicals Works of art, collectors' Inorganic chemicals; USA France pieces and antiques organic or inorganic compounds of precious metals, of rare earth metals, of radioactive elements or of isotopes Inorganic chemicals; Organic chemicals Works of art, collectors' USA France organic or inorganic pieces and antiques compounds of precious metals, of rare 16

earth metals, of radioactive elements or of isotopes Mineral fuels, min- Inorganic chemicals; or- Organic chemicals USA France eral oils and prod- ganic or inorganic comucts of their distilla- pounds of precious mettion; bituminous als, of rare earth metals, substances; mineral of radioactive elements waxes or of isotopes Mineral fuels, min- Organic chemicals Inorganic chemicals; USA France eral oils and prod- organic or inorganic ucts of their distilla- compounds of precious tion; bituminous metals, of rare earth substances; mineral metals, of radioactive waxes elements or of isotopes Inorganic chemicals; Organic chemicals Mineral fuels, mineral USA France organic or inorganic oils and products of compounds of pre- their distillation; bitucious metals, of rare minous substances; earth metals, of ra- mineral waxes dioactive elements or of isotopes Mineral fuels, min- Works of art, collectors' Pharmaceutical prod- USA France eral oils and prod- pieces and antiques ucts ucts of their distillation; bituminous substances; mineral waxes Pharmaceutical Works of art, collectors' Mineral fuels, mineral USA France products pieces and antiques oils and products of their distillation; bituminous substances; mineral waxes Glass and glassware Natural or cultured Ceramic products USA Gerpearls, precious or semi- many precious stones, precious metals, metals clad with precious metal, and articles thereof; imitation jewellery; coin Ceramic products Natural or cultured Glass and glassware USA Gerpearls, precious or semi- many precious stones, precious metals, metals clad with 17

precious metal, and articles thereof; imitation jewellery; coin Ceramic products Glass and glassware Natural or cultured USA Gerpearls, precious or sem- many iprecious stones, precious metals, metals clad with precious metal, and articles thereof; imitation jewellery; coin Glass and glassware Natural or cultured Articles of stone, plas- USA Gerpearls, precious or semi- ter, cement, asbestos, many precious stones, precious mica or similar materimetals, metals clad with als precious metal, and articles thereof; imitation jewellery; coin Articles of stone, Natural or cultured Glass and glassware USA Gerplaster, cement, as- pearls, precious or semi- many bestos, mica or simi- precious stones, precious lar materials metals, metals clad with precious metal, and articles thereof; imitation jewellery; coin Articles of stone, Glass and glassware Natural or cultured USA Gerplaster, cement, as- pearls, precious or sem- many bestos, mica or simi- iprecious stones, prelar materials cious metals, metals clad with precious metal, and articles thereof; imitation jewellery; coin Glass and glassware Natural or cultured Prepared feathers and USA Gerpearls, precious or semi- down and articles made many precious stones, precious of feathers or of down; metals, metals clad with artificial flowers; artiprecious metal, and arti- cles of human hair cles thereof; imitation jewellery; coin Prepared feathers Natural or cultured Glass and glassware USA Gerand down and arti- pearls, precious or semi- many cles made of feathers precious stones, precious or of down; artificial metals, metals clad with flowers; articles of human hair 18

precious metal, and articles thereof; imitation jewellery; coin Prepared feathers Glass and glassware Natural or cultured USA Gerand down and arti- pearls, precious or sem- many cles made of feathers iprecious stones, preor of down; artificial cious metals, metals flowers; articles of clad with precious human hair metal, and articles thereof; imitation jewellery; coin Glass and glassware Natural or cultured Umbrellas, sun umbrel- USA Gerpearls, precious or semi- las, walking sticks, seat many precious stones, precious sticks, whips, riding metals, metals clad with cropsand parts thereof precious metal, and articles thereof; imitation jewellery; coin Umbrellas, sun um- Natural or cultured Glass and glassware USA Gerbrellas, walking pearls, precious or semi- many sticks, seat sticks, precious stones, precious whips, riding crops metals, metals clad with and parts thereof precious metal, and articles thereof; imitation jewellery; coin Glass and glassware Natural or cultured Ceramic products USA Japan pearls, precious or semiprecious stones, precious metals, metals clad with precious metal, and articles thereof; imitation jewellery; coin Ceramic products Natural or cultured Glass and glassware USA Japan pearls, precious or semiprecious stones, precious metals, metals clad with precious metal, and articles thereof; imitation jewellery; coin Ceramic products Glass and glassware Natural or cultured USA Japan pearls, precious or semiprecious stones, precious metals, metals clad with precious metal, and articles 19

thereof; imitation jewellery; coin Glass and glassware Natural or cultured Articles of stone, plas- USA Japan pearls, precious or semi- ter, cement, asbestos, precious stones, precious mica or similar materimetals, metals clad with als precious metal, and articles thereof; imitation jewellery; coin Articles of stone, Natural or cultured Glass and glassware USA Japan plaster, cement, as- pearls, precious or semibestos, mica or simi- precious stones, precious lar materials metals, metals clad with precious metal, and articles thereof; imitation jewellery; coin Articles of stone, Glass and glassware Natural or cultured USA Japan plaster, cement, as- pearls, precious or sembestos, mica or simi- iprecious stones, prelar materials cious metals, metals clad with precious metal, and articles thereof; imitation jewellery; coin Glass and glassware Natural or cultured Prepared feathers and USA Japan pearls, precious or semi- down and articles made precious stones, precious of feathers or of down; metals, metals clad with artificial flowers; artiprecious metal, and arti- cles of human hair cles thereof; imitation jewellery; coin Prepared feathers Natural or cultured Glass and glassware USA Japan and down and arti- pearls, precious or semicles made of feathers precious stones, precious or of down; artificial metals, metals clad with flowers; articles of precious metal, and artihuman hair cles thereof; imitation jewellery; coin Prepared feathers Glass and glassware Natural or cultured USA Japan and down and arti- pearls, precious or semcles made of feathers iprecious stones, preor of down; artificial cious metals, metals flowers; articles of clad with precious human hair metal, and articles thereof; imitation jewellery; coin 20

Glass and glassware Natural or cultured Umbrellas, sun umbrel- USA Japan pearls, precious or semi- las, walking sticks, seat precious stones, precious sticks, whips, riding metals, metals clad with crops and parts thereof precious metal, and articles thereof; imitation jewellery; coin Umbrellas, sun um- Natural or cultured Glass and glassware USA Japan brellas, walking pearls, precious or semisticks, seat sticks, precious stones, precious whips, riding crops metals, metals clad with and parts thereof precious metal, and articles thereof; imitation jewellery; coin Umbrellas, sun um- Glass and glassware Natural or cultured USA Japan brellas, walking pearls, precious or semsticks, seat sticks, iprecious stones, prewhips, riding crops cious metals, metals and parts thereof clad with precious metal, and articles thereof; imitation jewellery; coin Table 5: Top Five Food and Ag Commodities for China (HS-6) Anteced- Conse- Antecedent 1 Antecedent 2 Consequent Sup- Con- Lift Count ents quent name name name port fidence {120220, {120210} Ground-nuts; Sugar confection- Ground-nuts; 0.3 0.81 2.33 593 170490} shelled, not ery; (excluding in shell, not roasted or oth- chewing gum, in- roasted or otherwise cooked, cluding white erwise cooked whether or not chocolate), not broken containing cocoa {120210, {120220} Ground-nuts; in Sugar confection- Ground-nuts; 0.3 0.9 2.31 593 170490} shell, not ery; (excluding shelled, not roasted or oth- chewing gum, in- roasted or otherwise cooked cluding white erwise cooked, chocolate), not whether or not containing cocoa broken {120210, {120220} Ground-nuts; in Vegetable prepa- Ground-nuts; 0.3 0.92 2.36 589 200310} shell, not rations; mush- shelled, not roasted or oth- rooms, prepared roasted or otherwise cooked or preserved oth- erwise cooked, erwise than by whether or not broken 21

vinegar or acetic acid {120220, {120210} Ground-nuts; Vegetable prepa- Ground-nuts; 0.3 0.82 2.34 589 200310} shelled, not rations; mush- in shell, not roasted or oth- rooms, prepared roasted or otherwise cooked, or preserved oth- erwise cooked whether or not erwise than by broken vinegar or acetic acid {070320, {120220} Vegetables, alli- Ground-nuts; in Ground-nuts; 0.3 0.91 2.34 586 120210} aceous; garlic, shell, not roasted shelled, not fresh or chilled or otherwise roasted or othcooked erwise cooked, whether or not broken The next section presents contextual methods, and discussions on how contextual analysis and AI can aid in detecting outlier events and understanding their effects. 7. Outlier Events and Contextual AI Outlier events or uncertainty affects economic outcomes. Uncertainty can arise from natural disasters, health pandemics or market disruptions as in the recent trade war between U.S. and China. Such uncertainty sets in motion a cascade of events: price changes, behavioral changes by producers and consumers, shipping and transportation issues, worker welfare and others. Moreover, the strength of cascading events varies by product or region of analysis. The three most critical adverse world incidents since the 1870s were World War II, the Great Depression in America, and World War I. Results from multiple studies suggest that the Great Influenza Pandemic of 1918-1920 is the next most important negative economic shock for the world (Barro et al., 2020). Not all outliers are created equal; the historical record suggests that the 1918 influenza was an outlier among outliers, with unusual circumstances including the co-occurrence of World War I. No other influenza pandemic on record had such devastatingly high mortality rates, with estimates ranging from 20 to 50 million excess deaths over the period 1918-20 (Fan et al., 2020). The ongoing Covid-19 pandemic is likely to be the next big outlier. Prior to Covid-19, researchers at the U.S. Centers for Disease Control and Prevention (CDC) calculate (using traditional models) that deaths in the United States could reach 207,000 and the initial cost to the economy could 22

approach $166 billion, or roughly 1.5 percent of GDP in case of an international pandemic similar to the 1918 outbreak (Garrett, 2008). Modeling and predicting the implications of such outlier events is an important endeavor. During these outlier events, analysis is performed in uncharted waters. Issues arise such as the need to use daily if not hourly data (but not monthly data) for pattern recognition and predictions. Additionally, decisions become timelier and need to be executed in a quick manner using real time analysis and on-demand analytics. AI methods are ideal in this context to not only help understand the impact of uncertainty or outliers but also provide on-time information to economic agents including policy makers. Deploying context (a prominent field within AI) to represent outlier events is complex but appropriate for the current coronavirus pandemic. The context within a dataset can be represented as features (Turney, 2002). Features in general fall into three categories: primary features, irrelevant features, and contextual features. Primary features are the traditional ones which are pertinent to a particular domain. Irrelevant features are features which occur randomly and can be safely removed, while contextual features are the ones to pay close attention to. The above categorization helps in eliminating irrelevant data but additional work is needed to clearly define context: Recognition and Exploitation of Contextual Clues via Incremental Meta-Learning, IML (Widmer, 1996), which is a two-level learning model in which a Bayesian classifier is used for context classification, and meta algorithms are used to detect contextual changes. An alternative to IML is context-sensitive feature selection (Domingos, 1997), which out performs traditional feature selection such as forward and backward sequential selection. Dominogos’s (1997) method uses a clustering approach to select locally-relevant features. Bergadano et al. (1992) introduced a two-tier contextual classification adjustment method called POSIEDON. The first tier captures the basic properties of context, and the second tier captures property modifications and context dependencies. Context injections however, have been more successful when they are applied to specific domains. For example, adding context to data has significantly improved the accuracy of algorithms for solving Natural Language Processing (NLP) problems. Dinh et al. (2012) combined the output from the classifier with a set of words manually labeled with context. A transformation-based learning algorithm was then used to generate new rules for the classifier. Their approach increased the contextual accuracy of their application by 4.8% as well as in software testing, i.e. significant improvements in time and quality of testing results due to context (Batarseh, 2014). 23

The issue of deriving context from data for outlier detection however, is challenging since Williams (2018) pointed out that data science algorithms could have an opacity problem when ignoring the context. This can cause models to be racist or sexist (for example). It is often observed that Google translator refers to women as ‘he said’ or ‘he wrote’ when translating from Spanish to English. This finding was also verified by Google Inc. Another opacity example is a word embedding algorithm which classifies European names as pleasant and African American names as unpleasant (Zou et al., 2018). If a reductionist approach is considered, adding or removing data can surely redefine context, especially in the case of outlier events. It is observed however, that most real-world data science projects use incomplete data (Sesa and Syed, 2016) (Kang, 2013). Data incompleteness occurs within one of the following categorizations: 1) Missing Completely at Random (MCAR), 2) Missing at Random (MAR), and 3) Missing not at Random (MNAR). MAR depends on the observed data, but not on unobserved data while MCAR depends neither on observed data nor unobserved data (Schafer and Graham, 2002) (Graham, 2009). There are various methods to handle missing data issues which includes list wise or pair wise detections, multiple imputation, mean/median/mode imputation, regression imputation, as well as learning without handling missing data. All the aforementioned methods require high quality data, since several types of bias can occur in any phase of the data science lifecycle or while extracting context. Bias can begin during data collection, data cleaning, modeling, or any other phase. Biases which arise in the data are independent of the sample size or statistical significance, and they can directly affect the context of the results or the model. They also affect the association between variables, and in extreme cases, they can even reflect the opposite of a true association or correlation (Pannucci, 2010). Based on reviewing multiple works in data science, the most commonly observed bias is class imbalance due to covariate shifts. Class imbalance is represented by the un-equal ratio of categories which can occur due to changes in the distribution of data (covariate shifts). Class imbalance depends on four factors: 1) degree of class imbalance 2) the complexity of the concept represented by the data 3) the overall size of the training size and 4) the type of classifier (Japkowicz, 2002). Datasets with imbalance create difficulties in information retrieval, filtering tasks, and knowledge representation (Lewis and Ringuette, 1994) (Lewis and Catlett, 1994) – which (if not accounted for) may lead to misinformation in the agricultural or economic domain. We aim to explore with the mentioned methods and test them along with the presented AI methods during conventional and outlier times. For instance, variations of RL methods can lead to pointers in causality and endogeneity. Injecting contextual data from outlier events (i.e. relevant to a black swan situation such as 24

the Covid-19 pandemic) would lead to retraining of the models in a manner that would influence the patterns found. DL and RL models have the ability to seamlessly include outlier data and use it for predictions and classifications; a notion that we intend to explore and experiment with in our future work. 8. Traditional Economics and Other Conclusions This study proposed a novel approach to understanding international trade patterns using AI methods. While traditional trade studies for over a century have provided important insights, the emerging big data environment and ongoing outlier events necessitate a nimbler and data-driven approach. First, we laid out AI methods appropriate for predicting trade patterns: Linear Regression, K-means clustering, Pearson correlations, Time Series such as Autoregressive Integrated Moving Average (ARIMA), and supervised and unsupervised AI methods. Applying these methods to agricultural products and using beef as example, we demonstrated that AI methods provide improved predictions relative to traditional models. Within this application, clustering major economies provided even better predictions. Next, we outlined association rules that can identify paired purchases in international trade. Using aggregated trade data, we demonstrated that such rules can identify complementarities and substitutability in international trade transactions. Finally, we showed how contextual AI’s classification of features into regular, irregular and contextual along with bias elimination can aid in modeling the recent outlier events like the trade war and Covid-19 pandemic. Results from the big data framework are aimed to be presented in data dashboards to the farmer to update them on daily events during outliers, and to policy makers at agricultural agencies. A sample example of what we envision a farm dashboard would be is illustrated in Figure 7. Such dashboards allow farmers to take decisions on irrigation, seeding, weather modeling, among many other timely frequent decisions. A key objective of quantitative economic analyses is to uncover relationships – e.g. demand, supply, prices or trade – for use in making predictions or forecasts of future outcomes. However, when the current systems generates forecasts for decision making, they require a range of ad hoc, expert-driven or a combination of simple forecasting models supplemented by subject matter expertise to econometricsbased methods and mega-models, i.e. applied general equilibrium. Employing such approaches, many international institutions and government agencies project economic variables including trade flows to 25

inform decisions in national and multilateral contexts (such as the World Economic Outlook – International Monetary Fund). These predictions are highly valued by producer and consumer groups as well as policymakers in making decisions. However, some of these predictions based on a combination of simple linear models and expert judgment, have limitations (Isengildina-Massa et al. 2011). Little guidance exists on theoretical modeling of trade policy uncertainty and its implications for producer and consumer behavior. As a result, ad hoc approaches to incorporating uncertainty can create specification bias in quantifying economic relationships and consequently, less precise outcomes on future agricultural trade patterns. The later, i.e. less precise forecasts, impacts producer and consumer decisions as well as government expenditures. These mega-models draw information from a variety of sources, e.g. elasticities, which can introduce additional specification errors or mismatch data distributions. Figure 7: Precision Agriculture Tools for Farmers during Outlier Events (Sfiligo, 2016) In sum, traditional models – ad hoc, econometrics or mega-models – have been challenged both on modeling uncertainties, and providing accurate and on-time information for policy and decision making. AI methods have the ability to provide solutions to these drawbacks. While better predictions are a key ingredient in decision-making of economic agents and public officials, the AI approaches can address 26

additional dimensions not necessarily dealt with in traditional studies. For instance, there has been a challenge to achieve consensus at international bodies such as the United Nations, World Trade Organization and others since 2000. For instance, the Doha round of multilateral trade negotiations initiated in 2001 has not been concluded as of today. Some have argued that G20, a group of 20 nations that account for much of world’s GDP and population, has become increasingly assertive on multilateral policies. For instance, In Annex II of G20’s Strategy for Global Trade Growth (SGTG, 2020), the G20 declares developing a world trade outlook indicator, and promoting further e-commerce development. That includes an initiative on an electronic World Trade Platform (e-WTP). The AI models illustrated in this paper can allow for contrasting between decisions made by G20 versus engaging all countries like at the United Nations. In sum, the experimental work in this article indicates the high relevance of AI for predicting trade patterns with a greater accuracy than traditional approaches. Future work will expand the scope of commodities as well as AI methods, e.g. causality and reinforcement learning, to simulate trade outcomes under alternative policy scenarios including outlier events in recent times. References 1. Batarseh, F.; and Yang, R. 2018. Federal Data Science: Transforming Government and Agricultural Policy using Artificial Intelligence. Elsevier’s Academic Press. ISBN: 0128124431. 2. Batarseh, F., Yang, R, 2020. Data Democracy: At the Nexus of Artificial Intelligence, Software Development, and Knowledge Engineering. Elsevier’s Academic Press. ISBN: 9780128183663. 3. The White House. 2008. Open Data Initiative. 4. Gevel, J.; and Noussair, C. 2013. The Nexus between Artificial Intelligence and Economics. Springer Briefs in Economics. DOI: 10.1007/978-3-642-33648-5_1 5. Gunning, D. 2017. Explainable Artificial Intelligence. DARPA’s update on XAI. 6. National Bureau of Economic Research (NBER), Available from: https://www.nber.org/ 7. Reddy, C.; Aggarwal, C. 2015. Healthcare Data Analytics. Taylor and Francis Group CRC Press. ISBN: 1482232111. 8. Batarseh, F., and Latif., E. 2015. Assessing the Quality of Service Using Big Data Analytics: With Application to Healthcare. Journal of Big Data Research. ISSN: 2214-5796. Vol. 4. pp. 13- 24 27

9. Niemi, D.; Pea, R.; Saxberg, B.; Clark, R. 2018. Learning Analytics in Education. Information Age Publishing. ISBN: 978-64113-369-2. 10. Alamar, B. 2013. Sports Analytics. Columbia University Press. ISBN: 0231162928. 11. Storm, H.; Baylis, K.; Heckelei, T. 2019. Machine Learning in Agricultural and Applied Economics. Proceedings of the European Review of Agricultural Economics. 12. Feng, L.; Zhang, J. 2014. Application of Artificial Neural Networks in Tendency Forecasting of Economic Growth. Proceedings of Economic Modeling. pp. 76.80 13. Abadie, A.; Diamond, A.; Hainmueller, J. 2010. Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program. Journal of the American Statistical Association. Vol. 105. pp. 493-505. 14. Milacic, L.; Jovic, S.; Vujovic, T.; Miljkovic, J. 2016. Application of Artificial Neural Network with Extreme Learning Machine for Economic Growth Estimation. Elseviers Journal of Physica: Statistical Mechanics and its Applications. pp. 285-288. 15. Kordanuli, B.; Barjaktarovic, L.; Jeremic, L.; Alizamir, M. 2016. Appraisal of Artificial Neural Network for Forecasting of Economic Parameters. Elseviers Journal of Physica: Statistical Mechanics and its Applications. pp. 515-519. 16. Falat, L.; Pancikova, L. 2015. Quantitative Modelling in Economics with Advanced Artificial Neural Networks. Proceedings of the Business Economics and Management Conference. pp. 194-201. 17. Barro, R., Ursúa, J., and Weng, J. The Coronavirus and the Great Influenza Pandemic: Lessons from the “Spanish Flu” for the Coronavirus’s Potential Effects on Mortality and Economic Activity. NBER Working Paper No. 26866. 2020. JEL No. E1,I0,O4 18. Sullivan, K., China's tariffs on Australian barley could see export market lose hundreds of millions of dollars, 2020, ABC news report, Available at: https://www.abc.net.au/news/2020-05- 19/barley-china-australia-tariffs-trade-dispute-explained/12263298 19. Eichenbaum, M., Rebelo, S., and Trabandt, M. The Macroeconomics of Epidemics. NBER Working Paper No. 26882. 2020. JEL No. E1,H0,I1 20. Fan, V., Jamison, D., and Summers, L., The Inclusive Cost of Pandemic Influenza Risk. Working Paper 22137. National Bureau of Labor Statistics (NBER). JEL No. H51,I15,I18 21. Center for Disease Control and Prevention, Available at: http://www.cdc.gov, 2019 22. Foreign Agricultural Service. 2019. Available from: https://apps.fas.usda.gov/Gats/default.aspx 28

23. World Bank, WITS. 2019. Available from: https://wits.worldbank.org/ 24. Python Libraries. 2019. Available from: https://docs.python.org/3/library/index.html 25. Ke, G.; Meng, Q.; Finley, T.; Wang, T.; Chen, W.; Ma, W.; Ye, Q.; and Liu, T. 2017. LightGBM: A Highly Efficient Gradient Boosting. 31st Conference on Neural Information Processing Systems (NIPS). pp. 1-9 26. Piatetsky-Shapiro G. Discovery, Analysis, and Presentation of Strong Rules. From Knowledge Discovery in Databases. 1991. ISBN: 978-0-262-66070-9. 27. Piatetsky-Shapiro, WJ Frawley (eds.), Knowledge Discovery in Databases. AAAI/MIT Press, Cambridge, MA. 1991. 28. Agrawal R, Imielinski T, Swami A. Mining Association Rules between Sets of Items in Large Databases. In Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, pp. 207–216. ACM Press. URL: http://doi.acm.org/10.1145/170035.170072. (1993). 29. Turney, P.D., 2002. The management of context-sensitive features: A review of strategies. In 13th International Conference on Machine Learning, Workshop on Learning in Context-Sensitive Domains, Bari, Italy, pp. 60-66. 30. Widmer, G., 1996. Recognition and exploitation of contextual clues via incremental meta-learning (Extended version). In The 13th International Conference on Machine Learning, Morgan Kaufmann, San Francisco, pp. 525-533. 31. Domingos, P., 1997. Context-sensitive feature selection for lazy learners. In Lazy learning, pp. 227-253. Springer, Dordrecht. 32. Bergadano, F., Matwin, S., Michalski, R.S. and Zhang, J., 1992. Learning two-tiered descriptions of flexible concepts: The POSEIDON system. In Machine Learning, Vol. 8(1), pp. 5-43. 33. Dinh, P.H., Nguyen, N.K. and Le, A.C., 2012. Combining statistical machine learning with transformation rule learning for Vietnamese word sense disambiguation. In Computing and Communication Technologies, Research, Innovation, and Vision for the Future (RIVF), pp. 1-6. 34. Batarseh, F.A., 2014. Context-driven testing on the cloud. In Context in Computing, pp. 25-44. Springer, New York, NY. 35. Mary-Anne Williams, 2018. Risky bias in artificial intelligence. The Australian Academy of Technology and Engineering. Retrieved from: https://www.atse.org.au/content/news/risky-biasin-artificial-intelligence.aspx 29

36. Zou, J. and Schiebinger, L., 2018. AI can be sexist and racist -it’s time to make it fair. Retrieved from: https://www.nature.com/articles/d41586-018-05707-8 37. Sessa, J. and Syed, D., 2016. Techniques to deal with missing data. In Electronic Devices, Systems and Applications (ICEDSA) 5th International Conference, pp. 1-4. 38. Kang, H., 2013. The prevention and handling of the missing data. In Korean journal of anesthesiology, Vol. 64(5), pp. 402-406. 39. Schafer, J.L. and Graham, J.W., 2002. Missing data: our view of the state of the art. In Psychological meth-ods, Vol. 7(2), pp. 147. 40. Graham, J.W., 2009. Missing data analysis: Making it work in the real world. In Annual review of psychology, Vol. 60, pp. 549-576. 41. Pannucci, C.J. and Wilkins, E.G., 2010. Identifying and avoiding bias in research. In Plastic and reconstructive surgery, Vol. 126 (2), pp. 619. 42. Japkowicz, N. and Stephen, S., 2002. The class imbalance problem: A systematic study. In Intelligent data analysis, Vol. 6(5), pp. 429-449. 43. Lewis, D.D. and Ringuette, M., 1994. A comparison of two learning algorithms for text categorization. In Third annual symposium on document analysis and information retrieval, Vol. 33, pp. 81-93. 44. Lewis, D.D. and Catlett, J., 1994. Heterogeneous un-certainty sampling for supervised learning. In Machine Learning, 1994, pp. 148-156 45. Gopinath, M., Batarseh, F.A., and Beckman, J. Machine Learning in Gravity Models: An Application to Agricultural Trade. NBER Working Paper No. 27151. May 2020. 46. Sfiligo, E. 2016. Precision Agriculture: Top 10 technologies. Available at: https://www.therobotreport.com/top-10-technologies-in-precision-agriculture/ 47. SGTG, 2020, In Annex II of G20’s Strategy for Global Trade Growth. Available at: https://trade.ec.europa.eu/doclib/docs/2016/july/tradoc_154789.pdf 48. Isengildina‐Massa, O., S. Irwin, D.L. Good, and L. Massa. Empirical confidence intervals for USDA commodity price forecasts. Applied Economics 43:3789‐3803. (2011). 30