Synthetic control emerges as a useful data science tool to test policy interventions in economics and social sciences

Jan. 9, 2023

In the last quarter of this year, news organizations have been reporting that the United Kingdom is headed toward a recession, with economists saying that Brexit, the economic decoupling of the United Kingdom and the European Union, is a major factor.

The economic downturn has not been a surprise to many economists. In fact, in 2017, a group of them deployed an emerging method called synthetic control in order to estimate economic costs. Synthetic control is when one group is compared to another group that has been synthetically created and treated with an intervention. In this case, the researchers compared the UK economy with and without Brexit.

This study, The Economic Consequences of the Brexit Vote, was published in 2017 by the Centre For Macroeconomics. The experiment showed that the economic loss from Brexit would be comparable to the downturn in the 2008 financial crisis, which has been borne out in recent news reports.

This real-life example was one of many that many researchers used at a synthetic control workshop earlier this year, from June 2nd to 3rd, at Princeton University. The two-day workshop’s aim was to educate researchers on how to use the technique, which has proven useful in modeling the impact of socio-economic trends and policies, such as Brexit. Videos, slides, and the agenda from the event can be found here.

The Center for Statistics and Machine Learning (CSML), DataX and the Department of Operations Research and Financial Engineering (ORFE) sponsored the event. CSML oversees portions of DataX, which is tasked with spreading and deepening the use of machine learning across campus.

Matias Cattaneo, ORFE professor, and Alberto Abadie, professor of economics at Massachusetts Institute of Technology (MIT), led the event, which had talks from 20 speakers. They came from various academic institutions such as the Center for Monetary and Financial Studies in Madrid, Spain; University of California, Berkeley and San Diego; University of Pennsylvania; Columbia University; Tsinghua University; Stanford University; the University of Texas at Austin; Rutgers University; University of Chicago; Yale University; and Boston University.

The first day of the workshop was anchored by tutorials and introductions on synthetic control from Abadie and Devavrat Shah, a professor at MIT’s electrical engineering and computer science department.

Abadie first proposed the synthetic control technique in 2003 with co-author Javier Gardeazabal in a landmark paper, “The Economic Costs of Conflict: A Case Study of the Basque Country,” which found that “after the outbreak of terrorism in the late 1960's, per capita GDP in the Basque Country declined about 10 percentage points relative to a synthetic control region without terrorism.”

“I am originally from Basque Country in Spain. There were a lot of newspapers talking about the impact of terrorism, but there were no numbers to back up those claims. So we decided to focus on this issue,” said Abadie.

Abadie showed how synthetic control has been used in economic and social science studies, such as studying the effects of legalized prostitution, immigration policy, and changes to the minimum wage. Shah used Brexit and the 2007 study mentioned earlier in this article to illustrate the efficacy of the synthetic control method.

And big tech companies have also used synthetic control to examine the impact of changes from proposed software or processes, said Abadie.

The second half of the first day had detailed presentations on synthetic control methods from Uros Seljak of Berkeley and Guido Imbens from Stanford. Imbens won the Nobel Prize in economics in 2021. 

“Synthetic control methods have become very popular,” Imben said. “And this a natural way to get good and credible estimates of policy effects, such as in the case of Brexit. To figure out how the impact of Brexit, it’s natural to look at other places that are similar to Britain and look how they fare without leaving the European Union.”

The second day had 11 speakers giving talks on different aspects of synthetic control. In one talk, Yingjie Feng, a professor at Tsinghua University, discussed uncertainty qualification for synthetic control. Uncertainty quantification is the process of analyzing and managing uncertainty in complex systems. The talk was based on two papers: the October publication of “Uncertainty Quantification in Synthetic Controls with Staggered Treatment Adoption,”which Feng had co-authored with Cattaneo, Filippo Palomba and Rocio Titiunik. And the second: “Prediction Intervals for Synthetic Control Methods,” published last year, with Feng, Cattaneo and Titiunik as co-authors, in the Journal of the American Statistical Association.

In one talk, Jinglong Zhao, a professor at Boston University, discussed synthetic controls for experimental design. In this case, he gave the example of a ride-sharing experiment examining the impact of a new driver-incentive pay program if it were to be deployed in a US city. Researchers could design a user-level randomized control trial for a city, but the drivers in the control group and drivers in the treated group may interfere with each other, Zhao said. This is where an experiment utilizing synthetic control can play an import role.

In another presentation, Dennis Shen, a postdoctoral fellow at Berkeley, discussed the use of synthetic interventions in a clinical study for treating Alzheimer’s disease. Shen said a pharmaceutical company performed a two-year study involving more than 1,000 patients who received four therapies, including one placebo. The study’s results pointed to all three therapies having an insignificant impact on patients. The company contacted Shen and other researchers to see if they could salvage any usable data from the clinical study. The research team utilized a similar method to synthetic control to examine if the three therapies had any impact on certain subsets of patients. They examined the question: “Can we estimate ADAS-COG score for each patient under each therapy?” ADAS-COG is a scale to quantify the level of cognition in Alzheimer patients.

Abadie was gratified by the event overall and the reception it received.

“Many people are working on synthetic control,” he said. “And this is an opportunity to exchange ideas, and to spur further progress in the field. It’s exciting that many young people are also involved.”