Impact Evaluation

class: center, middle, inverse, title-slide

# Impact Evaluation
### AECN 396/896-002

---

class: inverse, center, middle
name: mvr

.remark-slide-content.hljs-github h1 {
  margin-top: 5px;  
  margin-bottom: 25px;  
}

.remark-slide-content.hljs-github {
  padding-top: 10px;  
  padding-left: 30px;  
  padding-right: 30px;  
}

.panel-tabs {
  
  color: #841F27;
  margin-top: 0px;  
  margin-bottom: 0px;  
  margin-left: 0px;  
  padding-bottom: 0px;  
}

.panel-tab {
  margin-top: 0px;  
  margin-bottom: 0px;  
  margin-left: 3px;  
  margin-right: 3px;  
  padding-top: 0px;  
  padding-bottom: 0px;  
}

.panelset .panel-tabs .panel-tab {
  min-height: 40px;
}

.remark-slide th {
  border-bottom: 1px solid #ddd;
}

.remark-slide thead {
  border-bottom: 0px;
}

.gt_footnote {
  padding: 2px;  
}

.remark-slide table {
  border-collapse: collapse;
}

.remark-slide tbody {
  border-bottom: 2px solid #666;
}

.important {
  background-color: lightpink;
  border: 2px solid blue;
  font-weight: bold;
}

.remark-code {
  display: block;
  overflow-x: auto;
  padding: .5em;
  background: #ffe7e7;
}

.hljs-github .hljs {
  background: #f2f2fd;
}

.remark-inline-code {
  padding-top: 0px;
  padding-bottom: 0px;
  background-color: #e6e6e6;
}

.r.hljs.remark-code.remark-inline-code{
  font-size: 0.9em
}

.left-full {
  width: 80%;
  height: 92%;
  float: left;
}

.left-code {
  width: 38%;
  height: 92%;
  float: left;
}

.right-plot {
  width: 60%;
  float: right;
  padding-left: 1%;
}

.left5 {
  width: 49%;
  height: 92%;
  float: left;
}

.right5 {
  width: 49%;
  float: right;
  padding-left: 1%;
}

.left3 {
  width: 29%;
  height: 92%;
  float: left;
}

.right7 {
  width: 69%;
  float: right;
  padding-left: 1%;
}

.left4 {
  width: 38%;
  height: 92%;
  float: left;
}

.right6 {
  width: 60%;
  float: right;
  padding-left: 1%;
}

ul li{
  margin: 7px;
}

ul, li{
  margin-left: 15px; 
  padding-left: 0px; 
}

ol li{
  margin: 7px;
}

ol, li{
  margin-left: 15px; 
  padding-left: 0px; 
}

</style>

.full-width {
    display: flex;
    width: 100%;
    flex: 1 1 auto;
}
</style>

.blockquote p {
  margin-top: 0px;
  margin-bottom: 5px;
}
.blockquote > h1:first-of-type {
  margin-top: 0px;
  margin-bottom: 5px;
}
.blockquote > h2:first-of-type {
  margin-top: 0px;
  margin-bottom: 5px;
}
.blockquote > h3:first-of-type {
  margin-top: 0px;
  margin-bottom: 5px;
}
.blockquote > h4:first-of-type {
  margin-top: 0px;
  margin-bottom: 5px;
}

.text-shadow {
  text-shadow: 0 0 4px #424242;
}
</style>

.scroll-box-8 {
  height:8em;
  overflow-y: scroll;
}
.scroll-box-10 {
  height:10em;
  overflow-y: scroll;
}
.scroll-box-12 {
  height:12em;
  overflow-y: scroll;
}
.scroll-box-14 {
  height:14em;
  overflow-y: scroll;
}
.scroll-box-16 {
  height:16em;
  overflow-y: scroll;
}
.scroll-box-18 {
  height:18em;
  overflow-y: scroll;
}
.scroll-box-20 {
  height:20em;
  overflow-y: scroll;
}
.scroll-box-24 {
  height:24em;
  overflow-y: scroll;
}
.scroll-box-30 {
  height:30em;
  overflow-y: scroll;
}
.scroll-output {
  height: 90%;
  overflow-y: scroll;
}

</style>

# Impact (Program) Evaluation

---
class: middle

.content-box-green[**Definition**]

Impact (program) evaluation is a field of econometrics that focuses on estimating the impact of a program or event.

<br>

.content-box-green[**Examples**]

+ Groundwater use limit in Nebraska `$\Rightarrow$` water use 
+ Technology adoption (soil moisture sensor) `$\Rightarrow$` water use 
+ Crop insurance `$\Rightarrow$` input use 
+ Job training program `$\Rightarrow$` productivty 
+ Food Stamp `$\Rightarrow$` health, education, etc

---
class: middle

.content-box-red[**Key challenge**]

Most of the programs you are interested in are not randomized.

`$\downarrow$`

Selection Bias (endogeneity problem arising from self-selection into the program)

---
class: middle

.content-box-green[**Gold Standard**]

+ The best (if feasible) way to tackle the problem of selection bias in impact evaluation is randomized experiment, where who gets treated or not is determined randomly (you design a program or experiment and randomize treatment-control assignment)

+ This ensures that the treatment status (dummy variable indicating treated or not) is not correlated with the error term

<br>

.content-box-green[**Example**]

`$y \;\;(\mbox{income}) = \beta_0 + \beta_1 program \;\;(\mbox{financial aid}) + u$`

, where `$E[u|program]=0$` (the program is not correlated with the error term). OLS is just fine.

<br>

.content-box-red[**Problem**]

Many of the programs are simply not possible to randomize because of financial and/or ethical reasons.

`$\downarrow$`

We need to use data from an event that happened outside our control.

---
class: middle

# Natural (Quasi) Experiment

.content-box-green[**Definition**]

An event or policy change (often a change in government policy) that happens .blue[outside of the control of investigators], which changes the environment in which agents (individuals, families, firms, or cities) operate.

<br>

.content-box-green[**Challenges**]

The program is most likely correlated with the error term.

<br>

.content-box-green[**Lecture Objectives**]

+ Discuss different ways of estimating the impact of a program

+ Understand the strength and weakness of these methods

---
class: middle

# Example program

.content-box-green[**Incinerator Construction**]

+ rumored about the incinerator being built in North Andover, Massachusetts, began in 1978

+ construction started in 1981

<br>

.content-box-green[**Data collected**]

Housing prices in 1978 and 1981, and other variables (we observations before and after the incinerator construction)

---
class: middle
 
# Various Approaches

+ <span style = "color: blue;"> Approach 1 </span>: cross-sectional comparison of houses that are close to (treated) and far away from (control) to the incinerator .red[after] the incinerator was built (data in 1981)
  
+ <span style = "color: blue;"> Approach 2 </span>: comparison of the houses that are close to the incinerator before (control) and after (treated) the incinerator was built (data in 1978 and 1981)

+ <span style = "color: blue;"> Approach 3 </span>: comparison of differences (close by v.s. far away) in differences (before-after) of house prices (this method will become clearer later)

---
class: middle

# Approach 1

Run regression on the following model using the 1981 data (cross-sectional data)

`$rpice = \gamma_0 + \gamma_1 nearinc + u$`

+ `$rprice$`: house price in real terms (inflation-corrected)

+ `$nearinc$`: 1 if the house is near the incinerator, and 0 otherwise

+ `$\gamma_1$`: .blue[the difference between the mean house price of houses nearby the incinerator and the rest (not nearby) in 1981]

<br>

.content-box-green[**Question**]

Is `nearinc` endogenous?

---
class: middle

<template id="3ae758d9-14b8-4823-a0b0-292a680e58fd"><style>
.tabwid table{
  border-spacing:0px !important;
  border-collapse:collapse;
  line-height:1;
  margin-left:auto;
  margin-right:auto;
  border-width: 0;
  display: table;
  margin-top: 1.275em;
  margin-bottom: 1.275em;
  border-color: transparent;
}
.tabwid_left table{
  margin-left:0;
}
.tabwid_right table{
  margin-right:0;
}
.tabwid td {
    padding: 0;
}
.tabwid a {
  text-decoration: none;
}
.tabwid thead {
    background-color: transparent;
}
.tabwid tfoot {
    background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-d233440c{}.cl-d22dbbe0{font-family:'Helvetica';font-size:12pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-d22dbbea{font-family:'Helvetica';font-size:12pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(255, 0, 0, 1.00);background-color:transparent;}.cl-d22dd1c0{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-d22dfc90{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfc9a{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfc9b{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfca4{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfca5{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfcae{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfcaf{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfcb0{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfcb1{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfcb8{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfcb9{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfcba{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfcc2{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfcc3{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfccc{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d22dfcd6{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-d233440c'><thead><tr style="overflow-wrap:break-word;"><td class="cl-d22dfccc"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0"> </span></p></td><td class="cl-d22dfcd6"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">Model 1</span></p></td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-d22dfc9a"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">(Intercept)</span></p></td><td class="cl-d22dfc90"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">101.308***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d22dfca4"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0"></span></p></td><td class="cl-d22dfc9b"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">(3.093)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d22dfcae"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">nearinc</span></p></td><td class="cl-d22dfca5"><p class="cl-d22dd1c0"><span class="cl-d22dbbea">-30.688***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d22dfca4"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0"></span></p></td><td class="cl-d22dfc9b"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">(5.828)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d22dfcb0"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">Num.Obs.</span></p></td><td class="cl-d22dfcaf"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">142</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d22dfcb8"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">R2</span></p></td><td class="cl-d22dfcb1"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">0.165</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d22dfcba"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">Std.Errors</span></p></td><td class="cl-d22dfcb9"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">IID</span></p></td></tr></tbody><tfoot><tr style="overflow-wrap:break-word;"><td  colspan="2"class="cl-d22dfcc2"><p class="cl-d22dd1c0"><span class="cl-d22dbbe0">+ p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</span></p></td></tr></tfoot></table></div></template>
<div class="flextable-shadow-host" id="94a4bc30-3d13-4148-82c2-95260a58e9d4"></div>
<script>
var dest = document.getElementById("94a4bc30-3d13-4148-82c2-95260a58e9d4");
var template = document.getElementById("3ae758d9-14b8-4823-a0b0-292a680e58fd");
var caption = template.content.querySelector("caption");
if(caption) {
  caption.style.cssText = "display:block;text-align:center;";
  var newcapt = document.createElement("p");
  newcapt.appendChild(caption)
  dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

<br>

.content-box-green[**Question**]
Is this reliable?

---
class: middle

# Take a look at 1978

Run regression on the following model using the .red[1978] data (cross-sectional data)

`$rpice = \gamma_0 + \gamma_1 nearinc + u$`

`$\gamma_1$` represents the difference between the mean house price of houses nearby the incinerator and the rest (not nearby) .red[before] the incinerator was built.

---
class: middle

<template id="41e9e393-d4d0-4a40-83f3-7ccd338d8d4b"><style>
.tabwid table{
  border-spacing:0px !important;
  border-collapse:collapse;
  line-height:1;
  margin-left:auto;
  margin-right:auto;
  border-width: 0;
  display: table;
  margin-top: 1.275em;
  margin-bottom: 1.275em;
  border-color: transparent;
}
.tabwid_left table{
  margin-left:0;
}
.tabwid_right table{
  margin-right:0;
}
.tabwid td {
    padding: 0;
}
.tabwid a {
  text-decoration: none;
}
.tabwid thead {
    background-color: transparent;
}
.tabwid tfoot {
    background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-d257e76c{}.cl-d2525338{font-family:'Helvetica';font-size:12pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-d2525342{font-family:'Helvetica';font-size:12pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(255, 0, 0, 1.00);background-color:transparent;}.cl-d2526260{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-d2528fec{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d2528ff6{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d2529000{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d2529001{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d252900a{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d252900b{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d252900c{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d2529014{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d2529015{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d2529016{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d252901e{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d252901f{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d2529028{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d2529029{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d2529032{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d2529033{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-d257e76c'><thead><tr style="overflow-wrap:break-word;"><td class="cl-d2529032"><p class="cl-d2526260"><span class="cl-d2525338"> </span></p></td><td class="cl-d2529033"><p class="cl-d2526260"><span class="cl-d2525338">Model 1</span></p></td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-d2528ff6"><p class="cl-d2526260"><span class="cl-d2525338">(Intercept)</span></p></td><td class="cl-d2528fec"><p class="cl-d2526260"><span class="cl-d2525338">82.517***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d2529001"><p class="cl-d2526260"><span class="cl-d2525338"></span></p></td><td class="cl-d2529000"><p class="cl-d2526260"><span class="cl-d2525338">(2.654)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d252900b"><p class="cl-d2526260"><span class="cl-d2525338">nearinc</span></p></td><td class="cl-d252900a"><p class="cl-d2526260"><span class="cl-d2525342">-18.824***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d2529001"><p class="cl-d2526260"><span class="cl-d2525338"></span></p></td><td class="cl-d2529000"><p class="cl-d2526260"><span class="cl-d2525338">(4.745)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d2529014"><p class="cl-d2526260"><span class="cl-d2525338">Num.Obs.</span></p></td><td class="cl-d252900c"><p class="cl-d2526260"><span class="cl-d2525338">179</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d2529016"><p class="cl-d2526260"><span class="cl-d2525338">R2</span></p></td><td class="cl-d2529015"><p class="cl-d2526260"><span class="cl-d2525338">0.082</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d252901f"><p class="cl-d2526260"><span class="cl-d2525338">Std.Errors</span></p></td><td class="cl-d252901e"><p class="cl-d2526260"><span class="cl-d2525338">IID</span></p></td></tr></tbody><tfoot><tr style="overflow-wrap:break-word;"><td  colspan="2"class="cl-d2529028"><p class="cl-d2526260"><span class="cl-d2525338">+ p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</span></p></td></tr></tfoot></table></div></template>
<div class="flextable-shadow-host" id="608b1f70-f6e1-4da8-8989-797044e087eb"></div>
<script>
var dest = document.getElementById("608b1f70-f6e1-4da8-8989-797044e087eb");
var template = document.getElementById("41e9e393-d4d0-4a40-83f3-7ccd338d8d4b");
var caption = template.content.querySelector("caption");
if(caption) {
  caption.style.cssText = "display:block;text-align:center;";
  var newcapt = document.createElement("p");
  newcapt.appendChild(caption)
  dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

<br>

.content-box-red[**Critical**]

Houses nearby the incinerator were already lower than those houses that are not nearby...

---
class: middle

---
class: middle

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> treated </th>
   <th style="text-align:left;"> before </th>
   <th style="text-align:left;"> after </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> nearinc = 0 </td>
   <td style="text-align:left;"> $\gamma_0$ </td>
   <td style="text-align:left;"> $\gamma_0 + \alpha_0 + 0$ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> nearinc = 1 </td>
   <td style="text-align:left;"> $\gamma_1$ </td>
   <td style="text-align:left;"> $\gamma_1 + \alpha_1 + \beta $ </td>
  </tr>
</tbody>
</table>

<br>

+ `$\gamma_j$` is the average house price of those that are `$nearinc=j$` in 1978 (before)

+ `$\alpha_j$` is .blue[any] macro shocks .blue[other than the incinerator event] that happened between the before and after period to the houses that are `$nearinc=j$`

+ `$\beta$` is the true causal impact of the incinerator placement

---
class: middle

+ `$\gamma_j$` is the average house price of those that are `$nearinc=j$` in 1978 (before)

+ `$\alpha_j$` is .blue[any] macro shocks .blue[other than the incinerator event] that happened between the before and after period to the houses that are `$nearinc=j$`

+ `$\beta$` is the true causal impact of the incinerator placement

<br>

.content-box-green[**Question**]

So, what did we estimate with Approach 1?

<br>

.content-box-green[**Answer**]

`$E[rprice|nearinc = 1, year = 1981] - E[rprice|nearinc = 0, year = 1981]$`

`$= (\gamma_1 + \alpha_1 + \beta) - (\gamma_0 + \alpha_0 + 0)$`

`$= (\gamma_1 - \gamma_0)+ (\alpha_1 - \alpha_0) + \beta$`

+ `$\gamma_1 - \gamma_0$`: pre-existing differences in house price .red[before] the incinerator was built

+ `$\alpha_1 - \alpha_0$`: differences in the trends in housing price between the two groups

---
class: middle

+ `$\gamma_1 - \gamma_0$`: pre-existing differences in house price .red[before] the incinerator was built

+ `$\alpha_1 - \alpha_0$`: differences in the trends in housing price between the two groups

<br>

.content-box-green[**Question**]

So, when Approach 1 gives us unbiased estimation of the impact of the incinerator?

<br>

.content-box-green[**Answer**]

+ `$\gamma_1 = \gamma_0$`: the average house price between the two groups are the same before the incinerator was built

+ `$\alpha_1 - \alpha_0$`: the two groups experienced the same house price trend from 1978 to 1981

---
class: middle

# Approach 2

Compare of the houses that are close to the incinerator before (control) and after (treated) the incinerator was built (data in 1978 and 1981)

<br>

.content-box-green[**Data**]

Restrict the data to the houses that are near the incinerator

<br>

.content-box-green[**Model**]

`$rpice = \beta_0 + \beta_1 y81 + u$`

+ `$rprice$`: house price in real terms (inflation-corrected)

+ `$y81$`: 1 if the house is near the incinerator, and 0 otherwise

+ `$\beta_1$`: .blue[the difference in the mean house price of houses nearby the incinerator before and after the incinerator was built]

---
class: middle

<template id="31ac8368-1aef-4be6-bb8a-6f47ce06dd94"><style>
.tabwid table{
  border-spacing:0px !important;
  border-collapse:collapse;
  line-height:1;
  margin-left:auto;
  margin-right:auto;
  border-width: 0;
  display: table;
  margin-top: 1.275em;
  margin-bottom: 1.275em;
  border-color: transparent;
}
.tabwid_left table{
  margin-left:0;
}
.tabwid_right table{
  margin-right:0;
}
.tabwid td {
    padding: 0;
}
.tabwid a {
  text-decoration: none;
}
.tabwid thead {
    background-color: transparent;
}
.tabwid tfoot {
    background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-d8b9b4a8{}.cl-d8b4da46{font-family:'Helvetica';font-size:12pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-d8b4da50{font-family:'Helvetica';font-size:12pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(255, 0, 0, 1.00);background-color:transparent;}.cl-d8b4e784{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-d8b50d54{width:71.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d55{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d5e{width:71.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d68{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d69{width:71.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d6a{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d72{width:71.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d73{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d7c{width:71.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d86{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d87{width:71.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d88{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d90{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d91{width:71.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d92{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8b50d9a{width:71.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-d8b9b4a8'><thead><tr style="overflow-wrap:break-word;"><td class="cl-d8b50d92"><p class="cl-d8b4e784"><span class="cl-d8b4da46"> </span></p></td><td class="cl-d8b50d9a"><p class="cl-d8b4e784"><span class="cl-d8b4da46">Model 1</span></p></td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-d8b50d55"><p class="cl-d8b4e784"><span class="cl-d8b4da46">(Intercept)</span></p></td><td class="cl-d8b50d54"><p class="cl-d8b4e784"><span class="cl-d8b4da46">63.693***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8b50d68"><p class="cl-d8b4e784"><span class="cl-d8b4da46"></span></p></td><td class="cl-d8b50d5e"><p class="cl-d8b4e784"><span class="cl-d8b4da46">(5.296)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8b50d6a"><p class="cl-d8b4e784"><span class="cl-d8b4da46">y81</span></p></td><td class="cl-d8b50d69"><p class="cl-d8b4e784"><span class="cl-d8b4da50">6.926</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8b50d68"><p class="cl-d8b4e784"><span class="cl-d8b4da46"></span></p></td><td class="cl-d8b50d5e"><p class="cl-d8b4e784"><span class="cl-d8b4da46">(8.205)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8b50d73"><p class="cl-d8b4e784"><span class="cl-d8b4da46">Num.Obs.</span></p></td><td class="cl-d8b50d72"><p class="cl-d8b4e784"><span class="cl-d8b4da46">96</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8b50d86"><p class="cl-d8b4e784"><span class="cl-d8b4da46">R2</span></p></td><td class="cl-d8b50d7c"><p class="cl-d8b4e784"><span class="cl-d8b4da46">0.008</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8b50d88"><p class="cl-d8b4e784"><span class="cl-d8b4da46">Std.Errors</span></p></td><td class="cl-d8b50d87"><p class="cl-d8b4e784"><span class="cl-d8b4da46">IID</span></p></td></tr></tbody><tfoot><tr style="overflow-wrap:break-word;"><td  colspan="2"class="cl-d8b50d90"><p class="cl-d8b4e784"><span class="cl-d8b4da46">+ p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</span></p></td></tr></tfoot></table></div></template>
<div class="flextable-shadow-host" id="f3618d8d-914f-453f-984c-4998bf273b69"></div>
<script>
var dest = document.getElementById("f3618d8d-914f-453f-984c-4998bf273b69");
var template = document.getElementById("31ac8368-1aef-4be6-bb8a-6f47ce06dd94");
var caption = template.content.querySelector("caption");
if(caption) {
  caption.style.cssText = "display:block;text-align:center;";
  var newcapt = document.createElement("p");
  newcapt.appendChild(caption)
  dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

<br>

The incinerator increased the average house price (not statistically significant).

---
class: middle

<br>

+ `$\gamma_j$` is the average house price of those that are `$nearinc=j$` in 1978 (before)

+ `$\alpha_j$` is .blue[any] macro shocks .blue[other than the incinerator event] that happened between the before and after period to the houses that are `$nearinc=j$`

+ `$\beta$` is the true causal impact of the incinerator placement

<br>

.content-box-green[**Question**]

So, what did we estimate with Approach 2?

<br>

.content-box-green[**Answer**]

`$E[rprice|nearinc = 1, year = 1981] - E[rprice|nearinc = 1, year = 1978]$`

`$= (\gamma_1 + \alpha_1 + \beta) - \gamma_1$`

`$= \alpha_1 + \beta$`

---
class: middle

`$E[rprice|nearinc = 1, year = 1981] - E[rprice|nearinc = 1, year = 1978]$`

`$= (\gamma_1 + \alpha_1 + \beta) - \gamma_1$`

`$= \alpha_1 + \beta$`

<br>

.content-box-green[**Question**]

So, when Approach 2 gives us unbiased estimation of the impact of the incinerator?

<br>

.content-box-green[**Answer**]

`$\alpha_1 = 0$`: no trend in house price for the houses near the incinerator

Nothing else significant other than the incinerator happened 
between 1978 and 1981.

---
class: middle

# Approach 3

Compare of differences (close by v.s. far away) in differences (before-after) of house prices (this method will become clearer later)

+ Find the difference in the price of the houses .red[close to] the incinerator before and after the incinerator was built

+ Find the difference in house price (close to the incinerator) before and after the incinerator was built

+ Find the difference in the differences

<br>

.content-box-green[**Note**]

This method is called DID estimation method (Difference-in- differences).

---
class: middle

<br>

.content-box-green[**Data**]

All the observations

<br>

.content-box-green[**Model**]

`$rpice = \beta_0 + \beta_1 y81 + \beta_2 nearinc + \beta_3 nearinc \times y81 + u$`

+ `$\beta_3$`: the difference in differences

Let's confirm `$\beta_3$` indeed represents the difference in the differences.

---
class: middle

.content-box-green[**Model**]

`$rpice = \beta_0 + \beta_1 y81 + \beta_2 nearinc + \beta_3 nearinc \times y81 + u$`

<br>

.content-box-green[**Expected house price**]

+ `$E[rprice|year=1981, nearinc = 0] = \beta_0 + \beta_1$`

+ `$E[rprice|year=1981, nearinc = 1] = \beta_0 + \beta_1 + \beta_2 + \beta_3$`

+ `$E[rprice|year=1978, nearinc = 0] = \beta_0$`

+ `$E[rprice|year=1978, nearinc = 1] = \beta_0 + \beta_2$`

<br>

.content-box-green[**Differences**]

`$E[rprice|year=1981, nearinc = 1] - E[rprice|year=1978, nearinc = 1]$`

`$= (\beta_0 + \beta_1 + \beta_2 + \beta_3) - (\beta_0 + \beta_2)$`
`$= \beta_1 + \beta_3$`

`$E[rprice|year=1981, nearinc = 0] - E[rprice|year=1978, nearinc = 0]$`

`$= (\beta_0 + \beta_1) - \beta_0$`
`$= \beta_1$`

<br>

.content-box-green[**Difference in the differences**]

`$(\beta_1 + \beta_3) - \beta_1 = \beta_3$`

---
class: middle

<template id="98b6ab0b-fbc8-4d7e-8bbe-ec630020642e"><style>
.tabwid table{
  border-spacing:0px !important;
  border-collapse:collapse;
  line-height:1;
  margin-left:auto;
  margin-right:auto;
  border-width: 0;
  display: table;
  margin-top: 1.275em;
  margin-bottom: 1.275em;
  border-color: transparent;
}
.tabwid_left table{
  margin-left:0;
}
.tabwid_right table{
  margin-right:0;
}
.tabwid td {
    padding: 0;
}
.tabwid a {
  text-decoration: none;
}
.tabwid thead {
    background-color: transparent;
}
.tabwid tfoot {
    background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-d8df3458{}.cl-d8da31c4{font-family:'Helvetica';font-size:12pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-d8da31ce{font-family:'Helvetica';font-size:12pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(255, 0, 0, 1.00);background-color:transparent;}.cl-d8da3ed0{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-d8da6c20{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c2a{width:92.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c34{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c35{width:92.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c36{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c3e{width:92.9pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c48{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c49{width:92.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c52{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c53{width:92.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c54{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c5c{width:92.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c5d{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c5e{width:92.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c66{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c67{width:92.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c70{width:92.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c71{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c72{width:92.9pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8da6c7a{width:75.2pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-d8df3458'><thead><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c72"><p class="cl-d8da3ed0"><span class="cl-d8da31c4"> </span></p></td><td class="cl-d8da6c7a"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">Model 1</span></p></td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c2a"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">(Intercept)</span></p></td><td class="cl-d8da6c20"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">82.517***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c49"><p class="cl-d8da3ed0"><span class="cl-d8da31c4"></span></p></td><td class="cl-d8da6c48"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">(2.727)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c53"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">nearinc</span></p></td><td class="cl-d8da6c52"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">-18.824***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c49"><p class="cl-d8da3ed0"><span class="cl-d8da31c4"></span></p></td><td class="cl-d8da6c48"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">(4.875)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c5c"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">y81</span></p></td><td class="cl-d8da6c54"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">18.790***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c49"><p class="cl-d8da3ed0"><span class="cl-d8da31c4"></span></p></td><td class="cl-d8da6c48"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">(4.050)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c5e"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">nearinc × y81</span></p></td><td class="cl-d8da6c5d"><p class="cl-d8da3ed0"><span class="cl-d8da31ce">-11.864</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c49"><p class="cl-d8da3ed0"><span class="cl-d8da31c4"></span></p></td><td class="cl-d8da6c48"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">(7.457)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c67"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">Num.Obs.</span></p></td><td class="cl-d8da6c66"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">321</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c35"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">R2</span></p></td><td class="cl-d8da6c34"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">0.174</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8da6c3e"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">Std.Errors</span></p></td><td class="cl-d8da6c36"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">IID</span></p></td></tr></tbody><tfoot><tr style="overflow-wrap:break-word;"><td  colspan="2"class="cl-d8da6c70"><p class="cl-d8da3ed0"><span class="cl-d8da31c4">+ p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</span></p></td></tr></tfoot></table></div></template>
<div class="flextable-shadow-host" id="7473b407-2aad-4e7f-b951-cb8c30253ec0"></div>
<script>
var dest = document.getElementById("7473b407-2aad-4e7f-b951-cb8c30253ec0");
var template = document.getElementById("98b6ab0b-fbc8-4d7e-8bbe-ec630020642e");
var caption = template.content.querySelector("caption");
if(caption) {
  caption.style.cssText = "display:block;text-align:center;";
  var newcapt = document.createElement("p");
  newcapt.appendChild(caption)
  dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

<br>

The incinerator decreased the average house price (not statistically significant).

---
class: middle

<br>

.content-box-green[**Question**]

So, what did we estimate with Approach 3?

<br>

.content-box-green[**Answer**]

`$E[rprice|nearinc = 1, year = 1981] - E[rprice|nearinc = 1, year = 1978]$`

`$= (\gamma_1 + \alpha_1 + \beta) - \gamma_1 = \alpha_1 + \beta$`

`$E[rprice|nearinc = 0, year = 1981] - E[rprice|nearinc = 0, year = 1978]$`

`$= (\gamma_0 + \alpha_0) - \gamma_0 = \alpha_0$`

Differences `$= \alpha_1 - \alpha_0 + \beta$`

---
class: middle

.content-box-green[**Difference**]

Difference `$= \alpha_1 - \alpha_0 + \beta$`

<br>

.content-box-green[**Question**]

So, when Approach 3 gives us unbiased estimation of the impact of the incinerator?

<br>

.content-box-green[**Answer**]

+ `$\alpha_1 = \alpha_0$`: the two groups experienced the same trend in house price from 1978 to 1981

+ Unlike Approach 1, the pre-existing difference between the two group is not a problem as it is canceled out

---
class: middle

.content-box-green[**Key condition (common/parallel trend assumption)**]

`$\alpha_1 = \alpha_0$`: the two groups experienced the same trend in house price from 1978 to 1981

<br>
.content-box-green[**Common/parallel trend assumption in general**]

If no treatment had occurred, the difference between the treated group and the untreated group would have stayed the same in the post-treatment period as it was in the pre-treatment period.

<br>

.content-box-red[**Important**]

This condition/assumption is <span style = "color: red;"> NOT </span> testable because you never observe what would the treament group be like if it were not for the treatment (we will discuss this further)

---
class: middle

.content-box-green[**Approaches**]
    
Approach 1: `$(\gamma_1 - \gamma_0)+ (\alpha_1 - \alpha_0) + \beta$`

Approach 2: `$\alpha_1 + \beta$`

Approach 3: `$\alpha_1 - \alpha_0 + \beta$`

<br>

.content-box-red[**Important**]

+ None of these approaches are perfect.

+ It is hard to sell Approaches 1 and 2

+ Approach 3 (DID) is preferred over Approaches 1 and 2

+ But, Approach 3 is not certainly perfect and could definitely have a larger bias than Approaches 1 and 2

e.g., `$\alpha_1 = 5$` and `$\alpha_0 = - 5$`

---
class: middle

# DID: Another Example

.content-box-green[**Cholera**]

+ Back in mid 1800s', Cholera was believed to spread via air
+ John Snow believe it was actually through fecally-contaminated water

<br>

.content-box-green[**Setting**]

+ London's water needs were served by a number of competing companies, who got their water intake from different parts of the Thames river.

+ Water taken in from the parts of the Thames that were downstream of London contained everything that Londoners dumped in the river, including plenty of fecal matter from people infected with cholera.

<br>

.content-box-green[**Natural Experiment**]

+ Between those two periods of 1849 and 1854, a policy was enacted: the Lambeth Company was required by an Act of Parliament to move their water intake upstream of London.

---
class: middle

.content-box-green[**Treatment**]

+ Switch of where water is taken (downstream to upstream)

<br>

.content-box-green[**Before and After**]

+ "before" (1849): Lambeth took water downstream
+ "after" (1854): Lambeth took water upstream

<br>

.content-box-green[**Control and Treatment Groups**]

+ Control group: those who were not served by Lambeth 
+ Treatment group: those who were served by Lambeth

---
class: middle

.content-box-green[**Data**]

<template id="34f06708-3485-4048-9ce4-158de933f722"><style>
.tabwid table{
  border-spacing:0px !important;
  border-collapse:collapse;
  line-height:1;
  margin-left:auto;
  margin-right:auto;
  border-width: 0;
  display: table;
  margin-top: 1.275em;
  margin-bottom: 1.275em;
  border-color: transparent;
}
.tabwid_left table{
  margin-left:0;
}
.tabwid_right table{
  margin-right:0;
}
.tabwid td {
    padding: 0;
}
.tabwid a {
  text-decoration: none;
}
.tabwid thead {
    background-color: transparent;
}
.tabwid tfoot {
    background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-d8fd0fd2{}.cl-d8f89fd8{font-family:'Helvetica';font-size:11pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-d8f8ac9e{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-d8f8aca8{margin:0;text-align:right;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-d8f8d1ba{width:48.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8f8d1c4{width:110.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8f8d1c5{width:48.1pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8f8d1ce{width:110.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8f8d1cf{width:48.1pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d8f8d1d8{width:110.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-d8fd0fd2'><thead><tr style="overflow-wrap:break-word;"><td class="cl-d8f8d1d8"><p class="cl-d8f8ac9e"><span class="cl-d8f89fd8">Supplier</span></p></td><td class="cl-d8f8d1cf"><p class="cl-d8f8aca8"><span class="cl-d8f89fd8">1849</span></p></td><td class="cl-d8f8d1cf"><p class="cl-d8f8aca8"><span class="cl-d8f89fd8">1854</span></p></td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-d8f8d1c4"><p class="cl-d8f8ac9e"><span class="cl-d8f89fd8">Non-Lambeth only</span></p></td><td class="cl-d8f8d1ba"><p class="cl-d8f8aca8"><span class="cl-d8f89fd8">134.9</span></p></td><td class="cl-d8f8d1ba"><p class="cl-d8f8aca8"><span class="cl-d8f89fd8">130.1</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d8f8d1ce"><p class="cl-d8f8ac9e"><span class="cl-d8f89fd8">Lambeth + Others</span></p></td><td class="cl-d8f8d1c5"><p class="cl-d8f8aca8"><span class="cl-d8f89fd8">146.6</span></p></td><td class="cl-d8f8d1c5"><p class="cl-d8f8aca8"><span class="cl-d8f89fd8">84.9</span></p></td></tr></tbody></table></div></template>
<div class="flextable-shadow-host" id="2411c979-d7ee-48cf-bd99-88bfd88cc95d"></div>
<script>
var dest = document.getElementById("2411c979-d7ee-48cf-bd99-88bfd88cc95d");
var template = document.getElementById("34f06708-3485-4048-9ce4-158de933f722");
var caption = template.content.querySelector("caption");
if(caption) {
  caption.style.cssText = "display:block;text-align:center;";
  var newcapt = document.createElement("p");
  newcapt.appendChild(caption)
  dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

.content-box-green[**DID estimate**]

Estimate treatment effect is:

(84.9 - 130.1) - (146.6 - 134.9) = -56.9

---
class: middle

# DID using R

.content-box-green[**Well-levle groundwater use data in Kansas**]

```r
(
  lema_data <- readRDS("LEMA_data.rds")
)
```

```
##         site year    af_used in_LEMA    pr       et0       awc bulkdensity
##     1:   160 1991 195.328540       1 401.9 1047.3311 0.1859333    1.359156
##     2:   160 1992  62.390479       1 463.9  904.0815 0.1859333    1.359156
##     3:   160 1993  40.214699       1 615.6  842.7572 0.1859333    1.359156
##     4:   160 1994 155.113840       1 405.5 1028.4635 0.1859333    1.359156
##     5:   160 1995 103.093132       1 488.5  890.5116 0.1859333    1.359156
##    ---                                                                    
## 34121: 82261 2019   5.277566       0 598.4  961.0884 0.2078054    1.383747
## 34122: 82288 2018   0.007672       0 425.7 1069.7815 0.1823573    1.269078
## 34123: 82538 2017 195.000000       0 563.2 1027.4662 0.2063358    1.287400
## 34124: 82538 2018 158.000000       0 554.7 1092.2203 0.2063358    1.287400
## 34125: 82538 2019 136.000000       0 544.8 1011.2639 0.2063358    1.287400
```

<br>

.content-box-green[**Main variables**]

+ `site`: well
+ `af_used`: groundwater used (dependent variable)
+ `in_LEMA`: whether located inside the LEMA region or not
+ `year`: year

---
class: middle

.content-box-green[**Control and Treatment Units**]

+ (to be) treated: wells inside the red boundary (LEMA)
+ control: wells outside the red boundary (LEMA)

---
class: middle

.content-box-green[**Before and After**]

Effective 2013, wells located inside the LEMA can pump groundwater up to a certian amount

+ before: ~ 2012
+ after: 2013 ~

---
class: middle

Data transformation:

.content-box-green[**before or after**]

```r
lema_data <- mutate(lema_data, before_after = ifelse(year >= 2013, 1, 0))

filter(lema_data, site == 160, year > 2000)
```

```
##     site year   af_used in_LEMA    pr       et0       awc bulkdensity before_after
##  1:  160 2001 145.25044       1 446.5 1015.4708 0.1859333    1.359156            0
##  2:  160 2002 209.93951       1 197.7 1183.5617 0.1859333    1.359156            0
##  3:  160 2003 195.41140       1 312.2 1043.3746 0.1859333    1.359156            0
##  4:  160 2004  17.03601       1 414.5 1021.2781 0.1859333    1.359156            0
##  5:  160 2005 166.66206       1 434.0 1072.1092 0.1859333    1.359156            0
##  6:  160 2006 201.07350       1 319.4 1101.8365 0.1859333    1.359156            0
##  7:  160 2007 169.46396       1 365.8  999.9798 0.1859333    1.359156            0
##  8:  160 2008 117.59547       1 430.4 1017.9937 0.1859333    1.359156            0
##  9:  160 2009  85.44304       1 517.9  913.9900 0.1859333    1.359156            0
## 10:  160 2010 174.59790       1 390.0 1116.1719 0.1859333    1.359156            0
## 11:  160 2011 138.41817       1 426.5 1231.1252 0.1859333    1.359156            0
## 12:  160 2012 208.88136       1 245.3 1376.4707 0.1859333    1.359156            0
## 13:  160 2013 144.59185       1 325.8 1156.5505 0.1859333    1.359156            1
## 14:  160 2014  87.94449       1 428.5 1074.7903 0.1859333    1.359156            1
## 15:  160 2015  90.21000       1 458.1 1045.0393 0.1859333    1.359156            1
## 16:  160 2016 102.01000       1 532.2 1028.5500 0.1859333    1.359156            1
## 17:  160 2017  30.75000       1 577.0 1021.9370 0.1859333    1.359156            1
## 18:  160 2019  65.07085       1 663.4  989.8929 0.1859333    1.359156            1
```

.content-box-green[**(to be) treated or not**]

Whether wells are (to be) treated or not is already there in this dataset, represented by `in_LEMA`

---
class: middle

.content-box-green[**DID estimating equation (in general)**]

$$
`\begin{aligned}
y_{i,t} = \alpha_0 + \beta_1 before\_after_t + \beta_2 treated\_or\_not_i + \beta_3 before\_after_t \times treated\_or\_not_i + X_{i,t}\gamma + v_{i,t}
\end{aligned}`
$$

The variable of interest is `$\beta_3$`, which measures the impact of the treatment.

.content-box-green[**R code**]

```r
did_res <- feols(
  af_used ~ before_after + in_LEMA + I(before_after * in_LEMA) + pr + et0,
  cluster = ~site,
  data = lema_data
)
```

---
class: middle

<template id="b688f7a4-0191-4e22-acb6-fd7c9b084521"><style>
.tabwid table{
  border-spacing:0px !important;
  border-collapse:collapse;
  line-height:1;
  margin-left:auto;
  margin-right:auto;
  border-width: 0;
  display: table;
  margin-top: 1.275em;
  margin-bottom: 1.275em;
  border-color: transparent;
}
.tabwid_left table{
  margin-left:0;
}
.tabwid_right table{
  margin-right:0;
}
.tabwid td {
    padding: 0;
}
.tabwid a {
  text-decoration: none;
}
.tabwid thead {
    background-color: transparent;
}
.tabwid tfoot {
    background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-d95143ea{}.cl-d94c66f4{font-family:'Helvetica';font-size:12pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-d94c66fe{font-family:'Helvetica';font-size:12pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(255, 0, 0, 1.00);background-color:transparent;}.cl-d94c7306{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-d94c999e{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99a8{width:156.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99a9{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99b2{width:156.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99b3{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99bc{width:156.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99bd{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99c6{width:156.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99c7{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99c8{width:156.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99d0{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99d1{width:156.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99d2{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99da{width:156.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99e4{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99e5{width:156.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99e6{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99ee{width:156.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99ef{width:156.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99f0{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99f8{width:156.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d94c99f9{width:77.9pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-d95143ea'><thead><tr style="overflow-wrap:break-word;"><td class="cl-d94c99f8"><p class="cl-d94c7306"><span class="cl-d94c66f4"> </span></p></td><td class="cl-d94c99f9"><p class="cl-d94c7306"><span class="cl-d94c66f4">Model 1</span></p></td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-d94c99a8"><p class="cl-d94c7306"><span class="cl-d94c66f4">(Intercept)</span></p></td><td class="cl-d94c999e"><p class="cl-d94c7306"><span class="cl-d94c66f4">185.829***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99b2"><p class="cl-d94c7306"><span class="cl-d94c66f4"></span></p></td><td class="cl-d94c99a9"><p class="cl-d94c7306"><span class="cl-d94c66f4">(7.153)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99da"><p class="cl-d94c7306"><span class="cl-d94c66f4">before_after</span></p></td><td class="cl-d94c99d2"><p class="cl-d94c7306"><span class="cl-d94c66f4">-9.035***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99b2"><p class="cl-d94c7306"><span class="cl-d94c66f4"></span></p></td><td class="cl-d94c99a9"><p class="cl-d94c7306"><span class="cl-d94c66f4">(1.034)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99e5"><p class="cl-d94c7306"><span class="cl-d94c66f4">in_LEMA</span></p></td><td class="cl-d94c99e4"><p class="cl-d94c7306"><span class="cl-d94c66f4">30.002***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99b2"><p class="cl-d94c7306"><span class="cl-d94c66f4"></span></p></td><td class="cl-d94c99a9"><p class="cl-d94c7306"><span class="cl-d94c66f4">(3.225)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99b2"><p class="cl-d94c7306"><span class="cl-d94c66f4">I(before_after * in_LEMA)</span></p></td><td class="cl-d94c99a9"><p class="cl-d94c7306"><span class="cl-d94c66f4">-34.762***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99b2"><p class="cl-d94c7306"><span class="cl-d94c66f4"></span></p></td><td class="cl-d94c99a9"><p class="cl-d94c7306"><span class="cl-d94c66f4">(2.097)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99ee"><p class="cl-d94c7306"><span class="cl-d94c66f4">pr</span></p></td><td class="cl-d94c99e6"><p class="cl-d94c7306"><span class="cl-d94c66f4">-0.188***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99b2"><p class="cl-d94c7306"><span class="cl-d94c66f4"></span></p></td><td class="cl-d94c99a9"><p class="cl-d94c7306"><span class="cl-d94c66f4">(0.005)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99bc"><p class="cl-d94c7306"><span class="cl-d94c66f4">et0</span></p></td><td class="cl-d94c99b3"><p class="cl-d94c7306"><span class="cl-d94c66fe">0.014**</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99b2"><p class="cl-d94c7306"><span class="cl-d94c66f4"></span></p></td><td class="cl-d94c99a9"><p class="cl-d94c7306"><span class="cl-d94c66f4">(0.005)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99c6"><p class="cl-d94c7306"><span class="cl-d94c66f4">Num.Obs.</span></p></td><td class="cl-d94c99bd"><p class="cl-d94c7306"><span class="cl-d94c66f4">34125</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99c8"><p class="cl-d94c7306"><span class="cl-d94c66f4">R2</span></p></td><td class="cl-d94c99c7"><p class="cl-d94c7306"><span class="cl-d94c66f4">0.108</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d94c99d1"><p class="cl-d94c7306"><span class="cl-d94c66f4">Std.Errors</span></p></td><td class="cl-d94c99d0"><p class="cl-d94c7306"><span class="cl-d94c66f4">by: site</span></p></td></tr></tbody><tfoot><tr style="overflow-wrap:break-word;"><td  colspan="2"class="cl-d94c99ef"><p class="cl-d94c7306"><span class="cl-d94c66f4">+ p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</span></p></td></tr></tfoot></table></div></template>
<div class="flextable-shadow-host" id="d82c8c8c-7100-40e3-a8ce-08efac175372"></div>
<script>
var dest = document.getElementById("d82c8c8c-7100-40e3-a8ce-08efac175372");
var template = document.getElementById("b688f7a4-0191-4e22-acb6-fd7c9b084521");
var caption = template.content.querySelector("caption");
if(caption) {
  caption.style.cssText = "display:block;text-align:center;";
  var newcapt = document.createElement("p");
  newcapt.appendChild(caption)
  dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

---
class: middle

.content-box-green[**Note: individual fixed effects**]

DID does not require panel data. Two periods of cross-setional data are sufficient. But, if you have panel data, you can certainly include individual fixed effects, which would certainly help to control for time-invariant characteristics (both observed and unobserved)

<code class ='r hljs remark-code'>did_res_ife <- feols(<br>&nbsp;&nbsp;af_used ~ before_after + in_LEMA + I(before_after * in_LEMA) + pr + et0 <span style='background-color:#ffff7f'>| site</span>,<br>&nbsp;&nbsp;cluster = ~site,<br>&nbsp;&nbsp;data = lema_data<br>)</code>

Notice that `in_LEMA` was dropped due to perfect collinearity (this is not a problem). `in_LEMA` is effectively controlled for by including individual fixed effects.

<template id="0ca22591-ab25-471b-9236-05ef108794c2"><style>
.tabwid table{
  border-spacing:0px !important;
  border-collapse:collapse;
  line-height:1;
  margin-left:auto;
  margin-right:auto;
  border-width: 0;
  display: table;
  margin-top: 1.275em;
  margin-bottom: 1.275em;
  border-color: transparent;
}
.tabwid_left table{
  margin-left:0;
}
.tabwid_right table{
  margin-right:0;
}
.tabwid td {
    padding: 0;
}
.tabwid a {
  text-decoration: none;
}
.tabwid thead {
    background-color: transparent;
}
.tabwid tfoot {
    background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-d9ccb2c8{}.cl-d9c7f22e{font-family:'Helvetica';font-size:9pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-d9c7f238{font-family:'Helvetica';font-size:9pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(255, 0, 0, 1.00);background-color:transparent;}.cl-d9c7fe54{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-d9c82226{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c82230{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c82231{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c8223a{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c8223b{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c8223c{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c82244{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c8224e{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c8224f{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c82250{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c82258{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c82259{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c8225a{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c82262{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c82263{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c82264{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c8226c{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d9c8226d{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-d9ccb2c8'><thead><tr style="overflow-wrap:break-word;"><td class="cl-d9c8226c"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e"> </span></p></td><td class="cl-d9c8226d"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">Model 1</span></p></td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-d9c82230"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">before_after</span></p></td><td class="cl-d9c82226"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">-8.748***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d9c8224e"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e"></span></p></td><td class="cl-d9c82244"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">(0.838)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d9c8224e"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">I(before_after * in_LEMA)</span></p></td><td class="cl-d9c82244"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">-36.551***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d9c8224e"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e"></span></p></td><td class="cl-d9c82244"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">(1.961)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d9c82250"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">pr</span></p></td><td class="cl-d9c8224f"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">-0.185***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d9c8224e"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e"></span></p></td><td class="cl-d9c82244"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">(0.004)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d9c82259"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">et0</span></p></td><td class="cl-d9c82258"><p class="cl-d9c7fe54"><span class="cl-d9c7f238">0.019***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d9c8224e"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e"></span></p></td><td class="cl-d9c82244"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">(0.004)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d9c82262"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">Num.Obs.</span></p></td><td class="cl-d9c8225a"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">34125</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d9c8223a"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">R2</span></p></td><td class="cl-d9c82231"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">0.667</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d9c8223c"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">Std.Errors</span></p></td><td class="cl-d9c8223b"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">by: site</span></p></td></tr></tbody><tfoot><tr style="overflow-wrap:break-word;"><td  colspan="2"class="cl-d9c82263"><p class="cl-d9c7fe54"><span class="cl-d9c7f22e">+ p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</span></p></td></tr></tfoot></table></div></template>
<div class="flextable-shadow-host" id="f658229c-754a-4bc4-a7be-67a2c2850eb2"></div>
<script>
var dest = document.getElementById("f658229c-754a-4bc4-a7be-67a2c2850eb2");
var template = document.getElementById("0ca22591-ab25-471b-9236-05ef108794c2");
var caption = template.content.querySelector("caption");
if(caption) {
  caption.style.cssText = "display:block;text-align:center;";
  var newcapt = document.createElement("p");
  newcapt.appendChild(caption)
  dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

---
class: middle

.content-box-green[**Note: year fixed effects**]

If you have multiple years of observations in the before and after periods, you can (and should) include year fixed effects.

<code class ='r hljs remark-code'>did_res_yfe <- feols(<br>&nbsp;&nbsp;af_used ~ before_after + in_LEMA + I(before_after * in_LEMA) + pr + et0 | site <span style='background-color:#ffff7f'>+ year</span>,<br>&nbsp;&nbsp;cluster = ~site,<br>&nbsp;&nbsp;data = lema_data<br>)</code>

Notice that `before_after` was dropped due to perfect collinearity (this is not a problem). `before_after` is effectively controlled for by including year fixed effects. Indeed, year fixed effects provide a tighter controls on annual macro shocks.

<template id="3c0a6da4-2af3-4e9f-bb5c-4c2997a4870b"><style>
.tabwid table{
  border-spacing:0px !important;
  border-collapse:collapse;
  line-height:1;
  margin-left:auto;
  margin-right:auto;
  border-width: 0;
  display: table;
  margin-top: 1.275em;
  margin-bottom: 1.275em;
  border-color: transparent;
}
.tabwid_left table{
  margin-left:0;
}
.tabwid_right table{
  margin-right:0;
}
.tabwid td {
    padding: 0;
}
.tabwid a {
  text-decoration: none;
}
.tabwid thead {
    background-color: transparent;
}
.tabwid tfoot {
    background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-da17cd8a{}.cl-da132e60{font-family:'Helvetica';font-size:9pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-da132e6a{font-family:'Helvetica';font-size:9pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(255, 0, 0, 1.00);background-color:transparent;}.cl-da133aea{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-da135ef8{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f02{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f03{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f0c{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f0d{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f16{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f17{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f20{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f21{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f2a{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f2b{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f2c{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f34{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f35{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f3e{width:122.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-da135f3f{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-da17cd8a'><thead><tr style="overflow-wrap:break-word;"><td class="cl-da135f3e"><p class="cl-da133aea"><span class="cl-da132e60"> </span></p></td><td class="cl-da135f3f"><p class="cl-da133aea"><span class="cl-da132e60">Model 1</span></p></td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-da135ef8"><p class="cl-da133aea"><span class="cl-da132e60">I(before_after * in_LEMA)</span></p></td><td class="cl-da135f02"><p class="cl-da133aea"><span class="cl-da132e60">-37.150***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-da135ef8"><p class="cl-da133aea"><span class="cl-da132e60"></span></p></td><td class="cl-da135f02"><p class="cl-da133aea"><span class="cl-da132e60">(1.962)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-da135f0c"><p class="cl-da133aea"><span class="cl-da132e60">pr</span></p></td><td class="cl-da135f03"><p class="cl-da133aea"><span class="cl-da132e60">-0.115***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-da135ef8"><p class="cl-da133aea"><span class="cl-da132e60"></span></p></td><td class="cl-da135f02"><p class="cl-da133aea"><span class="cl-da132e60">(0.008)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-da135f0d"><p class="cl-da133aea"><span class="cl-da132e60">et0</span></p></td><td class="cl-da135f16"><p class="cl-da133aea"><span class="cl-da132e6a">0.053+</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-da135ef8"><p class="cl-da133aea"><span class="cl-da132e60"></span></p></td><td class="cl-da135f02"><p class="cl-da133aea"><span class="cl-da132e60">(0.031)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-da135f20"><p class="cl-da133aea"><span class="cl-da132e60">Num.Obs.</span></p></td><td class="cl-da135f17"><p class="cl-da133aea"><span class="cl-da132e60">34125</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-da135f21"><p class="cl-da133aea"><span class="cl-da132e60">R2</span></p></td><td class="cl-da135f2a"><p class="cl-da133aea"><span class="cl-da132e60">0.711</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-da135f2b"><p class="cl-da133aea"><span class="cl-da132e60">Std.Errors</span></p></td><td class="cl-da135f2c"><p class="cl-da133aea"><span class="cl-da132e60">by: site</span></p></td></tr></tbody><tfoot><tr style="overflow-wrap:break-word;"><td  colspan="2"class="cl-da135f34"><p class="cl-da133aea"><span class="cl-da132e60">+ p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</span></p></td></tr></tfoot></table></div></template>
<div class="flextable-shadow-host" id="9a387546-c1f1-4431-89ba-2ff67f9a38d6"></div>
<script>
var dest = document.getElementById("9a387546-c1f1-4431-89ba-2ff67f9a38d6");
var template = document.getElementById("3c0a6da4-2af3-4e9f-bb5c-4c2997a4870b");
var caption = template.content.querySelector("caption");
if(caption) {
  caption.style.cssText = "display:block;text-align:center;";
  var newcapt = document.createElement("p");
  newcapt.appendChild(caption)
  dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

---
class: middle

# How to argue your DID is good

.content-box-green[**Important**]

Selecting the right control group is important in DID. If the following conditions are satisfied, it is more plausible that the control and treatment groups would have had the same macro shock `$(\alpha_1 = \alpha_0)$` if it were not for the treatment.

+ There were no events that could significantly affect the dependent variable of the control group between the "before" and "after" period
+ The two groups are generally similar so other factors do not drive the differences between them
+ They had similar trajectories of the dependent variable prior to the treatment (possible if you have more than one years of data prior to the treatment)
  - this does <span style = "color: red;"> NOT </span> guarantee that the their trends <span style = "color: red;"> after </span> the treatment are similar

.content-box-green[**To do**]

+ Show the trajectory of the dependent variable
+ Run placebo tests

---
class: middle

---
class: middle

---
class: middle

So, how about our example?

Not too bad. We might want to consider starting from 1993.

---
class: middle

# Placebo tests

+ Look at only the pre-treatment periods
+ Pretend that a treatment happend sometime in the middle of the pre-treatment period to the actual treatment group
+ Estimate the impact of the fake treatment
+ Check if the estimated impact is stastitically insignificantly different from 0 
+ If statistically significant, that would mean there is likley to be something wrong with the parallel trende assumption

---
class: middle

Fake treament for the wells inside LEMA in 2000.

```r
pre_lema_data <-
  filter(lema_data, year <= 2012 & year >= 1993) %>%
  #* pretend that a treatment happend in 2000
  mutate(after_2000 = ifelse(year >= 2000, 1, 0))

did_res_placebo <-
  feols(
    af_used ~ I(after_2000 * in_LEMA) + pr + et0 | site + year,
    cluster = ~site,
    data = pre_lema_data
  )
```

<template id="dcbade3d-6572-43c9-ab74-287e85f28c13"><style>
.tabwid table{
  border-spacing:0px !important;
  border-collapse:collapse;
  line-height:1;
  margin-left:auto;
  margin-right:auto;
  border-width: 0;
  display: table;
  margin-top: 1.275em;
  margin-bottom: 1.275em;
  border-color: transparent;
}
.tabwid_left table{
  margin-left:0;
}
.tabwid_right table{
  margin-right:0;
}
.tabwid td {
    padding: 0;
}
.tabwid a {
  text-decoration: none;
}
.tabwid thead {
    background-color: transparent;
}
.tabwid tfoot {
    background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-dad69634{}.cl-dad1c15e{font-family:'Helvetica';font-size:9pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-dad1c168{font-family:'Helvetica';font-size:9pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(255, 0, 0, 1.00);background-color:transparent;}.cl-dad1ce42{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-dad1f3b8{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3b9{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3c2{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3c3{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3cc{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3cd{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3d6{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3d7{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3e0{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3e1{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3e2{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3ea{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3eb{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3f4{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3f5{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-dad1f3f6{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-dad69634'><thead><tr style="overflow-wrap:break-word;"><td class="cl-dad1f3f5"><p class="cl-dad1ce42"><span class="cl-dad1c15e"> </span></p></td><td class="cl-dad1f3f6"><p class="cl-dad1ce42"><span class="cl-dad1c15e">Model 1</span></p></td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-dad1f3b8"><p class="cl-dad1ce42"><span class="cl-dad1c15e">I(after_2000 * in_LEMA)</span></p></td><td class="cl-dad1f3b9"><p class="cl-dad1ce42"><span class="cl-dad1c168">1.988</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-dad1f3b8"><p class="cl-dad1ce42"><span class="cl-dad1c15e"></span></p></td><td class="cl-dad1f3b9"><p class="cl-dad1ce42"><span class="cl-dad1c15e">(2.589)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-dad1f3c3"><p class="cl-dad1ce42"><span class="cl-dad1c15e">pr</span></p></td><td class="cl-dad1f3c2"><p class="cl-dad1ce42"><span class="cl-dad1c15e">-0.214***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-dad1f3b8"><p class="cl-dad1ce42"><span class="cl-dad1c15e"></span></p></td><td class="cl-dad1f3b9"><p class="cl-dad1ce42"><span class="cl-dad1c15e">(0.017)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-dad1f3cc"><p class="cl-dad1ce42"><span class="cl-dad1c15e">et0</span></p></td><td class="cl-dad1f3cd"><p class="cl-dad1ce42"><span class="cl-dad1c15e">-0.005</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-dad1f3b8"><p class="cl-dad1ce42"><span class="cl-dad1c15e"></span></p></td><td class="cl-dad1f3b9"><p class="cl-dad1ce42"><span class="cl-dad1c15e">(0.046)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-dad1f3d7"><p class="cl-dad1ce42"><span class="cl-dad1c15e">Num.Obs.</span></p></td><td class="cl-dad1f3d6"><p class="cl-dad1ce42"><span class="cl-dad1c15e">23509</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-dad1f3e0"><p class="cl-dad1ce42"><span class="cl-dad1c15e">R2</span></p></td><td class="cl-dad1f3e1"><p class="cl-dad1ce42"><span class="cl-dad1c15e">0.728</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-dad1f3e2"><p class="cl-dad1ce42"><span class="cl-dad1c15e">Std.Errors</span></p></td><td class="cl-dad1f3ea"><p class="cl-dad1ce42"><span class="cl-dad1c15e">by: site</span></p></td></tr></tbody><tfoot><tr style="overflow-wrap:break-word;"><td  colspan="2"class="cl-dad1f3eb"><p class="cl-dad1ce42"><span class="cl-dad1c15e">+ p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</span></p></td></tr></tfoot></table></div></template>
<div class="flextable-shadow-host" id="3507323c-bb40-4974-8eed-00dc4b569e33"></div>
<script>
var dest = document.getElementById("3507323c-bb40-4974-8eed-00dc4b569e33");
var template = document.getElementById("dcbade3d-6572-43c9-ab74-287e85f28c13");
var caption = template.content.querySelector("caption");
if(caption) {
  caption.style.cssText = "display:block;text-align:center;";
  var newcapt = document.createElement("p");
  newcapt.appendChild(caption)
  dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

---
class: middle

Fake treament for the wells inside LEMA in 1995.

```r
pre_lema_data <-
  filter(lema_data, year <= 2012 & year >= 1993) %>%
  #* pretend that a treatment happend in 1995
  mutate(after_1995 = ifelse(year >= 1995, 1, 0))

did_res_placebo <-
  feols(
    af_used ~ I(after_1995 * in_LEMA) + pr + et0 | site + year,
    cluster = ~site,
    data = pre_lema_data
  )
```

<template id="b3ed1565-e76a-4da2-b87c-3456a9ab1bb5"><style>
.tabwid table{
  border-spacing:0px !important;
  border-collapse:collapse;
  line-height:1;
  margin-left:auto;
  margin-right:auto;
  border-width: 0;
  display: table;
  margin-top: 1.275em;
  margin-bottom: 1.275em;
  border-color: transparent;
}
.tabwid_left table{
  margin-left:0;
}
.tabwid_right table{
  margin-right:0;
}
.tabwid td {
    padding: 0;
}
.tabwid a {
  text-decoration: none;
}
.tabwid thead {
    background-color: transparent;
}
.tabwid tfoot {
    background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-db0a25ee{}.cl-db0430f8{font-family:'Helvetica';font-size:9pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-db04310c{font-family:'Helvetica';font-size:9pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(255, 0, 0, 1.00);background-color:transparent;}.cl-db043f58{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-db046366{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db046370{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db04637a{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db04637b{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db046384{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db046385{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db04638e{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db04638f{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db046398{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db046399{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db04639a{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db0463a2{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db0463a3{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db0463a4{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db0463ac{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-db0463b6{width:56.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-db0a25ee'><thead><tr style="overflow-wrap:break-word;"><td class="cl-db0463ac"><p class="cl-db043f58"><span class="cl-db0430f8"> </span></p></td><td class="cl-db0463b6"><p class="cl-db043f58"><span class="cl-db0430f8">Model 1</span></p></td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-db046366"><p class="cl-db043f58"><span class="cl-db0430f8">I(after_1995 * in_LEMA)</span></p></td><td class="cl-db046370"><p class="cl-db043f58"><span class="cl-db04310c">-2.353</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-db046366"><p class="cl-db043f58"><span class="cl-db0430f8"></span></p></td><td class="cl-db046370"><p class="cl-db043f58"><span class="cl-db0430f8">(2.911)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-db04637b"><p class="cl-db043f58"><span class="cl-db0430f8">pr</span></p></td><td class="cl-db04637a"><p class="cl-db043f58"><span class="cl-db0430f8">-0.214***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-db046366"><p class="cl-db043f58"><span class="cl-db0430f8"></span></p></td><td class="cl-db046370"><p class="cl-db043f58"><span class="cl-db0430f8">(0.017)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-db046384"><p class="cl-db043f58"><span class="cl-db0430f8">et0</span></p></td><td class="cl-db046385"><p class="cl-db043f58"><span class="cl-db0430f8">0.007</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-db046366"><p class="cl-db043f58"><span class="cl-db0430f8"></span></p></td><td class="cl-db046370"><p class="cl-db043f58"><span class="cl-db0430f8">(0.047)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-db04638f"><p class="cl-db043f58"><span class="cl-db0430f8">Num.Obs.</span></p></td><td class="cl-db04638e"><p class="cl-db043f58"><span class="cl-db0430f8">23509</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-db046398"><p class="cl-db043f58"><span class="cl-db0430f8">R2</span></p></td><td class="cl-db046399"><p class="cl-db043f58"><span class="cl-db0430f8">0.728</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-db04639a"><p class="cl-db043f58"><span class="cl-db0430f8">Std.Errors</span></p></td><td class="cl-db0463a2"><p class="cl-db043f58"><span class="cl-db0430f8">by: site</span></p></td></tr></tbody><tfoot><tr style="overflow-wrap:break-word;"><td  colspan="2"class="cl-db0463a3"><p class="cl-db043f58"><span class="cl-db0430f8">+ p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</span></p></td></tr></tfoot></table></div></template>
<div class="flextable-shadow-host" id="43957f00-e4ef-4fc3-a13c-5bba2f1e77a5"></div>
<script>
var dest = document.getElementById("43957f00-e4ef-4fc3-a13c-5bba2f1e77a5");
var template = document.getElementById("b3ed1565-e76a-4da2-b87c-3456a9ab1bb5");
var caption = template.content.querySelector("caption");
if(caption) {
  caption.style.cssText = "display:block;text-align:center;";
  var newcapt = document.createElement("p");
  newcapt.appendChild(caption)
  dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

You can try more years as the starting year of a fake treatment and see what happens.

---
class: middle

.content-box-green[**Note**]

Statistically insignificant estimated impacts of fake treatments bolster your claim about parallel trend assumption. But, it still does <span style = "color: red;"> NOT </span> guarantee the assumption is valid. Remember, the assumption is not testable.

---
class: middle

What if your data spans from 1991 to 2000 with a treatment occuring at 1993?

```r
pre_lema_data <-
  filter(lema_data, year <= 2000) %>%
  #* pretend that a treatment happend in 1993
  mutate(after_1993 = ifelse(year >= 1993, 1, 0))

did_res_placebo <-
  feols(
    af_used ~ I(after_1993 * in_LEMA) + pr + et0 | site + year,
    cluster = ~site,
    data = pre_lema_data
  )
```

<template id="da5d6b11-6509-4fab-8041-a81b68b865e6"><style>
.tabwid table{
  border-spacing:0px !important;
  border-collapse:collapse;
  line-height:1;
  margin-left:auto;
  margin-right:auto;
  border-width: 0;
  display: table;
  margin-top: 1.275em;
  margin-bottom: 1.275em;
  border-color: transparent;
}
.tabwid_left table{
  margin-left:0;
}
.tabwid_right table{
  margin-right:0;
}
.tabwid td {
    padding: 0;
}
.tabwid a {
  text-decoration: none;
}
.tabwid thead {
    background-color: transparent;
}
.tabwid tfoot {
    background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-d31808d0{}.cl-d3132f9a{font-family:'Helvetica';font-size:9pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-d3132fa4{font-family:'Helvetica';font-size:9pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(255, 0, 0, 1.00);background-color:transparent;}.cl-d3133d5a{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-d3136460{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d313646a{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d313646b{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d3136474{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d3136475{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d313647e{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d313647f{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d3136488{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 1pt solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d3136489{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d3136492{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d3136493{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d3136494{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d3136495{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d313649c{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(255, 255, 255, 0.00);border-top: 0 solid rgba(255, 255, 255, 0.00);border-left: 0 solid rgba(255, 255, 255, 0.00);border-right: 0 solid rgba(255, 255, 255, 0.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d313649d{width:117.1pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-d31364a6{width:61.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-d31808d0'><thead><tr style="overflow-wrap:break-word;"><td class="cl-d313649d"><p class="cl-d3133d5a"><span class="cl-d3132f9a"> </span></p></td><td class="cl-d31364a6"><p class="cl-d3133d5a"><span class="cl-d3132f9a">Model 1</span></p></td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-d3136460"><p class="cl-d3133d5a"><span class="cl-d3132f9a">I(after_1993 * in_LEMA)</span></p></td><td class="cl-d313646a"><p class="cl-d3133d5a"><span class="cl-d3132fa4">-16.386***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d3136460"><p class="cl-d3133d5a"><span class="cl-d3132f9a"></span></p></td><td class="cl-d313646a"><p class="cl-d3133d5a"><span class="cl-d3132f9a">(3.900)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d3136474"><p class="cl-d3133d5a"><span class="cl-d3132f9a">pr</span></p></td><td class="cl-d313646b"><p class="cl-d3133d5a"><span class="cl-d3132f9a">-0.121***</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d3136460"><p class="cl-d3133d5a"><span class="cl-d3132f9a"></span></p></td><td class="cl-d313646a"><p class="cl-d3133d5a"><span class="cl-d3132f9a">(0.026)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d3136475"><p class="cl-d3133d5a"><span class="cl-d3132f9a">et0</span></p></td><td class="cl-d313647e"><p class="cl-d3133d5a"><span class="cl-d3132f9a">0.114+</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d3136460"><p class="cl-d3133d5a"><span class="cl-d3132f9a"></span></p></td><td class="cl-d313646a"><p class="cl-d3133d5a"><span class="cl-d3132f9a">(0.065)</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d3136488"><p class="cl-d3133d5a"><span class="cl-d3132f9a">Num.Obs.</span></p></td><td class="cl-d313647f"><p class="cl-d3133d5a"><span class="cl-d3132f9a">11320</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d3136489"><p class="cl-d3133d5a"><span class="cl-d3132f9a">R2</span></p></td><td class="cl-d3136492"><p class="cl-d3133d5a"><span class="cl-d3132f9a">0.715</span></p></td></tr><tr style="overflow-wrap:break-word;"><td class="cl-d3136493"><p class="cl-d3133d5a"><span class="cl-d3132f9a">Std.Errors</span></p></td><td class="cl-d3136494"><p class="cl-d3133d5a"><span class="cl-d3132f9a">by: site</span></p></td></tr></tbody><tfoot><tr style="overflow-wrap:break-word;"><td  colspan="2"class="cl-d3136495"><p class="cl-d3133d5a"><span class="cl-d3132f9a">+ p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</span></p></td></tr></tfoot></table></div></template>
<div class="flextable-shadow-host" id="c85db50a-af45-4795-aaa9-933b2b8f90bc"></div>
<script>
var dest = document.getElementById("c85db50a-af45-4795-aaa9-933b2b8f90bc");
var template = document.getElementById("da5d6b11-6509-4fab-8041-a81b68b865e6");
var caption = template.content.querySelector("caption");
if(caption) {
  caption.style.cssText = "display:block;text-align:center;";
  var newcapt = document.createElement("p");
  newcapt.appendChild(caption)
  dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

+ So, this tells you that if it was a real treatment of which you want to understand the impact, then you would have suffered significant bias.
+ This clearly indicates that DID is by no means perfect and indeed can be very dangerous