11: Dealing with Endogeneity: Instrumental Variable
Instrumental Variable (IV) Approach
Endogeneity
\(E[u|x_k] \ne 0\) (the error term is not correlated with any of the independent variables)
Endogenous independent variable
If the error term is, for whatever reason, correlated with the independent variable \(x_k\), then we say that \(x_k\) is an endogenous independent variable.
- Omitted variable
- Selection
- Reverse causality
- Measurement error
You want to estimate the causal impact of education on income.
- Variable of interest: Education
- Dependent variable: Income
Causal diagram
We want to find a variable like \(Z\) in the diagram below:
- \(Z\) does NOT affect income directly
- \(Z\) is correlated with the variable of interest (education)
- does not matter which causes which (associattion is enough)
- \(Z\) is NOT correlated with any of the unobservable variables in the error term (including ability) that is making the vairable of interest (education) endogeneous.
- \(Z\) does not affect ability
- abiliyt does not affect \(Z\)
The Model
\(y=\beta_0 + \beta_1 x_1 + \beta_2 x_2 + u\)
- \(x_1\) is endogenous: \(E[u|x_1] \ne 0\) (or \(Cov(u,x_1)\ne 0\))
- \(x_2\) is exogenous: \(E[u|x_1] = 0\) (or \(Cov(u,x_1) = 0\))
Idea (very loosely put)
Bring in variable(s) ( Instrumental variable(s) ) that does NOT belong to the model, but IS related with the endogenous variable,
Using the instrumental variable(s) (which we denote by \(Z\)), make the endogenous variable exogenous, which we call instrumented variable(s)
Use the variation in the instrumented variable instead of the original endogenous variable to estimate the impact of the original variable
Idea
Using the instrumental variables, make the endogenous variable exogenous, which we call instrumented variable.
Step 1: mathematically
- Regress the endogenous variable \((x_1)\) on the instrumental variable(s) \((Z=\{z_1,z_2\}\), two instruments here) and all the other exogenous variables \((x_2\) here)
\(x_1 = \alpha_0 + \sigma_2 x_2 + \alpha_1 z_1 +\alpha_2 z_2 + v\)
- obtain the predicted value of \(x\) from the regression
\(\widehat{x}_1 = \widehat{\alpha}_0 + \widehat{\sigma}_2 x_2 + \widehat{\alpha}_1 z_1 + \widehat{\alpha}_2 z_2\)
Idea
Use the variation in the instrumented variable instead of the original endogenous variable to estimate the impact of the original variable
Step 2: Mathematically
Regress the dependent variable \((y)\) on the instrumented variable \((\widehat{x}_1)\),
\(y= \beta_0 + \beta_1 \widehat{x}_1+ \beta_2 x_2 + \varepsilon\)
to estimate the coefficient on \(x\) in the original model
Model
\(log(wage) = \beta_0 + \beta_1 educ + \beta_2 exper + (\beta_3 ability + v)\)
- Regress \(log(wage)\) on \(educ\) and \(exper\) \((ability\) not included because you do not observe it)
- \((\beta_3 ability + v)\) is the error term
- \(educ\) is considered endogenous (correlated with \(ability\))
- \(exper\) is considered exogenous (not correlated with \(ability\))
Instruments (Z)
Suppose you selected the following variables as instruments:
- IQ test score \((IQ)\)
- number of siblings \((sibs)\)
Regress \(educ\) on \(exper\), \(IQ\), and \(sibs\):
\[\begin{align*} educ = \alpha_0 + \alpha_1 exper + \alpha_2 IQ + \alpha_3 sibs + u \end{align*}\]Use the coefficient estimates on \(\alpha_0\), \(\alpha_1\), \(\alpha_2\), and \(\alpha_3\) to predict \(educ\) as a function of \(exper\), \(IQ\), and \(sibs\).
\[\begin{align*} \widehat{educ} = \widehat{\alpha_0} + \widehat{\alpha_1} exper + \widehat{\alpha_2} IQ + \widehat{\alpha_3} sibs \end{align*}\]Use \(\widehat{educ}\) in place of \(educ\) to estimate the model of interest:
\[\begin{align*} log(wage) = \beta_0 + \beta_1 \widehat{educ} + \beta_2 exper + u \end{align*}\]Weak instrument
Run the 1st stage regression
\[\begin{align*} educ = \alpha_0 + \alpha_1 exper + \alpha_2 sibs + \alpha_3 feduc + v \end{align*}\]Then, test the joint significance of \(\alpha_2\) and \(\alpha_3\) (\(F\)-test)
If excluded instruments \((sibs\) and \(feduc\), here) are jointly significant, then it would mean that \(sibs\) and \(feduc\) are not weak instruments, satisfying condition 1.
When we ran the IV estimation using fixest::feols() earlier, it automatically calculated the F-statistic for the weak instrument test.
Here, F-test for the null hypothesis of the excluded instruments (sibs and feduc) do not have any explanatory power on the endogenous variable (educ) beyond the included instrument (exper) is rejected.
Alternatively
You can access the iv_first_stage component of the regression results.
Data generation
Correlation
Estimation with the strong instrumental variable
Estimation with the weak instrumental variable
Question
Any notable differences?
Answer
The coefficient estimate on \(x\_end\) is far away from the true value in the weak instrument case.diagnostics (strong instrument)
diagnostics (weak instrument)
set.seed(238354)
B <- 1000 # the number of experiments
N <- 500 # number of observations
beta_hat_store <- matrix(0, B, 2) # storage of beta hat
for (i in 1:B) {
#--- data generation ---#
u_common <- runif(N)
z_common <- runif(N)
x_end <- u_common + z_common + runif(N)
z_strong <- z_common + runif(N)
z_weak <- 0.01 * z_common + 0.99995 * runif(N)
u <- u_common + runif(N)
y <- x_end + u
data <- data.table(y, x_end, z_strong, z_weak)
#--- IV estimation with a strong instrument ---#
iv_strong <- fixest::feols(y ~ 1 | x_end ~ z_strong, data = data)
beta_hat_store[i, 1] <- iv_strong$coefficients[2]
#--- IV estimation with a weak instrument ---#
iv_weak <- fixest::feols(y ~ 1 | x_end ~ z_weak, data = data)
beta_hat_store[i, 2] <- iv_weak$coefficients[2]
}Code
melted <- melt(data.table(beta_hat_store))
melted[variable == "V1", variable := "Strong"]
melted[variable == "V2", variable := "Weak"]
ggplot(data = melted[abs(value) < 5, ]) +
geom_density(aes(x = value, fill = variable), alpha = 0.3) +
geom_vline(xintercept = 1, color = "red") +
scale_fill_discrete(name = "") +
theme(
legend.position = "bottom"
)