Monte Carlo simulation is how we settle arguments in this course when the algebra gets hard to trust. You specify a complete, artificial data-generating process, use a random-number generator to draw a sample from that process, apply an econometric method, and repeat the exercise many times. Because you built the world, you know the true parameter and can see how the estimates behave around it.
Read the three uses on screen. First, we can confirm theory numerically. For example, theory says that OLS is unbiased when zero conditional mean and the other required assumptions hold. We can generate data that satisfy those assumptions and check whether repeated OLS estimates center on the coefficient we chose. Second, we can investigate a feature of real research data when no convenient theorem tells us what it will do. We reproduce that feature in a controlled simulation, change it while holding the rest of the process fixed, and watch the estimator respond. I use simulation this way in research as a diagnostic and design tool. Third, simulation helps you learn econometrics. A sampling distribution, bias, and variance become collections of actual numbers rather than only Greek letters.
It is a way to test econometric theories via simulation.
How is it used in econometrics?
Here is the question that motivates everything. In the regression model, zero conditional mean says that the average error is zero at every value of x. It rules out a systematic relationship between the observed regressor and the unobserved determinants collected in the error. We want to know what happens to OLS when that condition fails.
Could you simply take a real dataset and check? Open the answer and notice that both objects needed for a direct check are unobserved. You do not observe the true error, because the error includes all determinants of y that the model leaves out. You also do not observe the true coefficient beta. You observe beta-hat, one estimate from one sample, but without the truth you cannot label its gap as bias.
So real data hide both halves of the experiment: you cannot directly verify how the true error relates to x, and you cannot compare an estimate with the true parameter.
Suppose you are interested in checking what happens to OLS estimators if E[u|x]=0 (the error term and x are not correlated) is violated.
Question
Can you use the real data to do this?
Read the word in blue: you. You generate the data, and that single fact is what makes the whole method work. A data-generating process is a recipe specifying the parameter values, the distributions of the variables, the sample size, and any dependence among the variables. In real applications that recipe is unknown and must be inferred. Here, you write it down and make R follow it.
The first consequence on screen is that you know the true parameter. If you put a slope of one into the outcome equation, one remains the truth in every simulated sample. Each beta-hat will move because each sample is random, but you can compare the center of many beta-hats with one. That lets you distinguish ordinary sampling variation from systematic bias.
The second consequence is control. You can create two otherwise identical worlds and change only the feature you want to study, such as whether x and the error share a common component. Holding the coefficient, sample size, marginal distributions, and estimation command fixed prevents those other features from explaining the difference. This is a controlled experiment on an econometric method.
You generate data (you have control over how data are generated)
Before we simulate anything, we need numbers that behave like random draws. The blue word “approximate” matters. A pseudorandom-number generator is a deterministic algorithm with an internal state. Starting from a state, it produces a long sequence whose frequencies, dependence, and other statistical properties are designed to resemble genuinely random draws.
Run the first short cell. Say the call as “run-if five.” The argument five is the number of observations requested. Because the minimum and maximum arguments are omitted, R uses their defaults, zero and one. The result is five values from a Uniform zero-to-one distribution, so every interval of the same width inside zero to one has the same probability. The particular five values are not supposed to look evenly spaced. Randomness is a property of the process, not a promise about a tiny sample.
In the second cell, the assignment arrow stores ten thousand new Uniform draws in x, and “hist of x” groups them into bins. The horizontal axis records possible x values from zero to one. The vertical axis records the number of draws in each bin. Look for bars of roughly equal height across the range. They will not be exactly equal because ten thousand is still a finite random sample, but the approximately flat outline is the visible implication of a uniform density.
Pseudo random number generators (Pseudo RNG)
Algorithms for generating a sequence of numbers whose properties approximate the properties of sequences of random numbers
Examples
Draw from a uniform distribution:
Here is the practical consequence of “pseudo” rather than physically random. Once the generator’s algorithm and starting state are fixed, the entire future sequence is fixed. The first line says “set-seed two million three hundred eighty-seven thousand four hundred thirty-eight,” and that sets the starting state. The integer itself has no substantive meaning, and choosing a larger or more complicated seed does not make the simulation more random. It is simply a reproducible label for a position in the generator’s sequence.
Run the demonstration. The next call requests five Uniform zero-to-one draws, and with this seed the sequence begins approximately zero point zero four seven, zero point seven one two, zero point four zero seven, zero point two four two, and zero point three five seven. Run the whole cell again and you get the same five values, because the seed is reset before the draw. If you run only another random-draw command without resetting the seed, the generator advances and gives the next values in the sequence.
That answers the question at the bottom: setting a seed makes the entire analysis reproducible. A coauthor, a grader, or your future self can regenerate the same simulated data, estimates, tables, and figures.
Numbers drawn using pseudo random number generators are not truly random
Demonstration
Question
What benefits does setting a seed have?
The Normal distribution is the workhorse for the examples that follow, and “R-norm” draws from it. Start with the left column. The mathematical notation says x follows Normal zero comma one, where zero is the mean and one is the variance. The call “R-norm ten thousand” asks for ten thousand observations. Because no other arguments are supplied, R uses mean zero and standard deviation one. The assignment stores those observations in x, and the histogram puts x values on the horizontal axis and bin counts on the vertical axis. You should see a bell-shaped distribution centered near zero, with most values relatively close to zero and fewer values in either tail.
Now look at the right column. The notation says Normal two comma four, so the population mean is two and the population variance is four. The R call gives the named mean argument the value two, which shifts the center from zero to two, and gives the named S-D argument the value two, which sets the standard deviation to two. This is the common trap: mathematical Normal notation on this slide reports variance as its second number, but R’s S-D argument requires the square root of that variance. Since the square root of four is two, the code is consistent with the notation. The right histogram should therefore be centered near two and spread about twice as widely in standard-deviation units as the left one. Small asymmetries and uneven bars are sampling noise, not a change in the distribution.
x \sim N(0, 1)
x \sim N(2, 4)
Before the recipe, be clear about the situation the recipe is for. In real research you get one dataset. You run one regression on it, and you read off one number. That is the whole of your evidence, and this slide is about what that one number can and cannot settle.
The code builds exactly that situation, except that here we are the ones who decide what is true. Read the first line as “N gets one hundred,” so each dataset has one hundred observations. R-norm of N draws one hundred independent standard Normal values and stores them in x. The next R-norm of N draws one hundred more, independently, and stores them in u. Because those two calls share nothing, u carries no information about x, so the zero conditional mean condition holds by construction. Read the outcome line as “y gets one plus x plus u.” That sets the true intercept to one and the true slope on x to one. One is the truth in this world because we wrote it there.
The next line hands the two observed variables to fee-ols from the fixest package. Read the formula “y tilde x” as regressing y on x with an intercept. Notice that u is not passed to the regression. We know it, but the estimator does not, exactly as in real data. The last line selects the estimated coefficient on x out of the fitted model and prints it.
Run the cell. The printed number should land somewhere near one, but it will not be one. Now run it again, and again, four or five times. Every run draws a fresh x and a fresh u, so every run prints a different number. Nothing about the world changed between runs. The true slope is still one, the sample size is still one hundred, and the command is still the same. Only the random draw changed, and the answer moved anyway.
In real research you get one dataset. You run one regression and read off one number.
Here is that situation, except that we decide what is true: \beta_1 = 1.
Here is what happens when five different people each collect their own dataset from the world you just built. Same truth of one, same sample size of one hundred, same command, five separate random draws. The table reports what each of them would write in their paper: zero point nine five three, one point zero seven five, one point zero eight one, one point zero two zero, and zero point eight six four.
Look at the second column. All five are estimating the same quantity, and the true value is one. Not one of the five got it. The largest is about eight percent above the truth and the smallest is about fourteen percent below it, and none of them did anything wrong. Each ran a correct regression on a correctly collected sample.
So ask which of those five numbers is the right one. The honest answer is that the question is badly posed. Being right is not a property any single one of them has. Read the two questions on the lower half of the slide. Asking whether OLS is unbiased is asking where those five numbers, and the many more we did not draw, center. Asking how precise OLS is is asking how far apart they are. Both questions are about the whole collection of numbers, not about any individual entry in the table.
That is the difficulty with real data, and it is worth stating plainly. A real researcher gets the first row of this table and has no way to obtain a second one. There is only one world, it was sampled once, and the other rows do not exist. The collection that every econometric claim is about is exactly the thing you cannot observe.
Now read the last line. We built this world, so the constraint does not apply to us. We can draw a sixth row, a hundredth row, a thousandth row, as many as we are willing to wait for. Producing that table on purpose, and then reading its center and its spread, is what Monte Carlo simulation is. The next slide is the procedure for doing it.
Five people each draw their own dataset from the same world and run the same regression:
| dataset | \hat{\beta}_1 |
|---|---|
| 1 | 0.953 |
| 2 | 1.075 |
| 3 | 1.081 |
| 4 | 1.020 |
| 5 | 0.864 |
Why we simulate
With real data you get row 1 and no way to obtain row 2. But we built this world, so we can draw as many rows as we like — and then read off the center and the spread.
Here is the recipe, and every Monte Carlo exercise in this course follows it. In step one, specify the data-generating process completely: choose the true parameter values, sample size, distributions of the variables, and any dependence among them. In step two, generate one artificial dataset from that process. In step three, apply the estimator or procedure you want to study, such as OLS or a sample mean, and save its result. In step four, return to step two and repeat steps two and three with fresh random draws many times. In step five, summarize the saved estimates and compare them with the truth chosen in step one.
The question at the bottom asks why steps two and three are repeated while step one is not. Step one is the world we chose, and it has to stay fixed: if the true parameter or the sample size moved between repetitions, the collection of estimates would no longer describe one single situation. An estimator is a random variable because a new random sample produces a new estimate. One sample gives one draw from its sampling distribution. That single value cannot reveal the distribution’s center, spread, skewness, or tail behavior, so it cannot tell us whether a miss is ordinary sampling noise or systematic bias.
Thousands of independent replications approximate the sampling distribution. Their average estimates the estimator’s expected value, their standard deviation estimates its standard error under the chosen process, and their histogram shows its shape.
Question
Why do we repeat steps 2-3 many many times, but not step 1?
The sample mean is one over n times the sum of the observations. Calling it unbiased for the population mean means that if we repeatedly drew samples and recomputed that average, the center of all those sample means would equal the population expected value.
The equation on screen states this precisely. Take the expectation of one over n times the sum from i equals one through n of x sub i. By linearity of expectation, move the constant one over n outside and take the expectation of each term. Every x sub i is drawn from the same distribution, so every term has expected value E of x. Adding n copies and dividing by n leaves E of x. Independence is part of the sampling setup shown here and will help determine the variance of the mean, although linearity itself does not require independence.
Unbiasedness does not say that a particular sample mean equals the population mean. Almost every individual sample mean will differ. It says the positive and negative sampling errors balance in expectation across repetitions.
Question
Is sample mean really an unbiased estimator of the expected value?
That is, is E[\frac{1}{n}\sum_{i=1}^n x_i] = E[x], where x_i is an independent random draw from the same distribution?
Here are steps one through three for a single replication. The comment headings beginning with hash marks are ignored by R; they label the parts of the simulation for us. Read the first executable line as “x gets run-if one hundred.” Run-if one hundred generates one hundred independent Uniform zero-to-one values, and the assignment arrow stores that sample in x. The sample size is therefore one hundred. For a Uniform distribution from zero to one, the population mean is the midpoint, zero point five, and the population variance is one twelfth. We know that truth because we chose the distribution.
The next line calls “mean of x,” which adds the one hundred observations and divides by one hundred. The result is assigned to the object named mean-x. The final line contains only that object name, so R prints the stored sample mean. Run the cell. The answer should be near zero point five, but it will almost certainly not equal zero point five. Run it again and run-if advances to a fresh sample, so the printed mean changes.
That movement is sampling variation, not a defect. One result cannot tell us whether the estimator is centered correctly.
Step four is where the method earns its keep. The preceding cell produced one estimate from one random sample. Seeing that estimate above or below zero point five does not reveal whether the difference is bad luck or systematic bias. We therefore repeat the same data-generating and estimation steps many times, using fresh random numbers on each repetition, and save the resulting collection of estimates.
The blue word identifies the programming tool: a loop. A loop tells R to execute a block of commands once for each value in a sequence. In this simulation, each pass through the block will draw one hundred new Uniform values and compute one new sample mean. The parameter value, distribution, and within-sample size remain fixed. Only the random realization changes. That consistency is what makes the resulting collection an approximation to one well-defined sampling distribution.
This is a loop in isolation, doing nothing except printing its counter. The first assignment sets capital B equal to one thousand. B is our notation for the number of Monte Carlo replications. The expression on the loop line, spoken “one through B,” constructs the integer sequence one, two, three, and so on through one thousand.
Now read the loop header as “for each i in one through B.” On the first pass, R assigns one to i. It then executes every command between the opening and closing braces. Here the only command is “print i,” so R prints one. On the second pass, i becomes two and R prints two. This continues through i equal to nine hundred ninety-nine and then one thousand. After the body has run for the final value, the loop ends. The braces matter because they define the block to repeat; indentation helps humans see that structure but does not cause the repetition.
Run the cell and you will see one thousand printed scalar values scroll through the output. The words “print i” after the hash mark are a comment for the reader, and R ignores them.
R code
Verbally
For each of i in 1:B (1, 2, \dots, 1000), do print(i).
i takes the value of 1, and then print(1)i takes the value of 2, and then print(2)i takes the value of 999, and then print(999)i takes the value of 1000, and then print(1000)Now we have the real simulation loop. The cell option spoken “auto-run true” tells the browser to execute this cell automatically, so its stored results are available to the next tab. The first assignment sets B, the number of replications, to one thousand. This is not the sample size. Each replication will still contain one hundred observations.
The call “rep zero comma B” creates a vector containing B zeros, and the assignment gives it the descriptive name estimate-storage-mean. The zero is merely a placeholder, and B determines the vector’s length.
Now read the loop body. For each i from one through B, run-if one hundred generates a fresh sample of one hundred Uniform zero-to-one values and stores it in x. Mean of x computes that sample’s mean and stores it temporarily in mean-x. The last line says “estimate-storage-mean bracket i gets mean-x,” meaning the i-th position of the storage vector receives this replication’s mean. On iteration one, position one is replaced; on iteration two, position two is replaced; by iteration one thousand, every placeholder has been replaced by an estimate. The objects x and mean-x are overwritten on each pass, but no result is lost because it has already been filed in its indexed position.
Run the cell if it has not autorun. Nothing is printed because every executable line is an assignment or part of the loop. That silence is correct.
Step five compares the estimates with the truth. Because x was drawn from a Uniform zero-to-one distribution, the true population mean is zero point five. The first cell applies the mean function to the entire vector estimate-storage-mean. This is not another sample mean within one dataset. It is the Monte Carlo average of one thousand separate sample-mean estimates, our numerical approximation to the estimator’s expected value. It should be close to zero point five. In a verified run it was about zero point four nine nine, and an error around a thousandth is typical because the Monte Carlo standard error of this average is roughly zero point zero two nine divided by the square root of one thousand.
The second cell keeps all one thousand estimates visible. Read its first line as “data-frame, x equals estimate-storage-mean.” That makes a data frame whose x column contains the estimates. Gee-gee-plot uses that data, and geom-histogram maps the estimates to the horizontal axis, divides their range into thirty bins, fills the bars gray, and draws white borders between bins. The vertical axis is the number of estimates in each bin. The title reminds you that these are one thousand sample means, not ten thousand raw Uniform observations.
The first vertical line has x-intercept zero point five, is blue, and marks the truth. The second has x-intercept equal to the Monte Carlo average, is red and dashed, and marks the center of the simulated estimates. They should sit almost on top of each other. The histogram should also look approximately bell-shaped because each estimate averages one hundred independent values. Its spread is about zero point zero two nine; in the verified run, individual means ranged roughly from zero point four zero to zero point five eight. Those individual misses do not contradict unbiasedness. Unbiasedness is a statement about the center across repeated samples, not equality in any one sample.
Compare your estimates with the true parameter
runif()Now we reach the question that motivated the lecture. The model in the callout says y equals beta zero plus beta one times x plus u. Beta zero is the population intercept, beta one is the population slope, x is the observed explanatory variable, and u collects the unobserved determinants of y. The hat in beta-one-hat matters: beta one is the fixed truth built into the data-generating process, while beta-one-hat is the random OLS estimate computed from one sample.
Zero conditional mean says that the average value of u is zero for every value of x. When it holds, x carries no systematic information about the omitted component of y, and under the other OLS conditions the slope estimator is centered on beta one. The callout asks what happens when that condition does not equal zero. We are not changing the true coefficient. We are changing the relationship between x and the error, then asking where repeated beta-one-hats center.
Question
What happens to \hat{\beta}_1 if E[u|x]\ne 0 when estimating y=\beta_0+\beta_1 x + u?
Read the preparation first. B equals one thousand replications, N equals one hundred observations per replication, and “rep zero comma B” preallocates a vector with one position for each estimated slope. The simulation does not set a seed on this tab, so exact displayed values can change across runs, but the data-generating process and its theoretical implications do not.
Inside each loop iteration, R-norm of N creates N independent standard Normal draws called mu. The next R-norm of N creates another independent standard Normal component and adds mu to form x. A third independent standard Normal component is added to the same mu to form u. Sharing mu is the deliberate violation. When mu is high, both x and u tend to be high; when it is low, both tend to be low. Because all components are jointly Normal here, the conditional mean of u given x is one half times x, not zero.
The outcome line says y equals one plus x plus u. It therefore builds in a true intercept of one and a true slope on x of one. Read the next call as “data-frame, y equals y, x equals x.” It combines the two observed variables into a dataset with columns named y and x. The error u is intentionally not passed to the regression, just as it would not be observed in real data.
The fee-ols function from the fixest package fits OLS. Read the formula as “y tilde x,” meaning regress y on x with an intercept, and read the named data argument as telling the function where to find those variables. The fitted model is stored as reg. Its coefficient component is a named vector, and “reg dollar coefficient bracket x” selects the estimated coefficient on x rather than the intercept. Finally, “estimate-storage bracket i” stores that coefficient in the i-th position. Only the x-error relationship was broken; the coefficients, sample size, and estimator stay fixed, giving us the controlled comparison promised at the start.
Before looking at the output, predict it. Write x as an independent standard Normal component plus mu, and write u as a different independent standard Normal component plus the same mu. All three underlying components have variance one and are mutually independent.
Now expand the covariance of x and u. Cross-covariances between the independent components are zero. The only term shared by x and u is mu, so the surviving covariance is the variance of mu, which equals one. For the denominator, the variance of x is the variance of its private standard Normal component plus the variance of mu. Independence lets us add those variances, giving one plus one equals two.
The probability limit displayed at the bottom says beta-one-hat approaches the true beta one plus the covariance of x and u divided by the variance of x. The true slope was set to one. The covariance-to-variance ratio is one over two. Adding them gives one point five. Spoken economically, OLS attributes to x both the true one-unit effect and the positive movement in the omitted error that accompanies x. In this jointly Normal construction, the conditional mean of u given x is exactly one half x, so the regression function itself has slope one point five.
The prediction is therefore not merely “biased upward.” It is quantitative: the sampling distribution of the estimated slope should center near one point five rather than the true structural coefficient one. Write down one point five before moving on.
One note on the arrow, because it is not an equals sign. It means the value the estimate settles on as the sample grows, and we derive that idea in lecture eight. You have already seen the sample version of this same statement in lecture one, part three: beta-one-hat equals beta one plus one over S-S-T-x times the sum of x-i minus x-bar times u-i. Divide the top and the bottom of that error term by n and the top becomes the covariance of x and u while the bottom becomes the variance of x. So the displayed line is a familiar decomposition, rewritten in population quantities.
Both x and u contain \mu, so they are correlated by construction:
And the bias of \hat{\beta}_1 is governed by exactly that ratio:
\hat{\beta}_1 \rightarrow \beta_1 + \frac{Cov(x, u)}{Var(x)} = 1 + \frac{1}{2} = 1.5
The first cell applies the mean function to the one thousand stored slope estimates. That Monte Carlo average should be close to one point five. In a verified run of this data-generating process it was about one point five zero three, consistent with the covariance calculation on the previous tab. Small departures from one point five are Monte Carlo noise.
The plot then shows the full collection. Read the first call as “data-frame, x equals estimate-storage”; it puts the estimates in one column. The histogram maps that column to the horizontal axis, uses thirty bins, fills them gray, and separates them with white borders. The horizontal axis is the estimate of beta one; the vertical axis is the number of replications in each bin. The title identifies the one thousand estimates. A solid blue vertical line at one marks the true structural slope. A solid red vertical line at one point five marks the predicted center. The estimates pile up around the red line, while the blue truth lies well outside the bulk.
Compare this picture with the one in lecture 01-3, where zero conditional mean held and the histogram centered on the truth. Here the estimator is precise around the wrong target. That distinction also explains the final bullet. Raising N, the number of observations in each simulated dataset, reduces sampling variance and makes the histogram narrower, but the covariance-to-variance term remains one half. The distribution therefore tightens around one point five, not one. More data can make you increasingly confident in the wrong number when the identifying condition fails.
Unbiasedness of OLS estimator \rightarrow Seeing it), where E[u|x]=0 held and the histogram was centered on the truthThis example asks whether OLS estimates its own uncertainty correctly. The model at the top has outcome y, intercept beta zero, slope beta one, regressor x, and error u. The bullets specify a standard Normal regressor, a Normal error, and zero conditional mean, so x contains no information about the average error. Under homoskedastic errors with variance sigma squared, the conditional variance of beta-one-hat, holding the observed x values fixed, is sigma squared divided by S-S-T sub X.
The denominator S-S-T sub X means the sum, from i equals one through n, of x sub i minus x-bar, all squared. It measures the sample variation in x around its mean. More spread in x makes this denominator larger and the slope estimator more precise. Sigma squared is the population error variance. It is unknown in real data, so the second line replaces it with sigma-hat squared.
Look at that replacement carefully. For each observation, u-hat sub i is the OLS residual. Squaring and summing the residuals gives the residual sum of squares. Dividing by n minus two estimates sigma squared because this simple regression estimated two coefficients, an intercept and a slope, using two degrees of freedom. Multiplying that estimate by one over S-S-T sub X gives the estimated variance of beta-one-hat. A regression table’s standard error is the square root of this estimated variance.
The question “is it unbiased?” asks whether, with x held fixed and new errors repeatedly drawn, the average estimated variance equals the true conditional variance.
Model
\begin{aligned} y = \beta_0 + \beta_1 x + u \\ \end{aligned}Variance of the OLS estimator
True Variance of \hat{\beta}_1: V(\hat{\beta}_1) = \frac{\sigma^2}{\sum_{i=1}^n (x_i-\bar{x})^2} = \frac{\sigma^2}{SST_X}
Its estimator: \widehat{V(\hat{\beta}_1)} =\frac{\hat{\sigma}^2}{SST_X} = \frac{\sum_{i=1}^n \hat{u}_i^2}{n-2} \times \frac{1}{SST_X}
Question
Does the estimator really work? (Is it unbiased?)
Start with reproducibility. The first line says “set-seed nine hundred three thousand four hundred seventy-eight.” It fixes the random-number sequence, so the browser calculation matches the hidden R calculation used to print S-S-T sub X on the next tab. B is ten thousand replications and N is one hundred observations per replication. The two calls spoken “rep zero comma B” preallocate one storage vector for slope estimates and another for estimated slope variances.
The x line sits before the loop, which is a deliberate design choice. R-norm of N draws one hundred standard Normal x values once, and those exact values are reused in all ten thousand regressions. Read the next expression as “sum of x minus mean of x, squared.” It subtracts x-bar from every x, squares the deviations, and adds them to calculate S-S-T sub X. Holding x fixed matches the conditional variance formula on the preceding tab. If we redrew x each time, the true conditional variance would change across iterations and one fixed comparison line would no longer answer the same question.
Inside the loop, R-norm of N generates a fresh standard Normal vector on every pass, and multiplying by two gives u standard deviation two and variance four. Read the outcome line as “y gets one plus x plus u”; it sets the true intercept and slope to one. The following data-frame call supplies observed columns y and x to fee-ols from the fixest package, whose formula, spoken “y tilde x,” fits OLS with an intercept.
The coefficient line selects the named x entry from the fitted coefficient vector and stores it in position i. The variance line says “vee-cov of reg, bracket x comma x,” which returns the estimated coefficient variance-covariance matrix and extracts its row-x, column-x diagonal element. That element is an estimated variance, not its square-root standard error. With the conventional independent-and-identically-distributed variance calculation used here, it equals residual sum of squares divided by n minus two, then divided by S-S-T sub X. Ten thousand repetitions make the Monte Carlo average and density much more stable than one thousand would, while the fixed sample size N remains one hundred.
Here is the numerical comparison. The fixed x draw gives S-S-T sub X of about one hundred twelve point zero seven. The simulation code made u equal to two times a standard Normal, so sigma squared is two squared, or four. Substituting those values into the conditional variance formula gives four divided by one hundred twelve point zero seven, approximately zero point zero three five seven.
The cell says “mean of V-beta-storage,” averaging the ten thousand diagonal variance estimates saved from vee-cov. With the displayed seed, the verified result is approximately zero point zero three five eight. That is extremely close to the true value zero point zero three five seven; the small gap is simulation noise.
True Variance
V(\hat{\beta}) = 4/112.07 = 0.0357
Check
Your Estimates of Variance of \hat{\beta}_1?
One last look, because an average hides the sampling variation around it. Read the first call as “data-frame, x equals V-beta-storage.” It places the ten thousand estimated variances in a column named x. Gee-gee-plot passes that data to geom-density, which maps x to the horizontal axis and draws a smoothed density rather than histogram counts. The vertical axis is therefore density, scaled so the total area under the curve is one. Geom-v-line adds a reference line at four divided by S-S-T sub X, rounded to four decimal places, and theme-lecture applies the deck’s common figure styling.
The horizontal values are possible estimates of the variance of beta-one-hat. The true conditional variance is about zero point zero three five seven, so the reference line should pass near the center of the density. Its distribution is not exactly symmetric: it is bounded below by zero and its right tail extends farther than its left. The skewness is modest at ninety-eight degrees of freedom, but individual estimates can still land noticeably above or below the truth.
The average from the preceding tab lies very close to the true line, which is what unbiasedness promises. It does not promise that most estimates equal the truth, or that the single estimated variance in a real dataset is especially close. In real work you observe one draw from this distribution.
This optional exercise isolates something the preceding variance formula has already suggested: how does variation in x affect the precision of the OLS slope? The screen shows two simple-regression data-generating processes with the same intercept beta zero, the same slope beta one, and the same error distribution. The only difference is the explanatory variable used in each equation.
X one follows Normal zero comma one, so it has mean zero, variance one, and standard deviation one. X two follows Normal zero comma nine, so it has mean zero, variance nine, and standard deviation three. The parenthetical emphasizes the standard-deviation comparison because R’s R-norm function will require one and three as its named S-D arguments, even though the mathematical notation displays variances one and nine. The error u is standard Normal in both equations. Zero conditional mean holds for both x one and x two, so neither slope estimator has a systematic relationship with its error. Both should therefore center on the same true slope.
Using MC simulations, find out how the variation in x affects the OLS estimators
Model setup
\begin{align*} y = \beta_0 + \beta_1 x_1 + u \\ y = \beta_0 + \beta_1 x_2 + u \end{align*}The preparation sets B to one thousand replications and N to one hundred observations per equation. Read the storage call as “matrix zero comma B comma two.” It allocates an object with B rows and two columns: each row is one replication, column one will hold the slope from equation one, and column two the slope from equation two.
Within each loop pass, the first R-norm call has N as the number of draws and a named S-D argument of one, so it draws x one with standard deviation one. The second has N draws and S-D three, so it draws x two with standard deviation three and variance nine. A third R-norm of N draws the standard Normal error u. The same u vector enters both outcomes, making the two simulated experiments paired within an iteration, but each regressor is independently drawn and satisfies zero conditional mean. Read the first outcome line as “y one gets one plus x one plus u.” It and the corresponding y two line both set the true intercept and slope to one.
Data-table from the data.table package combines the four named columns into one table. The package name written before the double colon tells R exactly where to find that function. Each fee-ols call from fixest fits one OLS regression with an intercept: y one on x one, then y two on x two, using the shared table supplied through the named data argument. The coefficient component is named, so bracket x one or bracket x two selects the slope. Matrix indexing uses i for the row and one or two for the column, filing both estimates from each iteration side by side.
After the loop, say “estimate-storage bracket all rows comma column one.” The blank position before the comma selects every row, and the one selects column one; that vector is named beta-one-s. Column two similarly becomes beta-two-s. The two mean calls should both return values close to one, confirming that changing x variation did not create bias. The two S-D calls measure the spread of the simulated sampling distributions. The x-two estimates should have about one third the standard deviation of the x-one estimates. In a verified one-thousand-replication run, the ratio was about two point eight three rather than exactly three, an ordinary finite-simulation difference from the theoretical approximation.
Here is the same result as a picture. The first two lines create separate data tables. In each, the column x contains one equation’s stored coefficient estimates, and the column called type contains the label “Equation 1” or “Equation 2” for every row. R-bind stacks those tables into one long dataset so a single plot can distinguish the two groups. The figure-width and figure-height options set the drawing canvas to nine by six point five inches; they affect presentation size, not the simulation or estimates.
In gee-gee-plot, geom-density maps coefficient estimates to the horizontal axis and group labels to fill color. It draws one smoothed density per equation. The alpha argument equals zero point five, making each fill half-transparent so you can see the overlap. Scale-fill-discrete receives an empty name, which removes the legend title. X-lab names the horizontal axis “Coefficient Estimate,” the lecture theme supplies common styling, and the final theme call places the legend below the plot. The vertical axis is density, so each curve has total area one; compare centers and widths, not raw heights as counts.
Both densities should center near the true slope of one. That is the unbiasedness result and is not what separates them. Equation one, whose x has variance one, produces the wider density. Equation two, whose x has variance nine, produces a dramatically narrower density, with roughly one third the standard deviation and one ninth the variance. The sample size, error variance, coefficient, and identifying condition are the same, so the difference comes from the denominator S-S-T sub X. More variation in an independent variable is information about its slope. If x barely moves, OLS has little leverage for separating changes in y associated with x from noise.
Here is a problem that comes up in every applied project, and one for which you do not yet have a theorem. Your data is measured with error. Farmers recall last year’s yield without complete records, survey respondents round or misreport their income, and rainfall at a particular field is interpolated from readings at surrounding stations. In each case the number recorded in your dataset is the true quantity plus a mistake. The question is whether that matters, and whether it matters differently depending on which variable carries the mistake.
Read the model. Y equals beta zero plus beta one times x plus u, the true slope beta one is one, the true regressor x is standard Normal, and the error u is standard Normal and independent of x. Everything OLS asks for holds in this world. If you could observe x and y themselves, OLS would be unbiased.
Now read the two things you might observe instead. In the first, y-noisy equals y plus e, where e is an independent standard Normal draw: the mistake lands in the outcome, and the regressor is recorded perfectly. In the second, x-noisy equals x plus e, drawn the same way: the mistake lands in the regressor, and the outcome is recorded perfectly. The quantity of noise is identical in the two cases. The only difference between them is which variable carries it.
Work through the seven numbered instructions. Step one asks you to commit to a prediction before writing code, and it is the most important step on this slide. For each of the two cases, write down on paper whether you expect the estimates to center at one, above one, or below one. A prediction you have written down is the only way to find out whether you understood the problem, because an explanation always sounds reasonable after you have seen the answer.
Steps two through four build the simulation with the machinery from Example 2. Set B to one thousand replications and N to one hundred observations, and allocate a storage with B rows and two columns, one column per case. Inside the loop, draw x, draw u, and build y exactly as the model says. Then create the two contaminated variables by adding a fresh R-norm of N to each. Fresh matters: draw the noise separately for each, rather than reusing one vector, so the two cases are independent of each other.
Step five runs the two regressions. The first regresses y-noisy on x, which is the case where the outcome is mismeasured. The second regresses y on x-noisy, which is the case where the regressor is mismeasured. Store the first slope in column one and the second in column two.
Steps six and seven read the result. Take the mean of each column to find where that collection of estimates centers, and the standard deviation of each to find how far apart the estimates are. Then answer the three questions at the end: which case is biased, in which direction, and which case produced the tighter collection of estimates. Pay attention to whether those two are the same case, because the answer decides whether a narrow distribution is something you should be pleased about.
Read the note at the bottom before you write the regression. When you regress y on x-noisy, the coefficient you want is stored under the name x-noisy, not x, because fixest names a coefficient after the variable that appears in the formula.
Your explanatory variable is measured with error . Recalled yields, rounded survey income, rainfall interpolated from nearby stations: the number in your dataset is the true one plus a mistake. Does it matter which variable carries the mistake?
Model setup
\begin{align*} y = \beta_0 + \beta_1 x + u \end{align*}Instructions
B <- 1000 and N <- 100, and create a storage with B rows and 2 columns.x, u, and y as the model says.y_noisy and x_noisy by adding a fresh rnorm(N) to each (draw the noise separately, not once).y_noisy on x in column 1, and y on x_noisy in column 2.mean() of each column (where do they center?) and sd() of each (how far apart are the estimates?).One thing to watch
fixest names a coefficient after the variable in the formula, so in the second regression the slope is stored under "x_noisy", not "x".
Work here
Compare this against what you wrote. The preparation sets B to one thousand replications and N to one hundred observations, and the call spoken “matrix zero comma B comma two” allocates a storage with one row per replication and one column per case.
Inside the loop, R-norm of N draws the true regressor x, another R-norm of N draws the error u, and the outcome line reads “y gets one plus x plus u,” putting the true intercept and the true slope of one into the world. Nothing is wrong with this data. The next two lines add a fresh R-norm of N to y to make y-noisy, and a separate fresh R-norm of N to x to make x-noisy. Because those are two separate calls, the mistake in the outcome and the mistake in the regressor are independent of each other, and each has variance one.
The data-frame call collects all four variables so both regressions can read from the same table. The first fee-ols call has the formula “y-noisy tilde x,” which is the case where the outcome carries the mistake, and its slope goes into column one. The second has the formula “y tilde x-noisy,” which is the case where the regressor carries it, and its slope goes into column two under the name x-noisy.
After the loop the two columns are given the names noise-in-y and noise-in-x, and the four summary calls print where each collection centers and how far apart its estimates are.
Now read the numbers, because the two cases behave completely differently. In a verified one-thousand-replication run, the noise-in-y estimates centered at one point zero zero one with a standard deviation of zero point one three eight. That collection is centered on the truth. Putting a mistake in the outcome cost nothing on average. It simply added another independent piece to the error term, which widened the collection of estimates without moving it.
The noise-in-x estimates centered at zero point four nine eight with a standard deviation of zero point zero eight eight. That is not noise around one. It is roughly half of one, and it is half every time you run it. A mistake in the regressor shrinks the estimated slope toward zero systematically. The reason is that the recorded regressor is only partly the true one, so a one-unit change in what you recorded corresponds to less than a one-unit change in what actually drives the outcome, and OLS reports the smaller number.
Compare the two standard deviations before you leave this tab, because this is the part most people get backwards. Zero point zero eight eight is smaller than zero point one three eight. The biased case produced the tighter collection of estimates. If you had one dataset from each and only their standard errors to judge by, the one that was wrong by half would look like the more precise result. A narrow distribution tells you how repeatable an answer is, not whether it is the right answer.
Here is the same result as a picture, which makes the comparison harder to argue with. The first two lines create separate tables, each holding one case’s stored estimates in a column named x and a label in a column named type. R-bind stacks them into one long dataset so a single plot can draw both.
In gee-gee-plot, geom-density maps the estimates to the horizontal axis and the label to fill color, drawing one smoothed curve per case, and the alpha argument of zero point five makes each fill half transparent so the two can overlap. Geom-v-line adds a solid blue vertical line at one, which is the true slope, following the same convention as the earlier figures in this deck. Scale-fill-discrete with an empty name removes the legend title, x-lab names the horizontal axis, and the final theme call puts the legend below the plot. The vertical axis is density, so each curve encloses an area of one. Compare centers and widths, not heights.
Read the picture. The noise-in-outcome curve is centered on the blue line. Its estimates land on both sides of the truth, sometimes noticeably far from it, but there is no systematic tendency to miss in one direction. That is what an unbiased estimator looks like when you have made its job harder.
The noise-in-regressor curve sits well to the left, centered near zero point five, and the blue line marking the truth falls outside the bulk of it entirely. It is also the narrower of the two curves. Put those two facts together and you have the warning this exercise exists to deliver. The narrow curve is the wrong one. Width tells you how much an answer would move if you collected your data again. It says nothing at all about whether the thing you are centered on is the quantity you wanted.
One more thing worth noticing before you move on. Both of these came from a world where every condition OLS asks for held for the true variables. Nothing was endogenous by construction, no variable was omitted, and the sample was a clean random draw. All that happened is that a recording mistake moved from one column of the dataset to another.