Let’s set up exactly what this lecture is asking. An estimator is a rule, or a function, that takes sample data as its input and produces an estimate, an actual number, as its output. Look at the OLS slope estimator on screen. Beta-one-hat is the estimate of the unknown population slope beta one. In the numerator, we sum, from observation i equals one through n, the product of each x value’s deviation from x-bar and each y value’s deviation from y-bar. Here x-bar and y-bar are the two sample means. In the denominator, we sum the squared deviations of x from x-bar. Once you insert the observed x’s and y’s, this formula returns one number.
A small-sample property describes the behavior of that rule before we see a particular dataset or calculate a particular estimate. Imagine repeatedly drawing samples from the same population and applying the rule each time. The estimates form a sampling distribution. A property such as unbiasedness or variance tells us something about that distribution. The red phrase, “prior to,” is essential. A small-sample property does not inspect one realized estimate and certify that it is close to the truth. It tells you what to expect from the estimator across all possible samples under stated assumptions.
The word “small” can be misleading. These are finite-sample results, so they hold for a sample of any fixed size, small or large. That contrasts with asymptotic properties, which describe what happens as n grows without bound. It also differs from the algebraic properties from the earlier lecture. An algebraic property, such as the OLS residuals summing to zero when an intercept is included, holds mechanically in the sample you actually have. A statistical small-sample property concerns repeated hypothetical samples and depends on assumptions about how the data were generated. With that distinction in place, flip to the OLS tab and ask which statistical properties make this particular rule attractive.
What is an estimator?
A function of data that produces an estimate (actual number) of a parameter of interest once you plug in actual values of data
OLS estimators: \widehat{\beta}_1=\frac{\sum_{i=1}^n(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^n (x_i-\bar{x})^2}
What is small sample property?
Properties that hold whatever the sample size (small or large) is prior to obtaining actual estimates (before getting data)
Put more simply: what can you expect from the estimators before you actually get data and obtain estimates?
Difference between small sample property and the algebraic properties we looked at earlier?
Here is the question that motivates the whole lecture. OLS is only one possible rule for turning the information in a sample into estimates of beta zero and beta one. The red “a” is there to stop you from treating OLS as inevitable. We could invent other rules, use only part of the sample, assign different weights to observations, or deliberately trade a little bias for less variability. So we need a reason to prefer OLS.
The two bullets give us the criteria we will use. Unbiasedness asks about the center of the estimator’s sampling distribution. If we could repeat the sampling process indefinitely, would the average OLS estimate equal the true population parameter? Efficiency asks about the spread of that distribution. Among estimators that are comparable in the relevant sense, does OLS produce estimates that vary relatively little from sample to sample? An estimator can be centered correctly but very imprecise, so these are separate virtues.
The favorable verdict at the bottom is conditional, not unconditional. OLS is unbiased under assumptions that we will state explicitly, and it has the Gauss-Markov efficiency property under one additional assumption. The purpose of this lecture is to identify those assumptions, show where they enter the argument, and clarify what the conclusions do and do not let you say about the estimate from the one sample you actually observe.
OLS is just a way of using available information to obtain estimates. Does it have desirable properties? Why are we using it?
As it turns out, OLS is a very good way of using available information!!
Before doing this for OLS, let’s build the idea somewhere simpler. Suppose the parameter of interest is the population expected value of a random variable x. The estimator on screen is the sample mean: one over n times the sum of x-i from i equals one through n. It uses every observation and returns their arithmetic average. Under random sampling, its expected value equals the population mean, so it is an unbiased estimator even though an individual sample mean will almost never equal that parameter exactly.
Now read the code from top to bottom. The first assignment sets n, the number of observations, to one hundred. The call “r norm of n” then generates one hundred independent draws from a normal distribution. Because the mean and standard-deviation arguments are omitted, R uses its defaults, mean zero and standard deviation one. The comment records that this is a standard normal population, so the true mean we are trying to estimate is zero. Those simulated values are stored in x-seq. Finally, “mean of x-seq” adds the one hundred values and divides by one hundred, which is exactly the estimator displayed above the code.
Follow the direction in the callout and run the cell several times. Each run generates a fresh random sample, so the printed sample mean changes. It should usually be fairly near zero with one hundred observations, but it need not be zero, and occasionally it will be farther away. That sample-to-sample movement is not evidence that the estimator is biased. Bias is not the error in one run. To assess bias, we need the distribution produced by many runs and must ask where that entire distribution is centered. The next tab makes that repeated-sampling thought experiment visible.
What does unbiased even mean?
Let’s first look at a simple problem of estimating the expected value of a single variable (x) as a start.
R code: Sample Mean
Direction
Try running the codes multiple times and feel the tendency of the estimates.
Here is what the repeated-sampling experiment looks like when we perform it one thousand times. In every repetition, we draw one hundred observations from the same standard normal population and calculate their sample mean. The horizontal axis is the resulting sample mean, our estimate in that repetition. The vertical height of each histogram bar records how many of the one thousand estimates fall in that interval. The red vertical line at zero marks the true population mean.
Start with the top panel, labeled “unbiased.” The estimates are not all zero. They form a distribution because each random sample is different. But the pile is centered at the red truth line: estimates above zero balance estimates below zero in repeated samples. That is what unbiased means. In notation, if theta-hat denotes the estimator and theta denotes the true parameter, unbiasedness is the statement that the expected value of theta-hat equals theta. It is not the statement that theta-hat equals theta in every sample.
The bottom panel is constructed from exactly the same estimates after adding zero point two to every one. Its label therefore says “biased by zero point two.” The red line still marks the truth at zero, but the sampling distribution is now centered around zero point two. Notice that its shape and width are essentially the same as the unbiased distribution. Bias and variance are different dimensions: bias is displacement of the center from the truth, while variance is spread around the estimator’s own center. A biased estimator can be tightly clustered and look very convincing while being systematically wrong. We will return to the width of these distributions when we discuss efficiency.
Now translate that definition from the sample mean to the OLS slope estimator. Beta one is the true slope in the population model, and beta-one-hat is the slope calculated from a sample. The long expression inside the expectation is the OLS formula: in the numerator, sum each x deviation from x-bar times the corresponding y deviation from y-bar; in the denominator, sum each squared x deviation. The expectation operator asks for the average value that this formula would produce across repeated samples drawn by the same process.
The equality on the far right is the claim of unbiasedness: the expected value of beta-one-hat equals the true beta one. It does not claim that the ratio of two expectations is being taken, and it does not say the estimate from any particular sample equals beta one. The expectation applies to the entire random estimator, including all of the sample x’s, y’s, and sample means inside it.
Now focus on the blue phrase, “under certain conditions.” Unbiasedness is not an automatic algebraic feature of OLS. It is a statistical result that follows from assumptions about the population relationship and the sampling process. We state four such conditions in the Conditions tab and then show in the proof exactly which one removes the bias term. We concentrate on the slope because it is normally the economically meaningful parameter, the change in y associated with a one-unit change in x. The intercept also has an unbiasedness result under the same assumptions, but the slide deliberately sets it aside because intercepts are often not substantively interesting and our limited time is better spent on the slope.
Under certain conditions , OLS estimators are unbiased. That is,
\def\sumn{\sum_{i=1}^{n}} E[\widehat{\beta}_1]=E\Big[\frac{\sumn (x_i-\bar{x})(y_i-\bar{y})}{\sumn (x_i-\bar{x})^2}\Big]=\beta_1
(We do not talk about unbiasedness of \widehat{\beta}_0 because we are almost never interested in the intercept. Given the limited time we have, it is not worthwhile talking about it)
Let’s make the OLS claim visible before proving it. The simulation creates a population relationship in which the true intercept is ten and the true slope beta one is two. In each repetition, it draws n equals one hundred x values with substantial variation, draws independent mean-zero errors, constructs y as ten plus two times x plus u, runs the regression of y on x, and stores the estimated slope. Independence of x and u is built into this artificial world, so zero conditional mean holds by construction. The entire process is repeated B equals one thousand times.
The histogram collects those one thousand values of beta-one-hat. The horizontal axis is labeled “Estimate of beta one,” so moving right means a larger estimated slope. The vertical axis is the number of simulation repetitions falling into each of the forty bins. The blue outlines distinguish the histogram bars, and the red vertical line is placed at the true beta one, which is two. With the seed used on this slide, the estimates range roughly from one point four six to two point five three, and their average is about two point zero one. The exact collection is a simulation outcome, but the important visual feature is that the pile is centered at the red line.
Read the callout carefully. Individual estimates land on both sides of two, sometimes quite far from it. Any one of those values could have been the estimate from our only observed dataset. Unbiasedness does not promise that a particular estimate is correct or even close. It says that the estimation rule has no systematic tendency to miss in one direction across repeated samples. Now flip to Conditions, because that center-at-two result was engineered by assumptions, especially the way we generated u relative to x.
We did this for the sample mean two tabs ago. Here it is for the estimator this lecture is actually about: draw a fresh sample, run OLS, keep \widehat{\beta}_1, and repeat 1,000 times. The true \beta_1 is 2.
What to take from it
Individual estimates land all over the place — some well above 2, some well below. Unbiasedness is not a claim about any one of them. It is the claim that the red line sits at the centre of the pile.
Four conditions give us the finite-sample unbiasedness result, and this tab states SLR point one, linear in parameters. The population model says y equals beta zero plus beta one times x plus u. Here y is the dependent variable we want to explain, x is the explanatory variable, beta zero is the population intercept, beta one is the population slope, and u is the error or disturbance. The error collects every determinant of y that is not represented by the systematic part beta zero plus beta one x.
This is an assumption about the population relationship that generated the data, not a statement that the plotted sample points must lie exactly on a line. The points generally do not lie on the line because u differs across observations. The slope beta one says how the systematic part of y changes when x rises by one unit. The intercept beta zero is that systematic value when x equals zero, whether or not zero is meaningful in the application.
Also be precise about the phrase “linear in parameters.” Beta zero and beta one enter as coefficients without being squared, logged, or otherwise transformed. The explanatory variable itself can later appear as x squared, log x, or another known transformation and the model can still be linear in its parameters. That extension comes later in the course. For this simple-regression lecture, the maintained population model has one x and one slope. If this representation is not appropriate, the beta one whose unbiased estimation we are about to prove may not describe the relationship you intend to study. The note at the bottom simply tells you that SLR point one and the numbering used here follow Wooldridge’s textbook.
Linear in Parameters
In the population model, the dependent variable, y, is related to the independent variable, x, and the error (or disturbance), u, as
y=\beta_0+\beta_1 x+u
Note: This definition is from the textbook by Wooldridge
SLR point two concerns how observations enter the sample. The notation in the callout describes n observed pairs, x-i and y-i, indexed from i equals one through n, that follow the same population model. A random sample means the observations are independently drawn from the target population and each draw follows that same distribution. This lets us connect sample averages and repeated-sample expectations to population quantities, and it lets the zero-conditional-mean condition for one population draw carry over observation by observation.
The income and education example shows why the target population matters. If your target is all workers but you observe only people with income above twenty-five thousand dollars, inclusion depends directly on y, the outcome. The low-income part of the population is absent by design. The relationship between education and income within that selected group need not equal the relationship in the full population, so you cannot casually interpret the selected sample as random from all workers.
The two red terms distinguish kinds of non-random sampling. Exogenous sampling means selection is unrelated to the unobserved determinants of the outcome once the relevant explanatory variables are accounted for. Such a design may still change which population your conclusions describe, but it need not bias the conditional slope for that population. Endogenous sampling means inclusion is related to the outcome or to the unobserved error in a way that distorts the relationship we want. Sampling only above an income threshold is the warning example. The exact consequences depend on the selection rule, so we will treat sampling problems more carefully later. For now, do not reduce “I have many observations” to “I have a random sample.” Sample size cannot repair systematic selection.
Random sampling
We have a random sample of size n, {(x_i,y_i):i=1,2,\dots,n}, following the population model.
Non-random sampling
SLR point three requires variation in the explanatory variable. The notation in the callout means that the observed values x-one through x-n are not all identical. This is the gentlest assumption because you can check it directly in the data, and because its failure makes estimation impossible rather than silently misleading.
Look back at the OLS slope formula. Its denominator is the sum, over i, of x-i minus x-bar squared. We call that quantity SST-x. If every observation has the same x value, then that common value is also x-bar. Every deviation is zero, SST-x is zero, and beta-one-hat would require division by zero. There is no finite slope estimate.
The intuition is the same as the algebra. A slope describes how y changes as x changes. If x never changes in your sample, no comparison in the data reveals that relationship. For example, if every field received exactly the same fertilizer rate, that sample cannot tell you how yield responds to fertilizer, no matter how many fields you observed. Software will normally mark the slope as unidentified or remove the constant regressor. Later, when we study variance, you will see a graded version of this issue: even if x is not perfectly constant, very little variation in x makes the slope estimate imprecise.
Variation in covariates
The sample outcomes on x, namely, {x_i,i=1,\dots,n}, are not all the same value.
SLR point four is zero conditional mean, and it is the crucial identifying assumption. The first equation says the expected value of u conditional on x equals zero. In spoken terms, take all population units with any particular value of x and average the omitted influences collected in u. That average must be zero for every x value. Equivalently, knowing x must not give you a systematic prediction about whether the error is positive or negative. This allows the conditional mean of y given x to be beta zero plus beta one x.
Under random sampling, each observed pair follows that population condition, giving the second equation, expected u-i given x-i equals zero. For the proof, we condition on the full set of sample x values, bold x. Independence across randomly sampled observations means the other observations’ x values add no information about u-i, so expected u-i given bold x is also zero. That is the expectation we will use to make the error term disappear.
The important callout separates zero conditional mean from zero correlation. If expected u given x is zero at every x, then the covariance between x and u is zero, so x and u are uncorrelated. The reverse need not hold. Correlation measures only linear association. U can depend systematically on x in a nonlinear way while having zero covariance, in which case expected u given x is not zero. Therefore checking a sample correlation, even if u were observable, would not establish SLR point four.
Substantively, this assumption rules out omitted determinants of y that are systematically related to x, reverse causality that makes x respond to y, and some forms of measurement error and selection. Unlike variation in x, it cannot be verified from the observed x and y alone because u is unobserved. You must defend it using knowledge of how the data and treatment were generated. That is why this condition, and methods for dealing with its failure, occupy so much of econometrics.
Zero conditional mean
The error term u has an expected value of zero given any value of the explanatory variable. In other words,
E[u|x]=0
Along with random sampling condition, this implies that
E[u_i|x_i]=0
Necessary but not sufficient
E[u|x]=0 implies that x is uncorrelated with u. Zero correlation by itself does not imply E[u|x]=0.
This proof shows exactly why OLS is unbiased, so follow each equality rather than memorizing the final statement. The first line is the usual OLS slope estimator. Beta-one-hat equals the sum, from i equals one to n, of x-i minus x-bar times y-i minus y-bar, divided by the sum of x-i minus x-bar squared.
On the second line, y-bar disappears from the numerator. To see why, distribute x-i minus x-bar across y-i minus y-bar. One term is the sum of x-i minus x-bar times y-i. The other is minus y-bar times the sum of x-i minus x-bar. The deviations from a sample mean always sum to zero, so that second term is zero. This is an algebraic identity in every sample, not an assumption.
The third line introduces the abbreviation SST-x for the denominator, the total sum of squares of x: the sum of all squared deviations from x-bar. No mathematics changes there; the new name only makes the remaining lines readable.
On the fourth line, replace each observed y-i with beta zero plus beta one x-i plus u-i. That substitution comes from SLR point one, the population model. This is where the true parameters and the unobserved error enter the estimator’s formula. Finally, the fifth line distributes x-i minus x-bar across the three terms and splits one sum into three sums. Because beta zero and beta one are population constants, they can be placed inside or outside their respective sums. We now have an intercept component, a slope component, and an error component. Flip to Step 2 to simplify each of them.
\def\sumn{\sum_{i=1}^{n}} \begin{aligned} \widehat{\beta}_1 = & \frac{\sumn (x_i-\bar{x})(y_i-\bar{y})}{\sumn (x_i-\bar{x})^2} \\\\ = & \frac{\sumn (x_i-\bar{x})y_i}{\sumn (x_i-\bar{x})^2} \;\; \Big[\text{because }\sumn (x_i-\bar{x})\bar{y}=0\Big]\\\\ = & \frac{\sumn (x_i-\bar{x})y_i}{SST_x} \;\;\Big[\text{where,}\;\; SST_x=\sumn (x_i-\bar{x})^2\Big] \\\\ = & \frac{\sumn (x_i-\bar{x})(\beta_0+\beta_1 x_i+u_i)}{SST_x} \\\\ = & \frac{\sumn (x_i-\bar{x})\beta_0 +\sumn \beta_1(x_i-\bar{x})x_i+\sumn(x_i-\bar{x})u_i}{SST_x} \end{aligned}
Step 2 simplifies the three numerator pieces from the previous tab. Start with the first displayed line. The intercept component is beta zero times the sum of x-i minus x-bar. Since deviations from the sample mean sum to zero, that entire component vanishes.
The second displayed equation gives the identity needed for the slope component. We want to simplify the sum of x-i minus x-bar times x-i. Write x-i as x-i minus x-bar plus x-bar. Multiplying gives x-i minus x-bar squared, plus x-bar times x-i minus x-bar. When we sum over i, the second part is x-bar times a sum of deviations, so it is zero. The part that remains is the sum of squared deviations, exactly SST-x. This identity is again pure sample algebra.
Substitute both results into the final display. The numerator becomes beta one times SST-x plus the sum of x-i minus x-bar times u-i. Dividing every term by SST-x makes the first fraction beta one, because SST-x cancels. SLR point three guarantees that SST-x is not zero, so this division is legitimate.
We have now reached the key decomposition: beta-one-hat equals the true beta one plus one over SST-x times the sum of x deviations multiplied by their errors. The estimation error, beta-one-hat minus beta one, is therefore a weighted combination of the u-i’s, with weights determined by where each x-i lies relative to x-bar. All of the possible bias must come from the average behavior of that remaining error component. Step 3 takes its expectation.
\def\sumn{\sum_{i=1}^{n}} \begin{aligned} \widehat{\beta}_1 = & \frac{\sumn (x_i-\bar{x})\beta_0 + \beta_1 \sumn (x_i-\bar{x})x_i+\sumn (x_i-\bar{x})u_i}{SST_x} \end{aligned}
\def\sumn{\sum_{i=1}^{n}} \begin{aligned} \text{Since } & \sumn (x_i-\bar{x})=0\;\; \text{and}\\ & \sumn (x_i-\bar{x})x_i=\sumn (x_i-\bar{x})^2=SST_x, \end{aligned}
\def\sumn{\sum_{i=1}^{n}} \begin{aligned} \widehat{\beta}_1 = \frac{\beta_1 SST_x+\sumn (x_i-\bar{x})u_i}{SST_x} = \beta_1+(1/SST_x)\sumn (x_i-\bar{x})u_i \end{aligned}
The first line repeats the decomposition we just derived: beta-one-hat is the true beta one plus one over SST-x times the sum of x-i minus x-bar times u-i. Think of it as truth plus a sample-dependent error term. To establish unbiasedness, we must show that this second term has expected value zero.
The next display takes an expectation conditional on bold x, meaning the full collection x-one through x-n observed in the sample. Conditional on those x values, x-bar and SST-x are fixed numbers, as are all the weights x-i minus x-bar. Beta one is a fixed population parameter, so its conditional expectation is just beta one. Linearity of expectation lets us move one over SST-x outside the expectation, split the expectation across the sum, and leave only expected u-i given bold x inside. That justifies the move from the first conditional-expectation line to the second.
Now SLR point four does the decisive work. Together with random sampling, it gives expected u-i conditional on bold x equals zero for every observation. Replace each conditional error expectation by zero and the entire weighted sum vanishes. What remains is expected beta-one-hat given bold x equals beta one. This is conditional unbiasedness: for any admissible set of x values, repeated realizations of the errors center the slope estimator on the truth.
The last line applies the law of iterated expectations. First take the expected conditional expectation over the possible bold-x samples. Since the inner expectation is always beta one, the unconditional expectation of beta-one-hat is also beta one. That is the desired small-sample result.
The callout tells you what to remember when the proof is no longer in front of you. SLR point four makes the bias term disappear. If zero conditional mean fails, expected u-i given bold x need not be zero, so the weighted error component can survive in expectation. Its sign and size then determine the bias. This decomposition is the foundation for later discussions of omitted-variable bias, endogeneity, and strategies that restore credible identification.
\def\sumn{\sum_{i=1}^{n}} \widehat{\beta}_1 = \beta_1+(1/SST_x)\sumn (x_i-\bar{x})u_i
The estimator is the truth plus a term built entirely out of the errors. Taking the expectation of \widehat{\beta}_1 conditional on \mathbf{x}=\{x_1,\dots,x_n\},
\def\sumn{\sum_{i=1}^{n}} \begin{align*} \Rightarrow E[\widehat{\beta}_1|\mathbf{x}] = & E[\beta_1|\mathbf{x}]+E[(1/SST_x)\sumn (x_i-\bar{x})u_i|\mathbf{x}] \\\\ = & \beta_1 + (1/SST_x)\sumn (x_i-\bar{x}) E[u_i|\mathbf{x}] \end{align*}
So, if condition 4 (E[u_i|\mathbf{x}]=0) is satisfied, that second term vanishes:
\def\Ex{E_{\mathbf{x}}} \begin{align*} E[\widehat{\beta}_1|\mathbf{x}] = & \beta_1 \\\\ \Ex\big[E[\widehat{\beta}_1|\mathbf{x}]\big] = & E[\widehat{\beta}_1] = \beta_1 \end{align*}
Note
Notice which condition did the work: SLR.4 , zero conditional mean. Break that one and the second term does not vanish, and its value is the bias.
So what does the proof require from you as an applied researcher? The three bullets describe connected skills. First, you must judge whether the assumptions are credible in the particular setting. SLR point three can be checked from the observed variation in x, but random sampling and especially zero conditional mean require an argument about who entered the data, how x was determined, and what relevant forces sit in u. A regression command cannot make that judgment for you.
Second, when an assumption fails, you need to understand the consequence and choose a correction when one exists. A lack of x variation may require a different sample or research design. Non-random selection may require redefining the target population or modeling selection. Failure of zero conditional mean may require additional controls, a randomized experiment, fixed effects, instrumental variables, or another design we will study later. Different violations require different responses; there is no universal “fix regression” button.
Third, both tasks depend on substantive knowledge. You need to know the institutions, timing, incentives, measurement process, and behavior that generated the variables. That knowledge tells you which omitted factors are plausible, whether they are related to x, and whether a proposed remedy is believable. This is why good empiricists are not merely good at algebra or software, and why economists specialize in fields. The estimator supplies a conditional guarantee. Your contextual reasoning determines whether its conditions describe the application.
Good empiricists
have ability to judge if the above conditions are satisfied for the particular context you are working on
have ability to correct (if possible) for the problems associated with the violations of any of the above conditions
knows the context well so you can make appropriate judgments
Let’s practice that judgment on the simple house-price regression. The equation says price equals beta zero plus beta one times lot size plus u. Price is measured in dollars and is the dependent variable. Lot size is the explanatory variable. Beta one is the population change in predicted house price associated with one additional unit of lot size under this model. Beta zero is the predicted price at a lot size of zero, which is needed to position the line even though that intercept may have little economic meaning. The error u contains every determinant of price omitted from this one-variable equation.
The first question asks you to make that error concrete. It could include location and school district, floor area, bedrooms and bathrooms, age, physical condition, renovations, garage space, neighborhood amenities, local taxes, and the timing of the sale. It also includes measurement error and idiosyncratic bargaining or seller circumstances. Saying “everything else” is mathematically compact, but an assumption can only be evaluated after you name the important pieces.
The second question is whether the conditional mean of that collection is zero at every lot size. The slide gives the rough correlation version to build intuition, but remember that zero conditional mean is stronger than zero correlation. Large lots may be concentrated in suburbs or wealthier neighborhoods; they may also accompany larger, newer, or higher-quality houses. If those omitted price determinants systematically rise or fall with lot size, expected u given lot size is not zero. Then the simple OLS slope mixes the effect associated with lot size with the effects of those omitted characteristics, and the unbiasedness proof no longer goes through. The regression still produces a number. What fails is the causal or structural interpretation you hoped beta one would have. This is the practical reason we spent time on SLR point four.
Reconsider the following example
price=\beta_0+\beta_1\times lotsize + u
Questions
I want to close this section by being precise about what unbiasedness does not give you. The first bullet says it tells you nothing definitive about the numerical estimate from the one sample in front of you. Unbiasedness is the equation expected beta-one-hat equals beta one across repeated samples. It is not the equation beta-one-hat equals beta one in this sample, and it does not let you recover the realized estimation error beta-one-hat minus beta one because beta one is unknown.
The second bullet follows directly. Even when every assumption holds and the estimator is perfectly unbiased, random sampling can produce an unlucky dataset whose slope estimate lies far from the true slope. The simulation pile showed exactly that: estimates on both sides of two, including some much farther away, all generated by the same unbiased procedure. Our actual sample could correspond to any one draw from such a distribution. We cannot rerun history to see where it sits, and there is no diagnostic that certifies this particular coefficient is close to beta one.
What unbiasedness buys is a guarantee against systematic overestimation or underestimation by the procedure. That is valuable, but incomplete. We also want to know how widely estimates tend to scatter, because a narrow unbiased distribution makes a large miss less likely than a wide unbiased distribution. That leads directly to the variance of the OLS estimator on the next slide.
Unbiasedness property of OLS estimators says nothing about the estimate that we obtain for a given sample
It is always possible that we could obtain an unlucky sample that would give us a point estimate far from \beta_1, and we can never know for sure whether this is the case.
That brings us to the second property and the natural next question. If OLS estimates change from sample to sample, how much do they change? The first bullet calls the estimators random variables because beta-zero-hat and beta-one-hat are functions of random sample data. Before the sample arrives, y, x, and the unobserved disturbances u can take different realized values, so the estimates calculated from them are not yet known. After we observe one sample, each estimate becomes a fixed number, but it is still one realization of a random estimator.
The variance of an estimator measures its repeated-sampling spread around its own expected value. A small variance means different samples tend to produce similar estimates. A large variance means the estimate is sensitive to which observations happened to be drawn. Connect this to the last section: unbiasedness tells us whether the sampling distribution is centered at the true parameter; variance tells us how tightly or loosely the distribution is packed around that center. We prefer an estimator that is both centered on the truth and tightly concentrated, but one property does not imply the other.
The notation in the third bullet, variance of beta-hat OLS, can refer to the variance of either OLS coefficient estimator, beta-zero-hat or beta-one-hat. This lecture will focus on the slope. We will first compare two familiar estimators to see why spread matters, then derive the slope-variance formula and ask what features of the data make that variance larger or smaller.
OLS estimators are random variables because y, x, and u are random variables (this just means that you do not know the estimates until you get samples).
Variance of OLS estimators is a measure of how much spread in estimates (realized values) you will get.
We let Var(\widehat{\beta}_{OLS}) denote the variance of the OLS estimators of \beta_0 and \beta_1.
Here is why variance matters even when two estimators have exactly the same bias. Both displayed formulas estimate the population expected value of x. Theta-smart is one over n times the sum of all x-i from i equals one through n, with n equal to one thousand. It is the ordinary sample mean using every observation. Theta-naive is one tenth times the sum of only x-one through x-ten. It ignores observations eleven through one thousand even though they are available.
Under random sampling, both estimators are unbiased. Each observation has expected value equal to the population mean, so averaging either one thousand observations or ten observations leaves that same expectation. Unbiasedness therefore cannot distinguish them. Their variances can. If each x has variance sigma-x-squared and the observations are independent, theta-smart has variance sigma-x-squared over one thousand, while theta-naive has variance sigma-x-squared over ten. The naive estimator’s variance is one hundred times as large, and its standard deviation is ten times as large.
The figure shows that difference using one thousand simulated samples from a standard normal population. The horizontal axis is the sample-mean estimate. The vertical axis is density, so each filled curve has total area one rather than showing a raw count. Use the legend at the bottom to match each fill to the Smart and Naive estimators. Both distributions are centered near the true mean of zero, confirming that both rules are unbiased. But the Smart curve is a narrow, tall spike near zero, while the Naive curve is a much wider, flatter hill. In this run their simulated standard deviations are about zero point zero three one and zero point three one two, almost exactly the ten-to-one theoretical ratio. Using more independent information makes the estimator much more precise. That is the sense in which one unbiased estimator can be better than another.
Consider two estimators of E[x]:
\begin{align*} \theta_{smart} = & \frac{1}{n} \sum_{i=1}^{n} x_i \;\;(n=1000) \\\\ \theta_{naive} = & \frac{1}{10} \sum_{i=1}^{10} x_i \end{align*}Both are unbiased — the naive one simply throws away 990 of the 1000 observations.
Variance of the estimators
Now look at the true sampling variance of the OLS slope. The statement above the formula gives all of its conditions. We retain SLR point one through SLR point four from the unbiasedness result, and we add that the conditional variance of u given x equals sigma squared. Sigma squared is one constant positive number, so the spread of the error is assumed to be the same at every x. We will name and visualize that assumption in the next section.
Conditional on the sample x values, the variance of beta-one-hat is sigma squared divided by the sum, from i equals one through n, of x-i minus x-bar squared. The denominator is SST-x, the total sample variation in the explanatory variable. The formula is written without the conditioning bar on this slide, but the x values in SST-x are treated as fixed when we derive and use this expression. “True” means the formula contains the population error variance sigma squared, not an estimate of it.
The second display defines the standard error. Here standard error means the standard deviation of the estimator’s sampling distribution. Taking the square root of sigma squared over SST-x gives sigma over the square root of SST-x. Sigma is the standard deviation of the population error, and root SST-x scales it by the amount of identifying variation in x. Variance is measured in squared coefficient units; the standard error is in the same units as beta-one-hat, which makes it easier to compare with the estimated coefficient.
Do not confuse this true standard error with the number software prints. Sigma is unknown because u is unobserved, so the true quantity cannot be calculated directly. Later on this slide deck we replace sigma squared with an estimator and obtain an estimated standard error. That estimated standard error is the key input to confidence intervals and hypothesis tests, so the distinction between a coefficient’s sampling uncertainty and the coefficient itself will matter throughout the course.
(True) Variance of the OLS Estimator
If Var(u|x)=\sigma^2 and the four conditions (we used to prove unbiasedness of the OLS estimator) are satisfied,
\def\sumn{\sum_{i=1}^{n}} \begin{align*} Var(\widehat{\beta}_1) = \frac{\sigma^2}{\sumn (x_i-\bar{x})^2}=\frac{\sigma^2}{SST_x} \end{align*}
(TRUE) Standard Error of the OLS Estimator
The standard error of the OLS estimator is just a square root of the variance of the OLS estimator. We use se(\widehat{\beta}_1) to denote it.
\begin{aligned} se(\widehat{\beta}_1) = \sqrt{Var(\widehat{\beta}_1)} = \frac{\sigma}{\sqrt{SST_x}} \end{aligned}
This slide reads the comparative statics directly from variance of beta-one-hat conditional on x equals sigma squared over SST-x. Start with the numerator. Sigma squared is the conditional variance of u, the collection of influences on y that our model does not explain. Holding the x values fixed, a larger sigma squared makes the slope variance larger in the same proportion. The data cloud is noisier around the population line, so different samples can support more different fitted slopes. A smaller error variance makes the relationship easier to estimate precisely.
Now look at the denominator. SST-x is the sum of squared distances between each x-i and x-bar. When the x values are more spread out, SST-x is larger and the variance of beta-one-hat is smaller. Observations far apart in x give us a clearer comparison of how y changes with x. When all x values cluster tightly together, many different slopes can fit nearly equally well, so the slope estimate is less precise. At the extreme of no variation, SST-x is zero and SLR point three fails.
The experiment bullet turns that result into design advice. If you can assign treatment levels, using well-separated values of x can reveal the response slope more precisely than concentrating every treatment near the same value, subject to scientific, ethical, and practical constraints. Increasing the number of observations also tends to increase SST-x when new observations add genuine x variation. But the last bullet is realistic: with observational data, you usually inherit the support and distribution of x and cannot spread it yourself. Also remember that changing x’s measurement units changes both SST-x and the numerical units of the slope, so a smaller numerical variance after rescaling is not free information. The meaningful lesson is more independent variation in the underlying explanatory variable.
Variance of the OLS estimators
Var(\widehat{\beta}_1|x) = \sigma^2/SST_x
What can you learn from this equation?
the variance of OLS estimators is smaller (larger) if the variance of error term is smaller (larger)
the greater (smaller) the variation in the covariate x, the smaller (larger) the variance of OLS estimators
These two terms describe the conditional variance of the error. Homoskedasticity means variance of u given x equals the same constant sigma squared for every possible x. If you group otherwise comparable observations by x, the unobserved influences on y have the same spread in every group. The “homo” part means same, and “skedasticity” refers to variance or dispersion.
Heteroskedasticity means that conditional spread changes with x. The slide writes variance of u given x equals f of x, meaning some function of x rather than one constant. The variance might rise with x, fall with x, or change in a more complicated pattern. We do not need to know the exact function to say the errors are heteroskedastic.
Keep this separate from SLR point four. Zero conditional mean concerns the center of the error distribution: expected u given x equals zero. Homoskedasticity concerns its spread around that center. Errors can have mean zero at every x while their variance changes with x. They can also have constant variance while their conditional mean is wrong. Homoskedasticity does not require u to be normally distributed, either. Under zero conditional mean, heteroskedasticity does not by itself bias the OLS coefficients, but it changes their variance and invalidates the homoskedastic standard-error formula. Flip to Visualization to see the distinction as a change in the thickness of the data cloud.
Homoskedasticity
The error u has the same variance given any value of the covariate x (Var(u|x)=\sigma^2)
Heteroskedasticity
The variance of the error u differs depending on the value of x (Var(u|x)=f(x))
This figure puts the two error structures into otherwise identical data-generating processes. The horizontal axis is x, taking values from forty through two hundred. The vertical axis is y. In both panels, the systematic population relationship is y equals one hundred plus zero point five times x, shown by the same upward-sloping black line. For every x value, the simulation adds many mean-zero normal errors, so the center of the point cloud remains on that line in both panels.
Look first at the top facet labeled Homoskedastic. The error standard deviation is twenty at every x. As you move from left to right, the vertical band of points stays approximately the same thickness. Some observations lie above the line and some below, but their typical distance from it does not systematically depend on x. That is variance of u given x equals a constant.
Now look at the lower Heteroskedastic facet. Here the error standard deviation increases with x. At x equals forty it is sixteen, while at x equals two hundred it is eighty. The points therefore hug the population line more closely on the left and fan farther above and below it on the right. The conditional mean is still zero, so the fan remains centered on the same line. The horizontal and vertical reference lines at zero simply mark the coordinate origins; the displayed x values are positive, so the vertical origin may lie outside the visible data region.
The visual lesson is about spread, not center. Under the other simple-regression assumptions, heteroskedasticity alone does not systematically move the OLS slope away from beta one. It does mean that observations carry different amounts of noise, that the simple sigma-squared-over-SST-x variance formula is no longer correct, and that OLS no longer has the same Gauss-Markov efficiency claim. The next tab shows the same fan shape in actual house-price data.
The previous tab used simulated points whose variance pattern we chose. This figure uses the HousePrices data, so now the horizontal axis is observed lot size and the vertical axis is house price in dollars. Each point is one house sale. The data contain five hundred forty-six observations, with lot sizes ranging from about one thousand six hundred fifty to sixteen thousand two hundred and prices from twenty-five thousand to one hundred ninety thousand dollars.
The smooth black line is the fitted OLS regression of price on lot size. It summarizes the positive association in the sample: larger lots tend to be attached to more expensive houses. The code suppresses the confidence band, so the line itself, not a shaded interval, is what you see. Do not read that line as a causal effect without revisiting the omitted-variable concerns from the earlier house-price slide.
For heteroskedasticity, focus on vertical spread around the line. On the left, where lots are small, observed prices occupy a relatively narrow vertical band. As lot size increases, the cloud opens up. Houses with large lots include both moderately priced and very expensive properties, so their vertical deviations from the fitted relationship are much more dispersed. That widening fan is evidence that variance of u given lot size is unlikely to be one constant.
The pattern is economically plausible because scale often increases the range of possible outcomes: large-lot properties can differ greatly in floor area, location, quality, and amenities. Similar variance patterns appear with income, firm size, expenditure, and many other economic variables. That is why the red word “widens” and the final sentence matter. Homoskedasticity is a useful benchmark for deriving formulas, but heteroskedasticity is common in real data, so later we will use standard errors designed to remain valid when the conditional variance changes.
The previous tab was manufactured. Here is the same thing in real data: look at how the vertical spread of the points widens as lot size grows. Small lots sell within a narrow band; large lots are all over the place. That is heteroskedasticity, and it is the normal case, not the exception.
The Gauss-Markov theorem gives the formal efficiency result behind the claim that OLS is a good way to use the available information. The first callout lists the assumptions: SLR point one through SLR point four, which gave us unbiasedness, plus SLR point five, homoskedasticity. Under those conditions, the OLS coefficient estimators are BLUE, the best linear unbiased estimators.
Each word limits the conclusion. “Unbiased” means the estimator’s expectation equals the true coefficient. “Linear” here means the estimator can be written as a linear function, a weighted sum, of the observed y values, with weights that may depend on the x values. It does not merely mean that the population regression line is linear in x. “Best” means smallest conditional variance among estimators in that linear-and-unbiased class. For the slope, no other linear unbiased rule has a lower variance than beta-one-hat for the same x values. This is an efficiency comparison, not a statement that the estimate is correct in the realized sample.
Now read the second callout, because BLUE is often repeated without its qualifiers. The theorem does not say OLS beats every imaginable estimator under every loss criterion. An estimator that permits some bias can have smaller variance and potentially smaller mean squared error, which combines squared bias and variance. Nor does the theorem say OLS remains efficient when homoskedasticity fails. With heteroskedastic errors, OLS can remain unbiased under SLR point one through four, but another appropriately weighted linear unbiased estimator may be more efficient. The value of the theorem is precise: under its assumptions, you cannot improve variance within the stated class while retaining linearity and unbiasedness.
Gauss-Markov Theorem
Under conditions SLR.1 through SLR.4 and the homoskedasticity assumption (SLR.5), OLS estimators are the best linear unbiased estimators (BLUEs)
In other words,
No other unbiased linear estimators have smaller variance than the OLS estimators (desirable efficiency property of OLS)
Read the qualifiers
“Best” here means smallest variance, and only within the class of estimators that are both linear in y and unbiased. It does not say OLS beats everything. Drop the unbiasedness requirement and estimators with smaller variance do exist — that is the trade we make in more advanced methods.
This tab separates two consequences that are often conflated. First, we do not need homoskedasticity to prove that OLS is unbiased. Go back to the three-step proof: after rewriting beta-one-hat as beta one plus a weighted sum of errors, zero conditional mean made the expectation of that sum vanish. Nowhere did we require the errors to have equal variance. Therefore heteroskedasticity by itself does not create coefficient bias when SLR point one through four still hold.
Second, homoskedasticity does matter for the variance and standard-error formula used on the previous slide. If variance of u given x changes with x, the true variance of beta-one-hat is not simply sigma squared over SST-x because there is no single conditional sigma squared to place in the numerator. Using that formula anyway generally produces the wrong standard error. Since a significance test compares an estimate with its estimated sampling uncertainty, a wrong standard error can produce misleading t statistics, p-values, and confidence intervals.
That is why the lower bullets say heteroskedasticity has important implications for estimating variance and for significance testing. The point estimate from OLS can still be centered correctly, while our conventional measure of confidence in it is wrong. In most economic applications, constant error variance is not a plausible default, as the house-price fan just illustrated. Later in the course we will derive and use heteroskedasticity-robust standard errors, which change the estimated uncertainty without changing the OLS coefficient itself.
We do NOT need the homoskedasticity condition to prove that OLS estimators are unbiased
In most applications, homoskedasticity condition is not satisfied, which has important implications on:
( A lot more on this issue later )
This is a short tab, but it supplies the motivation for everything that follows. We derived the conditional variance of beta-one-hat as sigma squared over SST-x under homoskedasticity. SST-x can be calculated from observed x values, but sigma squared is the variance of the unobserved population error, so the true variance and true standard error of the slope are unknown. We need sample-based estimates of them.
The line on screen tells you why. Once we can estimate the variance of beta-one-hat, we can quantify how much the slope estimate would vary across samples and test hypotheses about beta one. A common null hypothesis is beta one equals zero, meaning no population linear relationship. We compare beta-one-hat’s distance from that null value with its estimated standard error. The resulting t statistic and its reference distribution produce a p-value, and the same estimated uncertainty produces confidence intervals. Formal testing comes later, but the plumbing begins here: estimate sigma squared, insert it into the slope-variance formula, and take a square root to obtain the estimated standard error printed beside a regression coefficient.
Once you estimate Var(\widehat{\beta}_1|x), you can test the statistical significance of \widehat{\beta}_1 (More on this later)
The problem is laid out one step at a time. First, under homoskedasticity, variance of beta-one-hat conditional on x equals sigma squared over SST-x. We observe every x-i, so we can calculate x-bar, each squared deviation, and SST-x. If sigma squared were known, the slope variance would therefore be known too.
Second, sigma squared is variance of u-i. By definition, variance of u-i equals expected u-i squared minus expected u-i, all squared. Zero conditional mean implies zero unconditional mean, so the second part is zero and sigma squared equals expected u-i squared. If we could observe the errors, one over n times the sum of u-i squared would average n identically distributed quantities whose expectation is sigma squared. Under the stated sampling and homoskedasticity assumptions, it would be an unbiased estimator of the error variance.
The obstacle in the fourth bullet is that an error is not observed. U-i is the difference between y-i and the true population regression line, and the true beta zero and beta one are unknown. What we do observe after fitting OLS is the residual u-hat-i, the difference between y-i and the fitted sample line. Open the first fragment and the callout asks whether residuals can stand in for errors.
The proposal uses the result referenced in blue and proved on the final tab: conditional on the x values, the expected difference between u-hat-i and u-i is zero under OLS unbiasedness. That makes substitution sound tempting. The next fragment proposes one over n times the sum of squared residuals as an estimator of sigma squared.
But the final fragment gives the catch. An expected difference of zero in levels does not imply equal expected squares. Residuals are created by fitting the same sample, and OLS chooses the intercept and slope specifically to make their squared sum as small as possible. They are therefore constrained and, on average, have too little squared magnitude to treat all n of them as free errors. Dividing their squared sum by n biases the estimate of sigma squared downward. The next tab identifies the two constraints, and the following tab supplies the exact correction.
The problem
We know that Var(\widehat{\beta}_1|x) = \sigma^2/SST_x.
You can calculate SST_x because x is observable. So, as long as we know \sigma^2, which is Var(u) (the variance of the error term), then we know Var(\widehat{\beta}_1|x).
Since Var(u_i)=\sigma^2=E[u_i^2] \;\; \Big( Var(u_i)\equiv E[u_i^2]-E[u_i]^2 \Big), \frac{1}{n}\sum_{i=1}^n u_i^2 is an unbiased estimator of Var(u_i)
Unfortunately, we don’t observe u_i (error)
But,
We observe \widehat{u_i} (residuals)!! Can we use residuals instead?
The proposal
We know E[\widehat{u}_i-u_i]=0 (the proof is in the (Math) Residual vs error tab at the end of this slide), so, why don’t we use \widehat{u}_i (observable) in place of u_i (unobservable)?
Proposed Estimator of \sigma^2
\frac{1}{n}\sum_{i=1}^n \widehat{u}_i^2
Unfortunately, \frac{1}{n}\sum_{i=1}^n \hat{u}_i^2 is a biased estimator of \sigma^2
The reason for the correction comes from the first-order conditions of the least-squares minimization problem. OLS chooses beta-zero-hat and beta-one-hat to minimize the sum of squared residuals. Differentiating that objective with respect to the intercept gives the first equation: the sum of u-hat-i from i equals one through n is zero. Differentiating with respect to the slope gives the second: the sum of x-i times u-hat-i is zero. These are algebraic properties of the fitted OLS residuals whenever the model includes an intercept.
Those two equations are two linear restrictions on the n residual values. Subject to variation in x, if you know n minus two suitably chosen residuals, the two restrictions determine the remaining two. So the residuals cannot vary as n unconstrained pieces of information. Their sum must be zero, and their x-weighted sum must also be zero.
The degrees-of-freedom language summarizes this bookkeeping. We began with n outcome observations, then used the sample to estimate two regression parameters, the intercept and the slope. Each estimated parameter consumes one degree of freedom, leaving n minus two residual degrees of freedom. That is why treating the residual sum of squares as though it contained n free error realizations makes its average too small. In a regression with k estimated coefficients including the intercept, the same logic leaves n minus k residual degrees of freedom. Here k equals two, so the correct denominator will be n minus two.
FOCs of the minimization problem OLS solves
\begin{align*} \sum_{i=1}^n \widehat{u}_i=0\;\; \text{and}\;\; \sum_{i=1}^n x_i\widehat{u}_i=0\notag \end{align*}The callout gives the corrected estimator of the error variance. Sigma-hat squared equals one over n minus two times the sum, from i equals one through n, of u-hat-i squared. The numerator is the residual sum of squares. The denominator is the residual degrees of freedom after estimating beta zero and beta one. Under SLR point one through five, the expected residual sum of squares conditional on x is n minus two times sigma squared. Dividing by n minus two therefore gives expected sigma-hat squared equal to sigma squared, which is the unbiasedness statement shown in parentheses.
The next display inserts that estimate into the true slope-variance formula. The estimated variance of beta-one-hat, indicated by the outer hat, equals sigma-hat squared over SST-x. Keep the levels of uncertainty distinct: beta-one-hat estimates the population slope beta one, while the hatted variance estimates how beta-one-hat would vary across repeated samples.
The following display moves from variance to standard error. The true standard error is sigma over root SST-x. Because sigma is unknown, replace it with the square root of sigma-hat squared. Thus the estimated standard error of beta-one-hat equals root sigma-hat squared divided by root SST-x. This is an estimate of the standard deviation of an estimator, and it is expressed in the same units as the slope coefficient.
The last callout previews its use. Testing will compare beta-one-hat with a hypothesized beta one after scaling their difference by this estimated standard error. Every formula on this tab uses the homoskedastic variance assumption. When that assumption fails, the n-minus-two estimate still summarizes an average residual variance, but sigma-hat squared over SST-x is not the generally valid sampling variance of the slope. That is why robust standard errors become important later.
Unbiased estimator of \sigma^2
\widehat{\sigma}^2=\frac{1}{n-2}\sum_{i=1}^n \widehat{u}_i^2 \;\;\;\;\;\;(E[\frac{1}{n-2}\sum_{i=1}^n \widehat{u}_i^2]=\sigma^2)
Hereafter we use \widehat{Var(\widehat{\beta}_1)} to denote the estimated variance of the OLS estimator \widehat{\beta}_1, and it is defined as
\widehat{Var(\widehat{\beta}_1)} = \widehat{\sigma}^2/SST_x
Since se(\widehat{\beta}_1)=\sigma/\sqrt{SST_x}, the natural estimator of se(\widehat{\beta_1}) ( standard error of \widehat{\beta}_1 ) is
\widehat{se(\widehat{\beta}_1)} =\sqrt{\widehat{\sigma}^2}/\sqrt{SST_x},
Note
Later, we use \widehat{se(\hat{\beta_1})} for testing.
Here is how the calculation appears in R. The commented line at the top gives the generic base-R pattern: “ell-em, open parenthesis, dependent variable tilde independent variable, comma data equals data-name.” The tilde separates the outcome on the left from the explanatory variables on the right, and the data argument tells R which data frame contains those variable names.
The live code uses the fixest package explicitly. Read the first call as “fix-est colon colon fee-ols.” The double-colon notation selects the feols function from fixest without attaching the whole package. Inside the function, “price tilde lot-size” requests an OLS regression with price as the dependent variable, lot size as the single explanatory variable, and an intercept included by default. “Data equals HousePrices” tells the function to find both columns in the HousePrices dataset. The fitted model object, including coefficients, residuals, and the information needed for inference, is stored as uni-reg.
The next line calls “summary of uni-reg,” which prints the regression table. In the executed output there are five hundred forty-six observations. The lot-size coefficient estimate is about six point five nine nine, and its IID standard error is about zero point four four six. “IID” tells you this default calculation uses the independent, identically distributed, homoskedastic formula developed on these slides. The intercept estimate and its standard error appear on the row above.
Look across the lot-size row. The Estimate column is beta-one-hat. The Std. Error column is the estimated standard error we just derived. The t value, about fourteen point eight, is the estimate divided by its standard error for the null hypothesis beta one equals zero. The very small p-value in the next column is based on that statistic. We will explain that test formally later. For now, recognize that the standard-error column condenses the residual sum of squares, the n-minus-two correction, and the variation in lot size into one measure of sampling uncertainty. Also remember the warning from the previous slide: the house-price plot looked heteroskedastic, so an IID standard error is a teaching benchmark, not the inferential choice we will ultimately prefer.
This final tab proves the statement used on the Proposal tab, and it also clarifies the difference between an error and a residual. The first pair of equations writes the same observed y-i in two ways. The population equation is y-i equals beta zero plus beta one x-i plus u-i. Beta zero and beta one are the true, unknown parameters, and u-i is the unobserved population error. The fitted-sample equation is y-i equals beta-zero-hat plus beta-one-hat x-i plus u-hat-i. The hats mark quantities calculated from the sample, and u-hat-i is the observable residual once the regression is fitted.
Now follow the algebra in the second display. The first line solves the fitted equation for the residual: u-hat-i equals y-i minus beta-zero-hat minus beta-one-hat x-i. The second line substitutes the population expression beta zero plus beta one x-i plus u-i for y-i. Subtract u-i from both sides and group like terms. That gives u-hat-i minus u-i equals beta zero minus beta-zero-hat plus beta one minus beta-one-hat, all times x-i for the slope part. In words, the difference between residual and error comes entirely from using the estimated line instead of the unknown population line.
The last line takes expectations conditional on bold x, the complete set of sample x values. Conditional on bold x, each x-i is fixed. Under the assumptions used earlier, both OLS coefficient estimators are conditionally unbiased: expected beta-zero-hat given bold x equals beta zero and expected beta-one-hat given bold x equals beta one. Therefore each coefficient-estimation difference has conditional expectation zero, and expected u-hat-i minus u-i given bold x equals zero.
This is a statement about the difference in levels, not equality observation by observation. A residual generally is not the corresponding error because the fitted coefficients generally differ from the true coefficients. It is also not a statement that their squares have equal expectations. OLS fitting imposes two restrictions and shrinks the residual sum of squares relative to the unobserved error sum of squares. That is exactly why replacing u-i with u-hat-i and dividing by n was biased, and why the n-minus-two degrees-of-freedom correction on the Unbiased estimator tab was necessary.
Error and Residual
\begin{align*} y_i = \beta_0+\beta_1 x_i + u_i \\ y_i = \hat{\beta}_0+\hat{\beta}_1 x_i + \hat{u}_i \end{align*}Residuals as unbiased estimators of error
\begin{align*} \hat{u}_i & = y_i -\hat{\beta}_0-\hat{\beta}_1 x_i \\ \hat{u}_i & = \beta_0+\beta_1 x_i + u_i -\hat{\beta}_0-\hat{\beta}_1 x_i \\ \Rightarrow \hat{u}_i -u_i & = (\beta_0-\hat{\beta}_0)+(\beta_1-\hat{\beta}_1) x_i \\ \Rightarrow E[\hat{u}_i-u_i|\mathbf{x}] & = E[(\beta_0-\hat{\beta}_0)+(\beta_1-\hat{\beta}_1)x_i|\mathbf{x}]=0 \end{align*}