Glossary Math & Statistics

Linear regression

Linear regression fits the straight line that best predicts one variable from another by minimising the squared vertical distances to the points.

Last reviewed:

AdSense Placeholder
Slot: header_reference_page

In more detail

The slope is Σ(x − x̄)(y − ȳ) / Σ(x − x̄)² and the intercept is ȳ − slope × x̄. "Least squares" means the line minimises the sum of the squared residuals, the vertical gaps between each point and the line. The correlation coefficient r (from −1 to 1) measures how closely the points follow a straight line, and R² = r² is the share of the variation in y that the line accounts for.

Why it matters

Regression is used to predict and to estimate trends, but it fits a straight line to anything, curved or not, so plot the data first. Predictions outside the range of the observed x values (extrapolation) are guesses, and a single outlier can drag the line. A strong fit does not show that one variable causes the other.

Example

Points (1, 2), (2, 4), (3, 5), (4, 4), (5, 5):

x̄ = 3, ȳ = 4
Σ(x − x̄)(y − ȳ) = 6     Σ(x − x̄)² = 10
slope = 6 / 10 = 0.6    intercept = 4 − 0.6 × 3 = 2.2
line: y = 0.6x + 2.2
r = 0.775, R² = 0.60    residual sum of squares = 2.4

At x = 6 the line predicts 5.8, a modest step outside the data; at x = 10 it predicts 8.2, which nothing in the data supports.

Common confusion

R² is not a p-value: a high R² from five points can still be chance, and a tiny R² can be highly significant in a huge sample.

AdSense Placeholder
Slot: footer_leaderboard