Bivariate Data

⚡ Key Takeaways

Bivariate data consists of paired observations of two variables measured on the same individuals, used to investigate whether and how the two variables are related. Key result in words: the strength and direction of a linear relationship between two numerical variables is captured by the correlation coefficient $r$ , and the relationship can be modelled by a least-squares regression line. Applies wherever you want to study association — height vs. weight, study time vs. exam score, temperature vs. ice-cream sales. Developed mainly by Francis Galton and Karl Pearson in the late 19th century.

💡 The Core Insight

A single pair of numbers tells you nothing about a relationship; a cloud of pairs does. By plotting all pairs as points (a scatterplot) you can see a relationship, and by computing how the two variables co-vary about their means you can measure it. The central quantity is covariance — the average of the products of paired deviations from the means — which, once you divide out the two standard deviations, becomes the dimensionless, scale-free correlation coefficient between $-1$ and $+1$ .

DefinitionBivariate Data and Its Building Blocks

A bivariate data set of size $n$ is a list of ordered pairs

1

\quad \{(x_1,y_1),(x_2,y_2),\dots,(x_n,y_n)\}.

Derivation hint: this is a definition — each individual contributes one $x$ -value and one $y$ -value measured simultaneously. We summarise each variable by its mean and standard deviation:

2

\quad \bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i,\qquad \bar{y}=\frac{1}{n}\sum_{i=1}^{n}y_i.

Derivation hint: ordinary arithmetic averages applied separately to the $x$ -list and the $y$ -list. The sample covariance measures how the two vary together:

3

\quad s_{xy}=\frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y}).

Derivation hint: average the product of each point's $x$ -deviation and $y$ -deviation; positive products (both above or both below their means) push covariance up. The (Pearson) correlation coefficient is the standardised covariance:

4

\quad r=\frac{s_{xy}}{s_x\,s_y}=\frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sqrt{\sum_{i=1}^{n}(x_i-\bar{x})^2}\,\sqrt{\sum_{i=1}^{n}(y_i-\bar{y})^2}}.

Derivation hint: divide covariance by the product of the two standard deviations; the $(n-1)$ factors cancel between numerator and denominator, giving a unitless number. Here $s_x,s_y$ are the standard deviations of $x$ and $y$ , and the symbols are: $n$ = number of pairs, $x_i,y_i$ = the values of the two variables for individual $i$ , $\bar x,\bar y$ = means, $s_{xy}$ = covariance, $r$ = correlation coefficient.

TheoremProperties of the Correlation Coefficient

For any bivariate data set:

5

\quad -1\le r\le +1.

Derivation hint: follows from the Cauchy–Schwarz inequality applied to the deviation vectors $u_i=x_i-\bar x$ and $v_i=y_i-\bar y$ . Furthermore $r=+1$ (resp. $r=-1$ ) exactly when the points lie on a straight line of positive (resp. negative) slope, and $r$ is unchanged by any linear rescaling $x\mapsto a x+b,\ y\mapsto c y+d$ with $a,c>0$ .

TheoremLeast-Squares Regression Line

The line $\hat y = a + bx$ minimising the sum of squared vertical residuals $\sum(y_i-\hat y_i)^2$ has

6

\quad b=\frac{s_{xy}}{s_x^2}=r\,\frac{s_y}{s_x},\qquad a=\bar{y}-b\,\bar{x}.

Derivation hint: set the partial derivatives of $\sum(y_i-a-bx_i)^2$ with respect to $a$ and $b$ to zero and solve the resulting two "normal equations." The proportion of variation in $y$ explained by the line is the coefficient of determination $r^2$ .

ProofDeriving the Least-Squares Slope and Intercept

We minimise the total squared error as a function of intercept $a$ and slope $b$ : 1. Define $S(a,b)=\sum_{i=1}^{n}\bigl(y_i-a-bx_i\bigr)^2$ . 2. Differentiate with respect to $a$ and set to zero:

7

\quad \frac{\partial S}{\partial a}=-2\sum_{i=1}^{n}(y_i-a-bx_i)=0.

Derivation hint: chain rule on each squared term; the constant $-2$ factors out. 3. Dividing by $-2$ and rearranging gives $\sum y_i = na + b\sum x_i$ , i.e. $\bar y = a + b\bar x$ , so $a=\bar y - b\bar x$ . The line passes through $(\bar x,\bar y)$ . 4. Differentiate with respect to $b$ and set to zero:

8

\quad \frac{\partial S}{\partial b}=-2\sum_{i=1}^{n}x_i(y_i-a-bx_i)=0.

Derivation hint: same chain rule, but the inner derivative with respect to $b$ brings down a factor $x_i$ . 5. Substitute $a=\bar y - b\bar x$ into step 4. After expanding and cancelling, the cross terms collapse to the covariance and variance:

9

\quad b=\frac{\sum (x_i-\bar x)(y_i-\bar y)}{\sum (x_i-\bar x)^2}=\frac{s_{xy}}{s_x^2}.

Derivation hint: use $\sum x_i(y_i-\bar y)=\sum(x_i-\bar x)(y_i-\bar y)$ , valid because $\sum(y_i-\bar y)=0$ . 6. Since $s_{xy}=r\,s_x s_y$ , we get $b=r\,s_y/s_x$ , completing equation [6]. $\blacksquare$

∎

mathematician · Karl Pearson | 1857–1936

Pearson, a polymath at University College London, transformed the loose Victorian study of "biometry" into rigorous mathematical statistics. Building on Francis Galton's idea of regression toward mediocrity, in 1896 he gave the product-moment correlation coefficient its modern algebraic form — equation [4] still bears the name Pearson's $r$ . He founded the world's first university statistics department and the journal Biometrika. His work, though scientifically foundational, was entangled with the eugenics movement of his era, a connection modern readers should view critically.

🧠 Intuition

Picture each data point's contribution to the covariance sum [3]. Split the scatterplot into four quadrants by drawing vertical and horizontal lines through the point of means $(\bar x,\bar y)$ . A point in the upper-right quadrant has both deviations positive, so its product is positive; a point in the lower-left has two negatives, also a positive product. Points in those two "agreeing" quadrants vote for a positive relationship. Points in the upper-left and lower-right quadrants have one positive and one negative deviation, contributing negative products and voting for a negative relationship. The covariance is the net tally of these votes. Dividing by $s_x s_y$ just converts the tally to a fixed scale from $-1$ to $+1$ , so a tall thin person and a short fat person produce the same $r$ as the same shapes measured in inches versus centimetres.

Figure

Scatterplot with least-squares line and the four deviation quadrants about the point of means

ExampleCorrelation between study hours and test score

Given five students' weekly study hours $x$ and test score $y$ :

$x$	2	3	5	7	8
$y$	50	60	70	80	90

Step 1 — means. $\bar x=(2+3+5+7+8)/5=5$ , $\bar y=(50+60+70+80+90)/5=70$ .

Step 2 — deviations and products.

$x_i-\bar x$	$y_i-\bar y$	product	$(x_i-\bar x)^2$	$(y_i-\bar y)^2$
$-3$	$-20$	$60$	$9$	$400$
$-2$	$-10$	$20$	$4$	$100$
$0$	$0$	$0$	$0$	$0$
$2$	$10$	$20$	$4$	$100$
$3$	$20$	$60$	$9$	$400$

Sums: $\sum(x-\bar x)(y-\bar y)=160$ , $\sum(x-\bar x)^2=26$ , $\sum(y-\bar y)^2=1000$ .

Step 3 — correlation (equation [4]).

r=\frac{160}{\sqrt{26}\,\sqrt{1000}}=\frac{160}{161.2}=0.992.

Step 4 — regression slope and intercept (equation [6]).

b=\frac{160}{26}=6.15,\qquad a=70-6.15(5)=39.2.

\boxed{r=0.99\ \text{(strong positive)},\qquad \hat y = 39.2 + 6.15\,x\ \text{(score per study-hour)}}

ExamplePredicting with the regression line

Given the line $\hat y = 39.2 + 6.15x$ from the previous example, predict the score of a student who studies 6 hours, and report how much of the variation in scores the model explains.

Step 1 — substitute $x=6$ .

\hat y = 39.2 + 6.15(6) = 39.2 + 36.9 = 76.1.

Step 2 — coefficient of determination. $r^2 = (0.992)^2 = 0.984$ .

Interpretation. About 98.4 % of the variation in test scores is accounted for by study hours under this linear model; the predicted score for 6 hours is roughly 76 points. Note $x=6$ lies inside the data range $[2,8]$ , so this is legitimate interpolation, not risky extrapolation.

\boxed{\hat y \approx 76\ \text{points},\qquad r^2 = 0.98}

⚠️ Correlation is not causation

A strong $r$ tells you two variables move together; it does not tell you that one causes the other. The relationship may be reversed, coincidental, or driven by a hidden confounding variable. Classic example: across towns, the number of churches correlates strongly with the number of bars — but neither causes the other; both are driven by population size. Only a controlled experiment or careful causal reasoning can establish causation.

⚠️ Do not extrapolate beyond the data

The regression line is only validated within the range of observed $x$ -values. Plugging in an $x$ far outside that range (extrapolation) can give absurd answers — e.g. extending a child's height-vs-age line to age 40 predicts a 5-metre adult. Always check that your prediction's $x$ lies inside the data spread.

CounterexampleZero correlation despite a perfect relationship

Take the symmetric data $(-2,4),(-1,1),(0,0),(1,1),(2,4)$ , which lies exactly on the parabola $y=x^2$ . Here $\bar x=0$ and the products $(x_i-\bar x)(y_i-\bar y)$ are $(-2)(1.8),(-1)(-1.2),(0)(-2.2),(1)(-1.2),(2)(1.8)$ , which sum to $-3.6-(-1.2)+0+(-1.2)+3.6 = 0$ . Therefore $r=0$ . The variables are perfectly (deterministically) related, yet Pearson's $r$ reports "no linear relationship" — proving $r$ measures only the linear component of association. Always look at the scatterplot, never trust $r$ alone.

connection

Bivariate analysis is the bridge between Univariate Data (single-variable summaries like mean and standard deviation, which feed directly into equations [2]–[4]) and Multivariate Statistics (three or more variables, partial correlation, multiple regression). The correlation coefficient is the sample estimate of the theoretical correlation studied in Probability Theory via the bivariate normal distribution. The least-squares method is one instance of Optimization / Calculus (setting derivatives to zero), and the algebra of deviation vectors links to Inner Product Spaces, where $r$ is literally the cosine of the angle between the centred data vectors.

🔬 Where this appears

⏳ History

- 1846: Auguste Bravais writes mathematical formulae for the joint distribution of errors, anticipating the correlation concept without naming it.

- 1877: Francis Galton, studying sweet-pea seeds, observes that offspring sizes "regress toward the mean" — coining regression.

- 1885: Galton presents the first regression line and correlation table for parent–child stature data, introducing the symbol

r

for "reversion/regression."

- 1888: Galton publishes Co-relations and their Measurement, defining correlation as a measurable quantity.

- 1896: Karl Pearson derives the modern product-moment correlation coefficient (equation [4]) and least-squares regression in rigorous algebraic form.

- 1900: Pearson introduces the chi-square test, extending association methods to categorical bivariate data.

- 1904: Charles Spearman invents the rank correlation coefficient for ordinal/monotonic data, sidestepping the linearity assumption.

- 1908–1925: William Gosset ("Student") and R. A. Fisher develop the sampling distribution of

r

and tests of its significance, founding inferential regression.

- 1930s onward: Fisher's Statistical Methods for Research Workers standardises regression and correlation across the experimental sciences.

open-problem · Beyond linear, beyond Pearson

Pearson's $r$ captures only linear association and is sensitive to outliers and to the assumption of an underlying bivariate-normal distribution. Several deeper questions remain practically important: (i) How best to quantify general dependence — modern measures such as distance correlation (Székely, 2007) and maximal information coefficient (MIC, 2011) aim to detect any relationship, but no single measure is universally agreed upon. (ii) The problem of distinguishing genuine causation from correlation in observational data is a live research frontier (Pearl's causal calculus, instrumental variables, randomised trials). (iii) When data are heavy-tailed or contaminated, robust regression methods (least absolute deviations, Theil–Sen estimator) are preferred, and choosing among them remains context-dependent. At the introductory level the cautionary lesson is simpler: always plot the data, beware outliers, and never confuse a tidy $r$ with a true model of reality.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also