Bivariate Quantitative Data

⚡ Key Takeaways

Bivariate quantitative data consists of paired numerical measurements $(x_i, y_i)$ on the same subjects, analyzed to detect and quantify relationships between two variables. The key result: a scatterplot reveals the form, direction, and strength of association, and the correlation coefficient $r$ summarizes linear association while the least-squares regression line predicts one variable from the other. Used everywhere two measurements covary — economics, biology, medicine, engineering. Foundations laid by Galton and Pearson (1880s–1900s).

💡 The Core Insight

When you measure two numbers on each subject, plot them as points. The pattern of the cloud of points — its shape (form), its tilt (direction), and its tightness (strength) — encodes the relationship. A straight-line trend is the simplest and most important pattern, and we can compress it into two numbers: a correlation that says how tightly the points hug a line, and a slope/intercept pair that says which line. Both come from a single quantity: how $x$ and $y$ vary together.

DefinitionBivariate Quantitative Data

A bivariate quantitative dataset is a set of $n$ ordered pairs

1

\quad \{(x_1,y_1),(x_2,y_2),\dots,(x_n,y_n)\}

Derivation hint: This is a definition, not a derivation — each pair records two numerical measurements on the same unit $i$ . where $x_i$ and $y_i$ are real-valued (quantitative) measurements on the same unit $i$ . The variable plotted horizontally ( $x$ ) is the explanatory (independent) variable; the variable plotted vertically ( $y$ ) is the response (dependent) variable. Both must be quantitative — measured on a numerical scale where arithmetic is meaningful.

DefinitionCovariance

The sample covariance measures how the two variables vary together:

2

\quad s_{xy} = \frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})

Derivation hint: Average the product of paired deviations from each mean; positive when both deviate in the same direction, negative when opposite. where $\bar{x}=\frac{1}{n}\sum x_i$ and $\bar{y}=\frac{1}{n}\sum y_i$ are the sample means. Covariance has units of $x$ times $y$ , which makes it hard to interpret directly.

DefinitionPearson Correlation Coefficient

The correlation coefficient $r$ standardizes covariance by the two standard deviations, producing a unitless number:

3

\quad r = \frac{s_{xy}}{s_x\,s_y} = \frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sqrt{\sum_{i=1}^{n}(x_i-\bar{x})^2}\,\sqrt{\sum_{i=1}^{n}(y_i-\bar{y})^2}}

Derivation hint: Divide covariance [2] by $s_x s_y$ ; the $(n-1)$ factors cancel, leaving the ratio of summed cross-products to the geometric mean of summed squares. It can be written compactly using standardized scores ( $z$ -scores):

4

\quad r = \frac{1}{n-1}\sum_{i=1}^{n}\left(\frac{x_i-\bar{x}}{s_x}\right)\left(\frac{y_i-\bar{y}}{s_y}\right)

Derivation hint: Substitute $z_{x_i}=(x_i-\bar x)/s_x$ and $z_{y_i}=(y_i-\bar y)/s_y$ into [3]; $r$ is the average product of standardized scores.

TheoremProperties of the Correlation Coefficient

For any bivariate quantitative dataset: 1. $-1 \le r \le 1$ always. 2. $r=+1$ exactly when all points lie on a line of positive slope; $r=-1$ when all lie on a line of negative slope. 3. $r$ is unitless and unchanged by changes of scale or origin (e.g. converting cm to inches). 4. $r$ is symmetric: the correlation of $x$ with $y$ equals that of $y$ with $x$ . 5. $r$ measures only linear association — it can be near $0$ for strong nonlinear relationships.

DefinitionLeast-Squares Regression Line

The least-squares regression line $\hat{y}=a+bx$ is the line minimizing the sum of squared vertical residuals. Its slope and intercept are

5

\quad b = \frac{s_{xy}}{s_x^2} = r\,\frac{s_y}{s_x}, \qquad a = \bar{y} - b\,\bar{x}

Derivation hint: Minimize $\sum(y_i - a - bx_i)^2$ by setting partial derivatives w.r.t. $a$ and $b$ to zero; solving the two normal equations gives these closed forms. The predicted value $\hat{y}_i = a + b x_i$ is the fitted value; the residual is $e_i = y_i - \hat{y}_i$ .

TheoremCoefficient of Determination

The fraction of variation in $y$ explained by the linear regression on $x$ equals $r^2$ :

6

\quad r^2 = 1 - \frac{\sum_{i=1}^{n}(y_i-\hat{y}_i)^2}{\sum_{i=1}^{n}(y_i-\bar{y})^2} = \frac{\text{explained variation}}{\text{total variation}}

Derivation hint: Decompose total sum of squares $\sum(y_i-\bar y)^2$ into explained $\sum(\hat y_i - \bar y)^2$ plus residual $\sum(y_i-\hat y_i)^2$ ; the explained fraction simplifies to $r^2$ .

ProofDeriving the Least-Squares Slope and Intercept

We seek $a,b$ minimizing the total squared vertical error

7

\quad S(a,b) = \sum_{i=1}^{n}\left(y_i - a - b x_i\right)^2

Derivation hint: This is the objective function — squared residuals summed over all points.

Step 1 — Intercept condition. Differentiate with respect to $a$ and set to zero:

8

\quad \frac{\partial S}{\partial a} = -2\sum_{i=1}^{n}(y_i - a - b x_i) = 0 \;\Rightarrow\; \sum y_i = na + b\sum x_i

Derivation hint: Chain rule on each squared term; the sum of residuals must be zero. Divide by $n$ to get $\bar y = a + b\bar x$ . This already gives $a = \bar{y} - b\bar{x}$ , so the line passes through the point of averages $(\bar{x},\bar{y})$ .

Step 2 — Slope condition. Differentiate with respect to $b$ and set to zero:

9

\quad \frac{\partial S}{\partial b} = -2\sum_{i=1}^{n}x_i(y_i - a - b x_i) = 0 \;\Rightarrow\; \sum x_i y_i = a\sum x_i + b\sum x_i^2

Derivation hint: The residuals must be uncorrelated with the $x$ -values; this is the second normal equation.

Step 3 — Solve. Substitute $a=\bar y - b\bar x$ into Step 2 and simplify using $\sum(x_i-\bar x)(y_i-\bar y)$ and $\sum(x_i-\bar x)^2$ :

10

\quad b = \frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2} = \frac{s_{xy}}{s_x^2}

Derivation hint: Algebraic rearrangement; the centered sums emerge once $\bar x,\bar y$ are subtracted. Dividing numerator and denominator by $(n-1)$ gives the covariance/variance form. Finally, since $s_{xy}=r\,s_x s_y$ , we get $b = r\,s_x s_y / s_x^2 = r\,s_y/s_x$ , completing equation [5]. $\blacksquare$

∎

mathematician · Francis Galton | 1822–1911

The English polymath and cousin of Charles Darwin, Galton invented the concepts of correlation and regression while studying heredity. Plotting parents' heights against children's heights, he noticed that tall parents tended to have tall — but on average less tall — children, a phenomenon he named "regression toward mediocrity" (now regression toward the mean). His 1886 paper introduced the regression line, and his student Karl Pearson formalized the correlation coefficient $r$ — still called the Pearson correlation — in the 1890s. Galton's statistical insights were genuine and lasting, even as his views on eugenics were later, rightly, discredited.

🧠 Intuition

Think of correlation as a measure of how much knowing $x$ helps you guess $y$ . If $r=0$ , knowing a person's $x$ tells you nothing about their $y$ — your best guess is always $\bar y$ . If $r=\pm 1$ , knowing $x$ tells you $y$ exactly. In between, $r^2$ tells you what fraction of the "spread" in $y$ you can account for: $r=0.8$ means $r^2=0.64$ , so $64\%$ of $y$ 's variation is explained by the line. The slope formula $b = r\,(s_y/s_x)$ also explains regression to the mean: when $x$ is $1$ standard deviation above average, the predicted $y$ is only $r$ standard deviations above average — and since $|r|<1$ in real data, $y$ is pulled back toward its mean. Outstanding parents tend to have merely good children; record-breaking seasons tend to be followed by ordinary ones. This is not a force of nature; it is pure arithmetic.

Figure

Scatterplot with least-squares regression line and residuals

```svg

```

ExampleComputing r and the Regression Line by Hand

Given: Five students' hours studied ( $x$ ) and exam scores ( $y$ ):

$x$	1	2	3	4	5
$y$	52	60	67	73	78

Step 1 — Means. $\bar x = 3$ , $\bar y = (52+60+67+73+78)/5 = 330/5 = 66$ .

Step 2 — Deviations and products.

$x_i-\bar x$	$y_i-\bar y$	product	$(x_i-\bar x)^2$	$(y_i-\bar y)^2$
−2	−14	28	4	196
−1	−6	6	1	36
0	1	0	0	1
1	7	7	1	49
2	12	24	4	144

Sums: $\sum(x-\bar x)(y-\bar y)=65$ , $\sum(x-\bar x)^2=10$ , $\sum(y-\bar y)^2=426$ .

Step 3 — Correlation [3].

r = \frac{65}{\sqrt{10}\cdot\sqrt{426}} = \frac{65}{\sqrt{4260}} = \frac{65}{65.27} = 0.9959

Step 4 — Slope and intercept [5].

b = \frac{65}{10} = 6.5, \qquad a = 66 - 6.5(3) = 66 - 19.5 = 46.5

Step 5 — Coefficient of determination [6]. $r^2 = 0.9959^2 = 0.992$ .

\boxed{\hat y = 46.5 + 6.5x,\quad r=0.996,\quad r^2=0.992\ (99.2\%\ \text{of score variation explained})}

Each extra hour of study predicts $6.5$ more points.

ExamplePrediction and Residual

Given: Using the line $\hat y = 46.5 + 6.5x$ from the previous example, predict the score for a student who studies $x=3$ hours, and find the residual since the actual score was $67$ .

Step 1 — Fitted value.

\hat y = 46.5 + 6.5(3) = 46.5 + 19.5 = 66\text{ points}

Step 2 — Residual.

e = y - \hat y = 67 - 66 = +1\text{ point}

Step 3 — Interpretation. The student scored $1$ point above the line's prediction; the positive residual means they outperformed the model. Predicting for $x=10$ hours would be extrapolation — outside the observed range $1$ – $5$ — and is unreliable.

\boxed{\hat y = 66\text{ points},\quad e = +1\text{ point}}

⚠️ Correlation Is Not Causation

A high $|r|$ means two variables move together — it does not mean one causes the other. Ice cream sales and drowning deaths are strongly correlated, but neither causes the other; both are driven by a lurking variable, hot weather. Establishing causation requires a controlled experiment or careful confounding adjustment, never correlation alone.

⚠️ r Only Measures Linear Association

A correlation near zero does not mean "no relationship." It means no linear relationship. Data tracing a perfect parabola $y=x^2$ symmetric about $x=0$ has $r=0$ exactly, yet $y$ is perfectly determined by $x$ . Always plot the data first.

CounterexampleAnscombe's Quartet

In 1973 statistician Francis Anscombe constructed four datasets, each with $11$ points, that share identical summary statistics: $\bar x = 9$ , $\bar y = 7.5$ , $r = 0.816$ , regression line $\hat y = 3 + 0.5x$ , and $r^2 = 0.67$ . Yet plotted, they are utterly different: one is a clean linear trend, one is a perfect curve (nonlinear), one is a line dragged off course by a single outlier, and one is a vertical strip held together by one extreme point. The lesson: summary statistics never substitute for a scatterplot.

⚠️ Don't Extrapolate or Confuse the Roles of x and y

The regression line of $y$ on $x$ is generally different from the line of $x$ on $y$ — the two slopes are $r\,s_y/s_x$ and $r\,s_x/s_y$ , which are equal only when $|r|=1$ . Choose the explanatory variable deliberately. And never use the line to predict far outside the data's $x$ -range: a study-hours model fitted on $1$ – $5$ hours cannot validly predict the score for $100$ hours.

connection

Bivariate quantitative analysis sits at the heart of a web of statistical ideas. The covariance and correlation in [2]–[3] generalize the single-variable Standard Deviation to two variables. The least-squares method [5] is the simplest case of Linear Regression and extends to many predictors in multiple regression. The decomposition behind $r^2$ in [6] is an instance of Analysis of Variance. When both variables are random, correlation connects to the Bivariate Normal Distribution and the concept of Statistical Independence (independence implies $r=0$ , but not conversely). For categorical pairs, the analogous tool is the Two-Way Table and the chi-squared test rather than $r$ .

🔬 Where this appears

⏳ History

- 1795/1805: Carl Friedrich Gauss (used ~1795) and Adrien-Marie Legendre (published 1805) independently develop the method of least squares to fit astronomical orbits.

- 1809: Gauss links least squares to the normal distribution of errors in Theoria Motus.

- 1846: Auguste Bravais derives mathematical expressions resembling correlation while studying errors in two dimensions.

- 1877–1886: Francis Galton introduces regression and the regression line through studies of heredity (sweet peas, then human heights), coining "regression toward mediocrity."

- 1888: Galton publishes "Co-relations and Their Measurement," giving the first explicit index of correlation.

- 1896: Karl Pearson formalizes the product-moment correlation coefficient

r

in its modern algebraic form.

- 1908: William Sealy Gosset ("Student") develops small-sample inference, enabling significance tests for

r

and regression slopes.

- 1922–1925: Ronald A. Fisher places regression and correlation on rigorous inferential footing and develops the analysis of variance.

- 1973: Francis Anscombe publishes his quartet, dramatizing why graphs must accompany statistics.

open-problem

Several deep cautions surround bivariate analysis. Simpson's paradox shows that a correlation present in every subgroup can reverse direction when groups are pooled — so the "right" grouping is genuinely ambiguous and context-dependent. Ecological correlation (computed from group averages, e.g. state-level data) is typically far stronger than the individual-level correlation, and mistaking one for the other is the ecological fallacy. Establishing causal direction from observational bivariate data remains a frontier: modern causal-inference frameworks (Pearl's do-calculus, instrumental variables, randomized trials) exist precisely because $r$ cannot do it. Finally, $r$ is highly sensitive to outliers and to the range of $x$ sampled — restricting the range (range restriction) attenuates $r$ even when the underlying relationship is unchanged, a persistent problem in test validation.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also