Lecture: Correlation vs Linear Models
Relationships
What Is Correlation?
Correlation measures the strength and direction of a linear relationship between two numerical variables. It runs from −1 to +1:
- +1 — perfect positive
- 0 — no linear relationship
- −1 — perfect negative
The Pearson correlation coefficient:
\[r = \frac{\sum_{i}(X_i - \bar{X})(Y_i - \bar{Y})}{\sqrt{\sum_{i}(X_i - \bar{X})^2 \sum_{i}(Y_i - \bar{Y})^2}}\]
which is
\[r = \frac{\text{Covariance}(X, Y)}{s_X \cdot s_Y}\]
🗳️ DECIDE · Which One Has r Closest to Zero?
Four datasets, plotted on the right. No correlation coefficients shown — on purpose.
Which panel has the Pearson correlation closest to zero?
- Panel A
- Panel B
- Panel C
- Panel D
Warning
✍️ Write your letter down, then vote.
What r Can and Cannot See
Important
✅ The answer is (b) — Panel B.
- Panel B is the most tightly related pair on the slide: give me x and I can tell you y almost exactly.
- Its correlation is near zero anyway, because the relationship is an arch, not a line.
- Panel C looks like the messiest, so most rooms vote (c). It has a weak but genuine linear trend, so its r sits farther from zero than B’s.
- r measures one thing only: how close the points lie to a straight line.
- A perfect relationship with a curve in it reads as “no correlation.”
- Panel D is the other warning — a single far-out point can manufacture a large r out of a cloud with no trend in it.
Always plot the data before you trust a correlation coefficient.
Example 16.1 · Flipping the Bird
Nazca boobies (Sula granti): do the visits a chick receives from non-parent adults predict how aggressive it is as an adult?
Pearson's product-moment correlation
data: booby_df$visits_as_nestling and booby_df$future_aggression
t = 2.9603, df = 22, p-value = 0.007229
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.1660840 0.7710999
sample estimates:
cor
0.5337225 
Checking the Same Assumptions by Eye
The two tests do not agree with each other:
- visits, p = 0.396 — clean
- aggression, p = 0.047 — just under 0.05
Now look at the pictures. No strong skew, points close to both QQ lines, and the scatterplot was straight rather than curved.
Important
✅ This is the case the last slide asked you to predict
At n = 24, a normality p of 0.047 is a coin-flip verdict, not evidence of a real problem. W&S report Pearson’s r for these data, and so do we.
Compare that with the next dataset, where the test and the picture agree that something is wrong.
When the Assumption Fails — Example 16.5
W&S Example 16.5: people who had seen the Indian rope trick were asked, years later, how impressive it was — on a subjective score.
That score is bounded and lumpy, and it fails Shapiro badly. Three options when normality fails:
- Transform one or both variables (log, square root)
- Switch to a rank-based correlation — Spearman’s ρ or Kendall’s τ
- Hunt down the outlier that is doing the work
The longer ago someone saw the trick, the more impressive they remember it being.
Two Things That Move r Without Touching the Biology
Important
✅ The answer is (c).
Range restriction. r depends on the spread of X you sampled. Look at a narrow slice of a real relationship and r falls, even though the slope is unchanged. This is why correlations from studies with different ranges cannot be compared head to head.
Warning
⚠️ Attenuation
Measurement error in X or Y pushes r toward zero, never away from it. The true correlation is usually stronger than the one you observed — so (d) has the direction backwards.
The Simple Linear Regression Model
Regression models the response Y as a straight-line function of the predictor X.
The population model: \(Y = \alpha + \beta X + \varepsilon\)
- α is the intercept — the value of Y when X = 0
- β is the slope — the change in Y per one-unit change in X
- ε is the error — the random deviation of a point from the line
The sample equation: \(\hat{Y} = a + bX\), where a estimates α and b estimates β.
Method of least squares: of all possible lines, take the one that makes
\[\sum_{i=1}^{n}(y_i - \hat{y}_i)^2\]
as small as possible — the sum of the squared vertical distances.
Least squares picks the third one.
Fitting the Lion Model
Male lions accumulate black pigment on the nose as they age — so nose colour can age a lion in the field without anyone killing it.
(Intercept) proportion_black
0.8790062 10.6471194 Read the formula as response ~ predictor.
Example 17.3 · Prairie Biodiversity and Stability
Tilman et al. (2006): does a more diverse plant community produce more stable biomass? 161 plots, seeded with 1 to 16 species.
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.1982943 0.041298043 29.015766 2.075490e-65
species_number 0.0329259 0.004883691 6.742012 2.733315e-10
Where the Variation Goes
Take one data point. Its distance from the overall mean \(\bar{y}\) splits, exactly, into two pieces:
\[\underbrace{(y_i - \bar{y})}_{\text{total}} = \underbrace{(\hat{y}_i - \bar{y})}_{\text{explained}} + \underbrace{(y_i - \hat{y}_i)}_{\text{left over}}\]
Square those deviations and add them up over every point, and the same split holds for the whole dataset:
\[SS_{total} = SS_{regression} + SS_{residual}\]
- \(SS_{regression}\) — variation the line accounts for; df = 1
- \(SS_{residual}\) — variation it does not; df = n − 2
- \(SS_{total}\) — all of it; df = n − 1
🗳️ DECIDE · Which Fit Has the Bigger Sums of Squares?
Four fitted regressions, on the right.
Which statement is true?
(a) \(SS_{regression}\) is larger in (d) than in (c)
(b) \(SS_{residual}\) is larger in (b) than in (a)
(c) \(SS_{total}\) is larger in (a) than in (b)
(d) \(SS_{residual}\) is zero in (c)
Warning
✍️ One letter. Write it down, then argue it with your neighbour.
From Sums of Squares to Mean Squares
A sum of squares keeps growing as you add data, so it cannot be compared across studies. Divide by degrees of freedom and you get a mean square — a variance, which does not grow with n:
\[MS = \frac{SS}{df} \qquad F = \frac{MS_{regression}}{MS_{residual}}\]
- \(MS_{residual}\) = 2.785 — estimates the background variance σ²
- \(MS_{regression}\) = 138.54 — estimates that same σ², plus whatever the X–Y relationship adds
- If the slope is truly zero, both estimate the same thing and F ≈ 1. Here F = 49.75
Sums of squares add up. Mean squares do not.
Testing Regression Assumptions — Overview
Simple linear regression assumes:
- Linearity — the true relationship is a straight line
- Independence — observations do not depend on each other
- Homoscedasticity — equal scatter at every value of X
- Normality — the residuals are normally distributed
Stated formally: the error \(\varepsilon_i\) is normal, with a mean of 0 and the same variance, at every value of x.
Every Check Is About the Residuals
Every assumption check from here on uses the residuals, not the raw data. That is the single most common mix-up in regression.
The error \(\varepsilon_i\) is estimated by the residual \(e_i = y_i - \hat{y}_i\) — the same red segment you have seen on every lion plot today, and the same quantity that got squared and summed to build \(SS_{residual}\).
Least squares already made these as small as possible. The diagnostics ask whether what is left over looks like random noise.
Checking Linearity — Residuals vs. Fitted
What you want: a shapeless cloud, scattered evenly above and below the horizontal line at zero.
What is a problem: any curve in the trend line. A U-shape or an arch means a straight line is the wrong shape for these data — the exact failure Panel B showed back in the correlation half.
The plot doubles as a homoscedasticity check: if the cloud fans out or funnels in from left to right, the variance is not constant.
Checking Normality — QQ Plot of Residuals
The QQ plot compares your residuals with what perfectly normal residuals would look like.
What you want: points hugging the diagonal.
What is a problem: systematic curvature, or points peeling away at the ends. Tails are where non-normality shows up first, and tails are what p-values depend on.
Note we check the residuals, not age_years itself. The raw ages are right-skewed, and that is allowed. It is the residuals that have to behave.
What Violations Actually Look Like
When an assumption fails:
- Transform one or both variables (W&S §17.6) — a log on a curved or fanning relationship fixes both problems more often than not
- Weighted least squares for heteroscedasticity
- A non-linear model (§17.8), or a GLM in Week 11
🗳️ DECIDE · One Island
Gotelli & Ellison’s Galápagos data: 17 islands, plant species richness against island area, both log-transformed. The fit is solid — R² = 0.585, p = 3.50e-04.
Now one more point arrives: a large island reported with very few species (red).
What does that single point do?
(a) Very little — one point in 18 cannot matter much
(b) It raises R², because the range of X just got wider
(c) It roughly halves R² and pushes p to the edge of significance
(d) It can change R² but not the slope
Warning
✍️ One letter, before the next slide.
