Lecture 03 — Describing Our Data

Wrangling, summaries, statistics, and our first real comparisons

tidyverse

descriptive-stats

Filter, select, mutate, and arrange with the tidyverse pipe; compute mean, median, variance, SD, and SE; start comparing groups.

Author

Bill Perry

Published

July 5, 2026

Where we left off (Lecture 02)

Setup — R and Positron installed; project folders created (data/, figures/, scripts/)
R as a calculator — arithmetic, order of operations, assignment with <-
Vectors — c(), data types (numeric, character, logical)
Functions — calling built-in functions, reading help with ?
Loading data — read_excel() and read_csv(); inspecting with glimpse(), head(), dim()
Pipe — %>% reads as “then”; chains steps together
First ggplot — boxplot + geom_jitter(), saved as PNG

Note

✅ Key idea from Lecture 02

You can get data into R and make a plot. Today we learn to wrangle and describe what we actually have.

Goals for Today

Part 1 — Tidyverse wrangling:

filter() — pick rows by a condition
select() — pick columns by name
mutate() — create new columns
arrange() — sort rows
Chain them all into a pipeline

Part 2 — Descriptive statistics:

mean, median, variance, SD, SE
NA handling — sum(!is.na())
group_by() + summarize()
skimr for a fast full overview
Boxplot + mean ± SE plots

Tip

🖐 Try it yourself

By the end you will wrangle and describe our leaf data in one connected pipeline.

Our tools today:

readxl
tidyverse
skimr

References:

📖 R4DS Ch 3 — Data Transformation
📖 R4DS Ch 13 — Numbers
📖 R4DS Ch 18 — Missing Values
🌐 Data Carpentry

Naming conventions:

data frames → _df
plots → _plot
models → _model

How to Use These Slides — Predict · Type · Run

This lecture runs in four short chunks. After each chunk you switch to the activity worksheet and type the code yourself.

For every code block, do three things:

Predict — before it runs, say what you think the output will be
Type it out by hand — do not copy-paste
Run it and compare to your prediction

Note

✅ Why bother? (the evidence)

Predicting first forces your brain to retrieve what it knows. The gap between your guess and the real answer is what makes the idea stick.
Typing by hand builds the finger-memory and error-spotting that copy-paste skips — including the typos R actually throws at you.
Chunk → immediate practice keeps a new idea in working memory long enough to form a lasting schema. A 90-minute lecture overloads it; an 8-minute chunk does not.

Load Libraries and Data

# Load all packages at the top of every script ----------
library(readxl) # reading Excel files
library(tidyverse) # data wrangling + ggplot2
library(skimr) # fast descriptive summaries

# Read the leaf data from the data folder ---------------
tree_df <- read_excel("data/2026_06_25_tree_experiment_raw_data.xlsx")

Note

Install once — load every session

Run in the Console one time only: install.packages("skimr")
Then every session: library(skimr) activates it.

Inspect the Data

# Preview the first rows ----------------------------
head(tree_df)

# A tibble: 6 × 5
  index side  weight_g width_cm height_cm
  <dbl> <chr>    <dbl>    <dbl>     <dbl>
1     1 sunny        4       12        21
2     2 sunny        3       14        22
3     3 sunny        5       12        21
4     4 sunny        3       13        23
5     5 sunny        4       15        22
6     6 sunny        3       13        23

# Column names, types, first values -----------------
glimpse(tree_df)

Rows: 20
Columns: 5
$ index     <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 1…
$ side      <chr> "sunny", "sunny", "sunny", "sunny", "sunny", "sunny", "sunny…
$ weight_g  <dbl> 4, 3, 5, 3, 4, 3, 5, 4, 3, 4, 7, 8, 6, 9, 8, 7, 8, 9, 9, 7
$ width_cm  <dbl> 12, 14, 12, 13, 15, 13, 16, 13, 14, 15, 9, 18, 17, 18, 19, 1…
$ height_cm <dbl> 21, 22, 21, 23, 22, 23, 21, 24, 23, 23, 28, 29, 27, 28, 27, …

# Rows × columns
dim(tree_df)

[1] 20  5

<chr> = character
<dbl> = numeric

🧩 Chunk 1 of 4 · Wrangling Verbs

We will cover: filter(), select(), mutate(), arrange(), and chaining them into a pipeline.

Tip

🖐 After this chunk you will do Activity Parts 1–3. Keep the worksheet open beside you — you will type every verb yourself.

Part 1 · The Core Tidyverse Verbs

Five functions do most of the work in data wrangling:

Function	What it does
`filter()`	keep rows matching a condition
`select()`	keep columns by name
`mutate()`	add or change a column
`arrange()`	sort rows
`summarize()`	collapse rows to a summary

All take a data frame as first input (via %>%) and return a data frame.

📖 R4DS §3 — Data Transformation

Note

✅ Key idea

These five verbs are the tidyverse grammar for data. Learn them once and you can wrangle almost any dataset.

Think of them as building blocks:

filter()   → rows you want
select()   → columns you want
mutate()   → new columns you need
arrange()  → order the rows
summarize()→ collapse to summaries

`filter()` — Picking Rows

Note

🔮 Predict first: We have 20 leaves. Before you run it, guess how many rows filter(side == "sunny") returns. Write your number down, then run it and check.

# filter() keeps rows where the condition is TRUE -------

# Keep only sunny leaves
tree_df %>% filter(side == "sunny") %>% head()

# A tibble: 6 × 5
  index side  weight_g width_cm height_cm
  <dbl> <chr>    <dbl>    <dbl>     <dbl>
1     1 sunny        4       12        21
2     2 sunny        3       14        22
3     3 sunny        5       12        21
4     4 sunny        3       13        23
5     5 sunny        4       15        22
6     6 sunny        3       13        23

# Keep only leaves heavier than 5 g
tree_df %>% filter(weight_g > 5) %>% head()

# A tibble: 6 × 5
  index side  weight_g width_cm height_cm
  <dbl> <chr>    <dbl>    <dbl>     <dbl>
1    11 shady        7        9        28
2    12 shady        8       18        29
3    13 shady        6       17        27
4    14 shady        9       18        28
5    15 shady        8       19        27
6    16 shady        7       19        28

# Combine two conditions with & (AND)
tree_df %>% filter(side == "shady", weight_g > 7) %>% head()

# A tibble: 6 × 5
  index side  weight_g width_cm height_cm
  <dbl> <chr>    <dbl>    <dbl>     <dbl>
1    12 shady        8       18        29
2    14 shady        9       18        28
3    15 shady        8       19        27
4    17 shady        8       18        29
5    18 shady        9       17        27
6    19 shady        9       19        27

Common comparison operators:

Symbol	Meaning
`==`	exactly equal
`!=`	not equal
`>` `<`	greater / less than
`>=` `<=`	greater / less than or equal
is.na	return all NA rows
!is.na	return all rows that are NOT NA

Warning

⚠️ Watch out!

= assigns a value.
== tests equality.
filter(side = "sunny") → error.
filter(side == "sunny") → correct.

Live Demo — Watch It Break (on purpose)

I will type this wrong on purpose:

# One equals sign — a very common mistake
tree_df %>% filter(side = "sunny")

R stops and tells us:

Error in `filter()`:
! We detected a named input.
i This usually means that you've used `=` instead of `==`.

Now the fix — two equals signs:

tree_df %>% filter(side == "sunny")

Tip

✅ Why show a broken run?

Real coding is fixing errors. Watching me read the message, spot the =, and repair it teaches the debugging process that finished code samples always hide.

When you hit this exact error yourself, you will recognise it.

`select()` — Picking Columns

# select() keeps the columns you name ------------------

# Keep only side and weight
tree_df %>% select(side, weight_g) %>% head()

# A tibble: 6 × 2
  side  weight_g
  <chr>    <dbl>
1 sunny        4
2 sunny        3
3 sunny        5
4 sunny        3
5 sunny        4
6 sunny        3

# Drop the index column (use minus sign)
tree_df %>% select(-index) %>% head()

# A tibble: 6 × 4
  side  weight_g width_cm height_cm
  <chr>    <dbl>    <dbl>     <dbl>
1 sunny        4       12        21
2 sunny        3       14        22
3 sunny        5       12        21
4 sunny        3       13        23
5 sunny        4       15        22
6 sunny        3       13        23

# Keep columns that start with a word
tree_df %>% select(starts_with("weight")) %>% head()

# A tibble: 6 × 1
  weight_g
     <dbl>
1        4
2        3
3        5
4        3
5        4
6        3

Why select() matters:
- Real datasets often have 50–100+ columns
- You rarely need all of them
- select() keeps your workspace clean
Useful helpers:
- starts_with("x") — columns starting with “x”
- ends_with("_g") — columns ending with “_g”
- contains("cm") — columns containing “cm”

📖 R4DS §3.3

`mutate()` — Creating New Columns

Note

🔮 Predict first: weight_g * 1000 converts grams to milligrams. Before running: will mutate() replace weight_g or add a new column? How many columns will the result have?

# mutate() adds a new column to the data frame ---------

# Convert grams to milligrams
tree_df %>%
  mutate(weight_mg = weight_g * 1000) %>%
  head()

# A tibble: 6 × 6
  index side  weight_g width_cm height_cm weight_mg
  <dbl> <chr>    <dbl>    <dbl>     <dbl>     <dbl>
1     1 sunny        4       12        21      4000
2     2 sunny        3       14        22      3000
3     3 sunny        5       12        21      5000
4     4 sunny        3       13        23      3000
5     5 sunny        4       15        22      4000
6     6 sunny        3       13        23      3000

# Add a size category based on weight
tree_df %>%
  mutate(size_class = if_else(weight_g > 5, "large", "small")) %>%
  head()

# A tibble: 6 × 6
  index side  weight_g width_cm height_cm size_class
  <dbl> <chr>    <dbl>    <dbl>     <dbl> <chr>     
1     1 sunny        4       12        21 small     
2     2 sunny        3       14        22 small     
3     3 sunny        5       12        21 small     
4     4 sunny        3       13        23 small     
5     5 sunny        4       15        22 small     
6     6 sunny        3       13        23 small

# mutate() can reference multiple existing columns
tree_df %>%
  mutate(
    weight_mg = weight_g * 1000,
    area_approx = width_cm * height_cm # rough area estimate
  ) %>%
  head()

# A tibble: 6 × 7
  index side  weight_g width_cm height_cm weight_mg area_approx
  <dbl> <chr>    <dbl>    <dbl>     <dbl>     <dbl>       <dbl>
1     1 sunny        4       12        21      4000         252
2     2 sunny        3       14        22      3000         308
3     3 sunny        5       12        21      5000         252
4     4 sunny        3       13        23      3000         299
5     5 sunny        4       15        22      4000         330
6     6 sunny        3       13        23      3000         299

mutate() keeps all existing columns and adds new ones
Use if_else(condition, value_if_true, value_if_false) to create categories

Note

✅ Key idea

mutate() never removes columns — it adds to the right of your data frame.

To overwrite a column, just use the same name on the left: mutate(weight_g = round(weight_g, 1))

📖 R4DS §3.3

`arrange()` — Sorting Rows

# arrange() sorts rows by one or more columns ----------

# Lightest leaves first
tree_df %>% arrange(weight_g)

# A tibble: 20 × 5
   index side  weight_g width_cm height_cm
   <dbl> <chr>    <dbl>    <dbl>     <dbl>
 1     2 sunny        3       14        22
 2     4 sunny        3       13        23
 3     6 sunny        3       13        23
 4     9 sunny        3       14        23
 5     1 sunny        4       12        21
 6     5 sunny        4       15        22
 7     8 sunny        4       13        24
 8    10 sunny        4       15        23
 9     3 sunny        5       12        21
10     7 sunny        5       16        21
11    13 shady        6       17        27
12    11 shady        7        9        28
13    16 shady        7       19        28
14    20 shady        7       19        29
15    12 shady        8       18        29
16    15 shady        8       19        27
17    17 shady        8       18        29
18    14 shady        9       18        28
19    18 shady        9       17        27
20    19 shady        9       19        27

# Heaviest leaves first (use desc() to reverse)
tree_df %>% arrange(desc(weight_g))

# A tibble: 20 × 5
   index side  weight_g width_cm height_cm
   <dbl> <chr>    <dbl>    <dbl>     <dbl>
 1    14 shady        9       18        28
 2    18 shady        9       17        27
 3    19 shady        9       19        27
 4    12 shady        8       18        29
 5    15 shady        8       19        27
 6    17 shady        8       18        29
 7    11 shady        7        9        28
 8    16 shady        7       19        28
 9    20 shady        7       19        29
10    13 shady        6       17        27
11     3 sunny        5       12        21
12     7 sunny        5       16        21
13     1 sunny        4       12        21
14     5 sunny        4       15        22
15     8 sunny        4       13        24
16    10 sunny        4       15        23
17     2 sunny        3       14        22
18     4 sunny        3       13        23
19     6 sunny        3       13        23
20     9 sunny        3       14        23

# Sort by side, then by weight within each side
tree_df %>% arrange(side, desc(weight_g))

# A tibble: 20 × 5
   index side  weight_g width_cm height_cm
   <dbl> <chr>    <dbl>    <dbl>     <dbl>
 1    14 shady        9       18        28
 2    18 shady        9       17        27
 3    19 shady        9       19        27
 4    12 shady        8       18        29
 5    15 shady        8       19        27
 6    17 shady        8       18        29
 7    11 shady        7        9        28
 8    16 shady        7       19        28
 9    20 shady        7       19        29
10    13 shady        6       17        27
11     3 sunny        5       12        21
12     7 sunny        5       16        21
13     1 sunny        4       12        21
14     5 sunny        4       15        22
15     8 sunny        4       13        24
16    10 sunny        4       15        23
17     2 sunny        3       14        22
18     4 sunny        3       13        23
19     6 sunny        3       13        23
20     9 sunny        3       14        23

Default: ascending order (smallest first)
desc(column) → descending order (largest first)
Very useful before printing a table or checking outliers

Tip

arrange() is rarely used in the middle of an analysis pipeline — it is most useful at the end when you want to inspect your results in a specific order.

The Full Pipeline

# Chain all the verbs together -------------------------
# Read it top to bottom like a recipe

wrangled_df <- tree_df %>%
  filter(weight_g > 0) %>% # remove any zeros
  select(side, weight_g, height_cm) %>% # keep only needed cols
  mutate(
    weight_mg = weight_g * 1000,
    size_class = if_else(weight_g > 5, "large", "small")
  ) %>%
  arrange(side, desc(weight_g)) # sort by side, then weight

head(wrangled_df)

# A tibble: 6 × 5
  side  weight_g height_cm weight_mg size_class
  <chr>    <dbl>     <dbl>     <dbl> <chr>     
1 shady        9        28      9000 large     
2 shady        9        27      9000 large     
3 shady        9        27      9000 large     
4 shady        8        29      8000 large     
5 shady        8        27      8000 large     
6 shady        8        29      8000 large

Read the pipeline out loud:

Take tree_df,

then filter to non-zero weights,

then select three columns,

then add two new columns,

then sort by side and weight.

Note

✅ Key idea

The pipeline %>% is what makes tidyverse readable. You can see every step in order — unlike nested function calls that read inside-out.

🛑 Pause — Do Activity Parts 1–3 Now

Open the worksheet and work through filter, select, mutate, arrange, and the full pipeline. Type every line; predict before you run.

Note

Natural stopping point (optional two-session split)

Chunk 1 is all new code. Everything after this is new statistics. If class runs short, stop here and start next session at Chunk 2 — that keeps students from meeting new code and new statistics at the same moment, which is exactly what cognitive-load research warns against.

Part 2 · Our Scientific Question

Biological prediction: Leaves on the shady side of a tree will be larger and heavier to capture more of the limited light.

	Hypothesis
H₀ — null	No difference in leaf size between sunny and shady sides
Hₐ — alternate	Leaf size differs between sunny and shady sides

Plan for this unit:

Today — describe the data
Lecture 04 — formally test the hypotheses with a t-test

Statistics helps us decide which hypothesis is better supported by the data.

🧩 Chunk 2 of 4 · Centre — Mean & Median

We will cover: the mean, extracting one group with filter() + pull(), and the median.

Tip

🖐 After this chunk: Activity Part 5 (mean & median).

What Is the Mean?

\[\bar{x} = \frac{\sum_{i=1}^{n} x_i}{n}\]

Add up all values, divide by n
The balance point of the distribution
Sensitive to outliers — one very large value pulls the mean up

Example:

Values: 5, 6, 7, 8, 50 → mean = 15.2

Is 15.2 a fair summary? Not really — the outlier dominates!

Why it matters:

The most commonly reported statistic
Used in every t-test, ANOVA, and regression
Always pair it with a measure of spread (SD or SE)

📖 R4DS §13 — Numbers

Calculating the Mean — filter() and pull()

# Use filter() + pull() to extract one group's values --
sunny_wt <- tree_df %>%
  filter(side == "sunny") %>%
  pull(weight_g)

sunny_wt # look at the raw values

 [1] 4 3 5 3 4 3 5 4 3 4

# Calculate the mean -----------------------------------
mean_sunny <- mean(sunny_wt, na.rm = TRUE)
cat("Mean sunny weight:", round(mean_sunny, 2), "g\n")

Mean sunny weight: 3.8 g

filter(side == "sunny") — keeps only sunny rows
pull(weight_g) — extracts that column as a vector
na.rm = TRUE — removes any NA before computing

Key pattern:

filter() → picks rows

pull() → extracts a column as a vector

📖 R4DS §3 — Data Transformation

What Is the Median?

The middle value when data are sorted

Sorted data	Median
5, 6, 7, 8, 50	7
5, 6, 7, 8, 50, 51	(7 + 8) / 2 = 7.5

Not pulled by outliers → robust
If mean ≫ median → right-skewed
If mean ≈ median → roughly symmetric

# Median for each side ---------------------------------
shady_wt <- tree_df %>%
  filter(side == "shady") %>%
  pull(weight_g)

med_sunny <- median(sunny_wt, na.rm = TRUE)
med_shady <- median(shady_wt, na.rm = TRUE)

cat("Median sunny:", round(med_sunny, 2), "\n")

Median sunny: 4

cat("Median shady:", round(med_shady, 2), "\n")

Median shady: 8

🧩 Chunk 3 of 4 · Spread & Counting Correctly

We will cover: variance, SD, SE, the trap in length(), and the fix sum(!is.na()).

Tip

🖐 After this chunk: Activity Part 4 (the NA problem) and the SD / SE steps of Part 5.

What Is Variance?

\[s^2 = \frac{\sum_{i=1}^{n}(x_i - \bar{x})^2}{n - 1}\]

For each value, compute distance from mean: \((x_i - \bar{x})\)
Square those distances → all positive
Sum them up, divide by \(n - 1\)

Dividing by \(n - 1\) gives an unbiased estimate of population variance.

Larger variance = more spread = more uncertainty

# Variance for each side ------------------------------
var_sunny <- var(sunny_wt, na.rm = TRUE)
var_shady <- var(shady_wt, na.rm = TRUE)

cat("Variance sunny:", round(var_sunny, 2), "\n")

Variance sunny: 0.62

cat("Variance shady:", round(var_shady, 2), "\n")

Variance shady: 1.07

Note

Variance is in squared units (g²) — hard to interpret. That is why we use SD.

📖 R4DS §13.3

What Is Standard Deviation?

\[s = \sqrt{s^2} = \sqrt{\frac{\sum(x_i - \bar{x})^2}{n-1}}\]

The square root of variance → back in original units (g) ✓
For a normal distribution:
- ~68% of values fall within mean ± 1 SD
- ~95% fall within mean ± 2 SD
Reports how spread out individual data points are

# Standard deviation for each side -------------------
sd_sunny <- sd(sunny_wt, na.rm = TRUE)
sd_shady <- sd(shady_wt, na.rm = TRUE)

cat("SD sunny:", round(sd_sunny, 2), "g\n")

SD sunny: 0.79 g

cat("SD shady:", round(sd_shady, 2), "g\n")

SD shady: 1.03 g

What Is Standard Error?

\[SE = \frac{s}{\sqrt{n}}\]

Measures how precisely we know the mean
Not the spread of raw data — the uncertainty of the mean itself
Gets smaller as n increases → more data = more precise estimate

	Describes
SD	spread of individual data points
SE	precision of the sample mean

We report SE when making claims about group means.

# SE = SD / sqrt(n) — need correct n -----------------
n_sunny <- sum(!is.na(sunny_wt))
n_shady <- sum(!is.na(shady_wt))

se_sunny <- sd_sunny / sqrt(n_sunny)
se_shady <- sd_shady / sqrt(n_shady)

cat("n sunny  =", n_sunny, "\n")

n sunny  = 10

cat("SE sunny =", round(se_sunny, 2), "g\n\n")

SE sunny = 0.25 g

cat("n shady  =", n_shady, "\n")

n shady  = 10

cat("SE shady =", round(se_shady, 2), "g\n")

SE shady = 0.33 g

The Problem with `length()`

# length() counts ALL positions — including NAs -------
x <- c(4, 3, NA, 7, NA) # a vector with missing values

length(x) # returns 5 — but 2 are NA!

[1] 5

sum(is.na(x)) # how many NAs are there?

[1] 2

# Using length() in SE gives the WRONG answer ---------
wrong_n <- length(x)
wrong_se <- sd(x, na.rm = TRUE) / sqrt(wrong_n)

cat("Wrong n  =", wrong_n, "\n")

Wrong n  = 5

cat("Wrong SE =", round(wrong_se, 3), "\n")

Wrong SE = 0.931

length(x) counts all positions in the vector — including NA
This is a silent error — R will not warn you!
Using n = 5 in SE = SD / √n when n should be 3 gives the wrong answer

📖 R4DS §18 — Missing Values

Fix — `sum(!is.na())`

# Break it down step by step -------------------------
x <- c(4, 3, NA, 7, NA)

is.na(x) # TRUE where NA, FALSE where real

[1] FALSE FALSE  TRUE FALSE  TRUE

!is.na(x) # FLIP: TRUE where real, FALSE where NA

[1]  TRUE  TRUE FALSE  TRUE FALSE

sum(!is.na(x)) # count the TRUEs = number of real values

[1] 3

# Apply to our real leaf data ------------------------
n_sun_check <- sum(!is.na(sunny_wt))
cat("Non-missing sunny weights =", n_sun_check, "\n")

Non-missing sunny weights = 10

How it works step by step:

is.na(x) → TRUE for each NA
!is.na(x) → flips it: TRUE for real values
sum(...) → counts TRUEs

Tip

Best practice: always use sum(!is.na(x)) when you need n — even when you think there are no NAs.

All Stats — Base R, One Group at a Time

# All stats for the sunny side -----------------------
n_s <- sum(!is.na(sunny_wt))
mean_s <- mean(sunny_wt, na.rm = TRUE)
med_s <- median(sunny_wt, na.rm = TRUE)
sd_s <- sd(sunny_wt, na.rm = TRUE)
se_s <- sd_s / sqrt(n_s)

cat("--- Sunny Leaves ---\n")

--- Sunny Leaves ---

cat("n      =", n_s, "\n")

n      = 10

cat("Mean   =", round(mean_s, 2), "\n")

Mean   = 3.8

cat("Median =", round(med_s, 2), "\n")

Median = 4

cat("SD     =", round(sd_s, 2), "\n")

SD     = 0.79

cat("SE     =", round(se_s, 2), "\n")

SE     = 0.25

Every function takes na.rm = TRUE
cat() prints clean labelled output
This is clear but repetitive for many groups → that is why we use group_by() next

🧩 Chunk 4 of 4 · Tidy Stats & Plots

We will cover: group_by() + summarize(), skimr, and the boxplot / mean ± SE plots.

Tip

🖐 After this chunk: Activity Parts 6–9.

Stats the Tidy Way — `group_by()` + `summarize()`

Note

🔮 Predict first: How many rows will stats_df have? (Hint: how many values does side take?) Predict the number before you run it.

# Calculate all stats for both sides at once ----------
stats_df <- tree_df %>%
  group_by(side) %>%
  summarize(
    n = sum(!is.na(weight_g)),
    mean_wt = round(mean(weight_g, na.rm = TRUE), 2),
    med_wt = round(median(weight_g, na.rm = TRUE), 2),
    sd_wt = round(sd(weight_g, na.rm = TRUE), 2),
    se_wt = round(sd_wt / sqrt(n), 2)
  )

stats_df

# A tibble: 2 × 6
  side      n mean_wt med_wt sd_wt se_wt
  <chr> <int>   <dbl>  <dbl> <dbl> <dbl>
1 shady    10     7.8      8  1.03  0.33
2 sunny    10     3.8      4  0.79  0.25

group_by(side) — splits the data by the side column
summarize() — collapses each group to one row of stats
You get both groups in one clean table
You can reference a column just created (sd_wt → se_wt)

📖 R4DS §3.5 — summarize()

Multiple Variables at Once

# One group_by() pipe summarizes all measurements -----
size_stats_df <- tree_df %>%
  group_by(side) %>%
  summarize(
    n = sum(!is.na(weight_g)),
    mean_wt = round(mean(weight_g, na.rm = TRUE), 2),
    mean_ht = round(mean(height_cm, na.rm = TRUE), 2),
    mean_wd = round(mean(width_cm, na.rm = TRUE), 2)
  )

size_stats_df

# A tibble: 2 × 5
  side      n mean_wt mean_ht mean_wd
  <chr> <int>   <dbl>   <dbl>   <dbl>
1 shady    10     7.8    27.9    17.3
2 sunny    10     3.8    22.3    13.7

With one group_by() pipe you summarize all three measurements across both sides.

Do the numbers support our hypothesis?

Shady heavier? → check mean_wt
Shady taller? → check mean_ht
Shady wider? → check mean_wd

Fast Summaries with `skimr`

# skim() grouped by side — full column summaries ------
tree_df %>%
  group_by(side) %>%
  skim()

Data summary
Name	Piped data
Number of rows	20
Number of columns	5
_______________________
Column type frequency:
numeric	4
________________________
Group variables	side

Variable type: numeric

skim_variable	side	complete_rate	mean	sd	p0	p25	p50	p75	p100	hist
index	shady	1	15.5	3.03	11	13.25	15.5	17.75	20	▇▇▇▇▇
index	sunny	1	5.5	3.03	1	3.25	5.5	7.75	10	▇▇▇▇▇
weight_g	shady	1	7.8	1.03	6	7.00	8.0	8.75	9	▂▇▁▇▇
weight_g	sunny	1	3.8	0.79	3	3.00	4.0	4.00	5	▇▁▇▁▃
width_cm	shady	1	17.3	3.02	9	17.25	18.0	19.00	19	▁▁▁▂▇
width_cm	sunny	1	13.7	1.34	12	13.00	13.5	14.75	16	▅▇▅▅▂
height_cm	shady	1	27.9	0.88	27	27.00	28.0	28.75	29	▇▁▆▁▆
height_cm	sunny	1	22.3	1.06	21	21.25	22.5	23.00	24	▆▃▁▇▂

Per column and per group: n_missing, mean, sd, percentiles (p0 to p100), and a mini histogram.

Tip

skim() is your fastest first look at any dataset. Pair it with group_by() to compare groups instantly.

Boxplot — Basic Structure

# Basic boxplot — data is already long format ---------
tree_box_plot <- tree_df %>%
  ggplot(aes(x = side, y = weight_g, fill = side)) +
  geom_boxplot(alpha = 0.6, outlier.shape = NA) +
  labs(
    title = "Leaf Weight by Tree Side",
    x = "Side of Tree",
    y = "Leaf Weight (g)",
    fill = "Side"
  ) +
  theme_minimal() +
  theme(legend.position = "none")

tree_box_plot

Boxplot anatomy:

Middle line = median
Box edges = 25th and 75th percentile (IQR)
Whiskers = 1.5 × IQR
Dots beyond whiskers = outliers

Because our data is in long format, ggplot2 reads side straight off the x-axis — no reshaping needed.

📖 R4DS §9 — Layers

Boxplot — Overlay Individual Points

# Add raw data points over the boxplot ----------------
tree_jitter_plot <- tree_df %>%
  ggplot(aes(x = side, y = weight_g, fill = side)) +
  geom_boxplot(alpha = 0.5, outlier.shape = NA) +
  geom_point(
    position = position_jitter(width = 0.15, seed = 42),
    alpha = 0.6,
    size = 2.5
  ) +
  labs(
    title = "Leaf Weight by Tree Side",
    x = "Side of Tree",
    y = "Leaf Weight (g)",
    fill = "Side"
  ) +
  theme_minimal() +
  theme(legend.position = "none")

tree_jitter_plot

Key arguments:

position_jitter(width = 0.15) — spreads points left and right
seed = 42 — same jitter layout every render
alpha = 0.6 — semi-transparent to show overlap
size = 2.5 — point size

Tip

Always show raw points with a boxplot — with only 10 leaves per group, every point matters!

Mean ± SE Plot — `stat_summary()`

# stat_summary() computes and plots mean and SE -------
tree_mean_se_plot <- tree_df %>%
  ggplot(aes(x = side, y = weight_g, color = side)) +
  geom_point(
    position = position_jitter(width = 0.15, seed = 42),
    alpha = 0.35,
    size = 2
  ) +
  stat_summary(fun = mean, geom = "point", size = 4) +
  stat_summary(
    fun.data = mean_se,
    geom = "errorbar",
    width = 0.15,
    linewidth = 0.9
  ) +
  labs(
    title = "Mean ± SE Leaf Weight by Tree Side",
    x = "Side of Tree",
    y = "Leaf Weight (g)",
    color = "Side"
  ) +
  theme_minimal() +
  theme(legend.position = "none")

tree_mean_se_plot

Two stat_summary() layers:

call	what it draws
`fun = mean`	large point at the mean
`fun.data = mean_se`	error bars for ± 1 SE

Individual points in the background (alpha = 0.35)
Mean shown as a larger foreground point
Error bars = ± 1 SE

🛑 Pause — Do Activity Parts 6–9 Now

Tidy stats with group_by(), a fast skim(), and both plots. Predict each output, type it, then run it.

What We Learned Today

R wrangling skills:
- filter() — keep rows by condition
- select() — keep columns by name
- mutate() — add new calculated columns
- arrange() — sort rows
- Full pipeline: chain all verbs with %>%
Statistics concepts:
- Mean — balance point; sensitive to outliers
- Median — middle value; robust
- Variance / SD — spread of individual data
- SE — precision of the sample mean
More R skills:
- group_by() + summarize() — all stats for all groups at once
- sum(!is.na()) — the right way to count n
- skim() — instant full dataset overview
- geom_boxplot() + geom_point(), stat_summary()

References:

📖 R4DS §3 — Data Transformation
📖 R4DS §13 — Numbers
📖 R4DS §18 — Missing Values
🌐 Data Carpentry

Up next — Lecture 04:

Two-sample t-test
Checking assumptions (normality, equal variance)
Interpreting and reporting results

Other Formats

Where we left off (Lecture 02)

Goals for Today

How to Use These Slides — Predict · Type · Run

Load Libraries and Data

Inspect the Data

🧩 Chunk 1 of 4 · Wrangling Verbs

Part 1 · The Core Tidyverse Verbs

filter() — Picking Rows

Live Demo — Watch It Break (on purpose)

select() — Picking Columns

mutate() — Creating New Columns

arrange() — Sorting Rows

The Full Pipeline

🛑 Pause — Do Activity Parts 1–3 Now

Part 2 · Our Scientific Question

🧩 Chunk 2 of 4 · Centre — Mean & Median

What Is the Mean?

Calculating the Mean — filter() and pull()

What Is the Median?

🧩 Chunk 3 of 4 · Spread & Counting Correctly

What Is Variance?

What Is Standard Deviation?

What Is Standard Error?

The Problem with length()

Fix — sum(!is.na())

All Stats — Base R, One Group at a Time

🧩 Chunk 4 of 4 · Tidy Stats & Plots

Stats the Tidy Way — group_by() + summarize()

Multiple Variables at Once

Fast Summaries with skimr

Boxplot — Basic Structure

Boxplot — Overlay Individual Points

Mean ± SE Plot — stat_summary()

🛑 Pause — Do Activity Parts 6–9 Now

What We Learned Today

`filter()` — Picking Rows

`select()` — Picking Columns

`mutate()` — Creating New Columns

`arrange()` — Sorting Rows

The Problem with `length()`

Fix — `sum(!is.na())`

Stats the Tidy Way — `group_by()` + `summarize()`

Fast Summaries with `skimr`

Mean ± SE Plot — `stat_summary()`