What does reproducible mean?

Reproducible Workflows · Session 1

A. Stamm

Department of Mathematics Jean Leray, UMR CNRS 6629, Nantes University

1 Before the slides: what does “reproducible” mean to you?

1.1 Class brainstorm

No slides, no spoilers: just you. In pairs, spend 4 minutes writing down everything you associate with the word reproducible. Don’t filter, don’t judge:

  • What does a reproducible analysis look like?
  • When is an analysis not reproducible?
  • Have you ever tried to re-run someone else’s (or your own!) code months later? What happened?

We’ll collect your answers on the board, group them, and the session builds from there.

1.2 What research is: the IMRAD structure

Research, distilled, is a short chain of steps:

  1. A question worth asking.
  2. An experiment (or study) that collects data to answer it.
  3. Analysis methods, explained and applied to that data.
  4. Results presented and commented.
  5. A discussion that puts the results and the original question into perspective.

That is, in essence, what data science does every day: it turns step 2’s data into step 4’s results by doing step 3 (analysis) with code, statistics and visualization.

That structure is often called IMRAD: introduction, methods, results and discussion. Reproducibility means shipping the implementation details of every one of these steps, so a reader can rebuild them from scratch.

1.3 Research, by code: the data science cycle

This is the same IMRAD loop, run with code, the classic R for Data Science1 figure.

Hypothesis→ Experiment→ Data→ Conclusions every analysis starts from a question and ends by answering it.

1Import → 2Tidy → 3Transform → 4Visualize → 5Model → 6Communicate

That is IMRAD, by code. Import + Tidy reproduce your methods; Transform + Model + Visualize are the analysis; Communicate is the results. Every step runs on programming, the layer that makes the whole loop re-runnable — which is reproducibility.

2 What is reproducibility?

2.1 The reproducibility crisis

John Ioannidis’s 2005 paper, Why Most Published Research Findings Are False, used a mathematical model to show that most claimed findings are false positives: the spark of the modern reproducibility crisis.

Core concepts of the crisis

  • The 2005 paper: small samples, tiny effects, high bias, flexible designs make claimed findings statistically likely to be wrong.
  • Impact across fields: from medicine and biomedicine into psychology, the social sciences and the life sciences.

Key causes

  • “Publish or perish” culture: novelty rewarded over truth.
  • P-hacking: adjusting analysis until a result becomes significant.
  • Publication bias: positive results favored over negative or null findings.
  • Nothing shared: no data, code or protocol.

2.2 “Works here” vs “works anywhere”

The test is simple. Take an analysis you did last month and ask three questions:

  1. Will it re-run?: does the code execute from source, with no hidden clicks or hand-fixes?

  2. Will it re-run here?: does it reproduce on your machine tomorrow, when packages have updated?

  3. Will it re-run anywhere?: does it reproduce on a colleague’s machine, or in a year?

Everything in this course is in service of making each “will it re-run?” an unequivocal yes: for you, and for anyone else.

2.3 What breaks reproducibility

The enemies we’ll learn to fight, one per later session:

  • The human memory gap: “I clicked through the setup”, “I cleaned it by hand in Excel”. Code that lives nowhere but the analyst’s head.

  • The moving environment: install.packages() happily updates globally, so your analysis breaks when a dependency version silently changes.

  • The (dis)appearing data: reading a local file path only you have; data that can’t be re-downloaded.

  • The vanished machine: your laptop dies, your setup is gone.

The reproducibility habit answers all four at once: document your code, pin your environment, source your data, version everything.

2.4 What reproducible research looks like

Here is the shape of every analysis we build this course:

  1. Data is sourced by code (a documented package, an open-data portal), never by manual download.
  2. The analysis lives in a document (Quarto) that mixes prose and code, so seeing the result shows how it was made.
  3. Dependencies are pinned (uvr/uv, or renv), so the document re-runs tomorrow.
  4. Everything is versioned with git: every change, every step, reversible.
  5. It is published to a URL others can reach.

You already learned the R side: vectors, data frames, plotting. This course wraps that skill in a workflow so your results can be trusted, shared and rebuilt by anyone, anywhere.

2.5 The course at a glance

Over three sessions we climb from “it works on my machine” to “it’s published and reproducible”:

  • Session 1 · Write: your first Quarto documents: prose + code + results, transparent from source.

  • Session 2 · Source: source an open-data set (e.g. data.gouv.fr), pin it, version it with git, and start your portfolio book.

  • Session 3 · Publish: deepen git (pull requests, review), manage deps with uvr/uv, publish to Posit Connect, and finish Chapter 1.

By the end you’ll have a published, versioned, reproducible portfolio: and the habit of making every future analysis reproducible by default. Projects in other courses slot into the same book, so your portfolio grows over time.

3 Setting up your environment

3.1 Your toolbox

Check https://astamm.github.io/r-ensai/setup.html

R: the language. Installed from CRAN.

Positron: the IDE for both R and Python in this course.

Quarto: renders R or Python into documents, reports, slides and books: the heart of everything we do.

{gt}: turns a data frame into a captioned, styled report table inside your Quarto documents.

git + GitHub: version control, driven by usethis and gitcreds; we add this in Session 2.

uvr/uv: pins the Python (and R) dependencies behind a project, so it re-runs next year on another machine.

Posit Connect: where your published report goes live at a URL (Session 3).

3.2 Verify your install

Run these and compare outputs:

R.version.string   # e.g. "R version 4.6.1"
quarto --version
python3 --version
git --version

Rule of thumb. If any of these errors, resolve it now: a broken toolchain will surface in every later session. The step-by-step guides are in https://astamm.github.io/r-ensai/setup.html.

3.3 Think in terms of projects

Open your course folder as a project and work there: the working directory is that folder.

Every file path in R is resolved relative to your working directory unless you write an absolute one. A small, predictable layout keeps a project tidy:

# a tidy project layout

my-project/
├── data/
│   ├── raw/
│   └── processed/
└── scripts/
  • In Positron, open the folder as a project (File → Open Folder); a relative path like read_csv("data/raw/penguins.csv") then just works.
  • Prefer relative paths: they let a script run on any machine with the same folder layout.
  • Projects are the unit you’ll version, pin and publish in Sessions 2–3.

Thinking in projects, not loose files, is the first reproducibility habit: it makes your work portable, and it’s the container everything else (git, uvr, publishing) builds on.

3.4 Create a Quarto document

The simplest way in Positron: New → New File… → Quarto document.

A blank .qmd opens. Press Preview (or Cmd/Ctrl+Shift+K) to render; edit a heading (a line starting with ##) and the output updates. Most-used Markdown:

  • #/##/### headings, **bold**, *italic*, - bullets, [t](u) links

Live block (≈15 min). Together: create a document, render it, and open source (.qmd) and preview side by side. See Markdown basics for headings, emphasis and lists.

Your turn (lab 1, steps 0–1, ≈10 min). Create and render your own document, then write one short prose paragraph about yourself (a heading, a bold or italic word, a bullet list). Download lab-1-first-doc.qmd from the course home page; steps 2–5 come later.

4 Quarto documents

4.1 What is a Quarto document?

A .qmd file is plain text that mixes three things:

Markdown for prose: headings, lists, emphasis, links, just like a README.

Code cells that run R or Python when the document renders, with their output (numbers, figures, tables) written into the document.

YAML front matter at the top (title, format, …) controlling how the whole thing is rendered.

Render it once (quarto render, or Preview in Positron) and you get a self-contained report: the story and the evidence, generated from the same source.

4.2 Your first Quarto document

Minimal but complete, this is a reproducible analysis:

---
title: "My first report"
format: html
---

Here is the mean of `x`:

```{r}
x <- c(2, 4, 6, 8, 10)
mean(x)
```

Today is `r Sys.Date()`.

My first report

Here is the mean of x: 

x <- c(2, 4, 6, 8, 10)
mean(x)
[1] 6

Today is 2026-09-09.

Inline code (r Sys.Date()) runs R inside a sentence; a code cell (fenced {r}) prints its result below it. Both re-evaluate on every render.

4.3 The YAML header

YAML is the control panel at the top of every document:

---
title: "My first report"
author: "Your Name"
format: html
---
  • title, author: metadata shown at the top of the page.
  • format: what to build, html, pdf, docx, revealjs (these slides are a revealjs), or a whole book (Session 2).
  • Many more options: themes, table of contents, captions…

YAML uses strict key: value pairs with two-space indentation. Indentation mistakes are the single most common Quarto error: the error message points at the line. Full options: Front matter · formats: All formats.

4.4 Code cells: controlling what shows

An option is a #| comment on its own line inside the cell, e.g. #| echo: false:

Option Effect
label: fig-… / label: tbl-… cross-reference a figure or table
eval: false keep the code, don’t run it
echo: false hide the code, show the output
include: false show neither (setup code)
fig-cap: "…" / tbl-cap: "…" caption a figure or table
message/warning: false silence package-load chatter

By default a cell shows code + output; each #| option changes that. The full list: Code execution options.

4.5 One tool, many languages

Quarto is language-agnostic: the same .qmd renders R, Python, or both.

  • An R cell and a Python cell sit side by side in one document, each with the same options.
  • Which language to use is a property of each cell, not of the document.
  • R and Python dominate in data science, but Quarto also speaks Julia and Observable JS, plus more via Jupyter.
  • Your portfolio (Session 2) is Python-compatible by design: prose plus whichever language fits the analysis.

Render once and you get a single self-contained report, whatever the language. That portability is why this course’s toolchain covers both R and Python.

4.6 A Quarto document in action

Everything so far is easier to see than to describe. Try it for yourself:

  • Open the document you just created in Positron.
  • Press Preview (or Cmd/Ctrl+Shift+K) to render it.
  • Change a heading or a number, save, re-render, and the output updates to match the source.

The loop, edit source → render → see output, is the whole of Quarto. Everything else (figures, books, publishing) is that loop scaled up.

5 Structured reporting & transparent code

5.1 Literate programming

The idea at the heart of Quarto (prose and code woven together) has a name:

  • Literate programming (Knuth, 1984): an analysis document is written primarily for humans to read, with the code embedded so the results it discusses appear in place.
  • The reader follows your thinking; the machine follows your instructions.
  • The document is self-documenting: the evidence sits beside the claim it supports.

Contrast with the output-first habit: a completed PDF with no source is only half an analysis. Here, the source is the analysis.

5.2 Structure a report for a reader

A reproducible report is a document, not a dump of output. Structure it:

  1. Question first: what are you asking, in one sentence?
  2. Data: where it came from, how it was sourced (the code shows this).
  3. Cleaning decisions: what you renamed, recoded, dropped, and why.
  4. Figures & tables: each captioned and referred to in the text.
  5. Findings: what you found, honestly, limits included.

The caption habit. A figure without a caption is a picture; a captioned figure referred to in prose is evidence. You’ll practice this in Lab 2 and it becomes the backbone of your portfolio chapter.

5.3 Captions & cross-references

Quarto numbers and links your figures automatically. This boxplot is a live captioned figure:

#| label: fig-penguins
#| fig-cap: "Bill length across penguin species."
ggplot(palmerpenguins::penguins, aes(species, bill_length_mm)) +
  geom_boxplot(fill = "#2E7D6B")

Refer to it in prose with @fig-penguins:

Figure 1 shows that Adelie penguins have the shortest bills.

@fig-... is a cross-reference: the number updates automatically if you move figures around. Every good report reads like a conversation with its own figures.

5.4 Nice tables with {gt}

Markdown tables are plain; {gt} turns a data frame into a styled, captioned report table:

#| label: tbl-penguins-summary
#| tbl-cap: "Mean bill length (mm) by penguin species."
penguins |>
  dplyr::group_by(species) |>
  dplyr::summarise(mean_bill = mean(bill_length_mm, na.rm = TRUE)) |>
  gt::gt() |>
  gt::opt_stylize(style = 4)

Refer to it in prose like any figure, with @tbl-penguins-summary:

Table 1 is a gt() table, captioned with #| tbl-cap.

5.5 Your turn: finish lab 1

Take 30 min for lab 1, steps 2–5: download lab-1-first-doc.qmd from the course home page (steps 0–1 are done). Each remaining step maps to a slide you’ve just seen:

  • Step 2, inline code: Your first Quarto document;
  • Step 3, a code chunk with a figure and its caption: Code cells and Captions & cross-references;
  • Step 4, a table with a caption: Captions & cross-references;
  • Step 5, a heading structure: Structure a report for a reader.

5.6 The transparency loop

Because source is the analysis, any reader can ask “how did you make that?” and get a real answer.

  • This deck ships with a code popover: press c on any slide to see the source behind it.
  • That’s the habit we want you to internalize for your own work: the analysis answers its own “how”.
  • When someone asks “where does this number come from?”, the answer is “render the document”, not “from my head”.

Lab 2 (≈30 min): build a structured report. You’ll write a short reproducible report on the palmerpenguins data (with prose, a captioned figure, and a table), following the structure template lab-2-structured-report.qmd downloadable from the course home page.

6 Project preview: your portfolio book

6.1 The end goal

Everything this course points toward is a single artefact: your portfolio.

A Quarto book: not slides, not a single report, but a book that grows all semester.

Chapter 1 = your EDA: a real dataset you care about, fetched by code from an open-data portal of your choice, analysed and cleaned.

Versioned, pinned, published: under git, dependencies locked, reachable at a URL.

The scaffold is on the course home page as a downloadable zip: grab it, render it, make it yours. We set up your actual Chapter 1 in Session 2.

6.2 The semester plan

Here’s how your book fills up over the term:

  1. Session 1 (today): you learn the document mechanics; the book scaffold renders.
  2. Session 2: you source a dataset by code (any open-data portal, data.gouv.fr or your own country’s), clean it, and drop your EDA into Chapter 1.
  3. Session 3: you put the book under git, pin it with renv/uv, and publish it.
  4. Later courses: other instructors slot their work into the placeholder chapters, so the book keeps growing.

By semester’s end you hold a published, versioned, reproducible portfolio, and the skill to do it again for any analysis.

6.3 Three moves, one artefact

Write, structure, grow.

6.4 Write, structure, grow

Today you took the first reproducible steps.

Document

Your analysis lives in a .qmd: YAML, code cells, inline r. Source is the report.

Structure & review

Captions, cross-references, honest findings: a structured report (your lab-2) that a stranger can follow.

Grow the book

The scaffold preview: a portfolio book that will keep filling all semester, chapter by chapter.

6.5 Your Chapter 1 is next

From a rendering document to a book with content.

Grow the book: Chapter 1 is next

Every slide today built toward the same artefact: a book that grows. In Session 2 you source a real dataset by code (any open-data portal), clean it, and drop your EDA into Chapter 1 of the scaffold. That chapter is step two toward the versioned, pinned, published portfolio you’ll finish in Session 3.

6.6 Next time

Session 2: source your data, clean it, and build the book.

Between sessions: make sure quarto render works on a trivial document, and skim Quarto: Getting Started for the parts we didn’t cover.