lab 1 · EDA & data.gouv.fr (Qmd) — source
A. Stamm 
This page shows the source file of the lab below. Use the button to download it as a plain file and run it in your own workspace.
---
title: "Lab 1: source a dataset from an open-data portal"
subtitle: "Reproducible Workflows · Session 2"
author: "Your Name"
format: html
editor: source
---
> ~35 min. Your task: find a dataset you actually care about on an **open-data
> portal of your choice** ([data.gouv.fr](https://www.data.gouv.fr), or your
> own country's official portal), **source it with code** (no manual download),
> and run a quick exploratory analysis in this Quarto document. This same
> dataset will become **Chapter 1** of your portfolio in Lab 2; choose
> something you'd be proud to show all term.
>
> **Tools in this lab:** an open-data portal of your choice, plus the `{dplyr}`,
> `{ggplot2}`, `{janitor}`, `{naniar}`, `{skimr}`, `{gt}` packages (see Step 0).
## Step 0: know the packages
```wkq
#| label: setup
#| message: false
library(dplyr) # data wrangling
library(ggplot2) # plots
library(janitor) # clean_names(), tabyl()
library(naniar) # vis_miss(), gg_miss_var()
library(skimr) # skim()
library(gt) # report tables
```
The **only portal-specific bit** is how you *fetch* the data. Two options;
choose the one that fits your portal:
- **`{rdatagouv}`** (data.gouv.fr search/API). *An option, not a requirement.*
If you use data.gouv.fr and want it, install from GitHub:
```r
# in the console, once:
# install.packages("remotes")
# remotes::install_github("astamm/rdatagouv")
```
- **Any portal by URL**: no extra package needed. Most portals (e.g. your
country's) expose a downloadable file you can grab with `download.file()`.
Pick the file's direct URL from the portal's web page.
## Step 1: find a dataset
Browse your chosen portal for a topic you genuinely care about: transport,
sport, environment, culture, health…
**If using `{rdatagouv}`** on data.gouv.fr, you can search from R. Use
`type = "dataset"` to narrow to datasets (not reuses or organizations):
```wkq
#| label: search
#| eval: false
res <- search_datasets("air quality", type = "dataset")
res
```
Read the columns to find the dataset's `slug` and a description that matches
your interest. Pick **one**; you'll live with it for the whole term.
> **Pair up.** Spend 5 minutes describing to a classmate *why* you chose this
> dataset. A clear "because…" is the first sentence of your report.
## Step 2: source it with code
The rule is: **the fetch must be reproducible by anyone re-running the
document.** Record the dataset's page URL in your document; that's the seed
of your provenance statement.
**If using `{rdatagouv}`**, get the dataset's metadata, then find a
**resource** (the actual file) you can download by URL:
```wkq
#| label: fetch
#| eval: false
ds <- get_dataset("my-dataset-slug") # <- your slug here
ds$resources # list downloadable files
url <- ds$resources$url[1] # the first downloadable file
download.file(url, "data/my-data.csv")
```
**Any other portal**: find the direct download link on the dataset's web
page and fetch it the same way:
```wkq
#| label: download
#| eval: false
url <- "https://www.example-portal.org/.../my-data.csv" # <- the file's direct URL
download.file(url, "data/my-data.csv")
```
> Record the **dataset name, publisher, and URL** in a short sentence in your
> document. That sentence is your *provenance statement*: the reproducible
> habit in action.
## Step 3: quick EDA
Now the familiar loop from the slides:
```wkq
#| label: read
#| eval: false
dat <- readr::read_csv("data/my-data.csv", show_col_types = FALSE)
```
```wkq
#| label: skim
#| eval: false
skim(dat) # one-line glimpse
```
```wkq
#| label: names
#| eval: false
dat |> clean_names() |> names() # are the names already clean?
```
```wkq
#| label: tabs
#| eval: false
dat |> tabyl(<a-categorical-column>)
```
```wkq
#| label: missing
#| eval: false
vis_miss(dat) # where is the missingness?
```
## Step 4: record your cleaning decisions
Under a `## Cleaning decisions` heading, write 2–3 sentences:
- what you renamed / recoded / dropped, and **why**;
- how you'll handle missing values, and **why**;
- one thing that surprised you.
A defensible decision, written down, is what makes the difference between a
script and a reproducible analysis.
## Step 5: one figure with a caption
Add **one** short plot that answers a question you actually want answered, with
a real caption (`#| fig-cap:`). Keep it simple: a boxplot, histogram or line
plot over one variable is plenty. Refer to it in a sentence of prose.
## Done: check yourself
- Is the dataset **sourced by code** (`download.file`, with `{rdatagouv}` only
as an option), not downloaded by hand?
- Is there a **provenance sentence** (dataset, publisher, URL)?
- Do you have a `## Cleaning decisions` paragraph?
- Is there exactly **one captioned figure**, referred to in prose?
> **Stretch.** Turn your cleaning into a *function* or a `dplyr` pipeline kept
> in a separate `.R` file sourced by this document; so the cleaning is
> reusable and the document stays readable.