Lecture 12 Notes — Visualisation & the End-to-End Capstone
Learning outcomes: LO1–LO4
Companion files: slides L12, notebook Week12_Visualization_Capstone.ipynb, the Pipeline Guide
1. Charts are arguments
A truncated bar axis turns a 0.2-star gap between the best and worst city (4% of the 5-star scale) into a dramatic difference. Bars should start at zero. Other tricks to recognise: dual axes that fake correlation, 3-D effects, pies with many slices, cherry-picked windows, and size encodings that exaggerate (doubling a circle's radius quadruples its area).
2. Choosing a chart
| Question | Good chart | Avoid |
|---|---|---|
| Compare categories | Sorted horizontal bar | Pie with many slices |
| Trend | Line | 3-D area |
| Distribution | Histogram, box plot | Bar of means only |
| Two numbers | Scatter | Dual-axis line |
| Many groups | Small multiples | One tangled multi-line chart |
City means differ by 0.2 stars while the within-city standard deviation is about 1 star, five times larger: show the distribution when it matters.
3. Small multiples and dashboards
One panel per city with shared axes lets the eye compare shapes (quarterly swings of 0.11–0.25 stars). A dashboard has a headline KPI plus a few panels, each answering a stated question; decoration is removed.
4. Quality gates
A quality gate is an automated check that stops the pipeline (non-zero exit) when the data looks wrong. On the Travel-Review pipeline output all seven gates pass:
| Gate | Observed |
|---|---|
| Corrupt-line rate below 5% | 1.7% |
| Duplicate rate below 5% | 1.1% |
| Missing or invalid rating below 20% | 12.1% |
| Unparsed dates equal 0 | 0 |
| Mean rating within 1–5 | 3.60 |
| Quartiles ordered | 3.0 ≤ 3.5 ≤ 4.5 |
| Rating–sentiment correlation positive | 0.86 |
Fed deliberately corrupted statistics, three gates fail as they should. Wire the gate script into run_all.sh so a failure stops publication.
5. The capstone (20% of the grade)
Report structure: (1) problem and users, (2) data and its limitations, (3) pipeline design including cluster configuration, (4) data quality with before/after counts and gates, (5) at least two analytics techniques from Weeks 5–11 with uncertainty, (6) a dashboard following the principles above, (7) ethics and privacy, (8) limitations and next steps, (9) reproducibility.
6. Assessment
End-of-semester examination 70%; Travel-Review Analytics Pipeline 20%; concepts quiz 10%. Weekly labs are ungraded practice for all three.
7. Discussion questions
- Find a misleading chart in a news story and redraw it honestly.
- Which quality gate would you add for a streaming source, and what threshold?
- Which two analytics techniques best suit your capstone question, and why?