Plans for second edition

The second edition, planned to be out late 2025 will focus on correcting errors, improving some explanations and adding minor edits overall. There may be a single new chapter.

Error correction

The most important plan will be correcting typos and errors based on the errata page.

  • typos, errors
  • improve unclear sentences
  • add a few lines of explanation when needed

Data Analysis with AI

Each chapter will get a new section, which talks about how AI (LLM) might help one or more methods in the chapter. A section will be 0.5 to 1 page long and include a link to a chat. Examples

  • (Chapter 03) Data discovery with AI. We show a dataset and chat about variables, possible wrnagling and cleaning issues.
  • (Chapter 06) Look for statistics with AI. We discussed the t-test in the material. Now we ask how to run tests in more complicated scenarios


Each section will get a Beyond section replacing Further readings. This will be somewhat longer and link to an ever growing online version called Beyond: Directions to Frontier. Basically an extra 1-2 paragraphs focusing on helping readers towards what’s new.

Broad issues I’m thinking about

  1. Chapter 10 is too large, and is set to be bigger. Some say 07-09 is too slow. Some magic rearrangement?
  2. More maths. Some users advised to add more derivation to Under the Hood sections to avoid the need for another textbook in more advanced metrics classes. Okay, which ones?

Small improvements and additions

We plan several smaller improvements. Mostly adding some examples, better explanations. Also adding concepts based on feedback. Typically a few extra paragraphs, maybe a short new section.

Some ideas are:

Part 1

Chapter Topic Idea
02 variable naming Add a few para on naming variables, some ideas and when it’s really important. Extend 2.U1 or add 2.U2
03 3.9, 3.U1 Redo the theoretical distribution section. Bring pdf, cdf to main bit. Show pdf and cdf for normal, log-normal. Give more reason who they are useful when comparing cities, countries. Be more explicit re definitions of pareto, scale-free, power law, zipf’s law. Redo Pareto x axis
04 Dashboards What is a good dashboard, creating a simple one in shiny/quarto to show conditional means with hotel data
05 now short CS Maybe add a case study on estimating arrival time with simulation
06 t-test fro two samples One para and the formula for independent sample means

Part 2

Chapter Topic Idea
08 Practice of standard errors Discuss special cases. One source is Gelman’s JE bit but we had thhought about countlessly.
08 attenuation Add example for attenuation bias from Feodora Teti customs data paper, real policy implications
10? Frisch-Waughn-Lowell theorem Add a short section on FWL (no proof), with a case study application. The key application will be a graph, ie show show scatterplot despite controls. Maybe use earnings. Or even add a new case study on Mankiw-Romer-Weil QJE growth regressions. Deepnote
10,21 dataviz Add coeffplots
10,22 p-values Show tables with p-values and stars, add a para discussion ref back to p-hacking + both have pros and cons
10 Regression vs test U.x Discuss how regression may be the same as testing ideas
10 Hard q on confounders Suppose I have two random variables, y and x. If I’m allowed to construct a third random variable z, I can guarantee that a regression y = beta1 * x + beta2 * z will yield any value for beta1 I want Source
10 Exercise Read and discuss obesity gap by Economist
12 What is seasonality really Seasonality as human behavior. Example: Interest over time on Google Trends for Diet

Part 3

Chapter Topic Idea
13 r vs python results Add a few para/section on discussing that results that are borne out of algos without a close solution, will vary across platforms
13 loss Price prediction model trade-offs, loss function Kayak
14 var imp for OLS For linear models in prediction, add a few para, new section on variable importance
14 ln OLS correction More on what smearing does, when it’s better to use other formula, bias, MAE vs RMSE
14 Quant reg If MAE is target, qreg is a way. MAE vs RMSE discussion
14 Correlated predictors In any predictive model (OLS, RF), when we have many predictors that are correlated, we have problems: varimp and interpretation. Ideas: PCA, groupings, drop
16 SHAP for ML For the machine learning bit, add SHAP in addition to VIP
16 ensemble for OLS For linear models in prediction, we can also have an ensemble model, ln+log (as assignment for cars)
16 cloud comp Add run time in google colabs / amazon cloud for Table 16.4

Part 4

Chapter Topic Idea
19 Intro to causality friedmans-thermostat
19 SUTVA 2 para – Explicit about SUTVA
20 more on A/B test Add a bit more on experiments in large companies like UBER Pool more source1 Microsoft 2009, Kohavi HBR
21 More on RDD A more detailed example on RDD, maybe even a short case study
21 Good vs bad control Two example stories with discussion on controls, confounders, mechanism and collider
24 Add a new DiD Event study, maybe add one of new DiD method using the same case study

A new chapter

I am thinking about a single new chapter called “Different shapes of Data”.

  • This would basically give a short intro to non-tabular (ie not numbers) data:
    • (i) text,
    • (ii) spatial data/maps,
    • (iii) network data.
  • Give basic concepts, about 6-7 pages each including a short case study.

Case studies, data sources


We are open to suggestions! Plase make a suggestion for a minor change or a short addition you think would be helpful HERE. Also report errors, pls.