19 Documenting code and data with AI

20 Documenting code and data with AI

The first chapter where AI saves real time on real work. Documentation — READMEs, data dictionaries, code comments — is boring, important, and exactly the kind of task AI is good at. This chapter shows you how to use it without producing the kind of empty documentation no one reads.

20.1 What you will get from this chapter

A pattern for getting AI to draft documentation that is actually useful.
Three short, specific outputs every project should have.
A check that catches the most common AI documentation failure.

20.2 Three outputs every project needs

Most analytical projects benefit from three written artefacts. AI helps with all three.

1. A README. One page, at the root of the repo. What is this project? What runs in what order? Who do I ask if I am stuck? The next chapter — Documentation fundamentals — has the recipe.

2. A data dictionary. One row per variable. Name, type, units, range, meaning, missing-value convention. Lives next to the data, not in the README. Survives changes of personnel.

3. A short methodology note. Two paragraphs at the top of the analysis script (or in a METHODS.md). What the unit of observation is. What restrictions you applied. What you compute. This is the artefact that lets you, six months later, remember what the regression coefficient meant.

If your project has these three, you will have spent maybe an hour on documentation. The next person — including future-you — will save days.

20.3 The pattern: paste the thing, ask for the artefact

The right move with AI on documentation is to give it the actual thing and ask for the artefact you want.

Here is a working pattern, used through the book.

“Here is the head of my data file and the relevant pages of the codebook. Produce a Markdown table with one row per variable: column name, type, units, valid range, missing-value convention, one-line description. Do not invent variables that are not in the file. Where the codebook is ambiguous, mark the row ‘CHECK’ so I can resolve it.”

Three things make this prompt work.

You give the actual thing — the head of the file and the codebook pages — not a description of them.
You constrain the format — one row per variable, specific columns.
You give it permission to flag uncertainty — “mark the row CHECK”. The AI now has a way to tell you it does not know, instead of bluffing.

The third point is the difference between AI-drafted documentation that you trust and AI-drafted documentation that lies to you.

20.4 A worked example: the World Values Survey codebook

The Wave 7 codebook of the World Values Survey runs to several hundred pages and covers six hundred variables. By hand it is intimidating; with AI it becomes manageable.

Workflow that works:

Upload the codebook PDF and a small slice of the data (a thousand rows is plenty).
Pick a topic — “trust” or “gender attitudes” or “happiness.”
Ask the AI to list the variables in the codebook that map to the topic, with the response scales.
Spot-check three of them by hand against the codebook. Two minutes per variable.
If the spot-checks pass, ask for a Markdown table covering all of them.

You now have a working data dictionary for your topic in fifteen minutes. Doing this by hand would have taken half a day, and you would have made more mistakes than the AI did.

20.5 The most common failure, and the cheapest catch

The most common AI documentation failure is inventing a column or a value that is not in the data. The AI sees the spirit of the codebook and produces something that fits the spirit, even when the letter has changed.

The cheapest catch:

import pandas as pd
df = pd.read_csv("your-data.csv")

# Compare the AI-produced column list to reality
ai_columns = ["state", "age", "sex", "earnwke", "uhours"]   # paste from the AI's table
print("In data, not in AI list:", set(df.columns) - set(ai_columns))
print("In AI list, not in data:", set(ai_columns) - set(df.columns))

Run it. The two print statements should be empty. If they are not, the AI either missed columns (often fine) or invented columns (always investigate).

Total time: fifteen seconds. This single check would prevent half the documentation errors I have seen students hand in.

20.6 When not to outsource

Three places where you should write the prose yourself.

The “why this project” paragraph at the top of the README. AI does not know why your project exists. It will produce a generic description.
The limitations paragraph. What this analysis cannot tell you is yours to know. AI defaults to vague hedges that do not commit to anything.
The acknowledgements. You know who helped.

20.7 Where AI helps · Where AI bluffs

Helps. Drafting tables from a real codebook. Generating docstrings for functions you wrote. Translating notes into clean Markdown. Rewriting your own draft into something readable.

Bluffs. Inventing columns that are not in the data. Guessing units when the codebook does not say. Producing confident text about variables it has not actually read. The catch is always the same: compare the artefact to the source.

20.8 Keep this with you, not the AI

The “why this project” paragraph.
The limitations paragraph.
Any judgement call about what the data actually means. AI defaults to the most common reading; your reading may differ for good reason.

20.9 Try this

Pick a small CSV from a project of yours — anything with at least five columns and a hundred rows. Open your chat AI. Paste the head of the file. Ask for a Markdown data dictionary in the pattern above. Run the catch script.

If it passes the catch, congratulations: you have a working data dictionary in five minutes. Commit it to your repo. Future-you will thank you.

20.10 AI and me

How did AI support me here?
How did AI fail me?
How did AI extend me?

20.11 Where to go next

Documentation fundamentals for the principles that make a README actually useful, with or without AI.

--- title: "Documenting code and data with AI" --- # Documenting code and data with AI The first chapter where AI saves real time on real work. Documentation — READMEs, data dictionaries, code comments — is boring, important, and exactly the kind of task AI is good at. This chapter shows you how to use it without producing the kind of empty documentation no one reads. ## What you will get from this chapter - A pattern for getting AI to draft documentation that is actually useful. - Three short, specific outputs every project should have. - A check that catches the most common AI documentation failure. ## Three outputs every project needs Most analytical projects benefit from three written artefacts. AI helps with all three. **1. A README.** One page, at the root of the repo. *What is this project? What runs in what order? Who do I ask if I am stuck?* The next chapter — [Documentation fundamentals](06-doc-fundamentals.qmd) — has the recipe. **2. A data dictionary.** One row per variable. Name, type, units, range, meaning, missing-value convention. Lives next to the data, not in the README. Survives changes of personnel. **3. A short methodology note.** Two paragraphs at the top of the analysis script (or in a `METHODS.md`). What the unit of observation is. What restrictions you applied. What you compute. This is the artefact that lets you, six months later, remember what the regression coefficient meant. If your project has these three, you will have spent maybe an hour on documentation. The next person — including future-you — will save days. ## The pattern: paste the thing, ask for the artefact The right move with AI on documentation is to *give it the actual thing* and ask for the artefact you want. Here is a working pattern, used through the book. > *"Here is the head of my data file and the relevant pages of the codebook. Produce a Markdown table with one row per variable: column name, type, units, valid range, missing-value convention, one-line description. Do not invent variables that are not in the file. Where the codebook is ambiguous, mark the row 'CHECK' so I can resolve it."* Three things make this prompt work. - **You give the actual thing** — the head of the file and the codebook pages — not a description of them. - **You constrain the format** — one row per variable, specific columns. - **You give it permission to flag uncertainty** — *"mark the row CHECK"*. The AI now has a way to tell you it does not know, instead of bluffing. The third point is the difference between AI-drafted documentation that you trust and AI-drafted documentation that lies to you. ## A worked example: the World Values Survey codebook The Wave 7 codebook of the World Values Survey runs to several hundred pages and covers six hundred variables. By hand it is intimidating; with AI it becomes manageable. Workflow that works: 1. Upload the codebook PDF and a small slice of the data (a thousand rows is plenty). 2. Pick a topic — *"trust"* or *"gender attitudes"* or *"happiness."* 3. Ask the AI to list the variables in the codebook that map to the topic, with the response scales. 4. Spot-check three of them by hand against the codebook. Two minutes per variable. 5. If the spot-checks pass, ask for a Markdown table covering all of them. You now have a working data dictionary for your topic in fifteen minutes. Doing this by hand would have taken half a day, and you would have made more mistakes than the AI did. ## The most common failure, and the cheapest catch The most common AI documentation failure is **inventing a column or a value that is not in the data**. The AI sees the spirit of the codebook and produces something that fits the spirit, even when the letter has changed. The cheapest catch: ```python import pandas as pd df = pd.read_csv("your-data.csv") # Compare the AI-produced column list to reality ai_columns = ["state", "age", "sex", "earnwke", "uhours"] # paste from the AI's table print("In data, not in AI list:", set(df.columns) - set(ai_columns)) print("In AI list, not in data:", set(ai_columns) - set(df.columns)) ``` Run it. The two `print` statements should be empty. If they are not, the AI either missed columns (often fine) or invented columns (always investigate). Total time: fifteen seconds. This single check would prevent half the documentation errors I have seen students hand in. ## When *not* to outsource Three places where you should write the prose yourself. - **The "why this project" paragraph at the top of the README.** AI does not know why your project exists. It will produce a generic description. - **The limitations paragraph.** What this analysis cannot tell you is *yours* to know. AI defaults to vague hedges that do not commit to anything. - **The acknowledgements.** You know who helped. ## Where AI helps · Where AI bluffs **Helps.** Drafting tables from a real codebook. Generating docstrings for functions you wrote. Translating notes into clean Markdown. Rewriting your own draft into something readable. **Bluffs.** Inventing columns that are not in the data. Guessing units when the codebook does not say. Producing confident text about variables it has not actually read. The catch is always the same: compare the artefact to the source. ## Keep this with you, not the AI - The "why this project" paragraph. - The limitations paragraph. - Any judgement call about what *the data actually means*. AI defaults to the most common reading; your reading may differ for good reason. ## Try this Pick a small CSV from a project of yours — anything with at least five columns and a hundred rows. Open your chat AI. Paste the head of the file. Ask for a Markdown data dictionary in the pattern above. Run the catch script. If it passes the catch, congratulations: you have a working data dictionary in five minutes. Commit it to your repo. Future-you will thank you. ## AI and me 1. How did AI **support** me here? 2. How did AI **fail** me? 3. How did AI **extend** me? ## Where to go next [Documentation fundamentals](06-doc-fundamentals.qmd) for the principles that make a README actually useful, with or without AI.