Part 3 — Staying in Charge: Checking the Agent’s Work

Friendly Introduction to Agentic AI for Social Scientists

Published

November 9, 2026

Part 3 — Staying in Charge

Trust, verification and control: how to set the agent’s limits, see what it actually did, and stand behind work you did not do with your own hands

Before you start

This is Part 3 of the mini course Friendly Introduction to Agentic AI for Social Scientists. It assumes you have done Part 1 — Setup and Part 2 — First steps, so you have a project folder with AGENTS.md, the decision-log skill, and a first analysis.

Who is in charge here?

After the first real task, the same worries come up, in some form, with almost everyone. They are worth taking seriously.

Control. It does things on my computer. What stops it from touching files it should not?
Visibility. It works behind the scenes. How do I see what it actually did, the code included?
Judgment. It makes choices along the way. How do I know which ones, and which it made without asking me?
Trust. Its results look convincing. How can I put my name on them?

Behind these sits a bigger unease: if the agent does the work, where is my part in it? Is this the robots taking over, one folder at a time?

Our answer is no, but not automatically. The agent is a tool that acts, and a tool that acts needs supervision, the way a research assistant does. The supervision is your part: you set the limits, you decide what must be true before you accept a result, you read what it did, and you decide what goes out under your name. None of this needs coding skills. It needs the habits a careful researcher already has, applied to a new kind of assistant.

This part turns the four worries into four steps: limits (3.1), visibility (3.2), checks and decisions (3.3), and independent review (3.4). It takes about an hour, best done on the World Bank task from Step 2.4.

What you will have at the end of Part 3

A clear picture of what the agent can and cannot touch on your computer
Three ways to see what it did: the scripts, the changes, and every step it took
Checks written into the code, with a pass/fail table after each run
A routine for having the work reviewed by a fresh session
A few new lines in your AGENTS.md

Step 3.1 — Keep the agent in its folder

Files: the folder is a boundary. In Manual mode, the agent asks you before it reads or changes any file outside the project folder. So for files, the folder is a real boundary, and you are asked whenever it wants to cross it.

Commands: the one gap. To run code, the agent runs commands, and a command you approve can reach anywhere on your computer that you can. The safeguard here is you: in Manual mode every command waits for your approval. That is why we stay in Manual mode at the beginning.

Four habits close the gap in practice:

Read the paths in each command before you approve it. If you see a folder that is not your project (C:\Users\you\Desktop, ~/Documents/other-project, a drive letter you don’t recognize), click no and ask: “Why do you need to go outside the project folder?”
Know the legitimate exceptions. Installing Python in Step 1.4 puts tools in your user folder, and downloading data in Step 2.4 reaches the web. These are expected, and the agent tells you before it does them.
Work on copies. Put copies of your data and documents in the project folder; the originals stay where they are. Then even a bad mistake costs nothing.
Write it into AGENTS.md. Add this line under “Safety with my files”:

- Work only inside this project folder. If a step needs anything outside it,
  stop, tell me what and why, and wait for my OK.

For the technically curious: the sandbox (click to open)

On Mac and Linux, Claude Code can also enforce the folder boundary at the level of the operating system, so even an approved command cannot write outside the project. This is called the sandbox, and it is off by default. It does not work on Windows directly. You do not need it for this course; the sandboxing documentation explains it if you are interested.

Step 3.2 — See what it did

The agent does not have to work behind the scenes. There are three windows into its work, and it is good practice to use all three.

1. The scripts. Every data step should live in a script file in code/, so you can read it and rerun it. Your AGENTS.md already asks for numbered scripts. Add two lines that close the loopholes:

- Do every step that reads, changes or creates data in a script in code/,
  never as a one-off command, so that I can read and rerun it.
- Start each script with a short comment in plain language: what it reads,
  what it does, and what it writes.

You do not need to read Python to use this. Click a script name in the chat to open it, then ask: “Explain code/02_clean.py to me paragraph by paragraph, in plain language, as you would to a colleague who does not code.” Then ask about anything that sounds like a choice: “Why drop those rows? What else could you have done?”

2. The changes. After the agent edits files, a Changes card appears with line counts such as +12 -1. Click it to open the diff view: every changed file, with removed lines in red and added lines in green. To comment on a line, hover over it, click the + next to the line number, and type. Send your comments together (Ctrl+Enter on Windows, Cmd+Enter on Mac), and the agent revises.

3. The steps. By default the conversation shows each action as a short summary. To see every file it read and every command it ran, switch the Transcript view to Verbose: press Ctrl+O (it cycles through the views), or use the menu next to the session title. Use this when something surprises you and you want to know how the agent got there.

Step 3.3 — Decide the checks before the work

How much should you check, and when? Save the code, plan checks in advance, or ask to hear only about problems? The answer is all three, in this order.

First, before the work: agree on what must be true. In Plan mode, ask the agent to propose checks for each step. For social science data, the usual ones are:

the number of observations after each step (how many respondents, countries, years);
no duplicate IDs where there should be one row per person or country-year;
after a merge: how many rows matched, and how many did not, from each side;
values in a plausible range (age 18–99, shares between 0 and 100);
missing values per key variable;
totals that should match a published table.

You know your data, so add your own. These are the checks a good research assistant would run anyway; here you make them explicit.

Second, during the work: checks go into the code. Written in the scripts, they run every time, not just once. After each run, the agent saves one short table with every check and a pass or fail.

Third, after the work: report by exception, with two kinds of exception. A failed check means stop and tell me. And any choice you did not specify goes into DECISIONS.md, marked needs-review. This is what the decision-log skill from Step 2.3 is for.

Here is the prompt. Use it at the start of a task, in Plan mode:

Before you start, propose the checks that should hold after each step:
row counts, duplicate IDs, matches in merges, value ranges, missing values.
Wait for my OK on the list.
Then put the checks in the code, so they run every time. After each run,
save a short table of all checks with pass/fail in output/checks.md.
If a check fails, stop and tell me.
Any choice I did not specify goes into DECISIONS.md, marked needs-review.

Try it on the World Bank task from Step 2.4: start a new session and ask the agent to add checks to the existing scripts, using the prompt above.

Report by exception is not enough on its own

“No problems found” from the agent is not evidence that there are no problems. It only means the checks you agreed on passed. So keep doing the hand check from Step 2.4: pick two or three numbers and compare them with the source yourself. It takes five minutes, and it is the one check that does not depend on the agent.

Step 3.4 — A second pair of eyes

The session that did the work is not the best judge of it. It remembers its own reasoning and tends to defend it. A new session has none of that history. It reads the files the way a coauthor would.

When a piece of work is done, start a new session and send:

You are reviewing work that someone else did in this folder.
Read AGENTS.md, DECISIONS.md and the scripts in code/.
Rerun all scripts from scratch, in order. Do the outputs match the ones
already saved? Then list anything that looks wrong, untested or
undocumented, most serious first. Do not change any files.

Read the review the way you read a referee report: some points are serious, some are noise. Take the serious ones back to the agent, in a new session, one at a time.

The new lines for AGENTS.md, in one place

Copy these into your AGENTS.md (or ask the agent to add them):

## Checking

- Work only inside this project folder. If a step needs anything outside it,
  stop, tell me what and why, and wait for my OK.
- Do every step that reads, changes or creates data in a script in code/,
  never as a one-off command, so that I can read and rerun it.
- Start each script with a short comment in plain language: what it reads,
  what it does, and what it writes.
- Put checks in the code (row counts, duplicate IDs, merge matches, value
  ranges, missing values). After each run, save all checks with pass/fail
  in output/checks.md. If a check fails, stop and tell me.
- Any choice I did not specify goes into DECISIONS.md, marked needs-review.

Where to go next

Part 4 — Live demo: how to write a paper in one hour, the last part of this mini course.
Back to the course overview.

--- title: "Part 3 — Staying in Charge: Checking the Agent's Work" subtitle: "Friendly Introduction to Agentic AI for Social Scientists" date: "2026-11-09" --- ::::::: {.hero-section} :::::: {.container} ::: {.hero-title} Part 3 — Staying in Charge ::: ::: {.hero-subtitle} Trust, verification and control: how to set the agent's limits, see what it actually did, and stand behind work you did not do with your own hands ::: :::::: ::::::: ------------------------------------------------------------------------ ## Before you start This is Part 3 of the mini course [Friendly Introduction to Agentic AI for Social Scientists](index.qmd). It assumes you have done [Part 1 — Setup](setup.qmd) and [Part 2 — First steps](first-steps.qmd), so you have a project folder with `AGENTS.md`, the decision-log skill, and a first analysis. ## Who is in charge here? After the first real task, the same worries come up, in some form, with almost everyone. They are worth taking seriously. - **Control.** *It does things on my computer. What stops it from touching files it should not?* - **Visibility.** *It works behind the scenes. How do I see what it actually did, the code included?* - **Judgment.** *It makes choices along the way. How do I know which ones, and which it made without asking me?* - **Trust.** *Its results look convincing. How can I put my name on them?* Behind these sits a bigger unease: if the agent does the work, where is my part in it? Is this the robots taking over, one folder at a time? Our answer is no, but not automatically. The agent is a tool that acts, and a tool that acts needs supervision, the way a research assistant does. The supervision is your part: you set the limits, you decide what must be true before you accept a result, you read what it did, and you decide what goes out under your name. None of this needs coding skills. It needs the habits a careful researcher already has, applied to a new kind of assistant. This part turns the four worries into four steps: **limits** (3.1), **visibility** (3.2), **checks and decisions** (3.3), and **independent review** (3.4). It takes about an hour, best done on the World Bank task from Step 2.4. ::: {.callout-note} ## What you will have at the end of Part 3 - ☐ A clear picture of what the agent can and cannot touch on your computer - ☐ Three ways to see what it did: the scripts, the changes, and every step it took - ☐ Checks written into the code, with a pass/fail table after each run - ☐ A routine for having the work reviewed by a fresh session - ☐ A few new lines in your `AGENTS.md` ::: ------------------------------------------------------------------------ ## Step 3.1 — Keep the agent in its folder **Files: the folder is a boundary.** In **Manual** mode, the agent asks you before it reads or changes any file outside the project folder. So for files, the folder is a real boundary, and you are asked whenever it wants to cross it. **Commands: the one gap.** To run code, the agent runs commands, and a command you approve can reach anywhere on your computer that you can. The safeguard here is you: in Manual mode every command waits for your approval. That is why we stay in Manual mode at the beginning. Four habits close the gap in practice: 1. **Read the paths in each command before you approve it.** If you see a folder that is not your project (`C:\Users\you\Desktop`, `~/Documents/other-project`, a drive letter you don't recognize), click no and ask: *"Why do you need to go outside the project folder?"* 2. **Know the legitimate exceptions.** Installing Python in Step 1.4 puts tools in your user folder, and downloading data in Step 2.4 reaches the web. These are expected, and the agent tells you before it does them. 3. **Work on copies.** Put copies of your data and documents in the project folder; the originals stay where they are. Then even a bad mistake costs nothing. 4. **Write it into `AGENTS.md`.** Add this line under "Safety with my files": ``` text - Work only inside this project folder. If a step needs anything outside it, stop, tell me what and why, and wait for my OK. ``` ::: {.callout-note collapse="true"} ## For the technically curious: the sandbox (click to open) On Mac and Linux, Claude Code can also enforce the folder boundary at the level of the operating system, so even an approved command cannot write outside the project. This is called the **sandbox**, and it is off by default. It does not work on Windows directly. You do not need it for this course; the [sandboxing documentation](https://code.claude.com/docs/en/sandboxing) explains it if you are interested. ::: ------------------------------------------------------------------------ ## Step 3.2 — See what it did The agent does not have to work behind the scenes. There are three windows into its work, and it is good practice to use all three. **1. The scripts.** Every data step should live in a script file in `code/`, so you can read it and rerun it. Your `AGENTS.md` already asks for numbered scripts. Add two lines that close the loopholes: ``` text - Do every step that reads, changes or creates data in a script in code/, never as a one-off command, so that I can read and rerun it. - Start each script with a short comment in plain language: what it reads, what it does, and what it writes. ``` You do not need to read Python to use this. Click a script name in the chat to open it, then ask: *"Explain code/02_clean.py to me paragraph by paragraph, in plain language, as you would to a colleague who does not code."* Then ask about anything that sounds like a choice: *"Why drop those rows? What else could you have done?"* **2. The changes.** After the agent edits files, a **Changes** card appears with line counts such as `+12 -1`. Click it to open the **diff view**: every changed file, with removed lines in red and added lines in green. To comment on a line, hover over it, click the **+** next to the line number, and type. Send your comments together (**Ctrl+Enter** on Windows, **Cmd+Enter** on Mac), and the agent revises. **3. The steps.** By default the conversation shows each action as a short summary. To see every file it read and every command it ran, switch the **Transcript view** to **Verbose**: press **Ctrl+O** (it cycles through the views), or use the menu next to the session title. Use this when something surprises you and you want to know how the agent got there. ------------------------------------------------------------------------ ## Step 3.3 — Decide the checks before the work How much should you check, and when? Save the code, plan checks in advance, or ask to hear only about problems? The answer is all three, in this order. **First, before the work: agree on what must be true.** In Plan mode, ask the agent to propose checks for each step. For social science data, the usual ones are: - the number of observations after each step (how many respondents, countries, years); - no duplicate IDs where there should be one row per person or country-year; - after a merge: how many rows matched, and how many did not, from each side; - values in a plausible range (age 18–99, shares between 0 and 100); - missing values per key variable; - totals that should match a published table. You know your data, so add your own. These are the checks a good research assistant would run anyway; here you make them explicit. **Second, during the work: checks go into the code.** Written in the scripts, they run every time, not just once. After each run, the agent saves one short table with every check and a pass or fail. **Third, after the work: report by exception, with two kinds of exception.** A failed check means stop and tell me. And any choice you did not specify goes into `DECISIONS.md`, marked `needs-review`. This is what the decision-log skill from Step 2.3 is for. Here is the prompt. Use it at the start of a task, in Plan mode: ``` text Before you start, propose the checks that should hold after each step: row counts, duplicate IDs, matches in merges, value ranges, missing values. Wait for my OK on the list. Then put the checks in the code, so they run every time. After each run, save a short table of all checks with pass/fail in output/checks.md. If a check fails, stop and tell me. Any choice I did not specify goes into DECISIONS.md, marked needs-review. ``` **Try it** on the World Bank task from Step 2.4: start a new session and ask the agent to add checks to the existing scripts, using the prompt above. ::: {.callout-warning} ## Report by exception is not enough on its own "No problems found" from the agent is not evidence that there are no problems. It only means the checks you agreed on passed. So keep doing the hand check from Step 2.4: pick two or three numbers and compare them with the source yourself. It takes five minutes, and it is the one check that does not depend on the agent. ::: ------------------------------------------------------------------------ ## Step 3.4 — A second pair of eyes The session that did the work is not the best judge of it. It remembers its own reasoning and tends to defend it. A **new session** has none of that history. It reads the files the way a coauthor would. When a piece of work is done, start a new session and send: ``` text You are reviewing work that someone else did in this folder. Read AGENTS.md, DECISIONS.md and the scripts in code/. Rerun all scripts from scratch, in order. Do the outputs match the ones already saved? Then list anything that looks wrong, untested or undocumented, most serious first. Do not change any files. ``` Read the review the way you read a referee report: some points are serious, some are noise. Take the serious ones back to the agent, in a new session, one at a time. ------------------------------------------------------------------------ ## The new lines for AGENTS.md, in one place Copy these into your `AGENTS.md` (or ask the agent to add them): ``` text ## Checking - Work only inside this project folder. If a step needs anything outside it, stop, tell me what and why, and wait for my OK. - Do every step that reads, changes or creates data in a script in code/, never as a one-off command, so that I can read and rerun it. - Start each script with a short comment in plain language: what it reads, what it does, and what it writes. - Put checks in the code (row counts, duplicate IDs, merge matches, value ranges, missing values). After each run, save all checks with pass/fail in output/checks.md. If a check fails, stop and tell me. - Any choice I did not specify goes into DECISIONS.md, marked needs-review. ``` ------------------------------------------------------------------------ ## Where to go next - [Part 4 — Live demo: how to write a paper in one hour](live-demo.qmd), the last part of this mini course. - Back to the [course overview](index.qmd).