Project 4: AI Coding Agent

In Project 3 (docchat) you built a program that loads files into a conversation and answers questions about them. Its tools only read your computer: cat dumps a file, grep searches for text, and ls lists a directory, so the model can diagnose code but cannot edit it.

With tools for creating, editing, and deleting files, plus automatic git commits, docchat can edit a project like a small version of Claude Code or Cursor.

Due: Wednesday, December 2 at 11:59pm, see the schedule.

Learning objectives

1. extend smaller projects into larger projects
2. understand how AI agents work

Instructions

1. Your project must meet every specification from Project 3 (docchat). If you missed any points there, fix the issue now, or you will lose those points again on this project.

2. Create a new branch in your repository and do all of your work in it. You may not touch the main/master branch you submitted for docchat.

3. Coding tasks.

   When your program starts, it should run two checks:

   • Look for a .git folder in the current directory. If it is not present, print an error and stop.

   • Look for an AGENTS.md file in the current directory. If it is present, load it into the conversation using your cat tool.

     Note: AGENTS.md is a standard file that AI agents load when they start working in a repo. It is a README for the agent that lists project-specific instructions. You can find details and examples at https://agents.md/.

   Then add the following four tools.

   doctests

   • It takes a single argument, path.

   • It runs the doctests in path with the --verbose flag and returns the output.

     Note: Use --verbose so the LLM sees explicit confirmation that the tests pass.

     Note: We have not covered how to run doctests from inside a Python program; any approach is fine.

   write_file

   • It takes three arguments: path, contents, and commit_message.

   • It opens path and writes contents to it. The output file must be UTF-8 encoded.

   • Then it uses the git library in Python to run the equivalent of:

     $ git add <path>
     $ git commit -m "[docchat] <commit_message>"

     Note: Commit every change the agent makes so you can revert its work when it “goes Skynet.”

     

   • If the file is a Python file, run your doctests tool on it and return the output.

   write_files

   • It takes two arguments: files and commit_message.

   • files is a list of dictionaries, each with a path key and a contents key. Write each one the same way write_file does.

   • After writing all of the files, add and commit them together.

     Note: Implement the shared logic in write_files, then make write_file call it with a one-element list. Shared code is easier to test and change. This is the DRY principle, short for Don’t Repeat Yourself.

     

     Note: Keep write_file because LLMs use a specific tool more reliably than a general one. Its name and narrower schema make the appropriate call easier to select.

   rm

   • It takes a single argument, path.

   • It deletes the path using Python’s os.remove function.

   • It supports multiple files at once using globs.

   • It creates a commit with the file removed, using the commit message [docchat] rm <path>.

     Warning: Incorrect rm implementations have caused real data loss:

     • Meta’s director of AI alignment research, Summer Yue, had Claude accidentally delete her inbox through a bad combination of the rm and compact commands: https://www.pcmag.com/news/meta-security-researchers-openclaw-ai-agent-accidentally-deleted-her-emails.
     • Toy Story 2 was almost lost forever to a bad rm command combined with a glob: https://thenextweb.com/news/how-pixars-toy-story-2-was-deleted-twice-once-by-technology-and-again-for-its-own-good.

   Warning: Every one of these tools must validate its path before running, exactly like your read tools did: it must not run on absolute paths or on paths containing ... A bug in a read tool can leak a file to the model; the same bug here can let the agent delete files outside the project.

4. Repository organization.

   Add examples to your README that show the agent in action: creating, modifying, and deleting files, and making git commits along the way. Use shell commands like ls, cat, and git to show the before and after, and format the prompts clearly.

   This session shows docchat creating a file and committing it to the git repo:

   $ git checkout -b agent-work
   Switched to a new branch 'agent-work'
   $ ls -a
   .git  AGENTS.md  README.md
   $ git log --oneline
   c21103f (HEAD -> agent-work, master) init commit
   $ docchat
   chat> Create python code that implements the project in README.md
   Created the file hello_world.py
   chat> ^C
   $ ls -a
   .git  AGENTS.md  README.md  hello_world.py
   $ git log --oneline
   3cfb0a6 (HEAD -> agent-work) [docchat] create basic hello world python project
   c21103f (master) init commit

Grading rubric

This project is worth 20 points.

• Project 3 structure and required tools remain present (2 points, automated).
• submission.toml identifies a new branch and the Project 3 and submitted commit ids (1 point, automated).
• Startup checks require a git repository and load AGENTS.md through cat (3 points, automated).
• The doctests tool runs verbose doctests on a new fixture (2 points, automated).
• write_file and write_files write UTF-8 and make correctly prefixed git commits (5 points, automated).
• rm handles a glob and commits the deletion (2 points, automated).
• Every new tool uses the shared path-safety check (3 points, automated).
• The README demonstration clearly shows creating, changing, deleting, and committing files (2 points, instructor review).

Gradescope runs the 18 automated points inside a temporary git repository. No model call, package installation, or destructive operation reaches your computer or the network.

You may still complete any of the extra credit from previous projects, though extra credit you have already earned will not be counted twice. The extra credit below is new for this project.

Extra credit

• (1 pt) A pip_install tool. It takes a single parameter, library_name, and runs the pip3 install <library_name> command.

  Warning: A PyPI package can run arbitrary code during installation, so an agent with this tool could execute malicious code on your computer. The popular LiteLLM package recently suffered a supply-chain attack:

  • https://docs.litellm.ai/blog/security-hardening-april-2026
  • https://www.trendmicro.com/en_us/research/26/c/inside-litellm-supply-chain-compromise.html
  • https://console.groq.com/docs/litellm

  Open question: how would you prove that a piece of code is “safe”?

• (3 pts) The Ralph Wiggum loop. Whenever your doctests fail, force the agent to do another round of tool use, and only leave the loop once the doctests pass. You may leave the loop always on, or make it something you can turn on and off.

  Note: Anthropic recently renamed their Ralph Wiggum implementation to just “Ralph” for copyright reasons. You can follow along in their GitHub repo: https://github.com/anthropics/claude-plugins-official/commit/44328beed48874d8e00da6c4ca5daaa5f0f3183c.

• (5 pts) Have your agent complete the markdown compiler on its own. To claim this, include in your README a link to a branch of your markdown compiler project that contains only commits made by your AI agent. The easiest approach is to start a brand-new repo rather than reuse your existing one.

  Hint: The free Groq API’s Llama models have tight token limits and may struggle with this task. A current GPT or Claude Opus model generally performs better. An OpenRouter API key lets you experiment across models. This task should cost less than $0.10 with a current high-end model; $10 of credit leaves room to experiment.

• (5 pts) Let your agent update files instead of only overwriting them. Change write_file / write_files so that the contents parameter becomes optional and a diff parameter can be given instead. (A diff is the standard format for describing the changes to a file; applying one to produce the updated file is called patching the file.)

  Even current LLMs often generate invalid diffs because they are unreliable at identifying exact line positions. Standard diff and patch tools reject that output. Any workaround is acceptable. One option is wiggle, a tool built for applying these “broken” LLM-generated diffs.

  Updating only changed lines uses fewer tokens and avoids regenerating code that the LLM might accidentally alter, such as by dropping a ) that should match a (.

Submission

Push your work, then submit the new branch to the Project 4 programming assignment on Gradescope. Add a submission.toml file at the repository root:

[project]
branch = "agent-work"
base_commit = "abcdef0"       # commit submitted for Project 3
submitted_commit = "1234567"  # commit selected for Project 4

[features]
pip_install = false
ralph_loop = false
markdown_agent = false  # or the GitHub branch URL when claimed
patches = false

Set only the extra-credit features you completed. For markdown_agent, replace false with the GitHub /tree/BRANCH URL containing the agent-only commits. Unclaimed extra-credit tests are skipped.

In the Gradescope comments, submit a one-to-two-sentence explanation of what you believe your grade should be:

• If you completed any extra credit, say so.
• Describe any part of your project that does not work; this may earn you more lenient grading.
• If you are requesting points back from a previous project, you must say so explicitly.