Project 4: AI Coding Agent

Due: Wednesday, December 2 at 11:59pm · Worth: 20 points (+9 extra credit; maximum 29/20)

In Project 3 (docchat) you built a program that loads files into a conversation and answers questions about them. Its tools only read your computer: cat dumps a file, grep searches for text, and ls lists a directory, so the model can diagnose code but cannot edit it.

With tools for creating, editing, and deleting files, plus automatic git commits, docchat can edit a project like a small version of Claude Code or Cursor.

Start

Continue in the docchat repository from Project 3. Fix any missing Project 3 requirements first, then create and switch to a new branch named agent-work. Keep the Project 3 branch unchanged. From the docchat repository root, with its virtual environment active:

$ git switch -c agent-work
$ python -m pip install gitpython

Add gitpython to requirements.txt and the project dependencies in pyproject.toml; the installed library is imported as git. Keep the tools package entry from Project 3 and reinstall after changing the installed command. Keep the console command named chat, as in Project 3.

Learning Objectives

• Extend docchat with tools that edit files, run tests, and record changes in git.
• Trace how an agent uses tool results to choose its next action.

Build the Agent

When your program starts, it should run two checks:

• Check that the current directory is the root of a Git working tree. If it is not, print an error and stop before contacting the model. Use GitPython’s git.Repo('.') or git rev-parse --show-toplevel, and compare the repository root with the current directory. A submodule or linked working tree has a .git file, so requiring a .git directory would reject the previous project’s test repositories.

• Look for an AGENTS.md file in the current directory. If it is present, load it into the conversation using your cat tool.

  Note: AGENTS.md is a standard file that AI agents load when they start working in a repo. It is a README for the agent that lists project-specific instructions. You can find details and examples at https://agents.md/.

Then add the following four tools.

doctests

• It takes a single argument, path.

• It runs the doctests in path with the --verbose flag and returns the output.

  Note: Use --verbose so the LLM sees explicit confirmation that the tests pass.

  Use subprocess.run with an argument list beginning with sys.executable, "-m", "doctest", and "--verbose", followed by the path. Capture and return the output even when tests fail so the agent can use the failure report to fix its code.

write_file

• It takes three arguments: path, contents, and commit_message.

• It opens path and writes contents to it. The output file must be UTF-8 encoded.

• Then it uses the git library in Python to run the equivalent of:

  $ git add <path>
  $ git commit -m "[docchat] <commit_message>"

  Note: Commit every change the agent makes so you can revert its work when it “goes Skynet.”

  

• If the file is a Python file, run your doctests tool on it and return the output.

write_files

• It takes two arguments: files and commit_message.

• files is a list of dictionaries, each with a path key and a contents key. Write each one the same way write_file does.

• After writing all of the files, add and commit them together.

  Note: Implement the shared logic in write_files, then make write_file call it with a one-element list. Shared code is easier to test and change. This is the DRY principle, short for Don’t Repeat Yourself.

  

  Note: Keep write_file because LLMs use a specific tool more reliably than a general one. Its name and narrower schema make the appropriate call easier to select.

rm

• It takes a single argument, path.

• It deletes the path using Python’s os.remove function.

• It supports multiple files at once using globs.

• It creates a commit with the file removed, using the commit message [docchat] rm <path>.

  Test deletion in a disposable git repository containing only your test files. Inspect the glob’s matches before removing them, and handle an empty match without crashing.

Warning: Every one of these tools must validate its path before running, exactly like your read tools did: it must not run on absolute paths or on paths containing ... A bug in a read tool can leak a file to the model; the same bug here can let the agent delete files outside the project.

Repository Organization

Add examples to your README that show the agent in action: creating, modifying, and deleting files, and making git commits along the way. Use shell commands like ls, cat, and git to show the before and after, and format the prompts clearly.

This session shows docchat creating a file and committing it to the git repo:

$ git checkout -b agent-work
Switched to a new branch 'agent-work'
$ ls -a
.git  AGENTS.md  README.md
$ git log --oneline
c21103f (HEAD -> agent-work, master) init commit
$ chat
chat> Create python code that implements the project in README.md
Created the file hello_world.py
chat> ^C
$ ls -a
.git  AGENTS.md  README.md  hello_world.py
$ git log --oneline
3cfb0a6 (HEAD -> agent-work) [docchat] create basic hello world python project
c21103f (master) init commit

Before testing with a model, call each new tool directly in that disposable repository. Write a UTF-8 file, write two files in one call, test both passing and failing Python doctests, and remove two files with a glob. After each mutation, use git log -1 --stat and git status to check the commit and working tree. If git reports an unknown author, complete the name/email configuration in course setup before retrying. Register doctests, write_file, write_files, and rm in the same schema and dispatch registry as your read tools. Test one complete round trip with a fake model reply requesting write_file: the file should appear, one commit should be created, and the next model request should include the tool result with its tool_call_id. The grader checks this path as well as calling each tool directly.

For a first live demonstration in your disposable repository, give the agent a tiny Python file with one failing doctest and ask it to repair the function. Inspect the returned doctest report and the commit diff before trying a larger task.

Grading Rubric

This project is worth 20 points.

• Project 3 structure and required tools remain present (1 point, automated).
• The agent registers the new tools, executes a model-requested write_file, and returns its result to the model (1 point, automated).
• submission.toml identifies a new branch and the Project 3 and submitted commit ids (1 point, automated).
• Startup checks require a git repository and load AGENTS.md through cat (3 points, automated).
• The doctests tool runs verbose doctests on a new fixture (2 points, automated).
• write_file and write_files write UTF-8 and make correctly prefixed git commits (7 points, automated).
• rm handles a glob and commits the deletion (2 points, automated).
• Every new tool uses the shared path-safety check (3 points, automated).

Gradescope runs the 20 automated points inside a temporary git repository. No model call, package installation, or destructive operation reaches your computer or the network. The checks exercise both passing and failing doctests, file contents and commits, startup, and rejected paths.

The 20 base points are fully automatic. The executable behaviors in the rubric are required. The README demonstration, choice of Python library, internal delegation between helpers, and continued live-service operation are practice checks. You may choose any implementation that passes the published behavior tests. Use the same supported synchronous Groq/OpenAI interface, matching schema/parameter names, and dependencies as Project 3, with gitpython also available. Extra credit and scores from earlier assignments are handled on those assignments; this submission does not claim prior-project points.

Extra Credit

• pip_install (+1 point): a tool accepting library_name and invoking pip to install that library. The grader intercepts installation instead of accessing PyPI.
• Retry loop (+3 points): implement ralph_loop(step, check, max_rounds=5) in tools/ralph_loop.py. Each round calls step(), then check(). Return True immediately when check() is true; return False after max_rounds unsuccessful rounds. The callbacks let the grader exercise retry behavior without a live model. You can connect them to agent tool use and doctests in your own program.
• Patching (+5 points): both write_file and write_files accept a unified diff instead of full contents, preserve unaffected lines, and commit the result using the normal commit-message prefix. Add an optional diff argument to write_file, allowing contents to be omitted when a diff is supplied. For write_files, accept a diff key in place of contents in each file dictionary.

Having the agent build an entire Markdown compiler is an ungraded extension; no authorship or branch-history review is needed.

Submit

Keep the source and the project evidence table in submission.toml:

[project]
branch = "agent-work"
base_commit = "REPLACE_WITH_PROJECT_3_COMMIT"
submitted_commit = "REPLACE_WITH_SUBMITTED_COMMIT"

[features]
pip_install = true
ralph_loop = true
patches = true

Use agent-work and distinct 7–40 character hexadecimal commit IDs; omit unclaimed features. Set submitted_commit to your completed code commit, then make one final commit adding its ID to the manifest. The manifest records your development checkpoints; historical authorship is not graded. Commit and sync, then submit your repository and branch through Gradescope. The maximum is 29/20, all automatic.