Lab: More Project Setup

Due: Wednesday, November 4 at 11:59pm (one week after it is assigned) Worth: 8 points

Starter code: github.com/rtealwitter/lab-more-project-setup

This lab adds tests, GitHub Actions, and Python packaging to the LLM chat program from this week’s reading. The resulting chat.py becomes the basis of the docchat project.

Getting the code

Continue from the chat.py you wrote while following the reading. If that file does not work, use the reference chat.py in the starter repository.

1. Fork the starter repository, github.com/rtealwitter/lab-more-project-setup, to your own account with the Fork button.

2. Clone your fork and open the folder in VS Code:

   $ git clone https://github.com/<your-username>/lab-more-project-setup
   $ cd lab-more-project-setup

Testing your tests

So far, most doctests have been provided. When you write your own, you need to check whether they exercise the program’s code.

Code coverage measures which lines Python executes while running the doctests. A line that never runs has not been checked, so add tests that exercise it.

Start by installing three libraries:

$ pip install pytest coverage supply-chain-attack-poc

What they do:

• pytest is the standard testing library for Python; every advanced testing technique beyond plain doctests is built on it.
• coverage measures which lines your tests run.
• supply-chain-attack-poc is a proof-of-concept package that plays the Rick Astley video when installed. pypi.org is the package repository pip downloads from, and anyone can upload a package to it. Installing a package allows its author to run code on your machine, which is one way supply-chain malware reaches developers. By the end of this lab, you will upload your own package.

NOTE: PyPI stands for the Python Package Index, officially pronounced “pie-pee-eye.”

Run the coverage tool on your chat program:

$ coverage run -m pytest chat.py --doctest-modules
================================= test session starts =================================
platform linux -- Python 3.13.5, pytest-9.0.2, pluggy-1.6.0
collected 1 item

chat.py .                                                                       [100%]

================================== 1 passed in 1.26s ==================================

Coverage runs the doctests while measuring them, so fix any failing doctests before interpreting the report.

WARNING: This command creates a file called .coverage. That is a file we do not want in the repo, and you will lose points if it ends up on GitHub, so add it to your .gitignore.

View the results with a report:

$ coverage report -m
Name      Stmts   Miss  Cover   Missing
---------------------------------------
chat.py      24      9    62%   58-66
---------------------------------------
TOTAL        24      9    62%

The example report shows that the doctests ran 62% of the lines in chat.py; your result may differ. Generate an HTML report to identify the covered lines:

$ coverage html
Wrote HTML report to htmlcov/index.html

Click through to chat.py, and the tested lines are green while the untested lines are red:

The Chat class is green because its doctests execute every line. The REPL loop at the bottom is red because no test executes it:

    import readline
    chat = Chat()
    try:
        while True:
            user_input = input('chat> ')
            response = chat.send_message(user_input)
            print(response)
    except (KeyboardInterrupt, EOFError):
        print()

You tested this code manually in class, but the project needs an automatic test for the REPL.

First, refactor the REPL out of the if block and into its own function:

def repl():
    import readline
    chat = Chat()
    try:
        while True:
            user_input = input('chat> ')
            response = chat.send_message(user_input)
            print(response)
    except (KeyboardInterrupt, EOFError):
        print()


if __name__ == '__main__':
    repl()

The untested code is now in a named function that can have doctests. Try writing one before continuing.

Monkey patching

repl calls input, which reads from a keyboard that is unavailable during an automated doctest. Unlike a parameter, this input cannot be supplied directly in the test call.

Monkey patching temporarily replaces a function’s behavior from outside it. Replace the built-in input with a function that supplies scripted input to the REPL:

Open interactive Python and confirm that input works normally:

>>> x = input('chat> ')
chat> hello world
>>> print(x)
hello world

Define your own monkey_input and overwrite input with it:

>>> def monkey_input(prompt):
...     user_input = 'hola mundo'
...     print(f'{prompt}{user_input}')
...     return user_input
>>> input = monkey_input        # this is the monkey patch
>>> x = input('chat> ')
chat> hola mundo
>>> print(x)
hola mundo

After the patch, every call to input runs our version instead of the built-in one.

A monkey_input that always returns the same string would loop forever because the REPL stops only on KeyboardInterrupt. Instead, read a list of scripted inputs and raise KeyboardInterrupt when it is empty:

def monkey_input(prompt, user_inputs=['Hello, I am monkey.', 'Goodbye.']):
    try:
        user_input = user_inputs.pop(0)
        print(f'{prompt}{user_input}')
        return user_input
    except IndexError:
        raise KeyboardInterrupt

Assigning input = monkey_input inside a doctest does not change what repl calls. The input name inside the doctest lives in a different frame from the input that repl uses, so reassigning one does not touch the other. To patch it everywhere, modify the built-in directly through the builtins module:

>>> import builtins
>>> builtins.input = monkey_input

Combine these pieces in the repl docstring:

'''
>>> def monkey_input(prompt, user_inputs=['Hello, I am monkey.', 'Goodbye.']):
...     try:
...         user_input = user_inputs.pop(0)
...         print(f'{prompt}{user_input}')
...         return user_input
...     except IndexError:
...         raise KeyboardInterrupt
>>> import builtins
>>> builtins.input = monkey_input
>>> repl()
'''

Run the doctests and read the real output to fill in what repl prints, and remember to pin temperature=0 somewhere so that output is reproducible. With the test in place, rerun coverage:

$ coverage run -m pytest chat.py --doctest-modules
$ coverage report -m
Name      Stmts   Miss  Cover   Missing
---------------------------------------
chat.py      26      1    96%   79
---------------------------------------
TOTAL        26      1    96%

One line remains uncovered. Reaching 100% coverage can require tests whose value does not justify their maintenance cost:

For this lab, aim for greater than 90% coverage.

API keys in GitHub Actions

If you do not already have a GitHub repository for this project, create one and push your code. Add a GitHub Action that runs your doctests on every push.

The action will fail because the runner has no GROQ_API_KEY, which the doctests need to reach the API.

Do not put the key in the repository. GitHub Secrets stores encrypted values that a workflow can read at run time. Follow GitHub’s documentation and adapt the examples to your code:

1. Store your GROQ_API_KEY as a repository secret: Using secrets in GitHub Actions.
2. Modify your action to load that secret into an environment variable: the same guide, “Using secrets in a workflow”.

Once the doctests pass in the action, add the coverage commands to it as well:

$ coverage run -m pytest chat.py --doctest-modules
$ coverage report -m

This prints your coverage in the action’s log. The action must show greater than 90% coverage; Gradescope checks that you configured this threshold and provides the authoritative grade.

Publishing to PyPI

Follow this tutorial to publish your project to PyPI. Use the sections relevant to these requirements:

• You will write a pyproject.toml that declares a script pointing at your chat.py. Name the script chat, and that becomes the command people run after installing your package.
• A tool called twine does the actual upload.
• PyPI names are global, so pick a unique one, for example docchat-<yourname>.

WARNING: The upload tools create generated files that do not belong in the repository. GitHub’s maintained Python.gitignore includes patterns for them.

Integration tests

Doctests are unit tests: each checks one function or class in isolation. An integration test runs the components together through the same interface a user sees.

Here is a simple integration test for this project:

$ pip3 install <your-library-name>
$ chat <<'EOF'
I am bob.
What is my name?
EOF

The test installs your package from PyPI and runs the chat command on two scripted lines. It checks that the package installs and runs without error, but does not verify the exact reply. When the two piped-in lines run out, input() raises EOFError; that is why your repl catches EOFError alongside KeyboardInterrupt, so the program ends cleanly instead of crashing this test. Add a second GitHub Action, called integration-tests, that runs it.

NOTE: Integration tests for terminal programs can supply text input and inspect text output. Testing a web or mobile interface requires tools that perform interface actions and inspect the result.

Submitting

Resubmit until the tests pass. Your README.md must have:

• A green doctest badge, with the corresponding action showing greater than 90% coverage.
• A green integration-test badge.
• A one-to-two-sentence description of your project.
• A link to your PyPI project page, and that page must show your README (so your badges appear on PyPI too).

Remember that you lose points for any unnecessary files in the repo, so check your .gitignore one more time. Create a root-level submission.toml with the published package URL:

pypi_url = "https://pypi.org/project/your-package-name/"

When both badges are green, commit chat.py, pyproject.toml, README.md, the workflows, and submission.toml, then submit the repository and branch to the Gradescope Programming Assignment. GitHub Actions are preliminary feedback. Gradescope’s instructor-owned tests check the testable REPL structure, deterministic configuration, packaging, CI files, manifest, and repository cleanliness; that result is authoritative. External PyPI availability may be spot-checked separately because the grader itself runs offline.