Lab: The Messy Repo

Due: Wednesday, October 21 at 11:59pm (one week after it is assigned) Worth: 8 points

This lab uses the commands from the reading to organize a messy folder.

Repositories accumulate obsolete files, ambiguous names, and build output. Shell commands can organize or remove many files at once, making this kind of cleanup repeatable.

You will organize the repository with a few commands, then perform the first setup step for the docchat project.

Download: make_mess.sh, a script that builds your practice repo. You can instead fork the same sandbox at github.com/rtealwitter/lab-shell.

Make an empty folder, step into it, and run the script from the folder above it so the mess lands inside:

$ mkdir messy-repo
$ cd messy-repo
$ bash ../make_mess.sh
Built a messy practice repo in: /home/alice/messy-repo
Start by looking around with:  pwd  and  ls -a

The script only creates files in the folder where you run it. It deletes nothing, so you can rerun it to restore the practice files.

What you’ll practice.

1. Finding your way around a folder tree with pwd, ls, and cd.
2. Moving, copying, deleting, and creating files with cp, mv, rm, and mkdir.
3. Aiming a single command at many files at once with the * and ? wildcards.
4. Steering output with redirection (>, >>) and pipes (|).
5. Hunting through an entire project with grep -rn.
6. Standing up a brand-new project the way every project from here on begins.

Complete the four tasks in order in the same terminal, recording your results as you go.

Get your bearings

Before changing anything, use pwd to print your current location:

$ pwd
/home/alice/messy-repo

Use ls to list the current directory:

$ ls
README.md  cache.tmp  docs       hw1.py   hw2.py  lab02.py  lab04.py   src
build.tmp  debug.log  error.log  hw10.py  hw3.py  lab03.py  notes.txt  webpage

The directory contains homework files, lab files, folders, and temporary files. Add -a to show hidden dot-files. There are none yet, so the only additional names are . (the current directory) and .. (the parent directory):

$ ls -a
.  ..  README.md  build.tmp  cache.tmp  ...

Two of the entries above are folders. Step into one, look, and climb back out:

$ cd docs
$ ls
README_old.txt  archive
$ cd ..

cd docs moves one level down the tree, and cd .. moves back to the parent directory. Display a short file with cat:

$ cat README.md
# project

some old code we keep meaning to clean up.

Before moving files, make backups of anything you cannot afford to lose. cp takes a source and destination and leaves the original untouched:

$ cp lab04.py lab04.py.bak
$ ls lab04.py*
lab04.py  lab04.py.bak
$ rm lab04.py.bak

The example makes a backup, confirms that it exists, and removes it. rm has no undo or trash can, so check the filename before running it.

Task 1: Find the flag

One file in this repository contains a secret string of the form FLAG{...}. Find it and report its file and line number.

grep searches for a pattern. Add -r to search recursively through every subdirectory and -n to print the line number of each match:

$ grep -rn 'FLAG{' .

The single quotes around 'FLAG{' prevent the shell from interpreting the { before grep receives it. The . means the current directory and everything below it, so the command searches every file in the tree.

grep prints each hit as path/to/file:linenumber:the matching line (some versions add a leading ./ to the path). Record the flag’s file and line number for answers.txt.

Task 2: Tidy up

Before moving the homework, compare the two wildcards:

$ ls hw?.py
hw1.py  hw2.py  hw3.py
$ ls hw*.py
hw1.py  hw10.py  hw2.py  hw3.py

? matches one character, so hw?.py matches hw1.py but not hw10.py, because 10 has two characters. * matches any sequence of characters, so hw*.py matches all four files, including hw10.py.

A wildcard can match four files or four thousand. Before using one with mv or rm, run ls with the same pattern and check the list. The next command will act on those files.

Make a directory for the homework and move all four files with one mv command:

$ mkdir homework
$ mv hw*.py homework/
$ ls homework
hw1.py  hw10.py  hw2.py  hw3.py

The .tmp and .log files are build leftovers. Preview the wildcard before using rm:

$ ls *.tmp *.log
build.tmp  cache.tmp  debug.log  error.log

If the list contains anything you need, stop. Otherwise, delete the four files:

$ rm *.tmp *.log

List the remaining files:

$ ls
README.md  docs  homework  lab02.py  lab03.py  lab04.py  notes.txt  src  webpage

The homework is now in its own directory, and the temporary files are gone.

Task 3: Count things

Answer three questions about the repository and record each number for answers.txt.

First, how many lines of Python are sitting at the top of your repo? cat *.py prints those files one after another, and the pipe | feeds that stream straight into wc -l, which counts lines:

$ cat *.py | wc -l
17

Next, how much unfinished work is left in the whole project? Every loose end in this course is marked with a TODO comment, so count them recursively and pipe the result to wc -l:

$ grep -rn TODO . | wc -l
7

Count the Python files at the top level:

$ ls *.py | wc -l
3

There are four Python files under homework/ and more under src/, but ls *.py reports 3 and cat *.py counts only the three lab files at the top level. The * wildcard matches files only in the current directory. In contrast, grep -rn finds all 7 TODOs because -r searches subdirectories.

Task 4: Scaffold the docchat project

The docchat project begins by creating its directory and initializing a git repository:

$ mkdir docchat
$ cd docchat
$ git init
Initialized empty Git repository in /home/alice/messy-repo/docchat/.git/

Calling a language model costs money, so the docchat program needs an API key: a secret string that authorizes the call and bills it to your account. It lives in a file named .env. Create the file with redirection; > sends the output of echo into a new file instead of the screen:

$ echo "GROQ_API_KEY=gsk_your_key_goes_here" > .env

Never commit an API key to git because earlier versions remain in the repository history after later deletion. List .env in .gitignore so git does not track it. Create the file with >, then add a second line with >>, which appends instead of overwriting:

$ echo ".env" > .gitignore
$ echo "__pycache__/" >> .gitignore
$ cat .gitignore
.env
__pycache__/

> creates the file with one line, and >> adds a second without overwriting the first. Git will ignore both the secret key and Python’s cache directory.

Return to the parent repository:

$ cd ..

You will add code to this docchat directory later in the course.

The same commands in Project 3

Project 3 gives a language model tools named ls, cat, and grep. The model uses them to list files, read a file, and search across files before answering a question. You will implement those commands as Python tools for the agent.

Submitting

Record your findings in a file called answers.txt, built with the redirection you just practiced. Use > for the first line and >> for the rest, filling in the real numbers you got:

$ echo "flag location: docs/archive/fieldnotes.txt line 3" > answers.txt
$ echo "python lines at top level: 17" >> answers.txt
$ echo "unfinished markers in the tree: 7" >> answers.txt
$ echo "python files at top level: 3" >> answers.txt
$ cat answers.txt

Use “flag location” and “unfinished markers” rather than the literal search terms. If answers.txt contained FLAG{ or TODO, it would appear in later searches for those patterns.

Then save the three commands from Task 2 as a script called tidy.sh, so the tidy is repeatable:

$ cat tidy.sh
mkdir homework
mv hw*.py homework/
rm *.tmp *.log

Turn the whole thing into a repo and push it:

$ git init
$ git add answers.txt tidy.sh homework
$ git commit -m "tidy the messy repo"

Push to a repository on GitHub and submit its repository and branch to the Gradescope Programming Assignment. The required deliverables are answers.txt and tidy.sh; Gradescope runs tidy.sh in a fresh copy of the messy fixture and checks the answers with instructor-owned tests. That result is authoritative, and any GitHub Action is preliminary feedback. Fix, push, and resubmit until it passes. If a command did not work as expected locally, describe the problem in one sentence in answers.txt.