Lab: The Messy Repo

Due: Wednesday, October 21 at 11:59pm (one week after it is assigned) Worth: 8 points

Repositories accumulate obsolete files, ambiguous names, and build output. Shell commands can organize or remove many files at once, making this kind of cleanup repeatable.

You will organize the repository with a few commands, then perform the first setup step for the docchat project.

Starter repository: github.com/rtealwitter/lab-shell

Start

Select Use this template to create an independent lab-shell repository under your account, then clone that copy. It contains make_mess.sh, the script that builds the practice files.

From the repository root, make an empty work folder, step into it, and run the script from the folder above it so the mess lands inside:

$ mkdir messy-repo
$ cd messy-repo
$ bash ../make_mess.sh
Built a messy practice repo in: /home/alice/messy-repo
Start by looking around with:  pwd  and  ls -a

The script only creates files in the folder where you run it. It deletes nothing, so you can rerun it to restore the practice files.

What you’ll practice.

1. Finding your way around a folder tree with pwd, ls, and cd.
2. Moving, copying, deleting, and creating files with cp, mv, rm, and mkdir.
3. Aiming a single command at many files at once with the * and ? wildcards.
4. Steering output with redirection (>, >>) and pipes (|).
5. Hunting through an entire project with grep -rn.
6. Standing up a brand-new project the way every project from here on begins.

Complete the four tasks in order in the same terminal, recording your results as you go.

Get Your Bearings

Before changing anything, use pwd to print your current location:

$ pwd
/home/alice/messy-repo

Use ls to list the current directory:

$ ls
README.md  cache.tmp  docs       hw1.py   hw2.py  lab02.py  lab04.py   src
build.tmp  debug.log  error.log  hw10.py  hw3.py  lab03.py  notes.txt  webpage

The directory contains homework files, lab files, folders, and temporary files. Add -a to show hidden dot-files. There are none yet, so the only additional names are . (the current directory) and .. (the parent directory):

$ ls -a
.  ..  README.md  build.tmp  cache.tmp  ...

Two of the entries above are folders. Step into one, look, and climb back out:

$ cd docs
$ ls
README_old.txt  archive
$ cd ..

cd docs moves one level down the tree, and cd .. moves back to the parent directory. Display a short file with cat:

$ cat README.md
# project

some old code we keep meaning to clean up.

Before moving files, make backups of anything you cannot afford to lose. cp takes a source and destination and leaves the original untouched:

$ cp lab04.py lab04.py.bak
$ ls lab04.py*
lab04.py  lab04.py.bak
$ rm lab04.py.bak

The example makes a backup, confirms that it exists, and removes it. rm has no undo or trash can, so check the filename before running it.

Task 1: Find the Flag

One file in this repository contains a secret string of the form FLAG{...}. Find it and report its file and line number.

grep searches for a pattern. Add -r to search recursively through every subdirectory and -n to print the line number of each match:

$ grep -rn 'FLAG{' .

The single quotes around 'FLAG{' prevent the shell from interpreting the { before grep receives it. The . means the current directory and everything below it, so the command searches every file in the tree.

grep prints each hit as path/to/file:linenumber:the matching line (some versions add a leading ./ to the path). Record the flag’s file and line number for answers.txt.

Task 2: Tidy Up

Before moving the homework, compare the two wildcards:

$ ls hw?.py
hw1.py  hw2.py  hw3.py
$ ls hw*.py
hw1.py  hw10.py  hw2.py  hw3.py

? matches one character, so hw?.py matches hw1.py but not hw10.py, because 10 has two characters. * matches any sequence of characters, so hw*.py matches all four files, including hw10.py.

A wildcard can match four files or four thousand. Before using one with mv or rm, run ls with the same pattern and check the list. The next command will act on those files.

Make a directory for the homework and move all four files with one mv command:

$ mkdir homework
$ mv hw*.py homework/
$ ls homework
hw1.py  hw10.py  hw2.py  hw3.py

The .tmp and .log files are build leftovers. Preview the wildcard before using rm:

$ ls *.tmp *.log
build.tmp  cache.tmp  debug.log  error.log

If the list contains anything you need, stop. Otherwise, delete the four files:

$ rm *.tmp *.log

List the remaining files:

$ ls
README.md  docs  homework  lab02.py  lab03.py  lab04.py  notes.txt  src  webpage

The homework is now in its own directory, and the temporary files are gone.

Task 3: Count Things

Answer three questions about the repository and record each number for answers.txt.

First, how many lines of Python are sitting at the top of your repo? cat *.py prints those files one after another, and the pipe | feeds that stream straight into wc -l, which counts lines:

$ cat *.py | wc -l
17

Next, how much unfinished work is left in the whole project? Every loose end in this course is marked with a TODO comment, so count them recursively and pipe the result to wc -l:

$ grep -rn TODO . | wc -l
7

Count the Python files at the top level:

$ ls *.py | wc -l
3

There are four Python files under homework/ and more under src/, but ls *.py reports 3 and cat *.py counts only the three lab files at the top level. The * wildcard matches files only in the current directory. In contrast, grep -rn finds all 7 TODOs because -r searches subdirectories.

Task 4: Rehearse the docchat Scaffold

The docchat project will begin by creating a directory and initializing a git repository. Rehearse that setup in a separate sibling folder, not inside your lab-shell repository:

$ cd ../..
$ mkdir docchat-practice
$ cd docchat-practice
$ git init
Initialized empty Git repository in /home/alice/csci40/docchat-practice/.git/

Calling a language model costs money, so the docchat program needs an API key: a secret string that authorizes the call and bills it to your account. It lives in a file named .env. Create the file with redirection; > sends the output of echo into a new file instead of the screen:

$ echo "GROQ_API_KEY=gsk_your_key_goes_here" > .env

Never commit an API key to git because earlier versions remain in the repository history after later deletion. List .env in .gitignore so git does not track it. Create the file with >, then add a second line with >>, which appends instead of overwriting:

$ echo ".env" > .gitignore
$ echo "__pycache__/" >> .gitignore
$ cat .gitignore
.env
__pycache__/

> creates the file with one line, and >> adds a second without overwriting the first. Git will ignore both the secret key and Python’s cache directory.

Return to the messy-repo work folder inside your assignment repository:

$ cd ../lab-shell/messy-repo

The Same Commands in Project 3

Project 3 gives a language model tools named ls, cat, and grep. The model uses them to list files, read a file, and search across files before answering a question. You will implement those commands as Python tools for the agent.

Submit

Record your findings in answers.txt, using > for the first line and >> for the rest. Label these four answers clearly:

• the flag’s file path and line number;
• the number of Python lines in the top-level folder;
• the number of unfinished markers in the full tree;
• the number of top-level Python files.

Use “flag location” and “unfinished markers” rather than the literal search terms. If answers.txt contained FLAG{ or TODO, it would appear in later searches for those patterns.

Then save the three commands you wrote in Task 2 as tidy.sh, in the same order, so the tidy is repeatable.

The messy-repo folder is already inside your lab-shell assignment repository, so do not run git init here or create a nested repository.

The required deliverables are answers.txt and tidy.sh. If a command did not work as expected locally, describe the problem in one sentence in answers.txt.

Commit and sync answers.txt and tidy.sh, then on Gradescope choose GitHub and submit your lab-shell repository and the branch containing your commit.