Lab: Redacted

Due: Wednesday, November 18 at 11:59pm (one week after it is assigned) Worth: 4 points

This lab practices the regular-expression operations from the reading: re.search to ask is it in here?, re.findall to return every match, and re.sub to replace matches. You will write about a dozen patterns using classes, quantifiers, and anchors, then test them against messy, real-looking text.

Half of these functions extract data such as phone numbers, hashtags, and area codes from text. The other half redact email addresses, Social Security numbers, and all but the last four digits of a card. Both tasks require you to identify the pattern accurately.

The docchat project uses the same patterns. Its grep tool loops over every line of a file and calls re.search(pattern, line) to decide whether to keep it.

Do not write regex in your head, and do not trust the regex an LLM hands you. Regex answers that look plausible can still match the wrong text. Open regex101.com, paste in the text you actually have, and type your pattern while the site highlights every match and explains each piece. Adjust until the highlights are exactly the strings you want, then copy the pattern into your function.

Starter code: github.com/rtealwitter/lab-regex

The starter is a single file, lab_regex.py, holding about a dozen short functions whose bodies are blank. Each has a one-line description and a set of doctests; your job is to write the body so that every doctest passes. Work one function at a time and run the tests from your terminal:

$ python3 -m doctest lab_regex.py

Silence means every test passed. While a test is failing, the command prints the example, what it Expected, and what it Got, so you always know exactly which pattern to fix next. Write regex patterns as raw strings, such as r"\d+", so Python does not interpret their backslashes.

Warm-up: search, findall, sub

The warm-up compares the three operations using \d, which matches a single digit.

• has_digit(text) uses re.search to answer a yes/no question. re.search returns a match object when it finds the pattern and None when it does not; a match object is truthy and None is falsy, so wrapping it in bool(...) turns it into the True/False the doctest wants. This is the exact shape grep uses to decide whether a line matches.
• find_numbers(text) uses re.findall to return a list of every run of digits. The quantifier + (one or more) is what makes \d+ grab the whole number 10 instead of the two separate digits 1 and 0.
• mask_digits(text) uses re.sub to replace every digit with an X and return the new string.

The three functions return different types: a bool, a list, and a string.

🖼️ Meme: the three-way Spider-Man pointing meme, the three Spider-Men labeled re.search, re.findall, and re.sub, all pointing at the same raw string r"\d+".

Extraction

Extraction finds a specific kind of data within a larger page.

find_phone_numbers is the pattern straight from the reading, \d{3}-\d{3}-\d{4}, read left to right as three digits, dash, three digits, dash, four digits. find_urls is a looser one, where https? makes the trailing s optional so a single pattern catches both http:// and https://.

find_hashtags and find_mentions use a capture group to extract part of a match. Match the whole shape, #\w+, but wrap the part you want in parentheses, #(\w+). re.findall then returns the tag without its #. The examples use the course’s recurring tweet dataset:

>>> find_hashtags('Big rally tonight! #MAGA #America #MAGA')
['MAGA', 'America', 'MAGA']

The list keeps duplicates and preserves order. findall reports every match as it appears. area_code uses the same capture-group idea through re.search instead: one phone number goes in, and .group(1) reads out the piece the parentheses marked.

Your mention pattern, @(\w+), will happily “find a handle” inside an email address, because ada@example.com also contains an @ followed by word characters. Your doctests do not include that case, so they will pass even with this error. Build it on regex101 against text that contains both a mention and an email, and watch what lights up.

🖼️ Meme: a triumphant gold miner holding one gleaming nugget over a mountain of dirt, captioned “when re.findall finally returns something that isn’t [].”

Redaction

Redaction uses re.sub with a pattern shaped like the text you want removed. Organizations that handle personal data use it to strip PII from documents before sharing them.

redact_emails replaces the email shape from the reading with the literal string [REDACTED]. redact_ssns matches a US Social Security number, \d{3}-\d{2}-\d{4}, and mind the 2 in the middle, an SSN is not a phone number, then masks it as XXX-XX-XXXX. mask_last_four keeps the last four digits of a sixteen-digit card and replaces the rest with stars.

>>> redact_ssns('SSN 123-45-6789 on file')
'SSN XXX-XX-XXXX on file'

A redactor misses any format its pattern does not match. If your SSN pattern misses numbers typed with spaces instead of dashes, those numbers remain in the copy marked safe to share. Real redaction pipelines therefore use adversarial examples to test the patterns.

🖼️ Meme: a document with a thick black bar over every single line, captioned “my search history, now GDPR-compliant thanks to re.sub.”

Validation

Extraction and redaction ask is the pattern in here somewhere? Validation asks something stricter: is this entire string nothing but the pattern? Use anchors for this distinction. ^ pins the match to the start of the string and $ to the end, so ^...$ matches only when the whole string, start to finish, is the shape you described.

is_valid_email is the one anchored function. Without them, re.search finds a match anywhere. A string like ada@example.com and some junk would look valid because there is an email at the front of it:

>>> is_valid_email('ada@example.com')
True
>>> is_valid_email('ada@example.com and some extra text')
False

The second call must be False. The $ prevents the match from stopping before the end of the string. Login forms and comment boxes use anchored validation to reject malformed input.

Submitting

This lab uses the same loop as every doctest lab in the course. Work until the terminal falls silent:

$ python3 -m doctest lab_regex.py

Silence means every doctest passed; any output names the function, the Expected value, and what it Got, so you always know the next pattern to fix. Then commit and push your work:

$ git add lab_regex.py
$ git commit -m 'complete lab_regex.py'
$ git push

Your fork ships with a GitHub Actions workflow that reruns these doctests on every push. Enable Actions and use its result as preliminary feedback. Once it is green, submit the repository and branch to the Gradescope Programming Assignment. Gradescope runs instructor-owned pytest cases against lab_regex.py, including boundaries absent from the public examples, and its result is authoritative. There is no partial credit, so fix, push, and resubmit until every Gradescope test passes.

You do not need to memorize regex patterns. Build and test patterns against the text they must handle.