Regular Expressions

A scraped contact page contains phone numbers and email addresses mixed into ordinary sentences. We want a list of the contact details and a copy of the document with its phone numbers redacted. Searching for one known number will not find the others. We need to search for their format: a particular arrangement of digits, spaces, and punctuation. A regular expression (a regex) describes such a pattern so we can find, extract, or replace matching text.

You wrote related patterns in the shell. In ls *.log, the * matched any run of characters in a filename. The pattern access-2026-10-1?.log matched one character in the final position of the date. Those shell patterns are called globs. A regex can describe patterns in any text, not only filenames.

Regex has a well-earned reputation for looking like someone fell asleep on the keyboard.

Globs and Regex

In a glob, * matches any run of characters and ? matches exactly one character, so ?.txt matches a.txt but not ab.txt. Regex uses different symbols for the same two ideas.

The translation is:

• The glob ? (any one character) becomes . in regex.
• The glob * (any run of characters) becomes .* in regex.

The shell glob *.txt therefore becomes the regex .*\.txt when matching the whole filename. The filename’s literal dot becomes \., because a bare . means any character in regex. A backslash turns off a metacharacter’s special meaning. Regex also has syntax for patterns such as a digit, three of these, and this or that. These pieces make structured text patterns easier to express than filename wildcards do.

Selecting Contact Files

Before searching the text, we can use globs to choose which saved pages to read. In a new practice folder, create three empty sample files:

touch contact-a.txt contact-b.txt notes.txt
ls contact-?.txt
ls contact-[ab].txt
ls contact-[!b].txt

The first two listings select both contact files. The last selects only contact-a.txt. [ab] matches one character from the set, while [!b] matches one character outside it. A wildcard stays within one path component: *.txt selects files here, and archive/*.txt selects files inside archive.

The shell expands an unquoted glob into filenames before starting the command. Quoting a pattern preserves its literal text, so echo '*.txt' prints *.txt. This also affects a Python program’s arguments: python3 show_args.py *.txt passes each matching filename as a separate entry in sys.argv. The script name itself occupies sys.argv[0], as in the shell reading.

Python can select filenames directly with the standard-library glob module. Create select_contacts.py in the same folder:

import glob

print(sorted(glob.glob('contact-*.txt')))

Run python3 select_contacts.py. It prints ['contact-a.txt', 'contact-b.txt']. glob.glob returns a list of matching paths. sorted gives the report a predictable order. The pattern *.py would also match this script itself, since it is a Python file in the folder being searched.

Searching for a Pattern

Create contacts.py in a practice folder and add the Python blocks below in order. Run python3 contacts.py after each addition. The output beneath each block shows what those new lines print. The completed script contains the Python regex examples for comparison.

A short sentence lets us check where a search matched and what happens when it finds nothing. Python’s regexes live in the built-in re module, whose name abbreviates “regular expression.” re.search(pattern, text) looks for a pattern anywhere inside text. With no metacharacters, the pattern is an ordinary substring:

import re

print(re.search(r'cat', 'the cat sat on the mat'))
print(re.search(r'dog', 'the cat sat on the mat'))

The output is:

<re.Match object; span=(4, 7), match='cat'>
None

When the pattern is found, re.search returns a match object that reports where it matched. span=(4, 7) says the match runs from index 4 up to (but not including) 7, and match='cat' shows the exact text it found. When the pattern is not found, re.search returns None. The second print makes that result visible.

That match-or-None result works directly in an if, because a match object is truthy and None is falsy:

if re.search(r'cat', 'the cat sat'):
    print('found it')

The output is:

found it

Character Classes and Quantifiers

A literal pattern only finds text you could have found with in or str.find. Recognizing an unfamiliar phone number requires a pattern for digits. Try it on a short order number first. Metacharacters stand for a category of character rather than for themselves. The most useful one is \d, which matches any single digit:

print(re.search(r'\d', 'order #42 shipped'))

The output is:

<re.Match object; span=(7, 8), match='4'>

The pattern \d matched the 4 at index 7, the first digit, because re.search stops at the first match. The r in r"\d" marks a raw string, which tells Python to leave the backslash alone instead of reading \d as an escape sequence. Write regex patterns as raw strings to avoid conflicts between Python’s escapes and regex escapes.

The pattern found only the first digit of 42. To extract the whole number, it needs to match several digits. A quantifier specifies how many times the preceding pattern can match. The quantifier + means one or more, so \d+ grabs the longest run of digits:

print(re.search(r'\d+', 'order #42 shipped'))

The output is:

<re.Match object; span=(7, 9), match='42'>

The match is 42, spanning indices 7 to 9, because + kept going as long as it saw digits. The main pieces are:

• Character classes stand for a category: \d a digit, \w a word character (letter, digit, or underscore), \s whitespace, and . any character except a newline by default.
• Custom classes in square brackets match any one character you list: [aeiou] matches a vowel, [a-z] any lowercase letter, and [^0-9] any character that is not a digit (a leading ^ inside the brackets negates the class).
• Quantifiers say how many of the thing before them: + is one or more, * is zero or more, ? is zero or one (optional), {3} is exactly three, and {3,5} is between three and five.
• Anchors pin the match to a position: ^ means the start of the string and $ the end (or just before a final newline). Use re.fullmatch when every character must belong to the match.

Before combining these symbols into the phone pattern, try two small experiments with re.findall, which returns all matches. A custom class can pull every vowel out of a word. The ? quantifier makes a character optional, so one pattern can match two spellings of a word:

print(re.findall(r'[aeiou]', 'regular'))
print(re.findall(r'colou?r', 'color and colour'))

The output is:

['e', 'u', 'a']
['color', 'colour']

The u? says that a u may or may not be present, so colou?r matches both color and colour. The quantifier applies only to the piece immediately before it: in ab+, the b repeats, while (ab)+ repeats the pair. Python’s \d and \w include Unicode characters. Use [0-9] when you specifically need the ten ASCII digits.

Extracting Matches and Groups

re.search finds the first match. re.findall(pattern, text) returns a list of every matching substring. For example, a scraped contact page may contain several phone numbers:

text = 'call 909-621-8000 or 213-555-0199 today'
print(re.findall(r'\d{3}-\d{3}-\d{4}', text))

The output is:

['909-621-8000', '213-555-0199']

Read the pattern left to right: \d{3} is three digits, then a literal -, then \d{3}, another -, and \d{4} for the last four digits. findall returns both US phone numbers as a list.

Often we do not want the whole match, only one piece of it, like the area code. Wrap the piece you want in parentheses to make a group, and findall returns just the group:

print(re.findall(r'(\d{3})-\d{3}-\d{4}', text))

The output is:

['909', '213']

The parentheses around (\d{3}) mark the area code as the part to extract. The rest of the pattern still has to match, but findall returns only the text captured by the group.

With two capturing groups, each result is a tuple, an ordered grouping of the two captured strings:

print(re.findall(r'(\d{3})-(\d{3}-\d{4})', text))

The output is:

[('909', '621-8000'), ('213', '555-0199')]

Adding parentheses can therefore change the shape of the returned data, even when the same phone numbers match. Check whether the next step expects whole strings, one captured field, or tuples of fields.

A regex can also pull simplified email addresses from scraped text. This pattern asks for word characters, an @, more word characters, a dot, and a final run of word characters:

print(re.findall(r'\w+@\w+\.\w+', 'reach me at ada@example.com or grace@navy.mil'))

The output is:

['ada@example.com', 'grace@navy.mil']

The \. matches a literal dot. A bare . would match any character and accept text such as adaXexample. This simplified pattern rejects or misreads some valid email addresses, so use it only when it fits the text you have.

Matching a Whole String

The extraction gives us pieces of a larger document. If someone supplies a phone number in a form, we need a stricter check: the whole field must have the expected shape. Searching within text and checking an entire value are different jobs. re.fullmatch requires the whole string to fit the pattern:

print(bool(re.search(r'cat', 'the cat sat')))
print(bool(re.fullmatch(r'cat', 'the cat sat')))
print(bool(re.fullmatch(r'cat', 'cat')))

The output is:

True
False
True

The search finds cat somewhere in the sentence. The full match rejects the surrounding words and spaces. Use a search to locate data and a full match to check the complete shape of one field.

Try both operations on the same phone field, with and without an extra digit:

phone_pattern = r'\d{3}-\d{3}-\d{4}'
print(bool(re.fullmatch(phone_pattern, '909-621-8000')))
print(bool(re.search(phone_pattern, '909-621-80000')))
print(bool(re.fullmatch(phone_pattern, '909-621-80000')))

The output is:

True
True
False

A search can extract a number-shaped substring from a longer sequence. Our example document has numbers separated by words and spaces. Other input formats need their own boundary rules. The pattern describes this one hyphenated format, not every valid phone number.

Replacing Matches

Use the phone pattern to replace each number in the edited copy. re.sub(pattern, replacement, text) replaces every match with the replacement string and returns the new text. For example, it can redact every phone number in a document:

print(re.sub(r'\d{3}-\d{3}-\d{4}', '[redacted]', text))

The output is:

call [redacted] or [redacted] today

For more examples and details, see the Automate the Boring Stuff regex chapter and the Python re docs.

Alternatives with grep

Sometimes the contact report should keep a line mentioning either a phone or an email. The regex phone|email matches either word. This is called alternation. re.search(r'phone|email', text) can use it in Python.

We can search a saved text file from the terminal too. Save these lines as contact-notes.txt:

phone: 555-123-4567
address: 123 College Road
email: ada@example.com

Run:

grep -E 'phone|email' contact-notes.txt

It prints the phone and email lines. The -E requests extended regular expressions, including the | operator without an extra backslash. Keep the pattern quoted so the shell passes it to grep instead of interpreting its symbols itself. The -i option from the shell reading can be combined with it as grep -Ei to ignore case.

Testing Patterns

Before sharing the redacted document, check both the matches and the text left behind. A missed match leaves a phone number visible. An overly broad match removes unrelated text. You do not need to memorize every metacharacter, and dense regex is difficult to read at a glance.

Build a pattern against real text and watch what it catches. A tester such as regex101.com highlights every match and explains each part of the pattern. Paste in the text you actually have, adjust the pattern until the highlights are exactly the strings you want, and then copy it into your re.findall call. Include near misses as well as successes: a phone number with too few digits, different punctuation, an empty string, and extra text before or after it. Then run those examples with Python’s re module, since regex tools and languages do not all implement the same rules.

Generated Patterns

An LLM can produce a plausible regex that matches the wrong text. Test generated patterns against real examples before using them.

Determining what a pattern can and cannot match is a mathematical problem studied in formal-languages and complexity theory, usually in upper-division computer science courses rather than coding tutorials.

That theory also explains which patterns can be checked efficiently.

Keep the tests alongside the pattern so a later change to the accepted format does not undo the redaction.