Lab: Rosetta Stone

Due: Wednesday, December 2 at 11:59pm (moved past Thanksgiving break) · Worth: 8 points

Starter repository: github.com/rtealwitter/lab-formats

Hand the same tweets to another program and a spreadsheet user, with presentation settings a colleague can edit. The supplied handoff.py uses your converters to save JSON, read YAML or TOML settings, and export CSV. Inspect what happens to the records’ commas, quotes, newlines, and numeric like counts along the way. Use the reading as your reference for each format.

The Spider-Man-pointing meme: a Python dict and a JSON string point at each other, each accusing the other of being the real data. They are both right; the only question is who reads it next.

Start

Select Use this template, create lab-formats under your account, and clone it. Set up its Python environment, then install the starter’s dependencies:

$ python -m pip install -r requirements.txt

This installs PyYAML and, before Python 3.11, the tomli fallback for tomllib. Complete the ten functions in lab_formats.py using their doctests:

$ python -m doctest lab_formats.py

Silence means success; failures show Expected and Got. tweets.json, presentation.yaml, presentation.toml, and handoff.py supply ungraded practice data and checks. The driver stops with a helper’s name and expected type if it still returns None.

Records for the Next Program

Open tweets.json and implement these helpers:

from_json: parse JSON text into Python objects.
to_json: return JSON text with two-space indentation and sorted keys (indent=2, sort_keys=True).
roundtrip_json: check whether converting to JSON and back preserves the original object.
count_keys: count only a dict’s top-level keys.

The s in loads and dumps means string; load and dump take open files.

The sweating-guy-with-two-buttons meme: one button says json.load, the other json.loads. It is 2am, you are holding a string, and you are one keystroke from the wrong one.

Save and reload the tweets:

$ python handoff.py --stage json

Compare handoff-output/tweets.json with the input: formatting changes, but values and types should survive. The driver should report three fields in the first record and a successful round trip. Explain why its separate integer-key example fails the equality check: JSON object keys are strings.

Settings for the Reader

Open presentation.yaml and find the report title and username setting. Implement yaml_to_obj with yaml.safe_load; plain load can construct Python objects from a document. Then implement norway_value, which parses country: no and returns that field’s value.

'Change My Mind' meme: a man seated at a folding table behind a sign reading 'YAML is just Python-like JSON,' captioned 'change my mind.'

Bare no becomes False in PyYAML. The supplied config quotes "no" to preserve Norway’s country code as text. Check both cases:

$ python handoff.py --stage yaml

The display should begin Tweet handoff (country: no) and prefix each tweet with @username:. Change show_usernames to false, rerun, and restore it. The separate norway_value result should be False.

Meme: yaml.safe_load('country: no') returns {'country': False}. YAML interpreted the name of a country as a Boolean.

The Same Settings in TOML

presentation.toml contains the same settings. Implement toml_get to parse a string and return the requested top-level value; a [section] creates a nested dict. The standard library’s tomllib reads TOML but does not write it.

Trojan Horse meme: a giant wooden horse labeled 'TOML v1.1' being wheeled through a city gate, with armed soldiers labeled 'JSON' hidden inside its belly.

Compare the display using TOML:

$ python handoff.py --stage toml

With matching settings, it should match the YAML display. TOML requires quotes around strings: country = no is a parse error.

Records for the Spreadsheet

Complete the three CSV helpers using the reading’s CSV text example, which connects csv.DictReader and csv.DictWriter to strings through io.StringIO.

csv_to_rows: read CSV text into a list of dicts keyed by the header.
rows_to_csv: write row dicts as CSV text, using the first row’s key order and LF line endings (lineterminator='\n'). Let the CSV writer quote commas, quotes, and newlines.
json_to_csv_rows: parse a JSON array of flat records; wrap a lone object in a one-row list.

Export the tweets:

$ python handoff.py --stage csv

Inspect handoff-output/tweets.csv in an editor and, if available, a spreadsheet. Check that commas and newlines stay inside their cells. When the driver reloads the CSV, the first likes value becomes the string '2' rather than the integer 2. CSV has no built-in nested values or types; recovering the count requires an explicit conversion. Writing CSV also earns extra credit on Project 2.

Meme: Excel opens your gene-list CSV and helpfully rewrites the gene SEPT2 as a calendar date. The Norway problem has cousins in every corner of the type system.

Run the full handoff with either configuration file:

$ python handoff.py
$ python handoff.py --config toml

Both should export JSON and CSV and display the tweets with the selected settings.

A Larger Handoff

The docchat and agent projects exchange JSON with APIs. For datasets too large to load into memory, a database can select rows on disk; the next reading introduces SQL queries.

Submit

Rerun python -m doctest lab_formats.py and fix any failures. Commit and sync lab_formats.py, then submit your lab-formats repository and branch through Gradescope’s GitHub option. The first lab has the submission steps.