Lab: Analyzing Trump Tweets

Due: Wednesday, October 14 at 11:59pm (one week after it is assigned) Worth: 8 points

Download: lab_tweets.py

This lab has no starter repository or doctests. You’ll write an open-ended program, then Gradescope will inspect its required outputs with instructor-owned tests.

You’ll analyze every tweet Donald Trump sent from 2009 to 2018, and get practice loading datasets stored in JSON files, counting patterns in text, and making a plot in Python. The same dataset powers two well-known projects: an analysis of which tweets Trump wrote himself versus which his staffers sent, and a search engine for all of his tweets.

Your grade comes from instructor-owned Gradescope checks of a short README.md containing a formatted table and bar chart, along with the code that made them. These instructions also appear at the top of lab_tweets.py, so you can work from either place.

Part 0: set up the project

You’ll upload files to GitHub to submit this lab, so begin by creating a repository.

  1. Create a new GitHub repository through the GitHub website.
  2. Clone that repository onto your computer.
  3. Download lab_tweets.py and copy it into your project folder.

Part 1: download the data

The repository https://github.com/bpb27/trump_tweet_data_archive holds an archive of Trump’s tweets.

  1. Download the files named master_*.json.zip, where * is a year. There should be 10 of them, one for each year from 2009 to 2018.
  2. Unzip them into the project folder you made in Part 0. You’ll get a set of files named like master_2009.json.

This particular archive stops in 2018 because its maintainer moved to a newer archive that runs through the present (and includes messages sent on Truth Social). That newer data is a little more work to get at, so we use the older archive here. Using the latest data instead is worth extra credit; see the last section.

Part 2: analyze the data

Modify lab_tweets.py so that it:

  1. Opens each JSON file and loads it with the json library. Each file holds a list of tweets, and if you concatenate every file’s list together you get one list of every tweet Donald Trump ever sent.

  2. Prints the total number of tweets.

  3. Counts how many tweets contain each of these keywords: Obama, Trump, Mexico, Russia, and Fake News.

    Each keyword can appear with many different capitalizations, and your program must count it no matter how it’s written: OBAMA, obama, and ObAmA all count as an occurrence of Obama. Lower-case the tweet text and use the in keyword, as shown in the reading.

  4. Prints the count for each of these words.

A correct program produces these numbers:

len(tweets)= 36307
counts= {'trump': 13924, 'obama': 2712 ... }

Once those match, complete the remaining tasks:

  1. Choose at least 3 more interesting words or phrases of your own to count, and modify your program to display them.
  2. Calculate the percentage of tweets that contain each word (both your new words and the original five).
  3. Display the results in a Markdown table, with every word right-justified and every percent printed to two figures on each side of the decimal, as shown here:
| phrase            | percent of tweets |
| ----------------- | ----------------- |
|              daca | 00.17             |
|         fake news | 00.92             |
|  mainstream media | 00.06             |
|            mexico | 00.55             |
|             obama | 07.47             |
|            russia | 01.13             |
|             trump | 38.35             |
|              wall | 00.91             |

Use f-string formatting to align the text. The following doctests show the available formatting options:

>>> name = "Alice"
>>> f"{name:<10}"
'Alice     '
>>> f"{name:>10}"
'     Alice'
>>> f"{name:^10}"
'  Alice   '
>>> f"{name:*^10}"
'**Alice***'

>>> pi = 3.14159265
>>> f"{pi:.2f}"
'3.14'
>>> f"{pi:.4f}"
'3.1416'
>>> f"{pi:8.2f}"
'    3.14'
>>> f"{pi:08.2f}"
'00003.14'

  1. Plot the results in a bar graph.

    We haven’t covered plotting in class. Use a library’s documentation to learn how to make the graph. The usual choice is matplotlib; this w3schools tutorial provides an introduction. You’re also welcome to ask your favorite AI.

Submission

Write a properly formatted README.md that contains:

  1. the Markdown table your program produced,
  2. the image your program produced, and
  3. a short description (one sentence is fine) of the table and image above.

Make sure your repository also holds:

  1. your modified lab_tweets.py, and
  2. the image file your program saved.

When your repository has lab_tweets.py, the image, and README.md, submit the repository and branch to the Gradescope Programming Assignment. Gradescope’s instructor-owned tests check the analysis structure, required table, explanation, and embedded plot; that result is authoritative. Any GitHub Action is preliminary feedback. Fix, push, and resubmit until the required checks pass.

Extra credit

Two opportunities, worth one point each:

  1. Use the latest data. If you use the current dataset instead of the files in the 2009–2018 archive, you earn +1. Instructions for getting it are under “Can I have the data?” in the Trump Archive FAQ.
  2. Find something else in the data. Make a plot of something interesting that does not use the text key: what time of day does Trump tweet most often? what state does he tweet from most often? Add it to your repository for +1.