Lab: Analyzing Trump Tweets

Due: Wednesday, October 14 at 11:59pm (one week after it is assigned) Worth: 8 points

This lab has no starter code or doctests. You’ll write an open-ended program from scratch, then Gradescope will inspect its required outputs with instructor-owned tests.

You’ll analyze every tweet Donald Trump sent from 2009 to 2018, and get practice loading datasets stored in JSON files, counting patterns in text, and making a plot in Python. The same dataset powers two well-known projects: an analysis of which tweets Trump wrote himself versus which his staffers sent, and a search engine for all of his tweets.

Your grade comes from instructor-owned Gradescope checks of a short README.md containing a formatted table and bar chart, along with the code that made them. Those checks validate the submitted Python file, table values, and embedded image file. They do not rerun every possible analysis program or judge what the plot depicts, so check that your program produces the submitted results and that the image is a bar graph of those results.

Start

1. Create an empty repository named lab-tweets under your account and clone it.
2. In that repository, create an empty Python file named lab_tweets.py.

Part 1: Download the Data

The repository https://github.com/bpb27/trump_tweet_data_archive holds an archive of Trump’s tweets.

1. Download the files named master_*.json.zip, where * is a year. There should be 10 of them, one for each year from 2009 to 2018.
2. Unzip them into the project folder you made in Part 0. You’ll get a set of files named like master_2009.json.

This particular archive stops in 2018 because its maintainer moved to a newer archive that runs through the present (and includes messages sent on Truth Social). That newer data is a little more work to get at, so we use the older archive here. Using the latest data instead is worth extra credit; see the last section.

Part 2: Analyze the Data

Modify lab_tweets.py so that it:

1. Opens each JSON file and loads it with the json library. Each file holds a list of tweets, and if you concatenate every file’s list together you get one list of every tweet Donald Trump ever sent.

2. Prints the total number of tweets.

3. Counts how many tweets contain each of these keywords: Obama, Trump, Mexico, Russia, and Fake News.

   Each keyword can appear with many different capitalizations, and your program must count it no matter how it’s written: OBAMA, obama, and ObAmA all count as an occurrence of Obama. Lower-case the tweet text and use the in keyword, as shown in the reading.

4. Prints the count for each of these words.

A correct program produces these numbers:

len(tweets)= 36307
counts= {'trump': 13924, 'obama': 2712 ... }

5. Choose at least 3 more interesting words or phrases of your own to count, and modify your program to display them.
6. Calculate the percentage of tweets that contain each word (both your new words and the original five).
7. Display the results in a Markdown table, with every word right-justified and every percent printed to two figures on each side of the decimal, as shown here:

| phrase            | percent of tweets |
| ----------------- | ----------------- |
|              daca | 00.17             |
|         fake news | 00.92             |
|  mainstream media | 00.06             |
|            mexico | 00.55             |
|             obama | 07.47             |
|            russia | 01.13             |
|             trump | 38.35             |
|              wall | 00.91             |

Use f-string formatting to align the text. The following doctests show the available formatting options:

>>> name = "Alice"
>>> f"{name:<10}"
'Alice     '
>>> f"{name:>10}"
'     Alice'
>>> f"{name:^10}"
'  Alice   '
>>> f"{name:*^10}"
'**Alice***'

>>> pi = 3.14159265
>>> f"{pi:.2f}"
'3.14'
>>> f"{pi:.4f}"
'3.1416'
>>> f"{pi:8.2f}"
'    3.14'
>>> f"{pi:08.2f}"
'00003.14'

8. Plot the results in a bar graph.

   We haven’t covered plotting in class. Use a library’s documentation to learn how to make the graph. The usual choice is matplotlib; this w3schools tutorial provides an introduction. You’re also welcome to ask your favorite AI.

Submit

Write a properly formatted README.md that contains:

• the Markdown table your program produced;
• the image your program produced; and
• a short description (one sentence is fine) of the table and image above.

Make sure your repository also holds:

• your modified lab_tweets.py; and
• the image file your program saved.

Commit and sync lab_tweets.py, README.md, and the plot image, then on Gradescope choose GitHub and submit your lab-tweets repository and the branch containing your commit.

Extra Credit

Two opportunities, worth one point each:

• Use the latest data. If you use the current dataset instead of the files in the 2009–2018 archive, you earn +1. Instructions for getting it are under “Can I have the data?” in the Trump Archive FAQ.
• Find something else in the data. Make a plot of something interesting that does not use the text key: what time of day does Trump tweet most often? what state does he tweet from most often? Add it to your repository for +1.

If you use newer data, identify the dataset and its source in README.md; the 2009–2018 percentages above will no longer be the expected values. The instructor reviews the two extra-credit claims; the automatic checks do not award those points.