AI & Automation
Russian Troll Twitter Activity Classifier
Text-as-data analysis of IRA troll tweets: dictionary methods, sentiment/emotion profiling, and a Naive Bayes classifier that outperformed a classification tree (71.9% vs. 50% accuracy).
Role: Analyst (Columbia SIPA — Text as Data)
Situation
Internet Research Agency (IRA) troll accounts used distinct left- and right-coded personas to influence U.S. political conversation. Classifying those personas and measuring how they talked about elections is applied NLP evidence for disinformation analysis.
Decision question
Text-as-data analysis of IRA troll tweets: dictionary methods, sentiment/emotion profiling, and a Naive Bayes classifier that outperformed a classification tree (71.9% vs. 50% accuracy).
Data and constraints
See method and results below for sample, coverage, and documented limits.
Method
Built an elections keyword dictionary in quanteda, compared keyword use by LeftTroll vs. RightTroll accounts over time, and ran sentiment, profanity, and emotion profiling. Trained Naive Bayes and classification-tree models on tweet text to distinguish the two troll categories, then compared accuracy, precision, recall, and F1.
Tools
- R
- quanteda
- sentimentr
- caret
- rpart
- Naive Bayes
- ggplot2
Deliverables
- 17-page knitted analysis (March 2025)
- Dictionary, sentiment, and classifier comparison write-up
Outcome / decision value
- RightTroll accounts used election keywords more, with a spike in late 2016 around the U.S. presidential election.
- LeftTroll tweets had higher average sentiment and more profanity; RightTroll tweets had slightly negative average sentiment.
- Naive Bayes accuracy 71.9% (precision 71.6%, recall 75.0%, F1 73.2%) versus 50.0% for the classification tree.