Lecture 9 Notes — Sentiment & Text Analytics
Learning outcomes: LO3, LO4
Companion files: slides L09, notebook Week9_Sentiment_Text_Analytics.ipynb
The review text in our data is synthetic and templated, so accuracies are unrealistically high. The methods and failure modes transfer; the numbers do not.
1. Pre-processing
Lower-case → tokenise → remove punctuation and stop words → optional stemming. Each step discards information. Keep negations such as "not": removing them destroys meaning.
2. Word frequencies
A log-odds ratio between positive and negative reviews separates distinctive words. Most positive: lovely, garden, highly, recommended, friendly. Most negative: uncomfortable, overpriced, wifi, poor, rude.
3. Bag of words and TF-IDF
The document-term matrix here is 15,183 documents × 44 terms (87% zeros). Real corpora have 100,000+ terms and exceed 99% zeros, so sparse storage is essential. TF-IDF rescales counts:
tf-idf(t, d) = tf(t, d) × idf(t), idf(t) = ln((1 + N)/(1 + df(t))) + 1.
Common words get low weight; distinctive words high weight. Bigrams keep short phrases such as "not good".
4. Lexicon-based sentiment
Count positive and negative dictionary words. It needs no training data and is easy to explain. On positive vs negative reviews it is 100% accurate here (Spearman 0.82 with the rating), but it ignores negation and context.
5. A learned classifier
TF-IDF (unigrams and bigrams) into logistic regression reached accuracy and F1 of 1.00. This is a symptom of templated text; real review classifiers typically reach roughly 85–93%. The largest learned weights are an easy sanity check on what the model uses.
6. Hard cases
On eight hand-written hard sentences (negation, mixed opinions):
| Method | Accuracy |
|---|---|
| Plain lexicon | 62.5% |
| Lexicon + negation window | 87.5% |
| TF-IDF + logistic regression | 62.5% |
A simple negation window fixes many cases at no cost. The ML model cannot learn what its training data never contained. Test on hard cases, not only on a random split.
7. Topic modelling
NMF factorises the TF-IDF matrix into topics (word groups) and per-review topic weights. Humans must name topics from their top words. Mean rating by dominant topic ranges from 2.96 to 4.25, because topics here separate praise from complaint vocabulary.
8. Aspect scorecards
Group reviews by the aspects they mention. Reviews mentioning wifi (2.40) or showers (2.43) rate lowest; views and location (3.94) highest. A single score hides why; aspects show what to fix.
9. Spark ML
RegexTokenizer → CountVectorizer → IDF → LogisticRegression runs as one distributed pipeline (accuracy 1.00 on this data, matching scikit-learn). Word count at scale reuses the Week 4 pattern.
10. Risks
Negation and sarcasm; domain shift; mixed-language reviews (Sinhala, Tamil, English); fake or duplicated reviews; privacy of identifiable text.
11. Discussion questions
- Write three sentences that fool both the lexicon and the ML model.
- How would you collect a realistic labelled test set for Sri Lankan hotel reviews?
- Which aspect would you fix first for a hotel with a poor wifi score, and how would you measure success?
Next lecture: social network analytics.