PsyProxy
Datasets·Customer reviews·06__text_reviews__acl_imdb__binary_sentiment

ACL-IMDB movie-review sentiment

IMDB Movie Reviews (ACL) is a public corpus of 50,000 movie reviews scraped from IMDB, labeled positive or negative sentiment. The sampled texts from the corpus predominantly explore a variety of themes related to film critique , character development , and storytelling quality . Many reviews express strong opinions about the acting performances and directorial choices , often highlighting the disparity between expectations and the final product. Common sentiments include disappointment with sequels or remakes that fail to capture the essence of their predecessors, as well as frustration with clichéd plots and predictable narratives. Additionally, there are discussions surrounding the emotional impact of films, particularly those that tackle serious themes such as family dynamics and social issues , revealing a spectrum of viewer reactions from admiration to outright disdain. Overall, the texts reflect a rich tapestry of personal experiences and critical assessments that engage with the cinematic landscape. [Summary on 50 random texts by ChatGPT 4o Mini].

Distribution of Sentiment (positive vs negative)
1
2
25,000 at floor25,000 at ceiling
50,000
items
9,299
holdout n
Sentiment (positive vs negative)
target
Binary
kind
26
systems compared
Criterion validity

Reported holdout systems from the verified card

Binary classification uses FVE as the task-primary metric. Secondary columns keep the companion metrics visible so binary, ordinal, regression, and multiclass cards are not compared through one flattened score.

Source podium · FVE · 10 families
Gold
OpenAI (Rathje)
0.898
Silver
PsyProxy
0.646
Bronze
LIWC
0.305
Model-family mix
OpenAI / LLM · 3PsyProxy · 4Lexicon · 2Topic model · 2Baseline · 15

Best PsyProxy row is #2 overall among all model families on this card.

SystemFamilyVariantFVEAUCF1Primary scale
llmOpenAI Model gpt-4o-mini
OpenAI / LLMpermissive0.8970.9950.980
llmOpenAI Model gpt-5-nano
OpenAI / LLMpermissive0.8210.9900.958
llmOpenAI Model gpt-4.1-nano
OpenAI / LLMpermissive0.7930.9870.956
psyproxyPsyProxy — Behavioral Sciences Lens v0.5 · 1000d
PsyProxypermissive0.6460.9630.899
psyproxyPsyProxy — Health Lens v0.9 · 1100d
PsyProxypermissive0.6250.9580.895
psyproxyPsyProxy — Social Economics Lens v0.5 · 1000d
PsyProxypermissive0.6110.9550.892
psyproxyPsyProxy — Technology Lens v0.5 · 800d
PsyProxypermissive0.6060.9540.890
lexLinguistic Inquiry and Word Count (LIWC)
Lexiconpermissive0.3050.8510.773
lexValence Aware Dictionary and sEntiment Reasoner (VADER)
Lexiconpermissive0.2350.8130.734
topicHierarchical Dirichlet Process (tomotopy HDP)
Topic modelpermissive0.1320.7290.708
topicBERTopic
Topic modelpermissive0.1310.7330.729
baselineEmpath
Baselinepermissive0.1230.7340.678
baselineTool for the Automatic Analysis of Syntactic Sophistication and Complexity (TAASSC)
Baselinepermissive0.0910.7020.648
baselineTextDescriptives
Baselinepermissive0.0900.7050.646
baselineTool for the Automatic Analysis of Lexical Sophistication (TAALES)
Baselinepermissive0.0670.6780.633
baselineTool for the Automatic Analysis of Cohesion (TAACO)
Baselinepermissive0.0480.6500.608
baselineDisneyland TripAdvisor reviews continuous · via best lens
Baselinepermissive
baselineAmazon Video-Games reviews ordinal · via best lens
Baselinepermissive
baselineAmazon Video-Games reviews continuous · via best lens
Baselinepermissive
baselineDouban movie reviews (Chinese) ordinal · via best lens
Baselinepermissive
baselineAmazon Video-Games reviews binary · via best lens
Baselinepermissive
baselineDruglib drug reviews regression · via best lens
Baselinepermissive
baselineDisneyland TripAdvisor reviews binary · via best lens
Baselinepermissive
baselineDruglib drug reviews ordinal · via best lens
Baselinepermissive
baselineSentiment140 tweets binary · via best lens
Baselinepermissive
baselineLIAR fact-check statements ordinal · via best lens
Baselinepermissive