Datasets·AI·human_vs_random_ai_3lens

Human vs random-AI (Eval 17)

Human vs random-AI (3-lens) . PsyProxy quantitative evaluation. The excerpts reflect a diverse range of personal experiences and challenges, primarily revolving around relationships , mental health , and personal growth . Themes of communication and emotional support recur throughout, emphasizing the importance of understanding and empathy in navigating difficult situations, whether in romantic partnerships, friendships, or family dynamics. The tone varies from supportive and informative to candid and reflective, with many texts offering practical advice for improving self-confidence , managing stress, and fostering healthy boundaries. Additionally, there are discussions on societal issues such as financial struggles and the impact of substance abuse , showcasing the complexity of human experiences and the nuanced conversations surrounding them. [Summary on 50 random texts by ChatGPT 4o Mini].

Distribution of Human vs random-AI
1
2
3,014 at floor3,014 at ceiling
6,028
items
1,206
holdout n
Human vs random-AI
target
Binary
kind
15
systems compared
Criterion validity

Reported holdout systems from the verified card

Binary classification uses FVE as the task-primary metric. Secondary columns keep the companion metrics visible so binary, ordinal, regression, and multiclass cards are not compared through one flattened score.

Model-family mix
PsyProxy · 4Baseline · 11
SystemFamilyVariantFVEAUCF1Primary scale
psyproxyPsyProxy — Sociology Lens v0.5 · 1000d
PsyProxypermissive0.9170.9980.979
psyproxyPsyProxy — Behavioral Sciences Lens v0.5 · 1000d
PsyProxypermissive0.8830.9960.972
psyproxyPsyProxy — Technology Lens v0.5 · 800d
PsyProxypermissive0.8810.9960.969
psyproxyPsyProxy — Health Lens v0.9 · 1100d
PsyProxypermissive0.8640.9950.958
baselineDisneyland TripAdvisor reviews binary · via best lens
Baselinepermissive
baselineIMDB movie reviews (ACL) binary · via best lens
Baselinepermissive
baselineSentiment140 tweets binary · via best lens
Baselinepermissive
baselineLIAR fact-check statements ordinal · via best lens
Baselinepermissive
baselineDisneyland TripAdvisor reviews continuous · via best lens
Baselinepermissive
baselineAmazon Video-Games reviews continuous · via best lens
Baselinepermissive
baselineAmazon Video-Games reviews ordinal · via best lens
Baselinepermissive
baselineDouban movie reviews (Chinese) ordinal · via best lens
Baselinepermissive
baselineDruglib drug reviews ordinal · via best lens
Baselinepermissive
baselineAmazon Video-Games reviews binary · via best lens
Baselinepermissive
baselineDruglib drug reviews regression · via best lens
Baselinepermissive