benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: Ukraine teammates Yarmolenko and Stepanenko clash during Shaktar Donetsk v Dynamo Kiev, leading to mass brawl.
REPLY 1: For Euro 2016's sake, I hope this just destroys the dressing room for the tournament and they get turned over by everyone in their group.

POST 2: Perhaps she needs cash to support her newfound meth habit - she's not sleeping, she's paranoid, she's hearing things, she thinks massive wads of cash are only "a drop in the bucket" even though she works full time as a nurse; there literally is no other logical explanation.
REPLY 2: A lot of people seem to be suggesting manipulation, but I initially thought PTSD or some other hallucination-enducing condition.

POST 3: ofelia_loves_tseliot wrote: DAE think that we live in a meritocracy and that free market capitalism is an inviolable ahistorical utopia?
REPLY 3: SammyTheKitty wrote: I mean, I don't have any statistics or anything, but I know how people in poverty work because logic or reason or something

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

Official source

initial import