benchmarks.wiki

benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: He's really just a creeper you can tell by the look on her face but he must be her boss or something so she tries to act happy that he's bringing her coffee.
REPLY 1: She might think the camera/phone is a gun, and that half second before her realization is spent in complete and utter horror of death.

POST 2: There's this weird theory that the pope is like, leader of the Catholic Church and can interpret what God means
REPLY 2: In Germany, there was a wacky conspiracy theory in the early nineties: that after the end of the cold war and the collapse of the Soviet Union, the NSA had re-calibrated their facilities here in Germany to now spy on ordinary citizens, business leaders, and western politicians.

POST 3: The investigators are investigating this case as domestic terrorism, even though there is nothing hinting at this being terrorism.
REPLY 3: Every other local news station and could try music station around here is really throwing the word "terrorism" around.. Can't wait to see the response

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

task_id
50fcca0e-86d8-543d-973d-7f63929ee878
task_key
bbeh~5fsarc~5ftriples--full--50fcca0e-86d8-543d-973d-7f63929ee878
task_revision_id
1
config / split
bbeh_sarc_triples / full

Official source

initial import