benchmarks.wiki

benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: And the iPad was made in CHINA and Apple contributed money to the CLINTON FOUNDATION so, well, as you can see there is SMOKE and a FIRE
REPLY 1: Am I wrong in saying that every time I receive an email there is a copy on the senders machine and every time I send an email there is a copy on the recipients machine?

POST 2: "The EULA will not be updated with these allowances; instead, they will be a part of a larger document, the Commercial Use Guidelines, which defines acceptable commercial use of the Minecraft name, brand and assets, including Minecraft servers."
REPLY 2: So, we have to keep track of twitter accounts, blogs, reddit posts, and now, a customer service portal to know the full extended EULA?

POST 3: Devils advocate: Maybe he means his ex literally set him on fire and he looks like Freddy Kruger now.
REPLY 3: I look at them and see nothing but worthless cows but for some reason they don't appreciate a nice guy like me.

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

task_id
3a15c801-66ca-5ac7-ba09-5e0061616aa8
task_key
bbeh~5fsarc~5ftriples--full--3a15c801-66ca-5ac7-ba09-5e0061616aa8
task_revision_id
1
config / split
bbeh_sarc_triples / full

Official source

initial import