benchmarks.wiki

benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: If all of this hits live at once someday in 2016 I'm so coming back to play, good work BBurness and DB team :) Wish you a good christmas and well deserved rest.
REPLY 1: Yay, GK on its way to become Fracture-tier weapon If it goes on like this Vulkan will become the best gun for sniping over long distances

POST 2: Libertarian says he met a Thai sweatshop worker, worker was grateful to his captain of industry for creating his job.
REPLY 2: I honestly wouldn't doubt that, with a lot of sweatshop workers it's either a sweatshop or starvation, but where libertarians and I differ is I see this as an injustice and a sign that Thailand needs some serious economic reform.

POST 3: Man Who Recently Moved To Dallas Shot While Taking Pictures Of Snow: "He survived bombings but growing fears over terrorism groups like ISIS forced him to move somewhere safe."
REPLY 3: He really should have conducted more research before deciding on Texas as the safest destination in the US for a practicing Muslim.

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

task_id
141d8283-eaa4-52ba-b235-1aeab7b86af9
task_key
bbeh~5fsarc~5ftriples--full--141d8283-eaa4-52ba-b235-1aeab7b86af9
task_revision_id
1
config / split
bbeh_sarc_triples / full

Official source

initial import