benchmarks.wiki

benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: Republicans Will Rue Their Immigration Stance: "There are about 250 votes in the House to pass the Senate bill now, but most of those are Democrats, and Speaker John Boehner won't consider a measure opposed by the majority of his caucus".
REPLY 1: Cantor got primaries because he supported immigration reform, so the Republicans are not going to touch it with a 10-foot pole because they can't risk pissing off their racist and xenophobic base.

POST 2: Duke University Dean of Students on two intoxicated individuals engaging in sex: "Assuming it is a male and female, it is the responsibility in the case of the male to gain consent before proceeding with sex"
REPLY 2: #yesallwomen..... apparently you aren't confident enough to be held responsible for your half of a decision?

POST 3: As a guy I get it but seriously unreported rape and rapists who get let go is such a bigger and more serious issue that it's sort of insulting to even bring this up.
REPLY 3: This wouldn't possibly be used out of context or place fear in actual victims from reporting and prosecuting.

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

task_id
559b6dc0-a746-5f1f-ba02-3b939501959b
task_key
bbeh~5fsarc~5ftriples--full--559b6dc0-a746-5f1f-ba02-3b939501959b
task_revision_id
1
config / split
bbeh_sarc_triples / full

Official source

initial import