benchmarks.wiki

benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: I think it would upset me more if a person *looks* like a different gender than the one on the door than if they *are* a different gender.
REPLY 1: I am female to male transgender, I can confirm that I went through the difficult process of transitioning just so that I can stare at guy's dicks in the bathroom.

POST 2: Let us have a moment of silence for Myth tomes, and all the memories we shared: * 300 weekly myth cap * Only dropped from AK * waiting 2 hours to get into AK with a full party * Deciding "fuck getting the i90 chest piece, imma upgrade my Relic to +1" good times
REPLY 2: I'm really, -really- tired of tomestones and wish they'd just go away I better not see a single tomestone after the expansion

POST 3: But the engine of electro cars is better, they use energy better than normal cars, and its always better when the gases are not in populated areas (citys)
REPLY 3: It's a good job internal combustion engines and the oil drilling that they require is so environmentally benign then isn't it?

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

task_id
d755477d-f610-5ced-8c89-eb0b11c2c3b1
task_key
bbeh~5fsarc~5ftriples--full--d755477d-f610-5ced-8c89-eb0b11c2c3b1
task_revision_id
1
config / split
bbeh_sarc_triples / full

Official source

initial import