benchmarks.wiki

benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: Lol but the cure for cancer is being hidden by the government so we can spend more money to buy into this scheme.
REPLY 1: Yeah, that's not how knowledge works, that's ridiculous, knowledge is what is we already known, if we don't know "the cure for cancer" then it's because we haven't discovered, not because we don't have the password.

POST 2: TIL when The state of Virginia sent a bronze statue of George Washington to be displayed outside The National Gallery in London, they also sent a ton of earth to respect Washington's vow that he would never set foot on English soil.
REPLY 2: I would tell the Commonwealth of Virginia to eat a dick, we're going to weld a monocle and teacup to it.

POST 3: Yes I loved the complete lack of depth perception which made avoiding the raid-wiping mechanics incredibly easy.
REPLY 3: Not only visually but lore wise also, it's a huge point in azeroth history, we, mortals, defeated a envoy of the titans that was set to destroy the world, it's like slaping a angel in the face and say, it's our world now, say fuck off to your boss

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

task_id
f93a5d6e-ca9b-5ded-80d5-7fdcdc2b8940
task_key
bbeh~5fsarc~5ftriples--full--f93a5d6e-ca9b-5ded-80d5-7fdcdc2b8940
task_revision_id
1
config / split
bbeh_sarc_triples / full

Official source

initial import