benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: I see they finally nerfed that op Ursa gun, this is the change that inspires me to start playing again
REPLY 1: Don't want to complain after you took the time to do this, but not starting the Y-axis from zero is very misleading... Looks like those benchmark graphs tech companies manipulate to make a 1% look like a 10%.

POST 2: Because the community is full of stupid fucking Care Bears that don't think the wilderness should be dangerous and demand the reward without the risk
REPLY 2: "Owhmygod why pk the HC ironman, i know it's the wilderness but he was probably just doing some slayer, why pk in the wildy?

POST 3: I'm speechless as how she's treating her guest by letting him speak without interruption and say what he'd like to say.
REPLY 3: "Look I respect your point of view, I would hope that I would call my son a thug if he ever did such a thing" Drops mic

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

Official source

initial import