benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: North Beach: -several bars right on grant and green, Tupelo has live music every night, and eat a slice of pizza at golden boy...also, rogue ale house for their beer selection, and other corner is bar with great drink specials and good food called the square Soma: -tempest (my favorite bar in city) -louis (owned by same people as tempest) -city beer store (if you love beer) -bloodhound GG park: I actually gonna give to recs that are right at beginning of park in Cole valley; -kezar pub (great sports bar, get the chicken wings with bulleit bourbon sauce) -finnegans wake (they for ping pong table in backyard)
REPLY 1: Gee, if only I had the app that you're spamming on other subreddits, then I could figure out what to do and where to go.

POST 2: As soon as straight people stop throwing their sexuality in our faces and telling us we should be straight we'll get right on that.
REPLY 2: God, imagine if heterosexuality was actively promoted on every billboard, in every film and in every school playground.

POST 3: Considering the fact that we were told last week that it's coming in June, I am pretty sure this is a typo.
REPLY 3: Probably gonna get a big balancing update that will nerf my favorite characters and buff all the ones who give me trouble

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

Official source

initial import