benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: Complain about how you said "Well, that's the The International then" and make fun of you until new hatz
REPLY 1: Let's wait for the inevitable C9 disband and possible Secret disband, this year's post-TI roster changes will be the best.

POST 2: so when are the criminal charges for perpetrating this massive fraud scheme going to be handed down?
REPLY 2: Good, I hope they win, fraudulent crony banksters won't go to jail, but hopefully they will get hit where it hurts

POST 3: There should be a "friendly horn" in cars that makes a little chirp or something to thank other drivers.
REPLY 3: Horns and other warning devices You must not use the horn or any other warning device unless: Road rule in NSW say you can't use a horn unless You need to warn other road users that your vehicle is approaching You need to warn animals to get off the road The horn is being used as part of an anti-theft or alcohol interlock device fitted to your vehicle.

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

Official source

initial import