benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: Matt Sammon Called it on Twitter: "This performance by Noah Syndergaard is just setting up the inevitable game-winning homer by Conor Gillaspie."
REPLY 1: All you have to do is pick a random mediocre player on the Giants, say he'll be a postseason hero, and you'll probably end up being right

POST 2: Authorities say a Seattle police officer came upon two men in the midst of an altercation and shot one of them as officials were working to clean out a homeless encampment known as The Jungle.
REPLY 2: I guess that "Seattle Police Officer Saves Man From Being Stabbed to Death" just isn't catchy enough for some people.

POST 3: "You brown or Indian shouldn't be here" - Indian man's house trashed with dog poop and hate messages in Southern Colorado
REPLY 3: This was probably done by another sore loser that can't handle that Hillary lost the election trying to make Trump supporters look bad I seriously doubt anybody that voted for Donald Trump had anything to do with this

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

Official source

initial import