benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: Barry Logan should have offered to help the artists who had their copyrighted work illegally used sue the web developer for millions.
REPLY 1: Shit is easilly defended in a Canadian court if you never made a profit that douche company is using scare tactic like those Russians antivirus

POST 2: I'd laugh my ass off if he nominated the most conservative, backwards, regressive asshole in America just to watch the Republicans block them because they vowed to obstruct.
REPLY 2: Surely nothing like this would ever happen with Hillary in office, because she knows how to get stuff done.

POST 3: If left-wingers expressing their opinions makes you want to vote for Trump again then clearly your political views are purely spite-based and you have no business ever voting until you stop being stupid
REPLY 3: Ahh, it's really *our* fault for pointing out Trumps flaws, pointing out his flaws merely makes other people want to vote for him.

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

Official source

initial import