benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: although it *is* true that we all killed witches and jesus because we were jealous of those super powers.
REPLY 1: I understand that the majority of today's society is fucked, but how does this disgruntled whale relate to the point?

POST 2: That some self-hating Jews have progressivism as their true religion and thus can reconcile BDS support with their Jewish identity doesn't negate the obvious anti-Semitism of BDS.
REPLY 2: Okay, thanks for letting us know that vital information :) Thanks to this article we could all go to sleep knowing there are significantly less antisemites in this world than there were yesterday :D

POST 3: If I was Comcast I would make the cap 3GB like mobile phones so we can live in a innovative society instead of an oppressive one.
REPLY 3: And here I am begging Comcast to come to my neighborhood.... I'd vote for a president who would push national infrastructure upgrades and nothing else at this point.

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

Official source

initial import