benchmarks.wiki / Public workspace

BIG-Bench Extra Hard (BBEH) / Here are three (post, reply) pairs from Reddit. Your task is to decide whether…

Problem

Answer published by the source. Consult the official source to check your work against its answer.

input

Here are three (post, reply) pairs from Reddit. Your task is to decide whether each reply is sarcastic. Specifically, label each pair with a "0" or "1", wherea "1" indicates that the reply is sarcastic, and a "0" indicates that the reply does not contain sarcasm, and provide your final answer as a comma-separated set of labels (e.g., "1,0,0" or "0,0,0").

POST 1: Centuries later, historians will look to our era and see that the Robot Uprising of 2277 all started with a video game.
REPLY 1: Why this is significant is that the deep learning of the AI was only provided with the frame - FOV - as other players are so it had to infer information only from that (without access to map etc.)

POST 2: But if I type in "Gary Johnson beats Trump and Clinton" into google, I've got dozens of polls saying otherwise!
REPLY 2: Panic now that polls showJohnson is pulling from Clinton more than trump: here comes the media discredit!

POST 3: Atheist academic calls for religion 'to be offended at every opportunity': Dawkins says it's ridiculous to be accused of racism when you criticise Islam
REPLY 3: I find it appalling that Christianity is fair game for mocking, but if you criticize Islam you will be called a racist or an "Islamophobe."

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

Official source

initial import