{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2fbd508a-1241-553d-b8ec-4ab82afb95af","task_key":"default--validation--3238","task_revision_id":"1","upstream_id":"3238","short_description":"HellaSwag validation 3238","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to tell your boyfriend you disagree [title] Think of what to say before speaking. [step] When you disagree with someone, you may be quick to remark impulsively and offend the other person. Saying \\\" yeah, i hate (whatever you disagree on) because it's stupid and ugly \\\" is reason for the other person to get offended.\",\"endings\":\"[\\\"Being a quick thinker lets you think about the other person better and realise why you disagree. [substeps] If you consider yourself to be ill-mannered for being in a disagreement, then it's likely that you want to speak politely and respectfully toward the other person.\\\", '[substeps] Instead of saying, \\\" i hate (whatever you disagree on) because it\\\\'s stupid and ugly \\\", try to restate the reasons why you disagree. [title] Consider what you don\\\\'t like about someone.', \\\"Think about your own views before dismissing any on-topic disagreement. [substeps] You don't have to say anything enough without listing how you're feeling, but it can come off as aggressive, intimidating, or rude if someone else argues with you.\\\", 'It might be better to think hard about an intelligent reason that you disagree before sharing. [title] State your opinion clearly while taking ownership.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}