{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e993609a-071e-57cc-96ea-57193ad36062","task_key":"default--validation--17102","task_revision_id":"1","upstream_id":"17102","short_description":"HellaSwag validation 17102","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to face problems in your life [title] Acknowledge the problem. [step] It can be tempting to stay away from the issue that is causing you problems. However, avoiding the problem does not help solve it.\",\"endings\":\"['It can irritate the individual and lead to trouble for you. In fact, avoiding the problem can aggravate the individual, possibly even the underlying issue, by even sitting away from it.', \\\"Instead, accept that the problem exists and ask yourself some questions about it. What are the consequences of this problem? Who does it involve? [substeps] If you don't think you have a problem, but everyone tells you there is a problem, try to see if there is truth to it.\\\", 'You cannot force it to go away, and it will not help you at all. Instead, think about what you can learn about the problem.', 'By acknowledging your problem, you are making it less likely to happen again. [substeps] You can think, \\\" this is my problem, not my parents\\\\'.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}