{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"52340991-993a-5145-a348-3464f2f41ecd","task_key":"default--validation--42973","task_revision_id":"1","upstream_id":"42973","short_description":"HellaSwag validation 42973","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to get started with a course on eliademy [title] Create an account on eliademy.com. [step] You can also simply sign-up with your existing facebook, google or linkedin accounts. [title] Review the dashboard that you see at log-in.\",\"endings\":\"['[step] If it is your first login, you will only see a + create a new course button. [title] Enter the start date, end date, your course code and select a category for your course.', '[step] Similar to your own dashboard, it contains very different graphics, which will generally be on the \\\" all programs \\\" and \\\" systems \\\" page. The dashboard will include some important information about your course.', '[step] Sometimes an asterisk will appear in the bottom toolbar, to the right of the dashboard. This means the course is approved for the classes that are offered.', '[step] Find the \\\" likes \\\" and \\\" dislikes \\\" buttons. These buttons are marked clearly to the left.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}