{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c5e0b36a-7602-50fd-b3ea-c565d4575db3","task_key":"default--validation--34789","task_revision_id":"1","upstream_id":"34789","short_description":"HellaSwag validation 34789","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to prepare for a school dance (for girls ) [title] Decide if you're going to it or not. [step] School dances can be fun and memorable, but not everybody likes dancing in front of a lot of people. [title] Congratulations! [step] You've decided you're going to the dance.\",\"endings\":\"[\\\"How long does it take to make up your mind? If you know how many people will go with you, you can really stay with that group. If you're unsure, ask! If you're not sure, ask.\\\", 'Now all you have to do is relax relax. Do whatever brings you joy and has your heart racing.', 'If you need a ticket, buy one. Some schools also require you to show them your i.d.', 'Keep that in mind in case your parents and/or siblings are there to see it. [title] In addition, you should see your dance teacher.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}