{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2fdb6ecd-61f7-5ac7-99b7-2cc518ac2d9b","task_key":"default--validation--41603","task_revision_id":"1","upstream_id":"41603","short_description":"HellaSwag validation 41603","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to have a good night routine for school [title] Change into some comfortable clothes! [step] Wear some pajama pants and a loose top , [title] Do your chores. [step] It's very important that you get your chores done. We all know that if your parents expect you to handle some kind of chore and you don't, they won't be very happy with you.\",\"endings\":\"[\\\"[title] Start your homework as soon as you are done with your chores. [step] We all hate it, but it's the number one way to be prepared for school.\\\", 'So take care of your chores! [title] Make sure to let your phone alert you. [step] You should keep it in there and on you at all times.', 'Put everything in pile formation: pots, pans, utensils and dishware and your books. [title] Plan a good (and hygienic) meal in the evening.', \\\"[title] At school, that's it-getting your homework done on time or that stuff. [title] Always hold up the door open to let in more of the sunlight.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}