{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8d7cc802-f3e4-59f6-a4e3-9d7554ae2865","task_key":"default--validation--44740","task_revision_id":"1","upstream_id":"44740","short_description":"HellaSwag validation 44740","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to bathe a young child [title] Fill the tub with warm water. [step] Fill the tub about a quarter full. They don't use that much water, and unless they are soiled with dirt and grime, they really don't need that much.\",\"endings\":\"[\\\"You don't need to just fill the tub only halfway full. Fill the other half with very warm water, if you can.\\\", \\\"Make sure the water is not too hot or cold. To match the child's preference, have the child stick his/her hand in some lukewarm water, and ask them if they want it warmer.\\\", '[title] Add a little deicer water to get them used to the feeling of the water. [step] Add more or less water depending on their needs.', 'If you have a dishwasher, fill it to about the length and depth that they need to bathe. [title] Add a few drops of baby shampoo to the water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}