{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"5eb554dd-2947-5105-945b-52cb1185b982","task_key":"default--validation--493","task_revision_id":"2","upstream_id":"493","short_description":"HellaSwag validation 493","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"He starts mowing his lawn casually going over all the grass. His dog comes outside and starts to follow him a little while he's closer to the house. the yard\",\"endings\":\"['is slowly starting to look really really nice, once he is completely finished the yard looks great, nice and smooth.', 'is very neat and clean and very active under the circumstances.', 'gets mowed quicker by the dog.', 'eventually looks happy on the outside and appears to be going pretty happily.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}