{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"16335df9-7a49-51ab-95c7-d76837a295c4","task_key":"default--validation--35971","task_revision_id":"1","upstream_id":"35971","short_description":"HellaSwag validation 35971","config":"default","split":"validation","body":"{\"activity_label\":\"Doing crunches\",\"ctx\":\"A young lady is gripping a black and silver punching bag between her legs. once she\",\"endings\":\"['is comfortable in the room, she began punching and kicking the bag.', 'pulls it up, she brings it down exactly as she has demonstrated and clears the bag off the floor.', 'puts her whole hand on the bag, the force of it bounces her back and she walks out of frame.', 'has secured herself on the bag, she begins doing a set of crunches by pulling herself up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}