{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ef4f08f9-afb4-5fb3-bfe4-eaedcbc2cc78","task_key":"default--validation--15716","task_revision_id":"1","upstream_id":"15716","short_description":"HellaSwag validation 15716","config":"default","split":"validation","body":"{\"activity_label\":\"Doing crunches\",\"ctx\":\"This man is standing with his hands on his hips and he explains what exercises he will be doing throughout the video. first he\",\"endings\":\"['shows viewers how to properly do a crunch and his feet are apart, arms on his chest, and each time he goes up the elbows touch his legs.', 'adjusts the shelf to his right so that it is more comfortable and he can use it to balance on.', 'puts some man stilts on and he clips the stilts to his legs.', 'shows the correct form of the fitness machine, where you can begin by walking down the white line with ankles and neck rests facing upwards.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}