{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0661dd88-9c58-52b7-8bd0-6309fbf82b4b","task_key":"default--validation--8822","task_revision_id":"1","upstream_id":"8822","short_description":"HellaSwag validation 8822","config":"default","split":"validation","body":"{\"activity_label\":\"Polishing forniture\",\"ctx\":\"The chemical is placed at the sit of the wooden chair. The person scrubs the wooden chair with a brush. The person shows the brush bristles now browned. the person\",\"endings\":\"['rinses the wooden chair with the garden hose.', 'adjusts a light blue soap product on the item and rubs the item.', 'shines the metal handle of the metal polish brush on the shoe from the two round item placed on top of the chair.', 'set the stool on the side of the chair to show the underside of the chair.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}