{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"822475d0-f848-5142-b083-d220da8d818d","task_key":"default--validation--12482","task_revision_id":"1","upstream_id":"12482","short_description":"HellaSwag validation 12482","config":"default","split":"validation","body":"{\"activity_label\":\"Hand car wash\",\"ctx\":\"A white bmw 3 series is in a garage being cleaned by two men wearing safety jackets. once the front\",\"endings\":\"['of the car is cleaned the men put the back of the car in the back of the van could be seen in the garage.', 'is cleaned, the men move towards the back spraying the vehicle and come back to spray the rim and go back to the trunk to get all of the spots.', 'cover is lifted, the rear plate of the car is covered in white mud and a red bucket is shining away.', 'tires are taken off, the tires get stuck on parts of the car and starts to move.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}