{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"71327977-83e3-51be-a5d1-dc81ca4dadaa","task_key":"default--validation--45663","task_revision_id":"1","upstream_id":"45663","short_description":"HellaSwag validation 45663","config":"default","split":"validation","body":"{\"activity_label\":\"Snow tubing\",\"ctx\":\"The other two group members follow them in their tubes as they go down the mountain slope. the group members\",\"endings\":\"['pass a mountain fair or something on the way down.', 'are seen making impressive rafts down the mountain.', 'have difficulty climbing on the hillside as they carry the tubes down the hill.', 'enjoy themselves in the tubes are the go downhill.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}