{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3aa9a0c4-2eca-5280-9929-a4a267b526e5","task_key":"default--validation--10738","task_revision_id":"1","upstream_id":"10738","short_description":"HellaSwag validation 10738","config":"default","split":"validation","body":"{\"activity_label\":\"Calf roping\",\"ctx\":\"A cowboy is riding on a horse in a large arena and is swinging his rope. the view\",\"endings\":\"['turns to a view of a canyon with large rocks around.', 'cuts away to him jumping off of the horse and pulling a woman off.', \\\"changes to show a group of men with their horses and they're each holding onto their ropes and doing various things with them like tying and swinging them.\\\", 'changes to see the man fly over the arena.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}