{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40d00440-5db9-5d90-9c10-1b8cb0043ff9","task_key":"default--validation--12596","task_revision_id":"1","upstream_id":"12596","short_description":"HellaSwag validation 12596","config":"default","split":"validation","body":"{\"activity_label\":\"Snow tubing\",\"ctx\":\"People are pulled up a hill on a tug rope sitting on inter tubes. A family pulls their inter tubes along by hand and the top of the hill. a person\",\"endings\":\"['falls off of the hand and downhill.', 'falls off of the rope on the side along the side.', 'falls down and knocks off the tie.', 'sits in an inter tube at the top of a hill then rides down the slope laughing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}