{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"decdaec0-1625-5fe4-a9ae-96e7aef313fc","task_key":"default--validation--30638","task_revision_id":"1","upstream_id":"30638","short_description":"HellaSwag validation 30638","config":"default","split":"validation","body":"{\"activity_label\":\"Putting in contact lenses\",\"ctx\":\"Another news anchor also talks about the same lenses and how it has become a dangerous trend among teenagers. the lady news anchor\",\"endings\":\"['comes back with the male news anchor to continue the program.', 'holds on to the camera again to talk about it, and the really interesting more interesting reporter is also briefly talking about it.', 'uses various methods to apply a contact lens.', 'on court wearing purple sunglasses talks intermittently about it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}