{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25f88f77-2e18-55d9-bfde-b652588b5fdf","task_key":"default--validation--11931","task_revision_id":"1","upstream_id":"11931","short_description":"HellaSwag validation 11931","config":"default","split":"validation","body":"{\"activity_label\":\"Drinking coffee\",\"ctx\":\"Text appears on the screen with regards to coffee possibly being bad for your health. it\",\"endings\":\"['read \\\" coffee sours rapidly, \\\" and titled itself.', \\\"is then mention with text on the screen that it can lead to high blood pressure it is then mentioned that it can hurt your hair growth and that it doesn't have any vital nutrients in it.\\\", 'says \\\" hi martin \\\", along with \\\" exclusive! \\\".', 'also says to keep reading an email.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}