{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02a89e32-c598-53fa-8b4b-0225d28dc780","task_key":"default--validation--40790","task_revision_id":"1","upstream_id":"40790","short_description":"HellaSwag validation 40790","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"The lady from china has a second turn and also knocks over the bar and looks mad. the second lady from uzbekistan\",\"endings\":\"[\\\"rock has the third turn but isn't successful because the bar is stuck.\\\", 'performs and clears the bar and almost cries.', 'has her turn and then realizes she was supposed to have a turn and try to do the winning one but in the end fell off the bar.', 'has her third turn and she looks angry as she adds another five before leaving.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}