{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"47ce2e81-65cf-5420-847c-6126124e655a","task_key":"default--validation--31993","task_revision_id":"1","upstream_id":"31993","short_description":"HellaSwag validation 31993","config":"default","split":"validation","body":"{\"activity_label\":\"Playing badminton\",\"ctx\":\"Then the man wearing the black shirt serves to the boy whose wearing the gold shirt. then the boy who's wearing the white shirt\",\"endings\":\"['serves to the boy who has the gold shirt in his hands.', \\\"serves the serve to the player who's wearing the gold shirt.\\\", \\\"hits the shuttlecock and serves to the girl who's wearing the pink shirt and the game goes on and everyone keeps serving to each other and they all use different shuttlecocks.\\\", 'serves to put the blue balls in the hole.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}