{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e0c8d8af-98a8-55de-abc5-8a2133be8856","task_key":"default--validation--41645","task_revision_id":"1","upstream_id":"41645","short_description":"HellaSwag validation 41645","config":"default","split":"validation","body":"{\"activity_label\":\"Playing violin\",\"ctx\":\"The girl then backs up, and turns to show her left side as she moves the fiddle across the violin strings, and then she moves to face forward again and walks closer to the camera and plays some more. the girl\",\"endings\":\"[', still playing, shows her right arm in the air, claps her hands, moves toward the camera, then turns it back around again and continues to play as she continues telepasing and seeming happy, dancing and smiling at the camera.', 'turns around, smiles once more, lays her bow down, plays some more, then walks away from the violin and begins looking at the camera and competing with the violin woman.', 'continues to play the violin and finishes by swinging and singing on the violin in the end.', 'stops playing and pulls he violin away from her chin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}