{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fc6e8b55-33cf-589a-bace-9958a1a8bb25","task_key":"default--validation--8872","task_revision_id":"1","upstream_id":"8872","short_description":"HellaSwag validation 8872","config":"default","split":"validation","body":"{\"activity_label\":\"Baton twirling\",\"ctx\":\"The music begins and the girls do their routine of dancing, jumping, making different line formations and baton twirling as the crowd cheers them on. the group\",\"endings\":\"['ends their routine as they dramatically leave the gym one by one.', 'then moves around the ground doing swaying motions.', 'continues individually performing and ends without showing how the intro intro came to be.', 'finishes their routine and the instructor throws their sticks down in the sand.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}