{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c89875d6-f4f7-5b1f-8eca-d6c8f1f0577e","task_key":"default--validation--48046","task_revision_id":"1","upstream_id":"48046","short_description":"HellaSwag validation 48046","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A man is lifting weights in a weight room, pulling a large barbell up to his chest. He stands shaking before lifting it above his head, trying to hold it in place. he\",\"endings\":\"['stands and lets it fall slowly to the ground.', 'drops the barbell hard onto the ground.', 'drops the barbell and lowers it down messily.', 'then lifts it over his head to let it land on the floor.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}