{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"47ccc948-91d5-58ce-9a68-d00c499881ed","task_key":"default--validation--29414","task_revision_id":"1","upstream_id":"29414","short_description":"HellaSwag validation 29414","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"The man then lifts the weight over his head and stands up. The man drops the weight, pumps his fist, and walks off. the man\",\"endings\":\"['lifts the weight one more time, lifts it over his head, dumps it into the circle at the end of the circle and walks away.', 'returns and lifts the weight to his shoulders then over his head.', 'bends and gets on his knees, picks up a bowling ball, then drops it and goes and hits another bowling ball onto the ground behind him.', 'then begins acting funny as someone begins from behind.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}