{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3b24ac62-b87b-589f-9c8c-8f5975ca4e61","task_key":"default--validation--42848","task_revision_id":"1","upstream_id":"42848","short_description":"HellaSwag validation 42848","config":"default","split":"validation","body":"{\"activity_label\":\"Shot put\",\"ctx\":\"A man lifts a ball in the air and holds it against his neck and throws the ball far. The man holds the ball against his neck and twists his back and throws it. the man\",\"endings\":\"['grabs the ball and takes a big step behind him and throws the ball.', 'spins around and throws the ball, lands on his feet and throws it away.', \\\"continues swinging around where he doesn't score but there's people filming him.\\\", 'takes another shot his mouth, then drops the ball on the ground.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}