{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cc75fe9a-bf48-5e45-b70c-58123735e300","task_key":"default--validation--26329","task_revision_id":"1","upstream_id":"26329","short_description":"HellaSwag validation 26329","config":"default","split":"validation","body":"{\"activity_label\":\"Croquet\",\"ctx\":\"A man is seen hosting a news segment and leads into a group of people sitting with a dog. The men shake hands and begin playing a game of crochet. they\",\"endings\":\"['are then shown spinning the ball around a pole and moving their hands up and down.', 'hit the stick against the ball and the dog chops it around.', 'hit where the ball landed and the dog chases after it with a frisbee.', 'speak to the camera while continuing to play and others commenting on the game.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}