{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"57aac632-53bc-522c-99be-5e029a925505","task_key":"default--validation--11346","task_revision_id":"1","upstream_id":"11346","short_description":"HellaSwag validation 11346","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"An athlete is shown warming up in front of a pole while several others in the background run around a track. the man\",\"endings\":\"['eventually prepares for his jump and jumps over the pole.', \\\"then begins running down the track on the rod while the audience cheers him on and applause when he's finished.\\\", 'throws the pole in the air as his teammates run around to help him and he fails and falls down.', 'is then shown up close performing several routines including flips and handsprings as well as flipping and falling forward onto the mat.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}