{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"568eea01-745d-5151-8d57-323a8def9958","task_key":"default--validation--39971","task_revision_id":"1","upstream_id":"39971","short_description":"HellaSwag validation 39971","config":"default","split":"validation","body":"{\"activity_label\":\"Surfing\",\"ctx\":\"The man is then shown surfing along the water with others as well as landing in the water. the man\",\"endings\":\"['continues surfing along the water and ends with people standing and paddling around in the end and walking on the beach.', 'is shown on the machine once more and is shown again in slow motion and talking about various experiences.', 'is shown riding a wave and speaking colorful words on his phone.', 'is then shown surfing a long rocky body of water in a competition, with others surfing up against and above.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}