{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51c045fa-6204-5979-9be7-1db86e6638f3","task_key":"default--validation--18099","task_revision_id":"1","upstream_id":"18099","short_description":"HellaSwag validation 18099","config":"default","split":"validation","body":"{\"activity_label\":\"Starting a campfire\",\"ctx\":\"A man in a field strikes a match. He bends down and tries to start a fire, but it doesn't light. he\",\"endings\":\"['chases after the other people he strikes at, but they all run away.', 'walks away as another man comes to check the fire.', 'tries again with another match.', 'shades his eyes and walks, dragging his dog with him.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}