{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b318445e-c5c6-5439-a0c4-31ecfb1e4584","task_key":"default--validation--4126","task_revision_id":"1","upstream_id":"4126","short_description":"HellaSwag validation 4126","config":"default","split":"validation","body":"{\"activity_label\":\"Bathing dog\",\"ctx\":\"A young girl is seen scrubbing a small dog down with soap next to a tub full of water. she\",\"endings\":\"['shows off her rag and blow dryer while the dog mops the area around her.', 'pours more soap onto the dog and drags him over and leads into pictures of the dog all cleaned.', 'continues scrubbing the dog while the dogs look on.', 'continuously spits water all along the dog while the dog sits still.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}