{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4f40732d-a094-5e02-b1e1-4894fd3087af","task_key":"default--validation--14076","task_revision_id":"1","upstream_id":"14076","short_description":"HellaSwag validation 14076","config":"default","split":"validation","body":"{\"activity_label\":\"Ice fishing\",\"ctx\":\"The person walking to one of the hole and checks the fishing line to see if he caught nay fish. the guy\",\"endings\":\"['then pulls hard on the fishing line and drags it in.', 'is holding the fishing line fishing out of there hole pulling romaine with it of course.', 'walks back to the hole to see if he caught anything but his hooks were nowhere near the hole.', 'gather up the fish from their hole and grabs the pole and starts fishing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}