{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02accb06-a764-5a8e-ac49-a91626500a79","task_key":"default--validation--14075","task_revision_id":"1","upstream_id":"14075","short_description":"HellaSwag validation 14075","config":"default","split":"validation","body":"{\"activity_label\":\"Ice fishing\",\"ctx\":\"The person made many mid size hole in the ice and have fishing line hanging deep in the water underneath. the person walking to one of the hole\",\"endings\":\"['holes and pulled a from a hole in ice and wiped it with a rag.', 'holes and touch the hole with the rod.', 'in the ice for the fish to head for the ice.', 'and checks the fishing line to see if he caught nay fish.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}