{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fcb216f3-e2a4-5cc7-9ca4-43687125e8eb","task_key":"default--validation--39768","task_revision_id":"1","upstream_id":"39768","short_description":"HellaSwag validation 39768","config":"default","split":"validation","body":"{\"activity_label\":\"Painting furniture\",\"ctx\":\"A man standing at the table, he is showing the vaseline, a block of bronze or metal and a wooden brush. the man\",\"endings\":\"['then rolls up a cabinet board, he measures the panel with the brush.', 'rub the vaseline with the brush on all sides of the block, then spray paint it with white paint.', 'clears all the surfaces, in places the tables, wall and bed frame is scraped.', 'applies the product to the tattoo screen.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}