{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"3148e8ce-4fe8-53b1-8beb-e519a2e9387d","task_key":"default--validation--185","task_revision_id":"2","upstream_id":"185","short_description":"HellaSwag validation 185","config":"default","split":"validation","body":"{\"activity_label\":\"Sharpening knives\",\"ctx\":\"Then he takes a piece of bark and rubs the powered stone pieces onto it. The stone particles stick to the wet piece of wood. he\",\"endings\":\"['uses a small brush to polish one of the stone pieces.', 'then uses a strip of sealing tape to seal the board and then smooths the bottom of the board.', 'walks towards the camera.', 'then takes the knife and sharpens it against the wood piece.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}