{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"bb7d8214-bab7-5cdc-a80b-05de73b8b6c3","task_key":"default--validation--1200","task_revision_id":"2","upstream_id":"1200","short_description":"HellaSwag validation 1200","config":"default","split":"validation","body":"{\"activity_label\":\"Sharpening knives\",\"ctx\":\"She then shows an appliance on the table and tilts it upwards. She turns a dial on the appliance and sets it back down. she\",\"endings\":\"['shows several shots of the chair in the end.', 'picks the knife back up and places it on the appliance.', 'pokes the vacuum nozzle in the top seam of the vacuum cleaner and adds fuel to it while still keeping the vacuum nozzle up.', 'also shows how to use tubing no a string is attached to it and she explaining the use of the tubing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}