{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"a34c0a66-60d2-573d-a40a-879e67b00d67","task_key":"default--validation--1875","task_revision_id":"2","upstream_id":"1875","short_description":"HellaSwag validation 1875","config":"default","split":"validation","body":"{\"activity_label\":\"Sharpening knives\",\"ctx\":\"A cooker talks in a kitchen and shows the inside of a sharpener knife that has discs inside. Then, the man puts a knife in the sharpener knife to sharpen. after\",\"endings\":\"[', the man then paints the knife with the discs.', ', the man measures the sharpener and cut the discs on its knife, then sews the discs and carve them.', ', the man wash and dry the knife.', ', he cuts a slice in two and put them in a pan.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}