{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"9bb7e451-6bb2-5bbe-b1ef-f641a0f272f6","task_key":"default--validation--1045","task_revision_id":"2","upstream_id":"1045","short_description":"HellaSwag validation 1045","config":"default","split":"validation","body":"{\"activity_label\":\"Ironing clothes\",\"ctx\":\"A pair of female hands uses an iron on a piece of baby clothing. the woman\",\"endings\":\"['wets the garment and scrubs it with a cloth.', 'peels the old clothing off the piece and replaces it with dry material.', 'sets the iron aside and flips the clothing.', 'wraps the garment on top of the blouse, then irons it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}