{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c53b8f0-aa89-5484-a72d-e428c346e8eb","task_key":"default--validation--11581","task_revision_id":"1","upstream_id":"11581","short_description":"HellaSwag validation 11581","config":"default","split":"validation","body":"{\"activity_label\":\"Painting furniture\",\"ctx\":\"A person in a furniture store is talking about popular furniture made from reclaimed wood. He shows rustic, contemporary and traditional type of furniture that is in his store. he\",\"endings\":\"['demonstrates some of the kinds of flooring used in the furniture makeover.', 'discusses to other persons about the rafting experience.', 'continues talking in his store and shows the furniture recliner and parrot.', 'shows the high quality reclaimed wood and the different furniture made from it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}