{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8bb421ba-42c2-5e54-866c-2bd4fd02fcdb","task_key":"default--validation--11523","task_revision_id":"1","upstream_id":"11523","short_description":"HellaSwag validation 11523","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make oboe reeds [title] Purchase gouged cane from your music store. [substeps] Cane can be purchased 1 of 2 ways; either gouged or shaped and tubed. For the purpose of this article, we will start with gouged cane and shape it.\",\"endings\":\"['You do not need to be able to see the shape of the cane. The thickened ends are the musculature for the shanks.', 'Reel the long end into the other cane to form the nib, then cut it to a size of 6. Coat the cane with glaze or sugar.', '[title] Place the gouged cane in warm water and let it soak for 30 minutes. [substeps] This allows the cane to become more pliable for shaping.', 'The diameter of the broken cane will be as little as 15 mm long. The second method involves staking thin and sturdy planks of cane (less than 12 mm in diameter) together.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}