{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"88c3fb89-2490-5e1f-beb4-8e959a53eab2","task_key":"default--validation--35533","task_revision_id":"1","upstream_id":"35533","short_description":"HellaSwag validation 35533","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make apple jam [title] Peel and chop your apples. [step] Wash your apples and peel them. Core the apples to remove their stems.\",\"endings\":\"['Trim the apples into smaller pieces to use as jam. Use 3 to 4 large apple chunks in the jam.', \\\"Some may have chunks of skin if you cut them off. You'll need about two peeled apples for the total jam.\\\", 'Carefully dice the apples finely so they\\\\'re no larger than 1/4 \\\" cubes. Place all of the apples in a large 4-cup measuring cup.', 'From the core, you should cut off all the small, stringy flesh. You should also grate them on a separate cutting board.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}