{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5ecff0d5-4f38-5052-a8f1-c715c2a1d7ac","task_key":"default--validation--19807","task_revision_id":"1","upstream_id":"19807","short_description":"HellaSwag validation 19807","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to wash lettuce [title] Cut off the lettuce root on leaf lettuce. [step] Use a knife or your hands to break the root off of the lettuce and cut off any wilted spots. Separate the leaves of the lettuce with your hands.\",\"endings\":\"['Set the wilted lettuce aside while you prepare the last step. [substeps] You can purchase lettuce root at your local grocery store or online.', '[substeps] Ensure that all the parts are completely washed and dry. [title] Cut off the leaves to make the dressing dressing dressing.', \\\"If the stem breaks off after the cut through the stalk, you won't be able to wash the lettuce right away. [title] Collect or store the wilted leaves and stalk.\\\", \\\"[substeps] Be careful when using a knife so you don't hurt yourself. Make sure your fingers are away from the blade when you cut.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}