{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cf0af935-f463-5746-b389-5f3232d7ecf8","task_key":"default--validation--48277","task_revision_id":"1","upstream_id":"48277","short_description":"HellaSwag validation 48277","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make light applesauce [title] Wash and prepare the apples. [step] Wash 3 pounds (1.36 kg) of apples. Use an apple corer to remove the core and seeds of each apple.\",\"endings\":\"['Peel and core off the top and base of each apple. Cut off any foliage or skin on the apples.', 'Then, take a sharp, clean knife and cut each apple in half, just above the pit. After that, wash the apples under running water, which will soften them and make them easier to work with.', 'Use a sharp knife to carefully cut each apple into five or six pieces. Place the apples in a slow cooker.', 'Use another apple corer to remove the core and seeds from each apple. [substeps] Place apple corers on a hot, non-combustible pan to warm and roast overnight with the apples.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}