{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7550a839-9b08-5caa-a6e9-77a199c4d6cf","task_key":"default--validation--42270","task_revision_id":"1","upstream_id":"42270","short_description":"HellaSwag validation 42270","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make bracelets out of thread [title] Cut 3 equal sized strands of thread. [step] If you want to make a very basic design, you can use three strands of thread. In this example, we're using violet, pink and blue.\",\"endings\":\"['[title] Cut two lengths of thread about 3 times the size of your bracelet. [step] However, if you want to make a very basic design, you might not want to cut three lengths of thread.', 'The color of the thread is your choice. [substeps] To make violet, punch large holes in each end of the thread.', 'Since this project involves multiple strands, we want to leave some threads blank. In this case, you will use blue for each strand.', 'You may want to make a more elaborate design later on, but this pattern will teach some of the basics of knotting. [substeps] Cut out the strands to the length you wish.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}