{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"569ddece-ff73-51ea-95d7-e1b6ae5ce27a","task_key":"default--validation--23133","task_revision_id":"1","upstream_id":"23133","short_description":"HellaSwag validation 23133","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to revamp a sofa on a budget [title] Measure your sofa. [step] A \\\" slipcover \\\" is a commercially made cover for your sofa. In order to purchase the right one, you will need to measure your sofa.\",\"endings\":\"['If you will only use this cover, take your measurements to a professional seamstress. [substeps] Length and width determinations are a must, though you may need a second sheet if you want more padding.', 'The easiest way to measure is to wrap your sofa in masking tape and add an inch to the length of the cushion. Then, add an inch on either side of the sofa, and measure again the size of the sofa on both sides.', 'Using a tape measure, record the width and length of your couch, as well as the length of the arms. Additionally, you should look for any labels on your sofa.', 'You can purchase a slipcover at your local superstore or online. Your sofa must have a slipcover already made.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}