{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"542e5f8e-3307-5927-b9ee-bc918e68b09c","task_key":"default--validation--24823--variant-023a40bb3c41","task_revision_id":"1","upstream_id":"24823--variant-023a40bb3c41","short_description":"HellaSwag validation 24823","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to put on tights [title] Choose the right size of tights. [step] The most important step in putting on tights is to start with the correct size. Otherwise, you're going to have trouble no matter how you try to get them on.\",\"endings\":\"['[substeps] Tights are often sized according to weight and height. Check the packaging that comes with the tights to determine what size matches your weight and height range.', \\\"At the very least, you'll need a pair that will fit. The best standard size is ½-inch, for adult tights.\\\", '[substeps] Luckily, tights are easy to get onto your feet. However, they can also be heavy duty.', \\\"[substeps] All your tights should be in the correct size. If your tights aren't, your clothes will weigh down your feet, making your legs limp.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}