{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"91a0c1f5-36cc-55e8-a981-8a6521ec4e2f","task_key":"default--validation--26596","task_revision_id":"1","upstream_id":"26596","short_description":"HellaSwag validation 26596","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to unfade jeans [title] Weigh your jeans. [step] Use a food scale to weigh the jeans that you want to dye. Knowing the weight of the item of clothing will help you determine how much dye you need to use to re-dye the jeans.\",\"endings\":\"['[title] Use bleach to remove dirt and sediment from the jeans. [step] Mix the bleach with the water and rub the item of clothing with it to begin the process.', '[substeps] Do a number of math problems as you weigh them on a scale (such as you are 15 pounds or 240 lbs). It will help you to figure out how much dye you need to use.', '[substeps] If you are using a food scale, you will need to weigh both the inside and outside of the jeans. This will help the amount of dye blend better.', \\\"[substeps] If you don't own a scale you can estimate the weight of your jeans. Mens jeans will often weigh more than female jeans.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}