{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0246c021-ca30-514e-a9b2-0a8e211da99a","task_key":"default--validation--31900","task_revision_id":"1","upstream_id":"31900","short_description":"HellaSwag validation 31900","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to recognize quality khussa [title] Look for khussa made from high quality leather. [step] Many manufacturers will use inferior leather for the uppers of the khussa and some use poor quality leather for the entire shoe. To avoid this, the uppers should not be a different color leather than the soles.\",\"endings\":\"['The shoe should look stiff, not floppy (good khussa leather starts off very stiff and then breaks in to conform to your foot). The leather should appear thick, not thin.', 'New darf gravis leather should be considered an excellent choice for the uppers because it should not be confused with any other leather made from a lesser quality leather. Also, as a chip may result from compression, long tabs and bands should be removed for safety purposes.', \\\"[substeps] Nareth edition is produced from camel leather. If your shoe has any of the same colored or waxy leather that the uppers do, it's a good idea to buy an inferior texture and simply show it off to your child.\\\", '[title] Look for suede dams that have a high quality polyurethane finish. [step] These dams are often glossy, easily scratched or worn.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}