{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f4032d0d-45bd-5007-a198-d052e224987b","task_key":"default--validation--33316","task_revision_id":"1","upstream_id":"33316","short_description":"HellaSwag validation 33316","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use a cuticle pusher [title] Decide whether to get a cuticle pusher made of steel or wood. [step] Steel cuticle pushers are best for tough, hard cuticles. Wooden cuticle pushers are best for soft cuticles.\",\"endings\":\"['[substeps] Wood cuticles have been shown to taste best. They can also handle heat more effectively than wood cuticles.', 'Feel free to carry a variety of plastic container pushers over to a professional beauty supply store and purchase the correct kind. [title] Wet the skin with warm, soapy water.', 'Woods made of metal or plastic are not the best choice because they are difficult to work with. Metal cuticles have limited seam allowances and they do not give very much resistance.', '[title] Pick which tip you want your cuticle pusher to have. [step] Some have blunt rounded ends, others have sharp pointy ends or spoon-shaped ends and yet others are curved.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}