{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5943326d-a880-5ef9-b53d-fbcd359f7816","task_key":"default--validation--46463","task_revision_id":"1","upstream_id":"46463","short_description":"HellaSwag validation 46463","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply liquid metal eyeshadow [title] Get a liquid metal eyeshadow kit. [step] They are often labeled as \\\" metal foil. \\\" these kits contain a metallic, pressed powder eyeshadow and a clear, liquid primer.\",\"endings\":\"['[substeps] Look for a kit in a color that complement your metal eyeshadow, such as maroon. If your eyeshadow is neutral/deep toned, like metallic, you can also opt for opaque eyeshadow kits.', 'You can use these kits to mark your eyelids so you can make your eyeshadow sooner if you purchase one [title] Swipe the liquid metallic eyeshadow over your eyelid. [step] Tilt your head back and carefully swipe the powder eyeshadow over your eyelid.', '[title] Consider applying some basic eyeshadow first. [step] Liquid metal eyeshadow is best applied just on the lid or inner corners of your eye.', 'Fill them up with your liquid ingredients, including the powder and primer, and secure with a clip before waiting. [substeps] A tissue box is a good place to pack liquid eyeshadow.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}