{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9a88939f-33df-54cd-928c-149293d2897b","task_key":"default--validation--42073","task_revision_id":"1","upstream_id":"42073","short_description":"HellaSwag validation 42073","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to tightline eyes [title] Wash your hands. [step] Whenever you are going to be touching your face (especially around your eyes), wash your hands. Your hands can transfer dirt and oils to your skin, and your eyes are extremely susceptible to infection and irritation.\",\"endings\":\"['[title] Apply an eye primer to your lids. [step] Since tightlining requires applying eyeliner closer to your eyeball, make sure that your chosen eyeliner is waterproof.', 'Wear gloves after you are actually touching your eyes. This way, any work you put into your eyes does not transfer to your eyes.', '[substeps] Wash your hands without putting pressure on the skin with your fingers. It is important to keep your hand soft and loose so you can touch your whole face.', 'Use a washcloth and soap containing sulfates before and after touching your face. [substeps] This will keep your hands clean while you are working.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}