{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"441ebca3-dc18-502a-aa0a-cefaaba0132b","task_key":"default--validation--8794","task_revision_id":"1","upstream_id":"8794","short_description":"HellaSwag validation 8794","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make brightly colored eyelashes [title] Brush clear mascara onto your eyelashes. [step] You will be tapping the eyeshadow onto your lashes while the mascara is still wet, so it might be a good idea to do just one eye for now. If you do both eyes now, the mascara will dry out too fast, and the eyeshadow won't stick.\",\"endings\":\"[\\\"Don't worry; you won't need to keep doing it. Let it dry on both before moving on to the next step.\\\", 'But if you do both eyes in the same shade, you will have plenty of options, and there are different whole loads of options as many as you can pick from. You could decide on extensions in either color, or go one color at a time.', '[title] Sweep a soft, eyeshadow brush across the eyeshadow color you want to use. [step] Be sure to get a good amount on your brush, so that it sticks to the wet mascara.', 'Swipe a pea-sized amount of eyeshadow on the skin under your cheekbones. Place your forefinger in the center and gently tap it downward.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}