{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b02d1ae3-999e-53a6-8657-89ad69a6540c","task_key":"default--validation--34690","task_revision_id":"1","upstream_id":"34690","short_description":"HellaSwag validation 34690","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean makeup brushes with witch hazel [title] Gather all your brushes in a clean, convenient area. [step] Places such as the living room floor or the kitchen counter usually aren't acceptable if you live with others or have pets. If your bathroom is clean, putting them on the counter is a good idea, so long as you avoid chemicals or other things such as makeup and water.\",\"endings\":\"['[substeps] Collect any stains from your brushes you may have. Discard them if they are valuable or could cause allergic reactions.', '[substeps] Check your state laws to be sure. Only put your brushes in plain water if you want to keep them fresh.', '[title] Wet your brushes with tap water, or if you have it handy, distilled water. [step] Get as much makeup as you can out of the brush before you apply the witch hazel.', 'If you have a bathroom in an noisy home, it is recommended that you shower and rub with a non-irritating cleanser before using them. Otherwise, this will cause irritation and headache.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}