{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39ec8110-5d4e-55a5-b7b6-1390ca311324","task_key":"default--validation--29722","task_revision_id":"1","upstream_id":"29722","short_description":"HellaSwag validation 29722","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to make a secure home for an autistic child [title] Remove and replace fluorescent lights. [step] Many autistic people have severe sensory problems with fluorescent lights. If you have any fluorescent lighting in your home, replace it with soft, warm lamps or non-fluorescent fixtures.\",\"endings\":\"['[substeps] The reasons autistic people have difficulty with fluorescent lights differ depending on the person. For some, it is a visual disturbance.', 'See if any fluorescent fixtures you use advertise as helpful. [substeps] Say to your child, \\\" is there anything i can do to help with this problem? \\\" [title] Seek treatment.', '[substeps] A used or sparking bulb is usually okay; however, turning the lights off will require you to replace it in the middle of the night. Plug in incandescent lamps if you are not used to them.', 'Replacing fluorescent tubes with bulbs that are battery-powered could become a nuisance for autistic children. To avoid that possibility in the future, put up or install fluorescent panels in any rooms your child frequently frequents.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}