{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5303cb4c-4e80-5019-a7a4-1438c86fa4e2","task_key":"default--validation--23409","task_revision_id":"1","upstream_id":"23409","short_description":"HellaSwag validation 23409","config":"default","split":"validation","body":"{\"activity_label\":\"Hand washing clothes\",\"ctx\":\"A person puts a garment in a pail, then he adds detergent and water. Then, the man washes the cloth with his hand, and then pours water and rinse several times. next\",\"endings\":\"[', he wraps it with cloth and then water the cloth and cleans with an wipe, and then clean with hand.', ', the person carry the the garment to hang on a rope and dry.', ', the man adds detergent to the cloth, and spray his hand.', ', the man bedclothes the pail with the cloth to dry, then pour water on the cloth and rinse it again.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}