{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2162e360-cc81-5100-bfac-9d81bdd39c81","task_key":"default--validation--39083","task_revision_id":"1","upstream_id":"39083","short_description":"HellaSwag validation 39083","config":"default","split":"validation","body":"{\"activity_label\":\"Vacuuming floor\",\"ctx\":\"The video shows a demonstration of dyson vacuum cleaner and how well it can clean particles from the floor. there\",\"endings\":\"['are various graphics on the screen that include names and font and video of the machine.', \\\"'s rice grain, flour and other food particles scattered on the floor.\\\", 'are various solutions presented to the total vacuum product with her comment describing each implement as purchasing safety precautions onthank you very much for using it.', 'is one person that shows how to remove the particles from carpet.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}