{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3137a0e5-308a-570c-8d73-74f650f6a1d1","task_key":"default--validation--6516","task_revision_id":"1","upstream_id":"6516","short_description":"HellaSwag validation 6516","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to use an otoscope [title] Be gentle with the patient. [step] The ear is a very sensitive organ and can injure easily if improperly examined. Avoid pulling, pushing, or generally being rough with the patient you are examining.\",\"endings\":\"['This can calm your patient and minimize the risk of injury from sudden movements. [substeps] Ask your patient if the pressure is acceptable to them.', 'Sometimes the patient can already complain of pain. [substeps] If possible, limit the session to less than one hour and avoid poking or moving the device at all times.', 'Always provide a comfortable bottom to keep the patient from lifting their head. [substeps] The patient should be lying down with her head for the most part, with the mouth open and as close to their ear as possible.', '[substeps] If the patient is in pain, the eye feels like it is not open enough to receive light or sound. This puts strain on the ear and can obstruct the fisr and the magnifier when being examined.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}