{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2983dc54-4876-59bf-a18b-86c52ecb442b","task_key":"default--validation--7808","task_revision_id":"1","upstream_id":"7808","short_description":"HellaSwag validation 7808","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to survive an upset stomach on an airplane [title] Use a quick trick to reduce upset stomach. [step] Are you close to throwing up in the air? There are some small measures you can take that might help you feel better fast. [substeps] Put pressure on the inside of your wrist.\",\"endings\":\"[\\\"If you feel a deep ache on your wrist, lean forward in your seat and squeeze your wrist as tightly as you can. Don't flex your wrist, but consider moving your forearm.\\\", \\\"Press down, gently but firmly, with your fingers about two fingertip widths above your hand's bottom. Suck on hard candy to keep your saliva going.\\\", \\\"Make sure you are holding onto your hand like you would a medicine stick. If the inside feels bruised (again, don't throw up), you might have a headache or is actually suffering from food poisoning.\\\", 'Grab a bottle of water and feel the back of your wrist. Continue squeezing and twisting your wrist to increase blood flow.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}