{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99747844-8ae8-5ed0-9d62-939274c35c6b","task_key":"default--validation--31628","task_revision_id":"1","upstream_id":"31628","short_description":"HellaSwag validation 31628","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to choose the right chewing gum [title] Consider your insulin production. [step] Are you? If so, sugarless gum may be for you. [title] Think about your reaction to sugar.\",\"endings\":\"['[step] If you used sugar products, your saliva will react with an extra sugar than normal saliva. This may mean your gums are sensitive.', \\\"[step] Even if you're not diabetic, sugar can still be harmful to your health. Does diabetes run in your family? Does sugar make you? If so, you should avoid sugar as much as possible.\\\", '[step] Many people react more defensively when they experience a bite. Biting the inside of their cheeks kills the bacteria responsible for causing teeth misalignment.', '[step] Sugar-free gum stimulates your heart, lowers your blood pressure, and limits the amount of saliva going into your mouth. Avoid or limit sugarless gum if you are diabetic.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}