{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"58684e5c-a6d8-552e-a8cc-eb42970d0971","task_key":"default--validation--8914","task_revision_id":"1","upstream_id":"8914","short_description":"HellaSwag validation 8914","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to improve digestive health with fermented foods [title] Eat fermented beans. [step] Studies have linked fermented beans like tempeh (fermented soybeans) with improvements in bowel and menopausal health. Tempeh may also protect your heart against disease and protect you from some cancers.\",\"endings\":\"['Fermented beans may also reduce inflammation of the esophagus. [title] Introduce instant fermented vinegar in your diet.', 'Fermented beans contain beneficial bacteria that heal liver and kidney problems. However, they are not regulated and should not be ingested with fermented beverages.', '[substeps] Fermented beans, such as tempeh, are also known as purnate beans. Women should eat four to eight grams of fermented beans per day.', 'More research is needed to determine a daily recommended intake, but you should start by just including tempeh in your diet. [substeps] Tempeh is also a complete protein which means it has all the amino acids your body needs to make more protein.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}