{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cb103101-5963-51ef-99f7-993696d5f162","task_key":"default--validation--1621--variant-c5160ac2b554","task_revision_id":"1","upstream_id":"1621--variant-c5160ac2b554","short_description":"HellaSwag validation 1621","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to take care of twins [title] Remember that you are now eating for three : [step] Yourself and your two babies. [title] Incorporate folate and folic acid (the synthetic form of folate) into your diet. [step] Folate is a b vitamin, which is an essential ingredient during a pregnancy as it can help prevent serious abnormalities of the brain and spinal cord, and neural tube defects.\",\"endings\":\"[\\\"If you don't get enough folate in your diet, the risks of a preterm delivery are increased. [substeps] Whole grain cereals and trail mix.\\\", '[title] Drink plenty of water; not too much, but enough, to quench your thirst. [step] And be sure to drink almost all sodas, sports drinks, or your non-carbonated beverage.', 'Foods such as yogurt, fish and shellfish/fish milk are rich in these vitamins and folic acid. [substeps] If you are overweight or obese you can get more than three vitamins from food.', '[title] Add sources of fiber to your diet following a recent injection. [title] Be careful : [step] Sparkling water is recommended to prevent dehydration even during the pregnancy.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}