{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87b399df-50d1-57ed-a3c5-c86e8abd9217","task_key":"default--validation--50428","task_revision_id":"1","upstream_id":"50428","short_description":"HellaSwag validation 50428","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to choose a multivitamin for women [title] Ask your doctor if you have vitamin deficiencies. [step] Many people think they do not have vitamin deficiencies, when in reality, they may have one or several. Traditional, routine blood work does not test for all vitamins.\",\"endings\":\"['However, daily multivitamin recommendations can increase your chances of a vitamin deficiency if you have one. [substeps] You might say, \\\" would you recommend that i get a multivitamin from my doctor? \\\" also consider what type of food you eat and whether you are taking supplements for that reason.', 'In order to use a multivitamin, discuss this with your doctor. [substeps] Your doctor may do a much more general test or just refer you to a dietician for more information.', 'While there is no scientific consensus, taking a multivitamin may help you to fend off the symptoms of vitamin deficiencies, it is still recommended. [substeps] It is important to know what \\\" normal \\\" vitamins (including vitamin a, b12, and d4) mean to you.', \\\"It may not even test for vitamin d. you need to ask for these tests to ensure that they're being done.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}