{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c4914cd-43a6-529a-93c0-11a2f608f6fe","task_key":"default--validation--6467","task_revision_id":"1","upstream_id":"6467","short_description":"HellaSwag validation 6467","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to tell if you have metabolic syndrome [title] Measure your waistline. [step] If you have a large waistline, it can be a risk factor for metabolic syndrome. While a large waistline may often depend upon your height, there are some measurements which will usually tell you if you are at risk for metabolic syndrome.\",\"endings\":\"['[substeps] Sit on a small stool or a chair and measure your waist using a stopwatch or multimeter. Then, weigh it from top to bottom to find your waist size.', 'However, you should be mindful of your weight and body mass index (bmi) to determine whether you are at risk. Bmi is calculated based on your height.', '[substeps] For men, a waistline over 40 inches (100 cm) can be a risk factor for metabolic syndrome. A waistline over 35 inches (88.9 cm) is a risk for women.', '[substeps] To get an accurate measurement of your waistline, you first have to measure your waist length and chest height. Measure from your ribcage to the level of your shoulders.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}