{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"18398bde-d767-50b2-abe1-19a4c767f4df","task_key":"default--validation--4929","task_revision_id":"1","upstream_id":"4929","short_description":"HellaSwag validation 4929","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to compare body for life to the atkins diet [title] Eat 6 meals per day on the body for life diet. [substeps] Each meal should consist of 1 carbohydrate portion and 1 protein portion. The portions should match the size of your palm if you're eating meat, or the size of your closed fist for bulkier foods like fruits, vegetables and whole grains.\",\"endings\":\"['For a woman , 800 calories should make up the difference. 1 calorie 15 oz for every 1 ounce (14.', 'The total amount of protein your body needs per day should be around 20 g. Aim for 6 meals per day for life.', 'Some health professionals recommend eating 56 or more meals per day on the atkins diet. [title] Consume 3 cups of water daily on the atkins diet.', '[title] Follow the 4 phases of the atkins diet while eating 4 to 5 times per day. [substeps] Start the atkins diet by going through induction.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}