{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4dab2b74-4ef7-56c8-b21e-1fb976685d20","task_key":"default--validation--47868","task_revision_id":"1","upstream_id":"47868","short_description":"HellaSwag validation 47868","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to lose 20 pounds [title] Record your current eating patterns. [step] Take the first week to analyze your current diet habits. To do this, start a food diary.\",\"endings\":\"['This contains a daily log of how many calories you eat each day. Write down when you eat lunch or when you visit the gym for your regular gym classes.', '[substeps] A food diary means record your meals, snacks, activities, and daily routine, before working on weight loss. In this diary, include any foods you are avoiding and whether or not you are following a strict low-calorie or low-carb lifestyle program.', 'You should note the things you eat and do while you are struggling to lose weight, the times you exercise, and also the types of calories you cook and drink. Getting a journal will be a great help with this.', 'Record everything that you eat and drink for a week, be sure to include snacks and portion sizes. [substeps] As you record your food and drink intake, also record your mood.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}