{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4912014c-9a31-5c73-975a-f4f208cf8754","task_key":"default--validation--22062","task_revision_id":"1","upstream_id":"22062","short_description":"HellaSwag validation 22062","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to troubleshoot your workout [title] Get an expert to observe your workout. [step] If you've reached a plateau, a personal trainer may be able to help you take your workout to the next level. They can point out flaws in your technique and give you tips to get more out of your exercises.\",\"endings\":\"[\\\"[title] Do things that you might normally do for fun. [step] As often as possible, do things you're not much into doing.\\\", \\\"[substeps] If you've met someone who makes you workout more challenging, ask for your review. A trusted friend or family member may have really been there when you started and could help point out basic errors you could miss.\\\", \\\"[substeps] A personal trainer may ask you questions to get a sense of your diet, life activities, and other factors that might be impacting your workout. Answer these questions honestly-don't try to impress the personal trainer with how healthy and active you are, or you won't get the help you need.\\\", \\\"[substeps] Ask a family member or friend to help train with you so you can let them know what you're doing and how your workout is about to become more effective. [title] Contact your gym or business about any limitations or adjustments you've noticed.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}