{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"34e9b1f3-2d0c-5888-a32a-18adb9cda1f1","task_key":"default--validation--18099--variant-9af4032405b6","task_revision_id":"1","upstream_id":"18099--variant-9af4032405b6","short_description":"HellaSwag validation 18099","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to coach youth basketball [title] Watch a lot of basketball. [step] The best coaches are also great students of the game. Find basketball games wherever you can-on television, online, or down at the local recreation center-and study them.\",\"endings\":\"['Study their practices and techniques to get to know them better. [title] Teach your teammates how to \\\" fist bump \\\".', \\\"It's easy to get too immersed in the game or forget to think about the game at all. Catch up with the best basketball players you know and watch them play.\\\", 'Identify what plays or systems work best and which ones are less effective. Take plenty of notes, and go over them when the game is finished.', \\\"For the most part, get a lot of interest out of the get-go, and you'll see a lot of fun. Pay attention to every dribble, pitch, and hit.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}