{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"be8c204d-1006-5866-b22b-2c64e1fa58aa","task_key":"default--validation--1826","task_revision_id":"1","upstream_id":"1826","short_description":"HellaSwag validation 1826","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to show your parents a bad grade [title] Calm yourself down. [step] Don't get too worked up before showing your parents the grade. Anxiety and nervousness will cause you to overthink the situation.\",\"endings\":\"[\\\"Remember, this will one day be unimportant and a distant memory. The sooner you explain what has happened, the sooner it'll be over, and you can start looking towards doing better.\\\", 'Get at least ten minutes outside of school to fully calm yourself down. Try to read a book or take a walk instead.', 'Take a deep breath before you start showing your parents the grade. Do not let emotions get the best of you, and try to be calm and collected in the moment.', 'Make sure you are calm before you present your parents with the grade on your report card. Take a deep breath and remind yourself that you are collected and confident.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}