{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"112c7fda-4b7c-5b31-a61a-2b1c687a6f44","task_key":"default--validation--4170","task_revision_id":"1","upstream_id":"4170","short_description":"HellaSwag validation 4170","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to take perfect math notes [title] Ask questions when needed. [step] If you aren't certain about something your teacher said, or didn't understand the way something was presented, make sure to raise your hand and ask for clarification. The best notes in the world can't help you if you don't understand what you're taking notes on.\",\"endings\":\"['[title] Highlight key, relevant information in that piece. [step] If anything, highlighting the missing information prevents your teacher from trying to find the key and learning more quickly.', '[title] Smile when you hear the teacher ask you questions. [step] Smiling actually gives you a small boost in confidence.', \\\"Remember that the objective is to learn, so don't be afraid to ask questions. [substeps] Some professors may ask for you to hold your questions until after the lecture.\\\", 'Help your teacher out just by asking questions as needed. For example, if your teacher keeps talking about how a certain theorem is solved in a certain group of numbers, ask them how the answer was found.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}