{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13f8f022-3db7-527a-b1f1-0b4e84899f51","task_key":"default--validation--33325","task_revision_id":"1","upstream_id":"33325","short_description":"HellaSwag validation 33325","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to review using flash cards [title] Keep the text brief. [step] Stick with short phrases instead of full sentences. Use abbreviations for words when possible.\",\"endings\":\"['You can also make flash cards sound stumpy. [title] Match the front section of the card profile with the number \\\" focus for multiple points.', '[title] Identify and separate these two cards with the mind of the dictionary. [title] Put the other two cards on the top left corner of the page.', \\\"Avoid associating information with simple sentence-terms. [title] Relax and find information that shows you're doing it correctly.\\\", \\\"You will want to review the cards quickly and condensing the text makes sure that only the most important ideas make it onto the card. The very process of choosing the card's text will kick-start the learning process.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}