{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2dbb469f-5443-544f-9e35-d367d4bad14c","task_key":"default--validation--18261","task_revision_id":"1","upstream_id":"18261","short_description":"HellaSwag validation 18261","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to read like a historian [title] Find some background information about the author. [step] Before you even open the work, search online for the author's history. Find out where they graduated and when.\",\"endings\":\"['Meet with a respected historian to collect details about him or her. Research the author, and pay close attention to his or her background.', \\\"Historical period period descriptions will help you point you in the direction of where to look. [substeps] However, don't assume you can just drop hints.\\\", 'Look up their other publications and what they focused on. You might even find an article discussing why they chose to write this particular work and why they think it is important.', 'Find out what they studied in graduate school or what degrees they had in high school. [substeps] If the author has records to back up information, you may want to seek out records to help you back up your argument.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}