{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"219efcdc-27fc-5ea4-9a51-2d3065769c17","task_key":"default--validation--6025","task_revision_id":"1","upstream_id":"6025","short_description":"HellaSwag validation 6025","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to analyze case law [title] Read the case. [step] You should read the case through at least once from beginning to end until you attempt to figure out which facts are most important or analyze the court's holding. It's difficult to correctly determine what was central to the court's reasoning until you've read it all the way through.\",\"endings\":\"[\\\"[substeps] Some cases have some elements you can't analyze for yourself, and some cases don't. Research the court's or opposing counsel's discussion styles and determine what aspects are too complex to attempt to understand.\\\", \\\"You will most likely need to get a strong understanding of the case's arguments so you can formulate counterarguments based on statements made in the case. [substeps] Try to find the important counterarguments as soon as possible after reading the case.\\\", \\\"[substeps] The first time you read through a case, don't worry about trying to understand it. Just read for a sense of what's happening, who the major parties are, and what they want the court to do.\\\", \\\"You also should be able to catch basic information that might bother or confuse you later. [substeps] Your court's date of issue comes next.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}