{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"445ec2f5-c9c8-5079-8688-151277b9c0ae","task_key":"default--validation--14226","task_revision_id":"1","upstream_id":"14226","short_description":"HellaSwag validation 14226","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to cite legal research [title] Write the name of the case. [step] A legal citation to a court case begins with the title of the case itself, as it appears in the case reporter you're using. Generally, you should only use last names or business names.\",\"endings\":\"[\\\"Don't use the first word of the case you want to write with your first name. [substeps] If you don't know the full legal name of the case you're citing, skip ahead to the footnote and save it.\\\", 'However, you can add business names if you want or wish. [substeps] For example, using \\\" smith vs.', '[substeps] For example, you could write, \\\" this criminal defense \\\" rather than \\\" plaintiff (or defendant), \\\" in this case. [title] Collect citations from adaddendum.', '[substeps] Don\\\\'t include the phrase \\\" et al. \\\" to indicate there is more than one party to the suit.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}