{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fa52249b-c196-5b28-909a-cf6a094c9b9b","task_key":"default--validation--20352","task_revision_id":"1","upstream_id":"20352","short_description":"HellaSwag validation 20352","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to send a citizen complaint to a civil grand jury in california [title] Determine who your complaint will be against. [step] They are generally referred to as the \\\" respondent, \\\" and the person making the complaint is the \\\" complainant. \\\" [substeps] Keep in mind that county grand juries have no authority over state or federal agencies and private citizens, only county and local agencies or officials.\",\"endings\":\"['[title] Download the dmv complaint form from the dmv website. [step] You can download the form and bookmark it.', 'By contrast, the supreme court of the united states has no authority over the federal district of appeals to the supreme court (the district of appeals). States with a central hierarchy govern the district of appeals and other courts.', 'As an exception, they also may not have direct control. Plaintiffs are not represented by an independent party, and may not have direct jurisdiction over the case.', 'Take note of which county the agency of official is located in. [title] Determine which county grand jury would be appropriate to contact.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}