{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"343a54f2-6271-58d2-a74d-09fbfed97918","task_key":"default--validation--13924","task_revision_id":"1","upstream_id":"13924","short_description":"HellaSwag validation 13924","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write a letter of appeal [title] Locate the appropriate appellate court. [step] Every court sits in a hierarchy made up of other courts. You appeal a court's decision to the court that sits directly above it.\",\"endings\":\"[\\\"[substeps] For example, your state may have a court decision board and all other courts will report the case to your state's bench. The court clerk will typically pull the records and affidavits and give you a forwarding address.\\\", '[substeps] The supreme court has many rules you can use to locate a notary public in your county. The rules differ between courts, but the main rule in a local county is that you must admit or deny every allegation made in your letters.', 'If you do not know which court to appeal to, ask the judge after he has issued his ruling or check with the clerk court. [substeps] In the federal court system, district court decisions are appealed to the circuit courts, whose decisions may be appealed to the supreme court.', 'By finding the correct judge who feels he or she actually has the powers and authority to appeal the decision, you can best differentiate yourself from other courts. [substeps] Search the web to find local, state, and federal court appellate courts.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}