{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a5766cfe-852a-5714-9b08-ff3ba6597f92","task_key":"default--validation--41181","task_revision_id":"1","upstream_id":"41181","short_description":"HellaSwag validation 41181","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to divorce a wife who left the state [title] Find your wife's most recent address. [step] When you file divorce papers, you'll need to serve a copy on your wife. Typically, you'll need to make personal service on her or mail the papers to her address.\",\"endings\":\"['In any event, you need to know where she currently lives. [title] Check how long she has lived in her new home.', \\\"If you can't make service, you must first serve a copy on the wife. This service typically doesn't charge your wife or charge a fee for service.\\\", \\\"[title] Go to your state's bar association. [step] Most states have bar associations in the state where your wife lived, but some also have bar associations elsewhere.\\\", \\\"[substeps] In most cases, you'll have to pay a fee to have the sheriff's deputy mail or hand deliver the papers. The sheriff likely won't have much time to pay for the time you make service as you usually would.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}