{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7f5cbfc2-c20d-5f88-aa92-7cc8cc6cd9bf","task_key":"default--validation--35130","task_revision_id":"1","upstream_id":"35130","short_description":"HellaSwag validation 35130","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to treat both psoriatic arthritis and psoriasis [title] Talk to your doctor. [step] Consulting with your doctor should be your first response if you detect any symptoms of psoriasis or psoriatic arthritis. They will be able to diagnose your condition and advise you as to what actions you should take to manage it.\",\"endings\":\"['They will also give you a referral to a doctor. When you call your doctor, ask how long the symptoms have been going on.', \\\"[substeps] Don't neglect to mention any of your symptoms to your doctor. Your doctor will also want to know about your family history.\\\", '[title] Take over-the-counter medications as directed by your doctor. [step] Over-the-counter medications will not speed up your treatment as they are supposed to.', '[substeps] Your doctor can write you a prescription for antibiotics. If you are asking for antibiotics, your doctor may want to recommend 1.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}