{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d2b8ddbc-4bc7-5619-af52-5721e2ca795d","task_key":"default--validation--20087","task_revision_id":"1","upstream_id":"20087","short_description":"HellaSwag validation 20087","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to interpret blood gas results [title] Evaluate the results with your doctor. [step] The best way to interpret your blood results is by talking to your doctor. They understand the information and results better than anyone.\",\"endings\":\"['Making an assessment on your own can lead to misdiagnosis or complications from self-treatment. Ask your doctor any questions you may have about individual or total levels and what they may indicate.', '[substeps] Your doctor may tell you that your gas may be caused by something biological in nature. They may give you medication in larger doses.', '[substeps] If you are taking blood to test your thyroid, find the answer to a question that the doctor asks you. If the answer is not clear or you do not feel like describing the results with words or vocabulary, your doctor may ask more complex questions.', \\\"They'll be able to help make you more informed and make you aware of all possible possible issues with your blood gas. [substeps] Unfortunately, your doctor cannot direct you solely how your body reacts to gas.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}