{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"241f52a3-d403-5519-9c31-6b1f03b53ffc","task_key":"default--validation--46524","task_revision_id":"1","upstream_id":"46524","short_description":"HellaSwag validation 46524","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to take action against capital punishment in your state [title] Focus your action on a specific case or cases. [step] It may help you to focus your efforts on a couple of current capital cases in your state. Focusing your work on a couple of specific cases may be more effective than tackling several cases or state laws as a whole.\",\"endings\":\"['However, you should also stay on top of general issues related to capital punishment. Combining this knowledge can help you become a very informed participant in the action against these cases, which in turn may help alter capital punishment laws.', 'Some of the cases in your state are relatively minor compared to other cases in your state. Still, you should aim to focus your efforts on a very specific case at all costs.', '[substeps] You can look up places to locate a case. There is a wide variety of legal resources online that can help you locate the most common cases you may want to worry about.', \\\"[substeps] If you believe capital punishment in any of your cases is unjust, find a law that applies to those cases. Look at the organizations's forum links.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}