{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4d54a33-d5c7-531b-b3bf-210f940f6671","task_key":"default--validation--26208","task_revision_id":"1","upstream_id":"26208","short_description":"HellaSwag validation 26208","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to start a petition on the white house website [title] Identify your goal. [step] You can come up with a compelling petition by first identifying what you want to accomplish. Remember, you need other people to sign your petition, and they won't sign unless they understand your position on the issue.\",\"endings\":\"['Some examples of the type of petition you should include : [substeps] You will need to get married, have a child, and donate assets. You will need to agree to share capital to raise money for both of these purposes.', '[substeps] You should also try to research some other issues with which to petition. For example, if you want to get elected president of the united states, there might be some type of petition that will get you elected president.', '[title] Use forms that you already have on hand. [step] For example, you can go online and search for forms specifically for your age and gender.', 'Spend some time talking with other people about how you would like the white house to tackle an issue. [substeps] For example, you might be worried about climate change.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}