{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"94cff9d7-da00-5fef-9569-56e2a2af9e65","task_key":"default--validation--4225","task_revision_id":"1","upstream_id":"4225","short_description":"HellaSwag validation 4225","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to recruit political volunteers [title] Contact student groups at your local college. [step] Colleges are the perfect place to recruit volunteers. They have a large population of people who are young, energetic, and generally interested in trying new things.\",\"endings\":\"['Send an email to the student group you would like to join so that they can see you calling. Ask them if they would be willing to recommend someone who is adventurous enough to try things like knitting.', '[substeps] Most campuses have student-run organizations that may align with your cause. For examples, they may have campus republican or democrat clubs, an environmental organization, or a social justice group.', 'And the campus and your community will be nice to see. Point out that you attend campus organizations, such as the student go starter.', \\\"In many cases, they're willing to help out at a discounted rate. You may be able to find student groups near universities by checking on the word of mouth.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}