{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d124cfe-a62f-5a70-a11a-6cf87689ae58","task_key":"default--validation--11729","task_revision_id":"1","upstream_id":"11729","short_description":"HellaSwag validation 11729","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to keep up with the times [title] Subscribe to magazines and/or newspapers. [step] This depends on the subject for magazines. If you want to keep up with politics, do subscribe to a magazine about politics.\",\"endings\":\"[\\\"[substeps] One example of a magazine about political is that joe burnham's cold turkey campaign. One article wasn't about politics, but it was about current events.\\\", 'Or if you want to know the latest about games, subscribe to one of the gaming magazines. As for newspapers, find one you like.', 'If you are going to become a journalist, think about subscribing to articles on politics. With any experience, you will likely be better informed about politics.', \\\"You may not often find yourself writing a newspaper issue because it's a long-term favor. If that's the case for you, collect your magazine and, using an internet platform like newsfeed, get friends who might read the same issue as you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}