{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fe7e9225-6802-5400-ab63-f9f5af70b4cb","task_key":"default--validation--42133","task_revision_id":"1","upstream_id":"42133","short_description":"HellaSwag validation 42133","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to run the nyc marathon [title] Enter the drawing for a chance to secure a spot at the race. [step] Between mid-january and mid-february, apply for the drawing through the new york road runner website. The drawing is at the end of february.\",\"endings\":\"['Though this is the easiest way to secure a spot, it is not guaranteed. [substeps] In 2017, over 98 , 000 hopeful runners applied for the drawing and only 16 , 000 were accepted (a 16.', 'You may only need a copy, but be careful. If you choose to work with people from different races, you may not need a photo or two, or you may be required to put them in the car for an interview.', \\\"[title] Be clear about your race return policy. [step] While this is not a requirement for the race's discoverable location, you must know how to interpret more than the course and dietary necessities in your area.\\\", \\\"You can run the marathon if you're in a water country, but most national congress nominal awarding programs are only tickets to the winner a year. [substeps] Fulfill all applicable training requirements, meet etc.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}