{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"77aa7e76-e5a9-5b69-b859-c1e0a2549e45","task_key":"default--validation--16492","task_revision_id":"1","upstream_id":"16492","short_description":"HellaSwag validation 16492","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to appeal citations for animal law violations [title] Contact the agency that issued the citation. [step] The citation you received will have information for the agency that issued it, including an address and phone number you can use to respond to the citation. [substeps] In larger cities, you also may have the option of responding to the citation online.\",\"endings\":\"['The person or entity in question will then have to contact you. Call 1-800-574-1213 and request an address or phone number.', \\\"Typically your citation also will include a deadline by which you must respond if you intend to appeal the citation. Mark this deadline on your calendar and make sure you don't miss it or you may lose your ability to appeal.\\\", 'Other governmental entities, such as big box or financial institutions, may be able to answer your questions under oath. Under federal law, the number for that agency and the state it is located in may also be available on your website.', 'Follow the instructions for that particular service, as they can be found at. [title] Get a written response of any kind.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}