{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"347303c8-cbbf-5075-9663-8ebbec3d2c60","task_key":"default--validation--37916","task_revision_id":"1","upstream_id":"37916","short_description":"HellaSwag validation 37916","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to pass emissions [title] Check the emissions standards and procedures in your state. [step] Many states require periodic emissions tests to be performed to ensure that your car is running as clean as possible and not contributing to excessive air pollution. There are a lot of complicated numbers associated with the process that are probably not of interest to the average driver and vehicle owner, but you can research the environmental protection agency standards for all vehicles.\",\"endings\":\"['[substeps] Look for the following information on test company records: make, model, mileage, and engine history. These should be available from the health department.', '[substeps] Check your collision history and ensure you pass the agencies requirement. Check the emission standards that state regulations that put in place to get the tests used.', \\\"[substeps] You don't need to know the standards to have your vehicle tested. All you need to do is find a testing location in your area and schedule an appointment.\\\", '[substeps] Ohio, oregon, arkansas, and idaho require citizens to report their emissions to authorities. Colorado, texas, and utah require states with emissions standards of at least 18 over their driving average.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}