{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c3ef1677-6bd5-5238-aba8-752ccd60da3f","task_key":"default--validation--26877","task_revision_id":"1","upstream_id":"26877","short_description":"HellaSwag validation 26877","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to fix a brake fluid leak [title] Open the hood and check the brake fluid reservoir. [step] This reservoir is located on the driver's side toward the rear of the engine compartment. If fluid is low, then you may have a leak.\",\"endings\":\"[\\\"If you have noticed any cracks or discoloration in the reservoir, then you may need to treat it right away. Refer to your vehicle's driver's side manual if you have any doubt or complaints.\\\", '[title] Verify the leak by checking beneath the car for the presence of brake fluid. [step] The location of the brake fluid will also help you to locate the general placement of the leak.', \\\"If this is the case, check the owner's manual. [title] Change your oil temperature to 37.6 ° f (15.\\\", 'If it is higher, replace the fuel pump. [substeps] If the fluid is still low, then you will need to replace the fuel pump.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}