{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b864ce2e-849e-5132-aba9-e1866c1d0a4e","task_key":"default--validation--27000--variant-6eadd403820b","task_revision_id":"1","upstream_id":"27000--variant-6eadd403820b","short_description":"HellaSwag validation 27000","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to do external car detailing properly [title] The first thing you need to do properly is to wash down your car effectively. [step] There is no point trying to do external car detailing if you are just going to end up scratching your duco even more because you left dirt on your car. [substeps] You must first hose down your car with firm pressure.\",\"endings\":\"['This will remove most of the dirt from your engine and keep the engine running smoothly. Use high pressure washing machines to wash down your car.', 'Then you must begin to siphon off dirt. Once you have removed as much dirt as possible, you can then return to the vehicle to pick up trash.', 'Run a hose from your carburetor to the top of the engine block, wait for five minutes, then turn off the water and let the air out of the radiator. Remove the old gaskets and begin water washing down your engine cover.', \\\"Don't use a laser type wash from your hose, but rather a small funnel. Always start from top and work your way down.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}