{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1306d6d6-ff18-52ed-b676-ce88b3e1ebf2","task_key":"default--validation--3446","task_revision_id":"1","upstream_id":"3446","short_description":"HellaSwag validation 3446","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to brighten your car's yellowing headlights [title] Make a trip to your local auto parts store and pick up a headlight lens restorer. [title] Start off by washing the headlights and drying them. [step] This should be done in the shade.\",\"endings\":\"['[title] Apply the restorer in a small amount to a cotton towel and work it in deep small circles in a small area of your lens. [substeps] It should dry as you keep working it in.', 'You can wrap the headlight lens with plain aluminum foil and allow it to dry for a few hours but the glass will glow more in the dark. [substeps] Ensure your headlights are completely dry but not soaking wet.', \\\"Don't reuse any headlights that are covered with dirt and dust. You can use any headlight removal spray (such as tanner), but read how to clean top fresh headlights.\\\", \\\"Do not use bleach, as you won't be able to deliver the same shine you will due to the dirt. [title] Remove the blinkers and wash any dirt off of the headlights.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}