{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6bf9866c-29c3-5dbc-813e-85976c20f555","task_key":"default--validation--36927","task_revision_id":"1","upstream_id":"36927","short_description":"HellaSwag validation 36927","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to paint rusty steel [title] Loosen the rust with a wire brush. [step] You can either do it by hand or use a wire wheel brush attachment inserted into a hand drill. [substeps] If you must prepare a large rusty steel surface, use a sand blaster to remove the rust.\",\"endings\":\"['Sand is abrasive and has too much wear and wear to be released easily. [title] Pull the grating off the metal.', 'Wait until all of the rust has transferred to the surface, then begin sanding until the rust is gone. Avoid sanding into stubborn buildup or out of place rust.', 'A sandblaster uses compressed air and particles of sand to blow off the surface of the steel. [title] Sand the surface thoroughly with an 80-grit sandpaper, then with a finer 120-grit sandpaper.', 'Point the nozzle downward and hold the flame at an angle at the grimy surface of the steel. Wipe away rust with a damp rag dampened with rubbing alcohol, after approximately 3 to 4 minutes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}