{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"484f54b1-de79-58c1-9c41-01952d9eaf39","task_key":"default--validation--18951","task_revision_id":"1","upstream_id":"18951","short_description":"HellaSwag validation 18951","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to clean skateboard wheels [title] Remove the wheels from the board. [step] Turn the skateboard over so it is resting on the deck, with the wheels facing up. Use a ½ inch or 13 mm socket wrench to twist and remove the axle nuts on the outer sides of the skateboard wheels.\",\"endings\":\"['[substeps] They should be mounting more mount bolts. [title] Remove the gears on the top of the wheels.', 'Keep all wheel materials (nuts, washers, etc. ) together in one place so you can easily find and attach them later.', 'Slide the side axle nuts out of the lift of the skateboard as well. Some people prefer to use pliers to remove the wheels off the wheel.', 'If the wheels seem tight, you may need to try again. [substeps] The wheels may feel a bit loose.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}