{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"beda411b-01df-598d-9321-45e7cbd72590","task_key":"default--validation--34645","task_revision_id":"1","upstream_id":"34645","short_description":"HellaSwag validation 34645","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to go boating with your dog [title] Slowly get your dog acquainted with the boat. [step] Begin by bringing your dog out on the boat when you are at dock. Next time, try bringing him out to the dock, but turn the motor on to get him accustomed to the sound.\",\"endings\":\"['Slowly bring the boat closer to the dock and stop before it is clear to approach the boat as soon as your dog is lying on it. [substeps] Do not pull hard on the wheel as you move the boat around in the water.', 'Allow the engine to warm up, but do not encourage his attention. [substeps] Get your dog out and let him sit or lie down on the dock.', 'Then, take him on a short boat ride and then progressively longer rides. [substeps] This is important for getting your dog comfortable with the boat.', 'Try moving the motor occasionally as you get to shore. [substeps] As your dog gets used to the boat, he will be more likely to play on it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}