{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0a68a01a-84a0-5b58-8b1e-42894ff2a451","task_key":"default--validation--2851","task_revision_id":"1","upstream_id":"2851","short_description":"HellaSwag validation 2851","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to install entry doors [title] Remove the old door and the surrounding trim, molding, and door jamb. [step] You must remove everything covering the \\\" rough frame \\\" of the doorway. [substeps] The majority of doors for sale on the market today come pre-hung--that is, they are already mounted on a door jamb.\",\"endings\":\"['If you were to install a new door through its original frame--the whole thing would be finished simply on the inside frame. Look for other ways to go about this.', 'When installed correctly, they provide the tightest possible seal to keep out the elements. The door jamb is usually held in place with nails, screws, and caulk.', 'For the best results, you should remove them, the plaster, wood floor trim and doormolding. [title] Slide inside the door and insert the new door.', 'The good thing about plastic doors is that they are easy to remove. Some imperfections can be removed (nails are not necessary) with a phillips head screwdriver, or you can remove old door knobs by loosening the screws holding them in place.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}