{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cfea8899-c920-500a-a543-15df9e3b8928","task_key":"default--validation--46520","task_revision_id":"1","upstream_id":"46520","short_description":"HellaSwag validation 46520","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean a fiberglass shower [title] Empty out your shower. [step] Remove all bottles, soap, razors, and accessories from your shower. Wipe down the items to remove any grime that may be transferred back into the shower, especially mildew.\",\"endings\":\"[\\\"[substeps] This step helps ensure you don't touch or completely wash all of the surfaces that will be cleaned. Make sure the shower door is locked before you go any further.\\\", 'Clean using warm water, following the instructions in your kit. [title] Run a washcloth under warm water.', \\\"[title] Apply 30 minutes of soap from the shower's biggest container. [step] It's good to use a bucket full of water, not as huge container that can fit all the items from this large container.\\\", 'Mildew on the bottom of the bottles can immediately rub back onto your freshly clean shower surfaces. [substeps] If you have any empty or worn-out items, throw them away.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}