{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"43053428-ece1-565d-a4fd-86142d9a5f2b","task_key":"default--validation--2256","task_revision_id":"1","upstream_id":"2256","short_description":"HellaSwag validation 2256","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to empty a diaper genie [title] Open the genie. [step] Make sure your diaper genie rests squarely on the floor. Push the button on the front of your genie to separate the top portion of the genie from the bottom portion (you can let the top portion rest on the ground.\",\"endings\":\"[\\\"You won't need it for a while. ) [substeps] If there are any diapers remaining in the top portion of the genie, push them down into the bottom portion of the genie.\\\", ') screw off the top portion of the genie and press down firmly, then push the bottom portion of the genie open. [title] Slip your \\\" free leg \\\" inside the genie.', ') [title] Pull the opening of the genie from the side of the playstation. [step] Try flipping it on its side so that you can see the bottom portion.', ') [substeps] If you\\\\'re not sure how to open your genie, ask the attendant. [title] Put a \\\" corkscrew \\\" on the bottom portion of the genie.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}