{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"35b9a066-cde5-599f-bf92-7e305a57334c","task_key":"default--validation--37550","task_revision_id":"1","upstream_id":"37550","short_description":"HellaSwag validation 37550","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to remove sharpie [title] Lay the stained fabric face down on a layer of paper towels. [step] Make sure that you have enough paper towels to absorb the excess stain remover. These towels should also absorb some of the stain.\",\"endings\":\"['You may need more paper towels as you work, so keep a roll of paper towels handy. [substeps] You should also work on the stain on a surface that will not be damaged by the stain remover.', '[substeps] If you want to let your clothing soak up excess dye, put masking tape around the top edges of the fabric, such as the border or edges of a bath towel. The sticky side of the tape usually will be right in front of you, and the sticky side of the masking tape should be the center of the stain.', '[title] Lay the stained fabric along one of the two long sides of your blanket. [step] This will allow you to lift and fold the stained fabric easily and efficiently.', '[title] Allow the stained fabric to soak in the stain remover for 1 minute. [step] Use only white paper towels for this.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}