{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4bb7dd0f-2559-5d51-b805-a4b68e6e2210","task_key":"default--validation--32619","task_revision_id":"1","upstream_id":"32619","short_description":"HellaSwag validation 32619","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to care for tan leather shoes [title] Wipe down your leather shoes with a soft cloth after each wear. [step] Run the cloth over the surface of your shoes each time you take them off for the day. Doing this regularly will prevent dirt and grime from building up on your shoes.\",\"endings\":\"[\\\"[title] Remove any debris from your shoes after they've been in the sun. [step] Sand your shoes carefully after wearing them.\\\", '[title] Polish your leather shoes once a month. [step] A regular polishing routine will keep your shoes looking new.', 'On the other hand, if you have a lot of leather, dirt and grime can build up on the fabric. [substeps] Gently wipe up any dirt that has built up on your shoes each day with a soft cloth, such as a washcloth, towel, or soft cotton towel.', '[title] Apply a soft solution of a shoe stripper oil to the soles of your shoes. [step] Have the mixture slowly soak for at least 20 to 30 minutes, then rinse and dry with a clean towel.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}