{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b3d607ad-5417-5fd9-b647-ba85ffea4d2e","task_key":"default--validation--43274","task_revision_id":"1","upstream_id":"43274","short_description":"HellaSwag validation 43274","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buy comfortable shoes [title] Choose a reputable shoe retailer and good shoe brands. [step] Comfortable shoes are usually those that are well made and long lasting. While a cheap shoe may feel great when you first put it on, if it has a thin sole and poor supports, the initial sense of comfort will quickly wear off.\",\"endings\":\"[\\\"[title] Have the shoe retailer measure your foot. [step] Changes do occur over time, so it's a good idea to have this done every now and then even after you've stopped growing.\\\", '[title] Hold the front of the shoe up in between your fingers and slowly put your toe into the shoe until it is almost straight. [step] Now put your heel at the bottom of the shoe and walk the soles of your feet against the inside of the shoe.', 'There are several stores that specialize in shoes that will play it safe. [title] Look for shoes that are comfortable on you.', 'Search online for shoes at shoe stores that offer these types of shoes. [title] Be aware of the number of people wearing comfortable shoes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}