{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c12e625c-5451-581a-8bfd-574171a73dba","task_key":"default--validation--26317--variant-a1947d94b659","task_revision_id":"1","upstream_id":"26317--variant-a1947d94b659","short_description":"HellaSwag validation 26317","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buy toning shoes [title] Decide when and where you will wear your shoes. [step] Toning shoes' soles are designed differently for runners, walkers and everyday users. The 3 most common designs are rocker bottom soles, curved soles and collapsible heel soles.\",\"endings\":\"['If you\\\\'re foot is backwards, rocker toe will better allow you to find the \\\" heel \\\" section. Flip flops, collapsible sandals, durracers and high heels are also very popular choices.', 'These characteristics make them ideal for special exercise or everyday walk roles. [substeps] For example, you may wear rock bottom shoes, platform shoes, or street shoes if you absolutely must be in a high traffic area.', '[substeps] If you walk often, choose collapsible heel toning shoes. They provide the feeling of walking in sand, which can help to strengthen your legs as you go about your daily errands.', '[substeps] Most shoe stores will supply up to 20 or more different design sizes that you can purchase. Shoe stores may also have many different design sizes that are better to consider at different times of day.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}