{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"adf08263-3481-548e-83cb-8535bb465bdb","task_key":"default--validation--3671","task_revision_id":"1","upstream_id":"3671","short_description":"HellaSwag validation 3671","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to style mules [title] Pick mules that fit your feet comfortably. [step] You don't want to be walking around in shoes that hurt your feet, so make sure you try them on and walk around in them before making a purchase. For a nice fit, your foot should cover the shoe lining completely while not spilling over any of the shoe's edges.\",\"endings\":\"['So select a shoe with a few inches of extra tread to help draw the toe toward your foot. [substeps] Other styles of mules include lace or liners.', '[substeps] Sizes vary greatly from brand to brand, so you may want to experiment with more than one cobbler as well. Make sure you feel comfortable with these basic steps before attempting to style your own mules for the first time.', \\\"For a more individualized appearance, skip a pair of loose-heeled shoes. [substeps] You don't want a pair of low-heeled pumps that grind up the edges of your feet, so for this reason, pick flats.\\\", '[substeps] If your foot is sliding around in your shoe while walking, the shoe is too big. [title] Choose flat mules for a comfortable sole.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}