{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ddef565d-b144-548b-b6ed-5598bcbe9c88","task_key":"default--validation--46906","task_revision_id":"1","upstream_id":"46906","short_description":"HellaSwag validation 46906","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buy toms shoes online [title] Determine your toms shoe size. [step] Toms shoe sizes normally align with the shoe size you wear most often and are only available in a medium width. [substeps] Choose a toms shoe size identical to your size in casual or dress shoes.\",\"endings\":\"['Toms shoe styles and sizes are very similar to styles of casual and dress shoes. If you wear 2 different shoe sizes that vary depending on the shoe style, choose the smaller size when buying toms shoes.', 'If your shoes are too small to fit the shoe size you want, you may want to limit how much space you have to exercise. [title] Select a shoe for your target gait.', 'Your feet will be bigger within a two inches of the shoe, which allows for the sole to lay between your toes. [title] Make sure your shoes do not have wide toes.', 'This is because most soled shoes are a bit bigger than their regular size. Toms shoes are more likely to be wider, but still have more safety pin sizes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}