{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7acb7109-055e-59ab-80b0-e7466e452b8f","task_key":"default--validation--26950","task_revision_id":"1","upstream_id":"26950","short_description":"HellaSwag validation 26950","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose comfortable shoes [title] Look for shoes that feature designs built for comfort. [step] Some shoes forgo comfort for the sake of style. Though this might be fine for occasional wear, you want to look for those features when seeking a comfortable shoe.\",\"endings\":\"['[substeps] For instance, canvas shoes with low heel will be especially comfortable for walking to work or sharing an air conditioned apartment. For flat terrain, shoes with art prints to breathe and protection against rain or snow will definitely be best.', 'For example, a pair of high heels without straps or heels with straps that your foot would slide easily through will not go great with sneakers. Avoid any shoes with consonants that sound like they weigh things down.', 'These features include : [substeps] An achilles notch. The groove featured at the back of the shoe where the shoe meets the achilles tendon.', '[substeps] Some shoes have rubber soles as cushioning, which lets to reduce friction. It may also help if you have a smooth sole.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}