{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"263f4af8-c6e2-5e14-9793-e3ff5c69d157","task_key":"default--validation--9122","task_revision_id":"1","upstream_id":"9122","short_description":"HellaSwag validation 9122","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write a fan letter (preteen ) [title] Introduce yourself. [step] At the beginning of the letter, tell the person your name and why you are writing to them. Do you like their music or their movies? Do you look up to them and want to be like them? Did they help you deal with a problem you were having? Be honest and express your true feelings.\",\"endings\":\"[\\\"[substeps] Mention another fan's name if possible. If you're just starting a fan letter, don't mention race.\\\", 'If you want a fan letter then you need to be open about how you feel. You can read your cover letter but tell the person that you genuinely love music and movies, but that you are tired of discussing that.', 'Make it sound thoughtful and not lame or dumb. [substeps] This should be the first time you communicate directly with that person in the letter.', '[substeps] You may say, \\\" hi, my name is nicole. I love your music because it helped me get through a really hard time.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}