{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a01b034b-ceee-5353-9f10-0f1a1eb2404e","task_key":"default--validation--6887--variant-45504a128032","task_revision_id":"1","upstream_id":"6887--variant-45504a128032","short_description":"HellaSwag validation 6887","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write a speech introducing yourself [title] Make an outline of your speech. [step] Start by making a skeletal draft of your main points. Strip the speech down to its bare bones to determine what is most important to say, and in what order you should deliver those facts.\",\"endings\":\"['This is the basic structure which you will build your speech around. [substeps] State your name in the very first sentence of your speech.', '[substeps] Keep it short, as every paragraph should be at least 50 minutes long. You can take breaks to reflect on your speech, and add these to the outline.', \\\"[substeps] It's bad form to instead write in your own words. This creates a skeptical feel to the speech, and will only build on your ideas.\\\", 'For instance, write, \\\" speak for me. \\\" [substeps] If you choose to save the outline by writing out your introduction yourself, you can leave some detail to ease your audience into what your main points are.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}