{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4723421-7e57-5184-b65a-47fca82bd61d","task_key":"default--validation--34922","task_revision_id":"1","upstream_id":"34922","short_description":"HellaSwag validation 34922","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to edit a short story [title] Come to the story with fresh eyes. [step] To edit the story effectively, you need to try to read it with a fresh perspective. You may need to take a break and put the story away for a few days so you can think about other things or other stories.\",\"endings\":\"['By using the first few weeks of writing as a time to come to the story, you can identify some important discrepancies in the quality of the characters. [substeps] Try to avoid goops or costumes, since these can cause many of the scene changes to be confusing and overwhelming.', 'Then, after a few days, pull it out again and read it to yourself with fresh eyes. You may be surprised at how different it reads to you after you have not been staring at it for several hours straight.', 'Reading different short stories may help you uncover interesting key ideas and write stories based off of them. [substeps] Before you write a short story, read it over and over again.', '[substeps] Even if the original story was famously very bad, you can still read it a few times and figure out if something is off. Make sure you are really committed to revising the book before you take a break.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}