{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d35ef920-85e5-583c-9005-bee3b3a059d1","task_key":"default--validation--13870","task_revision_id":"1","upstream_id":"13870","short_description":"HellaSwag validation 13870","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to get an a without reading the book [title] If there is a movie adaptation of the book, watch it. [step] If there is more than one movie, watch as many versions as you can. Read the first and last chapter of the book, as well as the summary on the back or inside flap.\",\"endings\":\"[\\\"Do not change the story unless you have done so so in advance of the movie. If you haven't, next time.\\\", 'Go over the story and do some research to see if there are any parts you can learn, or at least to study the material over and over. [title] If you are purchasing the novel without reading it yet, go to your local library and talk to a librarian to visit a store that carries library books.', '[title] If the movie is over, read the original synopsis of the movie. [step] If the movie has been re-released, which it most likely has, read the full synopsis one or more times to make sure that you still understand and are reading the good stuff.', '[title] If the book has a wikipedia page, read it. [title] Go to the sparknotes and/or cliffsnotes websites.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}