{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"67946154-8cf5-545f-a06b-f047dde747cf","task_key":"default--validation--48650","task_revision_id":"1","upstream_id":"48650","short_description":"HellaSwag validation 48650","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to free write a book [title] Realize that you don't necessarily need a topic. [step] All you need is a solid character and a setting. [title] Discard the temptation to write following the \\\" rules \\\".\",\"endings\":\"[\\\"[title] Know that published books from major publishing companies (fcrs) do not represent every single writer and they report. [step] Content does not embodies every reader's opinion.\\\", '[step] For example, the rules would suggest writing a plan along these lines: \\\" plot: a, who lives in south carolina, has a strong desire to be the next american idol. If you write something using a topic so obvious and real as this one, it will be harder to write.', '[step] Remember that many humans overstay their boundaries, and in this cycle we give up to the \\\" rules \\\" to allow something to turn out better. In the end, the one for a believable story will be your fictional entity.', '[step] That is nonsense as they can be too broad. Just find the main main idea behind the book and make an action plan that works for you.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}