{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6be3ac98-2368-5933-b446-6fa448a8326e","task_key":"default--validation--21864","task_revision_id":"1","upstream_id":"21864","short_description":"HellaSwag validation 21864","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to write a letter from santa [title] Address the letter to the specific child. [step] If you want the child to believe the letter is really from, you should address it to that child. Mention the child's age in the letter as well.\",\"endings\":\"['Mention their name in the body of the letter at least twice in addition to addressing it to the specific child. [substeps] Thread details into the letter that show santa knows the child.', 'This will help the child feel needed and supported, thereby allowing them to properly grasp the letter\\\\'s purpose. [substeps] For example, the letter might say \\\" congratulations on santa\\\\'s birthday, jack.', 'This will help the child to believe that santa claus and katniss are not real entities and that the letter you are sending will fit their individual needs. [substeps] Explain to your child that this letter would give them great benefits and reduce feelings of helplessness and self-doubt.', \\\"You should address the letter by putting the child's father's name in the first line, followed by a period. [substeps] Address the letter to: santa claus-this is santa's father.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}