{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8b360031-1c8e-5b94-8bd6-119ef7f5e7cf","task_key":"default--validation--29638","task_revision_id":"1","upstream_id":"29638","short_description":"HellaSwag validation 29638","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write a cute note to a friend [title] Find the handwriting. [step] Find a fun type of penmanship (check out dafont.com) and use it! Practice it, writing the whole alphabet in capitals and lowercase. You can also write the sentence, \\\" the quick brown fox jumps over the lazy dog \\\", which uses every letter in the english alphabet.\",\"endings\":\"[\\\"[title] Paste, scratch, nail, paint, wet, smear. [step] Add many lines so others don't get in the way.\\\", '[title] Think of writing a complicated letter! [step] ) read characters from books in other languages. This step is useful for \\\" theme \\\".', '[substeps] If you want to write a sentimental or sincere note, use your normal handwriting or a more sophisticated or plain penmanship. Practice in your preferred writing utensil.', '[title] Describe the kitchen : [step] Make sure there is no blank space between the saucepan and stove top. Tray top is more desirable, as if you cooked in the big standing pan (the one with two glasses on top).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}