{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1e7372bf-2523-589b-82b1-0d62e4f5a50a","task_key":"default--validation--1107","task_revision_id":"1","upstream_id":"1107","short_description":"HellaSwag validation 1107","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to be a scene girl if your parents won't let you [title] Discuss the scene style with your parents. [step] Explain to them why you like it and how you'd like to adopt certain aspects of it. Make it clear that there are ways of adopting this style that are not permanent and that won't interfere with your school uniform presentation or other things that might worry them.\",\"endings\":\"['Keep going until you find the style you like the best. Then revise it to the \\\" scene \\\" style that you like best.', \\\"[title] Listen to your parent's concerns. [step] Try to work out in advance what is likely to worry them and have good, sensible solutions to offer them.\\\", '[substeps] You might say \\\" i think some of my friends respond well to this style and work best in it. \\\" [title] Get dressed like a scene girl.', '[substeps] If you really like the scene then tell your parents. If not, there are alternatives that are very difficult to adopt.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}