{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b273b5d3-3166-53b1-995b-106974e69fb6","task_key":"default--validation--11843","task_revision_id":"1","upstream_id":"11843","short_description":"HellaSwag validation 11843","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to date a guy from another school [title] Schedule your dates in advance. [step] It's easy to get wrapped up in daily life at school. Since you won't see your guy everyday, make sure to schedule time for each other after school or on weekends.\",\"endings\":\"['That way, you have a few days free to do something nice together and check in on each other. [substeps] You can also ask your guy when he needs something and send him your numbers.', \\\"Once school starts, prioritize being with him and avoiding being alone together. Try to schedule outings with his friends if you're in school together.\\\", 'Work out your other commitments so that you get to see him at least once a week, if not more. Having a plan to see each other will give you something to look forward to and make the relationship exciting.', '[substeps] Even if your guy spends all day on school, try scheduling dates with your friends. You can take advantage of your free time to do something fun together, like have a swim, walk your dog, or watch a play or movie.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}