{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40451f63-67e9-5cb6-9c7a-8742577b7e70","task_key":"default--validation--41130","task_revision_id":"1","upstream_id":"41130","short_description":"HellaSwag validation 41130","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be safe, be yourself and still have fun in high school [title] Shake the idea that being good is boring. [step] Being a good kid doesn't make you a goody-two shoes. Most of the time, following your parents' rules is better than breaking them.\",\"endings\":\"[\\\"The point of being a good kid is to be fun, but that doesn't mean letting your nerves get the best of you and letting them get away with acting badly. If you think acting bad might come with hefty consequences, learn to frame your actions and when they are most likely to apply, and vice versa.\\\", 'If your parents want you to do so, do it! Follow them step by step. How will this make your life easier at first? [title] Look through your surroundings.', 'It leads to less fighting and more appreciation. When your parents think of you as a good kid and can trust you, you get rewarded with more privileges.', '[substeps] Embrace your individuality no matter what. Much of school is about making good, smart decisions (and basically going against your parents ideas).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}