{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"19ead591-797e-5f82-a0a3-ab5cd41589a9","task_key":"default--validation--23079","task_revision_id":"1","upstream_id":"23079","short_description":"HellaSwag validation 23079","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be confident as a short person [title] Make the jokes first. [step] When you're short, people are going to make jokes about it. They make jokes to make you feel bad.\",\"endings\":\"[\\\"But if you're the first one to make jokes, if you participate in a little self-deprecating humor, they'll be much less likely to make them. It will also make them like you more because it will be clear that you have a good sense of humor, and people like that.\\\", \\\"And when you're arrogant or edgy, these jokes make you feel bad too. [substeps] And when you're happy, people are going to get to know you a lot better.\\\", \\\"If you giggle at a joke that is just stupid and not funny then people will likely laugh more. In general, you want to make sure that it's awkward so that it is not the top of your priority.\\\", \\\"They don't even know it, they're just trying to embarrass you. Make sure that jokes are about you and nothing about anyone else.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}