{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0484f534-c321-518c-85aa-55930ef9cd5c","task_key":"default--validation--43502","task_revision_id":"1","upstream_id":"43502","short_description":"HellaSwag validation 43502","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to deal with obsession as an autistic person [title] Admit to yourself that you have an obsession, and note what the obsession is. [step] This is the first step to dealing with it. [title] Trace your path to where the obsession started.\",\"endings\":\"['[step] Try to see if you can identify the initial source of your obsession, and where you got it from. For example, suppose you are obsessed with the number 28 and like to go around repeating it.', \\\"[step] For instance, what are you doing now? What do you want to do this year? What are your goals? See how that now makes you feel. If you've always loved going to the movies and watching them, does it scare you or completely fascinated you, or has made you jealous? [substeps] See how it's made you feel in the past.\\\", \\\"[step] If you don't know where to start you can check out this article. [title] Write down everything you think or feel that you are obsessed with one day, and that day you might not know your obsession that well.\\\", '[step] Begin by writing down your progress on an invisible sheet of paper. You can use the sheet of paper as a starting point, or you can use a tape measure.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}