{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"537ac98a-2de1-5e95-a25a-18f9dfa154e4","task_key":"default--validation--48082","task_revision_id":"1","upstream_id":"48082","short_description":"HellaSwag validation 48082","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to diss someone [title] Insult their looks. [substeps] \\\" i wish i could diss you, but nature already did. \\\" \\\" halloween is over now, you can take your mask off \\\" \\\" you look like the before picture.\",\"endings\":\"['\\\" \\\" you have the trademark black coat \\\" \\\" you always look ominous! \\\" [title] Annoy the person to lighten the mood. [step] The person may be looking to get your attention when you\\\\'re around, be annoying them so that you can start to talk yourself up.', 'Is your wig a good fit? \\\" \\\" your clothes...they are ugly, and your hair looks pathetic. \\\" \\\" i can flatter your look, but i can\\\\'t date this way.', '\\\" \\\" did you trade in your neck for another chin? \\\" \\\" was anyone else hurt in the accident that did this to you? \\\" \\\" i\\\\'ve seen someone that looked just like you before, but i had to pay admission. \\\" \\\" i could use your shadow as an umbrella.', '\\\" \\\" the real way is to banish everything that reminds me of you. \\\" \\\" holly, i wouldn\\\\'t do that, just trace it with chalk.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}