{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4824933a-1723-59c1-af54-db06ea534be6","task_key":"default--validation--11512","task_revision_id":"1","upstream_id":"11512","short_description":"HellaSwag validation 11512","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to contact mtv [title] Log in to your facebook account. [title] Go to mtv's official facebook profile at https: // www.facebook.com/mtv. [title] Scroll through mtv's timeline, and find a post you'd like to comment on.\",\"endings\":\"['[step] One of the posts listed is a post about a story you believe in. The post mentions who or what that story is about, or what you would like to talk about with your viewer.', '[title] Click on the \\\" comment \\\" tab at the top of the timeline. [title] Watch the post so you can see what it has to say.', '[step] At this time, mtv is accepting post comments only, and is not accepting messages from users on facebook. [title] Type your comment into the field provided, and press \\\" enter \\\" to post.', \\\"[title] Scroll down to the post you want to reply to. [step] ( note that your initial comment will not count against that post's date.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}