{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"08c937e4-af80-57d7-96f3-d602de31fa27","task_key":"default--validation--39118","task_revision_id":"1","upstream_id":"39118","short_description":"HellaSwag validation 39118","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to deal with a backstabbing friend [title] Ask your friend to have a private conversation. [step] Before you make any decisions, it's important that you get to the bottom of the situation. Tell your friend that you'd like to clear some things up, and that you'd appreciate them meeting with you.\",\"endings\":\"['[substeps] Say, \\\" billy, can we talk about some things? \\\" if they say no, accept them as a friend. Sometimes it\\\\'s okay to accept and maybe say, \\\" okay, then let\\\\'s talk about it.', '[substeps] Tell your friend that you\\\\'d like them to meet you in the event that you need to talk again. Say something like, \\\" i know it won\\\\'t be easy, but i\\\\'ve got something really bothering me that i need to talk with you about.', \\\"[substeps] If you don't see this friend at work or at school, ask them to meet you in a public place such as a cafe. It's best to meet in a public place so that neither of you feel uncomfortable.\\\", \\\"Doing so will make the interaction easier, but making the situation nasty won't do any favors for you at all. [substeps] If your friend requests to speak in private, let them.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}