{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b91e7310-f6ce-56a0-8b08-eeba787fb2a8","task_key":"default--validation--29963","task_revision_id":"1","upstream_id":"29963","short_description":"HellaSwag validation 29963","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to understand why he broke up with you [title] Contact your ex. [step] Give it a couple of weeks after the breakup. If you talk to him too soon, the conversation could prove too emotional to handle.\",\"endings\":\"['Once he listens to you openly and sees your concerns, he may propose taking matters to heart. [substeps] If he contacts you, you might say, \\\" hi, i wanted to ask how you feel about him.', \\\"It's a good idea to help him move on. He needs time to gain perspective, and the moment he doesn't react well, he'll be very hurt, frustrated, and possibly angry.\\\", \\\"After you've had some time to process, ask your ex to meet you to talk. [substeps] Make it clear you just want some closure.\\\", \\\"Let your ex know that you think it's over. [substeps] You could try calling him and asking him why he broke up with you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}