{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"70998293-ace7-511f-9fdd-2c03e824950b","task_key":"default--validation--2175","task_revision_id":"1","upstream_id":"2175","short_description":"HellaSwag validation 2175","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to tell someone you do not want to be friends [title] Schedule a time to meet in person. [step] Send her a text or an email to ask her to meet you in a neutral location. If you live in the same town, this is the best way to have the conversation about not being friends.\",\"endings\":\"['[substeps] If they ask you what you want to talk about, say something vague. For example, you might say, \\\" i just want to share some recent decisions with you.', 'If you live several towns over, you may go with a friend to meet up. [substeps] If you live far away from each other, meeting in person might help avoid having to talk face to face.', 'You can also share the same phone numbers so that it makes it easier to tell her you do not want to be friends. [substeps] Texting is ideal, as it allows the other person to have a chance to familiarize herself with the concept of being friends.', '[substeps] Tell her you are changing, and that you both need time to reflect about it. Make sure to set the timing so you both can process the news together.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}