{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f50408e6-f537-559d-940a-0c77c3b05121","task_key":"default--validation--43073","task_revision_id":"1","upstream_id":"43073","short_description":"HellaSwag validation 43073","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to get out of awkward situations [title] Wait for a lull in the conversation. [step] Conversations naturally have times where they drop off or have a lull in them. Typically this is when a new topic starts if you want to keep the conversation going, but it's also the perfect time to take the steps to end the conversation.\",\"endings\":\"['[substeps] Respect when a new topic rolls up. Oftentimes it simply goes from \\\" hey, i just missed some delicious chicken with buns on it, \\\" to \\\" awesome, see you around \\\" or \\\" let\\\\'s get out there and have some food.', 'Use transition words like so, anyway, or well to begin to say goodbye. [title] Summarize key points of the conversation.', 'If you see a lull in someone\\\\'s conversation, you can use it to your advantage. Maybe you can say something like, \\\" hey john, you seem to be a little standoffish lately.', '[substeps] A pause in a conversation indicates that you\\\\'re ready to say more. For example, your new friend might say, \\\" so you\\\\'re not doing well today.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}