{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d345a908-b084-5bb5-96b7-f73573560bda","task_key":"default--validation--40908","task_revision_id":"1","upstream_id":"40908","short_description":"HellaSwag validation 40908","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to talk to your kids about homosexuality [title] Expect to have several conversations. [step] Lgbt issues are complex, so you can't cram all of them into one 20-minute conversation. You will probably have quite a few conversations over their childhood, and that's normal and okay.\",\"endings\":\"['However, your children may want to talk about things they may not normally talk about with their peers or close friends. You may want to remember each discussion for 10 minutes, then go get them some special treats to discuss.', \\\"[substeps] If your child seems disinterested, stop talking. They won't learn much if they've tuned you out.\\\", \\\"Explain to them that homosexuality is normal, and that most importantly, they understand it. Don't feel like the conversation is a one-sided conversation-it's a whole conversation.\\\", \\\"Remember that lgbt communications are based around relationships. You and your partner may not communicate much, and that's ok.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}