{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dd5320d7-efab-5e8e-8f8f-bdd471902834","task_key":"default--validation--39617","task_revision_id":"1","upstream_id":"39617","short_description":"HellaSwag validation 39617","config":"default","split":"validation","body":"{\"activity_label\":\"Philosophy and Religion\",\"ctx\":\"[header] How to be an atheist parent [title] Raise them however you want. [step] They're your kids. There are worse kids out there.\",\"endings\":\"[\\\"You'll probably do more harm worrying about every little thing. Parenting is about using your instincts, experience and making good use of all the resources you have.\\\", \\\"It could make the difference in the long run, but because you are raising their opinions, they will almost certainly disagree with you. [title] Make sure you're neither at nor very religious (no church, etc).\\\", 'Let them act as parents as much as they want. Show them each that they can learn to be an atheist.', 'No one expects to be your \\\" godparents. \\\" try making them move in with your family, or set up a house for your new family.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}