{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7fad40e9-b436-52f6-b0d4-5a7bda21c525","task_key":"default--validation--35044","task_revision_id":"1","upstream_id":"35044","short_description":"HellaSwag validation 35044","config":"default","split":"validation","body":"{\"activity_label\":\"Philosophy and Religion\",\"ctx\":\"[header] How to start a women's bible study [title] Pick a place where you would like to have a bible study. [step] This place can be anywhere just so long as it's okay with the owner of the property. Some ideas are: at your house, in a park, in a coffee shop, at a restaurant, or at your church.\",\"endings\":\"['There are endless possibilities here! [title] Choose the best time to do the study. [step] The mornings tend to be calmer since the day has not yet begun and a lot of times, there are less people walking about.', '[title] Determine which member of your congregation you would like to make into the bible study. [step] You could choose anyone--people of all ages, religious faiths, or people from a conservative background.', '[substeps] You can also sign up for church services. There are usually lots of bible study/readings available at churches, library etc.', 'The location is important though, because people would go there for free if you passed away. Come up with plans for your study time by considering things like : [substeps] What you will be studying during your study time.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}