{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b98a41a8-dc66-5f4a-804e-bcadcb6deb94","task_key":"default--validation--12017","task_revision_id":"1","upstream_id":"12017","short_description":"HellaSwag validation 12017","config":"default","split":"validation","body":"{\"activity_label\":\"Philosophy and Religion\",\"ctx\":\"[header] How to attend a lutheran worship as a visitor [title] Research the church before attending. [step] Look at their website if they have one, or speak with a minister over the phone. Ask friends who attend the church and speak with them about it as well.\",\"endings\":\"[\\\"Speak openly with your favorite volunteers and fellow scholars in the church to access the members. Don't feel comfortable using a leave-out or visiting any of their worship sites yourself.\\\", 'Ask questions about the worship and what it does. Take note of any special intentions (goals, functions, history, or even past faith issues) the church might have.', '[title] Read about the denomination that a particular congregation belongs to. [step] Some are evangelical (such as the evangelical lutheran church in america), and others are more conservative (such as the lutheran church-missouri synod).', \\\"It's also a good idea to compare the church's history, which will help you get an idea about the chance you will have if you attend. Make sure that you still have time to attend the temple candle since you will need to attend it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}