{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6a21dea8-df63-55d7-8478-beef0c2f6ea1","task_key":"default--validation--15930--variant-21215c49b63d","task_revision_id":"1","upstream_id":"15930--variant-21215c49b63d","short_description":"HellaSwag validation 15930","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to choose catholic wedding music [title] Contact the music department at the church where you will be getting married. [step] The music director usually has a list of music specifically tailored for weddings that you can choose from. [substeps] If you are looking at the list and have no idea what these songs sound like, ask to meet with the organist to hear them played.\",\"endings\":\"['At this time, you can then choose the song specific music you want to sing. [title] Find an option that resonates with your values, religion, and musical tastes.', \\\"Then, try asking them to sing them for you. That way, the wedding party can get in sync with what they're listening to.\\\", '[title] Consider what each piece brings to the ceremony. [step] All signs should point to your relationship with god as individuals now becoming a couple.', 'The staff at the church actually will help you decide what sort of music you want to choose for the ceremony. [title] Research local music venues in the city.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}