{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fe61ad91-add9-53a0-978f-20ba74064fd8","task_key":"default--validation--45611","task_revision_id":"1","upstream_id":"45611","short_description":"HellaSwag validation 45611","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to remember song lyrics [title] Download two versions of the song. [step] When you sing along with an artist, you rely on their vocal cues to lead you from one verse to the next. This crutch prevents you from learning and retaining the lyrics.\",\"endings\":\"['So, download the two versions of the song from your computer. [substeps] Download the song through a keynote player, or access your music library and download it without a keynote player.', \\\"[substeps] Before you try to write down lyrics, take some time to have a cup of coffee or excuse yourself to put the two versions of the song in your own system. This will ensure that you don't forget every line.\\\", \\\"[substeps] Download the first version of the song so that you can familiarize yourself with the song before you start playing. Even if you don't have two parts, the download will definitely take your mind off of writing.\\\", 'This becomes problematic when you perform live and the cues are gone. One way to overcome this obstacle and memorize the lyrics is to sing along with the original track (on which the artist is singing) and the backing (instrumental) track.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}