{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7d14aa7-c822-5d90-aa1b-4f4eb140989e","task_key":"default--validation--12701","task_revision_id":"1","upstream_id":"12701","short_description":"HellaSwag validation 12701","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to make time to read a book when you're busy [title] Buy an audio version of a book. [step] It's usually the cheapest option, and readily available, so you can listen to it just about anywhere. Many public libraries now allow their cardholders to download electronic audiobooks to personal mp3 players.\",\"endings\":\"['These will be supported by many companies that pay for access to the recordings. [substeps] This is another great option, as large libraries often hold only a limited number of cds.', 'If your library doesn\\\\'t allow windows books, a \\\" free \\\" option is usually a better choice. Portable windows books are more readily available, though you may need to take them with you to school.', 'This may be beneficial for artists or inventors who might otherwise miss an instrumental playback. [substeps] Online book stores are also excellent options.', 'Alternatively, you can buy audio books on cds or specialized mp3 players. You could also buy the audio version of a book you already own.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}