{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"32eba0fe-cd3f-5a60-9971-de2aab05be3f","task_key":"default--validation--29558","task_revision_id":"1","upstream_id":"29558","short_description":"HellaSwag validation 29558","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose travel shoes [title] Think about activities you'll do on your trip. [step] When selecting travel shoes, review the activities you have planned for your trip. You want to pack shoes that will allow you to comfortably go through with your plans.\",\"endings\":\"[\\\"[substeps] If you're just going to be relaxing on your trip and not engaging in physically strenuous activities, there's no need to bring running shoes or hiking boots. A solid pair of tennis shoes and one or two pairs of dressier shoes are probably sufficient.\\\", \\\"If you'd like to bring hiking shoes, consider hiking boots and hiking socks. If you want to do outdoor hiking, you want hiking boots and hiking socks.\\\", \\\"Travel shoes can be a great way to save time because they'll help you go through difficult or unenjoyable destinations. [substeps] Travel shoes aren't very expensive so you may not have to splurge on them.\\\", 'Alternately, if you have many desires, you may want to pack three or four of those. Research different shoes you will be interested in and buy a selection that appeals to your interests.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}