{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bbc2ebb7-bd04-5730-847c-308a68d52a5e","task_key":"default--validation--9353","task_revision_id":"1","upstream_id":"9353","short_description":"HellaSwag validation 9353","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to plan a trip to brazil [title] Find out where you'll be staying. [step] This can either be an obvious answer, such as with a friend or relative who lives in brazil or it may take a little bit of time to figure out. If you don't know anyone there that you can stay with you can figure out if you want to stay at a hotel, in a hostel, or even.\",\"endings\":\"[\\\"[title] Get tickets to brazil! [step] It's the summer season. Travelling to brazil in only a week! Is a great idea.\\\", '[substeps] Several travel sites are available online to help you compare options and plan accommodations. [title] Get a passport or renew a previous one.', \\\"[title] Get ready! [step] All you need to do is apply your travel visa to brazil. When this is done, you'll be able to travel anywhere on the continent that you want to return to.\\\", 'Some of the most common methods of getting where you want to go include : [substeps] Book a hostel a mile ahead. You may even want to consider visiting all the while there.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}