{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0d55ba01-7636-5502-a773-985dfd1cf69e","task_key":"default--validation--42324","task_revision_id":"1","upstream_id":"42324","short_description":"HellaSwag validation 42324","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to plan a florida flydrive holiday [title] Make a decision about the type of flight you want. [step] Chartered flights are cheaper and are always direct, scheduled flights are more comfortable but sometimes have a connection. [title] Choose your airports.\",\"endings\":\"[\\\"[step] If you are flexible about your departure airport, you'll be able to get a better price. For orlando, chartered flights go into sanford (northern orlando), scheduled flights go into orlando international (southern orlando.\\\", '[step] Being in a secured area to talk over airport features will prevent security from getting into your airport. [title] Know what time of day you will fly.', \\\"[step] Ask the employees you are booking your flight with when they booked your flight. You don't want to be flying on a city or a railway system.\\\", '[step] Although you can allow for this trip from a hotel or airport or flight to an oa, you still need to find a well lit and public space. Airports are costly so it needs to be located well away from cities or states you live in, but also in the east coast of the united states.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}