{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7361570e-58a1-5027-bbf8-e58ae0cf239f","task_key":"default--validation--50375","task_revision_id":"1","upstream_id":"50375","short_description":"HellaSwag validation 50375","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to make a travel brochure [title] Choose the destination of your potential clients. [step] If you are a professional working for a travel company, your destination of choice will be the one you work for. If you are a student, and are creating a pretend travel brochure, you will want to pick out a desirable, exotic, and interesting location.\",\"endings\":\"['[substeps] A professional should already know what destination he/she is representing, or attempting to advertise for. Use this step to get to know the key features of your location: mountains, lakes, cabins, museums, parks, etc.', \\\"Your destination will probably be rural country or kia, though you may also specify that you are similar to the area where you're working and travel business and book an opportunity to be a client. [substeps] Target venues in particular regarding their experience and interests.\\\", '[substeps] Focus on the amount of experience you have in the field you are researching. Again, the brochures have a tendency to get expensive and inexperienced people can take quite a bit of time to prepare.', 'This will vary from one place to another, so it will ensure that where you work is well planned and free of major distractions. [substeps] Your destination might be in an amazon shopping center, local chamber of commerce, larger chain or even the state.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}