{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a1811826-b75a-534e-b473-207ae66796b7","task_key":"default--validation--48928","task_revision_id":"1","upstream_id":"48928","short_description":"HellaSwag validation 48928","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to see your flickr photos used on wikihow [title] Review these tips for wikihow photos. [title] Create a collection of photos. [substeps] Take photos of scenery or people when you travel.\",\"endings\":\"['Wildlife, lakes, rivers, wildlife and finding some secluded places are good choices because the photos made will be best shared with your friends. An image can be used to display photos.', \\\"Take photos at different activities, such as a concert or parade. Take photos while you're cooking or making a bed.\\\", \\\"Pull out old photos and post them with inspiration to remind yourself of your destinations. Remember to store the newly created photos in a folder or hidden location where they won't be easily lost.\\\", \\\"Take photos of animals if you are in a desert somewhere. Color-wash the photos if you would like, but this doesn't mean you must have to actually picture your photos.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}