{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bbe0c6fa-cdf7-5809-b75d-d6a40020c58a","task_key":"default--validation--28992","task_revision_id":"1","upstream_id":"28992","short_description":"HellaSwag validation 28992","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to sell photos to magazines [title] Use what you know. [step] You likely already love some magazines. Use that to your advantage, and find out to submit to them.\",\"endings\":\"[\\\"With anything you know, know that magazines can take your photos and print them at a moment's notice. With each company you make unique use of them to entice a larger company or a greater audience.\\\", 'Be open to the possibility of getting one on one contact with magazines that you think you should publish. Photography is a strength rather than an impediment, as people will see it as a weakness to you.', \\\"[title] Pick photos of your subjects, if that's your feeling. [step] Try not to agonize over what you will look like (considering how many pictures to submit).\\\", \\\"The more you know about the magazine, the more likely you'll find one of your photos that will be the perfect fit. [title] Visit a bookstore, magazine stand, or your local library.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}