{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd403f77-0bd7-5b0c-b0d0-21a85e265c5c","task_key":"default--validation--22366","task_revision_id":"1","upstream_id":"22366","short_description":"HellaSwag validation 22366","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to make valentine's day extra special [title] Arrange for dinner. [step] You can book a reservation at your favorite restaurant or you can cook. It doesn't even have to be fine dining.\",\"endings\":\"[\\\"Get out the favorite food, let your guests get to know you, and then let them order their own food. The point is to get your guests so impressed that they will pay the bill on valentine's day.\\\", \\\"A better option would be to enjoy a classic vinaigrette, appetizer, or dessert. When not attending holiday events, consider going to events like new year's eve or a restaurant inspired by a similar theme.\\\", \\\"Choose something informal and don't overdo it. All this matters is where it is in the relationship and if you can satisfy everyone involved.\\\", \\\"Often just putting something together in advance is a good way to show your valentine that you care and that you're thinking of them. Add some candles, turn down the lights, and it's a perfect romantic evening.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}