{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"49c93b6e-5313-5e99-9e36-2ee719a76f2a","task_key":"default--validation--48651--variant-6ee373313c72","task_revision_id":"1","upstream_id":"48651--variant-6ee373313c72","short_description":"HellaSwag validation 48651","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make mexican pizza [title] Preheat the oven and grease the pans. [step] To get the tortillas crispy and ready for the pizza, it's important to get your oven to the right temperature. Preheat it to 375 degrees fahrenheit (approximately 190 degrees celsius).\",\"endings\":\"[\\\"Get your baking sheets ready by lightly greasing them with non-stick cooking spray. [substeps] Pay close attention to your oven to be sure that it's at the right temperature.\\\", \\\"[substeps] The best temperature is specified above the stove top, as well as in the oven itself. Call a pizza artist if you aren't sure if your oven has the right temperature.\\\", \\\"Check the temperature on the oven's thermometer to see how well it's cooking. The inside should be browned, and the oven should beep or flash when it reaches 155 degrees f (66 degrees c).\\\", \\\"[substeps] If you have a cooler oven, you may want to preheat it to 360 degrees fahrenheit. This will give the tamales the best crisp, but if you'd rather only use a bowl or all your tortillas, you can store them at room temperature.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}