{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"63fbc619-0a06-592d-8e27-444a295f0559","task_key":"default--validation--8655","task_revision_id":"1","upstream_id":"8655","short_description":"HellaSwag validation 8655","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to bake a cake on a grill [title] Turn on the heat on a gas grill. [step] When working with a gas grill, you'll simply need to switch on the heat. If you've never baked on a grill before, it'll take some experimentation to get the grill to the right heat.\",\"endings\":\"['Start by turning on only one burner and go from there as needed to achieve the proper temperature. [title] Arrange your charcoals for a charcoal grill.', \\\"You'll get a range of temperature gauges and an oven thermometer. Your grill should be gas-operated or steam-operated.\\\", 'Here are some suggestions for what temperature to place your cake on your grill : [substeps] Not too high. Low settings can cause the cake to melt too fast.', \\\"Set the temperature at least 120 degrees fahrenheit (43 degrees celsius), depending on how cold you'd like the cake to be. [substeps] If you need to turn off the heat early on, simply turn the heat off right away.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}