{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"548e4b3c-dddd-54a6-af31-5bc49232e598","task_key":"default--validation--44043--variant-2946af69c276","task_revision_id":"1","upstream_id":"44043--variant-2946af69c276","short_description":"HellaSwag validation 44043","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make rutabaga fries [title] Preheat the oven and line a baking sheet. [step] To ensure that the oven is hot enough to bake the fries when they're ready, set the temperature to 400 degrees fahrenheit (200 degrees celsius) and allow it to heat fully. Next, use a piece of parchment paper to line a large baking sheet for the fries.\",\"endings\":\"[\\\"[substeps] You can substitute foil for the parchment paper if you prefer. It's important not to place the rutabaga fries directly on the baking sheet, though, because the edges will burn more quickly.\\\", '[substeps] Note that you may want to stick some heat resistant glass or steel foil over the front of the baking sheet. The glass or metal coating inside will prevent the fries from sticking together as they bake.', \\\"[substeps] It's okay if the sheet burns instead of browning at this point because the fries will be softer to fry. Avoid using waxy paper because it will degrade.\\\", '[substeps] You can use foil to transfer the fries from the oven to the greased baking sheet. A clean, oven-safe sheet is ideal.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}