{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"44ed14c1-0aac-5412-80fd-558c164ffd59","task_key":"default--validation--23122","task_revision_id":"1","upstream_id":"23122","short_description":"HellaSwag validation 23122","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to make mummy hot dogs [title] Preheat the oven to 375 ° fahrenheit or 190 ° celsius. [title] Fry the hot dogs. [step] Preheat the stovetop to medium heat and spread olive oil over a frying pan or skillet.\",\"endings\":\"['Place the ten hot dogs on the pan to cook for about 5-7 minutes, until they are lightly cooked but not thoroughly. Once cooked, remove the hot dogs using tongs and place them on a chopping board.', 'Coat the hot dogs with the olive oil and bake them for about 5 minutes. [substeps] Stir occasionally while they are frying.', 'Use tongs to work the hot dog pieces through the oil. You can also check for doneness in advance by dropping the hot dogs into the pan; they should brown completely.', 'Save 2-3 fried hot dogs for later. [title] Beat the cream with butter or margarine (heavy cream) and set aside.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}