{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c16ab5f6-306f-5d24-bb12-dbeb0e87df7f","task_key":"default--validation--29654","task_revision_id":"1","upstream_id":"29654","short_description":"HellaSwag validation 29654","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make fried smashed potatoes [title] Place the potatoes in a pot and cover with water. [step] Add 4 pounds (1.8 kg) of small red or yukon gold potatoes to a large pot. Fill the pot with enough cold water to cover the potatoes by at least 2-inches (5-cm).\",\"endings\":\"['[substeps] Fingerling potatoes also work well for the recipe. [title] Season the water with salt and bring to a boil.', 'Add salt to the water so the potatoes are submerged. The potatoes should smell fresh and fragrant as they are infused with freshly cooked onions and garlic.', 'Let the potatoes soak for 10 to 15 minutes. [substeps] Do not use 1 pound (0.45 kg) of frozen potatoes in this recipe.', '[substeps] Alternatively, if you prefer to cook the potatoes on their own, you can simply boil the water for the oven to bake them in. This should be enough time so that the potatoes can brown thoroughly and cook evenly.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}