{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8c6b37eb-27b4-58d1-bf6e-eae90a9f0a6a","task_key":"default--validation--50536","task_revision_id":"1","upstream_id":"50536","short_description":"HellaSwag validation 50536","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to preprare homemade tater tots [title] Peel the potatoes. [step] Wash and scrub the 2 pounds (908 grams) of potatoes, then use a potato peeler to completely peel them. Dispose of the peelings and place the potatoes in a large pot.\",\"endings\":\"['[title] Cook the potatoes over medium high heat. [step] Fill the pot of potatoes up with cold water until the water covers the potatoes by about one inch (2.54 cm).', '[substeps] When ready, you can steam your potatoes in the microwave, which is preferable for tater tots because of the high temperature, or you can steam them on the stovetop over low heat. Blanching prevents the potatoes from becoming soft when cooked.', 'Cover the pot with water and bring the water to a boil over medium heat. [substeps] If possible, soak the potatoes for 4-6 hours, then strain them.', \\\"Boil the potatoes until the skins peel off. [substeps] If you prefer, you can also cook the potatoes while they're steaming or by simply cooking them in olive oil.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}