{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c8b64bee-aa5a-5edb-83c8-7bce68d8c7cf","task_key":"default--validation--48351","task_revision_id":"1","upstream_id":"48351","short_description":"HellaSwag validation 48351","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to create chips and dip [title] Pre-heat the oven to 200 degrees, before-hand. [title] Wash the potatoes. [step] Make sure they are clean and any other bits which stick out have been peeled.\",\"endings\":\"['Though, leave the skin on if you can! [title] Cut them. [step] Cut the potatoes in half, half again, and half again! Cut all potatoes into eighths.', 'Either peel the potatoes or cut them up into small pieces. [title] Place 1 chicken head into the baking dish.', '[title] Place them in the oven and broil the potatoes for about 20 minutes. [step] This will give them their look, but not be their killing agent.', \\\"Split them in quarters, one or two shells. This way they are easier to dip and don't get moist during cooking.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}