{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"97250bcd-d93c-59cf-abda-9dbc06d875e6","task_key":"default--validation--18039","task_revision_id":"1","upstream_id":"18039","short_description":"HellaSwag validation 18039","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to sneak healthy food to kids [title] Add vegetables to pasta sauce. [step] When you are nearly finished boiling your pasta, add your child's favorite vegetable to the boiling pot. Drain the vegetable with the rest of the pasta and then add it to the pasta sauce.\",\"endings\":\"['[substeps] You might choose to add some broccoli to the pasta sauce. Broccoli is a great source of nutrients and vitamins like thiamin, fibre, vitamin k, and folate.', \\\"You don't need to keep the pasta as it boils to prevent it from getting watery. [substeps] You can omit vegetable puree for pasta sauce if you prefer.\\\", 'Kids often prefer pasta with vegetables because it is similar to noodles but with less liquid. [title] Put your pasta into the microwave oven.', \\\"The vegetables will make the pasta appear less greasy and prevent any of the sauce from dripping onto your child's arm. [title] Fish out other parts of the pasta.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}