{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"110bfbca-bbae-5d77-b515-34afe794001e","task_key":"default--validation--41143","task_revision_id":"1","upstream_id":"41143","short_description":"HellaSwag validation 41143","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to add umami to your cooking [title] Add cheese to pasta dishes and soups. [step] Parmesan has a salty, strong taste that only gets stronger with age. Add it to pasta sauces, either on top of the dish or into the sauce as it cooks, to create a deeper flavor.\",\"endings\":\"['[substeps] Add parmesan rinds to soups, like minestrone, to achieve the same effect. [title] Use anchovies to add the umami taste to a variety of dishes.', \\\"[substeps] You should add an even amount of cheese per dish (about 3/4 of this should always be grated). If adding it to pasta isn't convenient for you, get a handful of grated cheeses from the grocery store.\\\", 'Consider adding of your own cheese, as italian dishes use tomatoes and other fruits to flavor an array of foods. Pair tomato slices with a wedge of cheese and other parmesan mixes in place of pasta, substitute cheese for pasta, or add more avocado or spicier additions.', '[substeps] You can also add cheese on top of pasta dishes or soups. You can be creative and flavour your dishes using small amounts of cheese, but cheese is often the bread that goes with the pasta dish.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}