{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"54541037-4510-58cd-91f9-d24bd365bc8b","task_key":"default--validation--44345","task_revision_id":"1","upstream_id":"44345","short_description":"HellaSwag validation 44345","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to incorporate vegetables into a healthy breakfast [title] Make a vegetable scramble for breakfast. [step] A vegetable scramble is a simple dish made by sautéing vegetables in a shallow pan and then adding beaten eggs over them to hold the dish together. Begin by adding any vegetables you like, including onions, garlic, bell peppers, chili peppers, tomatoes, zucchini, broccoli, mushrooms, asparagus, or spinach.\",\"endings\":\"['Then, you can top it off with a variety of fruits, such as turnips, apples, bananas, and pears. This recipe is exceptionally simple to bake.', '[substeps] Combine any vegetables in the scramble as they are in the pan. For example, you could add tomatoes, broccoli, mushrooms, zucchini, or lettuce.', \\\"Cook over gentle heat or substitute beet stock. [substeps] Keep the amount of vegetables small so they don't crowd the pot.\\\", 'Once the vegetables are cooked down, pour one or two beaten eggs over the top, and allow it to cook thoroughly. [title] Incorporate extra vegetables into a hash brown preparation.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}