{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"84c013fb-3c40-5060-8cb7-d0f079a73b2e","task_key":"default--validation--15269","task_revision_id":"1","upstream_id":"15269","short_description":"HellaSwag validation 15269","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to stop enjoying carbs [title] Say \\\" no \\\" to white bread. [step] Bread is the most common simple carbohydrate we consume and the one that is easiest to remove. [substeps] Try lettuce wraps.\",\"endings\":\"['Lettuce wraps are the easiest toppings to remove, but they are equally effective at reducing the number of times you feel nauseous. Try olive oil and a fresh lemon and/or cucumber.', 'Many different greens are a great source of carbohydrates, even when you are in the habit of eating a lot of them. Try tomato in a stir fry, fish salad, or brownies in a lasagna, for example.', 'Wrap the top of a head of lettuce cut in quarters--simply roll one or two slices into a ball or catch them with a quick, powerful hand as the wrapper changes from red to black. Two other bread options to treat are giant intestine wraps.', 'Romaine lettuce works well to substitute for flat breads, while butter lettuce and radicchio make an excellent substitute for sandwich bread. Simple add sandwich filling to the lettuce and fold over and around, or add a second piece of lettuce to the top.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}