{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5b7406ac-6406-521d-87f7-36c8bcb4b225","task_key":"default--validation--31682","task_revision_id":"1","upstream_id":"31682","short_description":"HellaSwag validation 31682","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to cook fenugreek [title] Grind the seeds to add the flavor without cooking the seeds. [step] Fenugreek seeds are quite hard, so it can be difficult to grind them by hand with a mortar and pestle. Instead, put the seeds in a small coffee grinder and let the machine do the work for you.\",\"endings\":\"[\\\"Grind the fenugreek seeds until they're well-pureed. [substeps] Use a coffee grinder with a beater attachment or with the lid off to grind the fenugreek seeds to a fine powder on a flat surface.\\\", '[substeps] You can also try to buy fenugreek seeds that are already ground up, directly from the store. [title] Dry roast the seeds on the stove for a more edible addition to your dish.', 'Grind away until all of the seeds are ground, then add water to the grinder. [title] Add lemon juice to the grinder.', '[substeps] Fenugreek is often boiled by boiling the seeds for just about 10 minutes. It is finely ground into a fine powder and used as a spice for soups, salads, and other varieties of breads.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}