{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0756fd0d-2ce2-5dcc-9c21-49a5ac2d6802","task_key":"default--validation--30437","task_revision_id":"1","upstream_id":"30437","short_description":"HellaSwag validation 30437","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to grow valerian [title] Purchase valerian seedlings or starts from a garden shop. [step] They will usually be in the herb section of the store. Valerian seeds can be finicky, so this is the best way to ensure you will have young plants for your garden.\",\"endings\":\"['[title] Cut between 20 and 30 inches (60 and 59 cm) of valerian root. [step] Do not cut larger ones than four or five feet long.', '[title] Consider harvesting \\\" daughter \\\" or volunteer sections of the root system from older plants. [step] If valerian grows naturally in your area, you can usually find a number of young plants sprouting in the spring.', '[substeps] Try looking for seeds that are difficult to care for in the herb section of the store. Some seeds may be tightly-packed, so check the soil to find a few.', 'You can obtain valerian seeds online or sell them for a price somewhere between $20 and $40. [title] Prepare two large pots with a wire lid.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}