{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a509941a-0aba-5d01-9197-7d66dfd0f96f","task_key":"default--validation--42802","task_revision_id":"1","upstream_id":"42802","short_description":"HellaSwag validation 42802","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to use a coffee grinder for spices [title] Choose your grinder. [substeps] Not all coffee bean grinders are equal when it comes to grinding spices. Many spices are much smaller than the average coffee bean, and you're likely to be grinding fewer spices than beans at any given time.\",\"endings\":\"[\\\"[title] Measure your coffee grinder's length. [step] Generally, the longer the grinder, the worse it gets at grinding spices.\\\", 'So choose a small coffee grinder with a 1/2 to 1 cup capacity; any larger and you risk being unable to grind the spices. Also look for a grinder that has a sharp blade that is positioned close to the bottom of the grinder.', 'Sizes of grounds vary depending on the grinder and the brand of quality coffee bean being used, and the type of grinder you choose. Examples of popular brands include brown ground, black ground, and post grinder.', 'Look for grinders online and at your local coffee retail store, or ask your gardening or art supply store to show you how to make your own grinder from your online selection. [title] Place the coffee beans between ceramic or wooden stirrers.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}