{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a099a0bd-4f64-50f8-8243-c42d5b0bbb79","task_key":"default--validation--31441","task_revision_id":"1","upstream_id":"31441","short_description":"HellaSwag validation 31441","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to sprout quinoa [title] Gather your supplies. [step] For this project you'll need a fine mesh strainer , 2/3 cup of quinoa seeds, a bowl, water, a tray or plate, cloth, and a sealed glass container or plastic bag. [title] Rinse the quinoa seeds in a bowl.\",\"endings\":\"['[step] You need to wash the quinoa seeds to remove the soapy outer coating, which is known as saponin. Quinoa should always be rinsed prior to sprouting and cooking.', '[step] If your rinses for purposes of sprouting can be found at hardware stores, you can also purchase a mesh strainer for this project. [substeps] Fill the tray about halfway with fresh water.', '[step] Place the bowl into a strainer and remove any water that may interfere with the radish seeds. Rinse each seedling on a wire strainer for about 3 minutes, until no leaks are visible.', '[step] For the best results, rinse every single sunflower before trying to make it whole. Most quinoa should drain well into a tight strainer, but for ease of removing water, you can use your fingers or a sieve.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}