{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"658a08fa-8d3e-5aa1-bac3-59dfd3ebd240","task_key":"default--validation--42342","task_revision_id":"1","upstream_id":"42342","short_description":"HellaSwag validation 42342","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to breed mealworms [title] Purchase suitable bins. [step] You'll need shallow bins with smooth walls made of either glass or plastic, so that the worms and beetles aren't able to climb out. 10 gallon (37.9 l) fish aquariums work very well, as do plastic storage containers.\",\"endings\":\"[\\\"The containers need tops with tiny vents or airholes (wire mesh works, as does poking holes in the lid) that allow for airflow without letting the mealworms escape. Darkling beetles can't fly, but a lid is a safety precaution that i would have.\\\", \\\"However, be sure to also secure their containers with perfab wire to prevent the potential bin from leaking out during the hatching process. If they don't have enough room to cover any, weigh down the boxes further to allow them to move upwards without rotting.\\\", '[substeps] Plan to have at least 2 1-liter (. 6 l) bin containers for every six gallons (3.7 l) of fish you breed.', 'Decorations can also work, but opt for plastic. As with anything, you can use anything from tree ornaments to filleted and painted fish.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}