{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c21d123-7df6-5cca-95e6-58497948ab15","task_key":"default--validation--33220","task_revision_id":"1","upstream_id":"33220","short_description":"HellaSwag validation 33220","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to make a volcano with a film canister for a science fair project [title] Gather your materials. [step] You will need an empty film canister, modeling clay, and disposable cups to make the volcano. You will use baking soda and white vinegar to make the lava.\",\"endings\":\"['[substeps] Dry white vinegar works best for the volcano because of its consistency. You can also buy original vinegar, which would not be necessary if you used chocolate powder to make it, but baking soda works equally well.', '[substeps] Commercial volcano designs differ from those produced by film studios. Using a combination of boiling water and white vinegar is an alternative to using a chemical kettle.', 'Take the baking soda and vinegar mixture and put it into a smaller bowl. Bring it to a boil and place the white vinegar in a bowl.', 'Food coloring will color the lava if you choose to use it. You should also get a styrofoam or plastic tray to do the experiment on for easy clean up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}