{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a0724db2-a65b-5345-8adb-1aa7a43eb298","task_key":"default--validation--17621","task_revision_id":"1","upstream_id":"17621","short_description":"HellaSwag validation 17621","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to make a volcano [title] Lay down a sheet of wax paper. [title] Place a container at the center. [step] Use a container (soda can, mason jar, plastic bottle, etc.\",\"endings\":\"[') to catch any moisture, as a volcano can steam out debris more quickly. [substeps] Be sure to place the lid securely.', ') to hold the volcano (s). [title] Cut a candy thermometer from the wax paper.', ') to form the center of your volcano. This is where you will mix your lava! [title] Use clay to make the rest of the volcano.', \\\") to hold all the water in the volcano, if there are any. If there isn't a container, fill it with ice.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}