{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"27a57476-6a07-56d7-ae84-246aecb94416","task_key":"default--validation--21400","task_revision_id":"1","upstream_id":"21400","short_description":"HellaSwag validation 21400","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use a konjac sponge [title] Remove the konjac sponge from its airtight packaging. [step] Konjac sponges are most often sold in sealed plastic bags. Packaging them this way helps to ensure that they're not exposed to outside moisture prior to use.\",\"endings\":\"['[substeps] Try keeping your konjac sponge out of direct sunlight on a kitchen setting. It should be stored in a dry, cool, protected area, like in your garage.', \\\"[substeps] If storing konjac sponges in a sealed plastic bag, make sure that there are no holes in the bag or that it's airtight. If you do find any holes in your sponge, seal them in the container (s) you are working with.\\\", '[title] Remove the filter cover from the top portion of the sponge. [step] This usually involves a hook sticking off the bottom.', 'To open your new sponge, pull apart the plastic at one end of the bag. Check to make sure there are no visible defects or signs of damage.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}