{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81416d55-32e3-59fc-9fca-4349cc8af6c6","task_key":"default--validation--21847","task_revision_id":"1","upstream_id":"21847","short_description":"HellaSwag validation 21847","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to take off false eyelashes [title] Purchase a false lash glue remover. [step] Most false eyelash kits come with eyelash glue, but they don't often contain glue remover. However, you can find many brands of eyelash glue remover in the beauty department of your local department store.\",\"endings\":\"['[substeps] Pay close attention to the type of glue, which differs from product to product. A commercially produced eyelash glue remover can produce a super thick paste that does not even bother people who wear false eyelashes.', \\\"[substeps] If you can't find eyelash glue remover, try going to a drugstore that sells eyelash glue removers. [title] Put the frozen fixer on the lash-lifter and remove the lid.\\\", '[substeps] When buying a false eyelash glue remover, place a stick of glue on the upper lash line. You can then wipe the stick of glue off.', \\\"Choose a brand that meets your specific skin requirements; when possible, choose glue remover made by the same company that made the eyelash glue you used. [substeps] Always check to make sure you're not allergic to any of the ingredients.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}