{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"849a2b7e-3d3f-545f-97af-5a28a31fb06f","task_key":"default--validation--20785","task_revision_id":"1","upstream_id":"20785","short_description":"HellaSwag validation 20785","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to unclog a drain with salt and vinegar [title] Mix salt and vinegar. [step] In a small bowl, pour 1 cup of salt. Add 1 cup of vinegar.\",\"endings\":\"['Place 1 cup of distilled white vinegar in a small bowl. The salt and vinegar will help the solution penetrate the clogged drain.', 'Stir well so the salt soaks up all the vinegar. Continue stirring until the mixture is smooth and even.', 'Mix the salt in, going slowly but bringing the mixture to a boil. Always stir the solution well, making sure nothing gets broken.', 'If possible, give it an extra stir. The vinegar will set the clog, and it will give it a smoother initial scrubbing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}