{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ef7d3f7-2929-5441-8b15-451072f8910d","task_key":"default--validation--17823","task_revision_id":"1","upstream_id":"17823","short_description":"HellaSwag validation 17823","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to tie a bandana [title] Fold the bandana into a triangle. [step] Lay the bandana flat on a solid surface (e.g. a tabletop). Smooth out any bunches or wrinkles in the fabric.\",\"endings\":\"['[title] Tie and tear a thin piece of bandana or felt. [step] You want a short length that you can tuck into a pocket to prevent it from unraveling.', \\\"[substeps] Fold the bandana in half a few times to hide the creases and don't crease it. [title] Tie the bandana to your neck with a wide, straight motion.\\\", 'Unfold the bandana in the middle of your triangle. As you fold the bandana, make sure to fold both ends diagonally.', 'Bring two opposite corners of the bandana together to create a triangle. [title] Position the bandana on your head.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}