{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3bc8d825-e083-50a9-b379-f012c56d0006","task_key":"default--validation--35833","task_revision_id":"1","upstream_id":"35833","short_description":"HellaSwag validation 35833","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to make a rose and lavender perfume [title] Look round your garden for roses and lavender flowers. [step] Make sure they are fresh and smell great, before picking 1 good sized rose and about 3 lavender flower heads. [title] Remove the petals from the rose and the buds from the lavender.\",\"endings\":\"[\\\"[step] The flower heads can get quite dirty. Well, in the end, don't worry, it will all be fine.\\\", '[step] Be sure to follow the instructions provided on the packaging with a special tape to avoid damage. You may need to tape several roses within a 12 inches (30.', '[step] Place the petals and buds in a small bowl ready for washing. [title] Rinse the petals and lavender buds gently with water.', '[step] Some flower heads do this more easily than others, so make sure you have plenty. [title] Gather the roses to form a bunch of flowers.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}