{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a9423d94-d587-5448-8eaf-7710b3f0e539","task_key":"default--validation--6859","task_revision_id":"1","upstream_id":"6859","short_description":"HellaSwag validation 6859","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to get bug bites to stop itching [title] Cool the bite with a cold compress. [step] The cold will reduce the swelling and numb the area a little bit. [substeps] If you don't have an ice pack ready and waiting for use in the freezer, you can quickly make one by wrapping ice cubes or a package of frozen peas in a towel.\",\"endings\":\"['Put one wrapped towel over the bite site. Cover the area with a thin layer of ice for a couple minutes, then remove the wrap and let the cool completely.', 'Immediately apply ice to the affected area for approximately five minutes at a time. Repeat this process every ten minutes until the area stops itching.', 'Be careful not to hold the ice on your skin for too long. Limit it to 15 to 20 minutes at a time.', '[title] Apply a thin layer of ice over the affected area. [step] The cold will help neutralize the insect bite after the fact.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}