{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c9a767d-b8a3-5e3f-b6e4-ce3ff308be99","task_key":"default--validation--12909","task_revision_id":"1","upstream_id":"12909","short_description":"HellaSwag validation 12909","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to look good in a speedo [title] Get a speedo made of high quality, long lasting fabric. [step] Look for a speedo made of 100% polyester or a polyester blend with spandex. It will be more chlorine resistant and long lasting.\",\"endings\":\"['Make sure that the speedo has a quality grooved neck feature so that there is less pressure on your neck. [title] Try to find an olympic style speedo.', \\\"A speedo made of spandex has pure polyester threads that wick away from the speedo and they can be boiled at high temperatures while they are stored inside. Speedo's are generally very good at opening, hooking, shooting and spotting.\\\", 'Some speedos are also made in a high uv rating fabric, making them more resistant to the sun if you plan to swim outdoors. [substeps] Avoid speedos made of only spandex or lycra, as they tend to fade and get loose more quickly.', 'Make sure that your speedo is split down the middle, so that there are no sharp edges, gaps, or ruffles. [title] Match your speedo to your general personality.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}