{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c9ebdea-8d8b-53a8-941b-ddf4f59b2690","task_key":"default--validation--6643","task_revision_id":"1","upstream_id":"6643","short_description":"HellaSwag validation 6643","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to home tan [title] Pick out some tanning spray or lotion. [step] Drugstores have tons of different options for home tanning, and most of them won't break the bank. Select a type of home tanning spray of lotion that works for your skin tone and texture.\",\"endings\":\"['[substeps] You can find home tanning spray at your local hobby or drugstore. Make sure to choose the right kind for your skin type.', \\\"[substeps] Great brands include jelly sole and smartredge. [title] Adjust your workspace so that you're sitting quietly.\\\", \\\"[substeps] When using a spray, look closely at the product to make sure it doesn't contain artificial ingredients. Inside the bottle you'll probably see some packaging to help you decide which type of home tanning product works best for your skin tone.\\\", 'You probably only need one bottle for your body, but if you want to ensure full coverage, you may want to buy a backup. [substeps] Choose a tanning color just a few shades darker than your normal skin tone.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}