{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ee12025-515f-57f8-a157-bc42c0a7a8cc","task_key":"default--validation--18005","task_revision_id":"1","upstream_id":"18005","short_description":"HellaSwag validation 18005","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to shop for high heel shoes [title] Take your time. [step] Don't rush in buying the shoes. Yes, i know that they are so beautiful you can't help yourself.\",\"endings\":\"[\\\"However, , these shoes didn't completely sit right on your feet until they were promoted so you need to be fairly careful with them before you buy them. [title] Opt for high heels with toe-less shoals.\\\", \\\"But be detailed when you do browsing the aisles. Buy only one pair at time, otherwise you'll cause too much problems.\\\", 'But are they comfortable? Try them on in the shoe store. Wear them for about 10 minutes at a normal pace, not too slow (as we usually do in a store).', \\\"However, it doesn't mean that you should wait forever to find high heels. After all, time will help you achieve them! [title] Find out what level of support you want to get.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}