{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b66745f4-ba3b-5337-ba26-eb69ca00c1f3","task_key":"default--validation--9966","task_revision_id":"1","upstream_id":"9966","short_description":"HellaSwag validation 9966","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to get multiple admirers (women ) [title] Be a lady. [step] Although there are many females in the world, not all of them are the most pleasant creatures. In order to stand out from the crowd, treat others with kindness and love.\",\"endings\":\"[\\\"This shows the guys that you aren't afraid to hit the ball. They'll be impressed that you know how to walk, that you listen and enjoy being approached, and that you're a little funny.\\\", '[substeps] If you see a pretty woman sit next to another female, try to strike up a conversation. You could say, \\\" hi, i\\\\'m anna.', 'Additionally, follow the below tips : [substeps] Be polite to everyone. Despite how difficult this step may be, it is essential.', 'Are they nice to be around? Are they happy and amused? Do they have a rewarding life? Consider being gracious and kind to her among your friends. Doing the things that she wants, offering her favors, helping her at work or anything else shows that you care about her and are willing to support her.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}