{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"33ad2751-f82e-5f53-8fa6-e7d99ab57dc4","task_key":"default--validation--16075","task_revision_id":"1","upstream_id":"16075","short_description":"HellaSwag validation 16075","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be the best man [title] Choose the activities and location for the bachelor party. [step] The bachelor party is an opportunity to spend time with your friend and the rest of the friends in your close circle. Choose an activity and location which are informed by his interests and personality.\",\"endings\":\"['For instance, if your friend loves paintball, organize the party as a massive paintball battle. Between rounds of paintball you could enjoy some pizza and reminisce about your friend.', '[substeps] If your friend is a big fan of baseball or the movies, find a new venue to go to. If your friend loves sports, start researching local teams so you can find an appropriate place to go for the party.', \\\"[substeps] Outside of the bachelor party, a few of the other people your friend knows may be interested in going to the bachelor party and will like to spend time together. Switch roles if some of your friend's friends don't like that activity.\\\", 'Make the task as easy as possible for your friend, who is willing to take in an hour of lesson on even the least of the specifications. [substeps] Pick a location close to the party.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}