{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bcdd2fbc-9543-5372-8291-4c0282bc8edc","task_key":"default--validation--46464","task_revision_id":"1","upstream_id":"46464","short_description":"HellaSwag validation 46464","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use a stand up tanning bed [title] Find a reputable location that offers stand up tanning. [step] Stop by tanning salons in your area and ask them about the services they offer. Also search online to find the salon's website, which will list what services the salon offers, and customer reviews.\",\"endings\":\"['[substeps] Visit the locations listed to see if they make direct sales. If you are not able to visit a salon on your own, ask your friends if they would recommend the salon.', \\\"[title] Go in and sit down for a 15-20 minute consultation. [step] If you haven't been tanning before, or you'd rather sit in a private area, then go in.\\\", '[substeps] Ask to tour the salon before agreeing to a tanning session. Make sure the facility looks clean and has knowledgeable staff.', '[substeps] Tanning beds recommended can often be expensive. Remember, you can pay for the things you do get or the amenities the salon provides.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}