{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13b7cd36-7582-53c6-98c2-450adc19800f","task_key":"default--validation--26038","task_revision_id":"1","upstream_id":"26038","short_description":"HellaSwag validation 26038","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to build a skateboard [title] Find a suitable workspace. [step] Use a large, flat workspace where you can assemble your skateboard. You will be dealing with a lot of small parts like nuts, bolts, and washers.\",\"endings\":\"[\\\"Try a smaller space where you won't have a lot of room to work your skateboard. [substeps] Make sure to avoid storing large appliances, like refrigerators, like nuts, bolts, and washers.\\\", '[substeps] A large work table or open space on the floor works well. Just make sure that you have enough room to lay out all your parts and tools so you can easily find what you need.', 'Avoid any sharp buildings with heavy metal on them or that may cut the board if you fail to complete it properly. Changing the space you choose will also put you in the position you want to be on the board when you start making its ramp.', 'For example, you should only build a skateboard from a tire or deck. The frame needs plenty of space for the wheels to form.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}