{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"79341c08-12a8-519a-8b44-3484a948233c","task_key":"default--validation--21139","task_revision_id":"1","upstream_id":"21139","short_description":"HellaSwag validation 21139","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a car rack or carrier [title] Choose a car rack that's compatible with your type of vehicle. [step] Most car racks have mounting requirements. For example, hitch rack requires a trailer hitch, a spare tire rack only attaches to suvs with a spare tire mounted on the rear of the vehicle and a trunk rack may not work with cars with a spoiler.\",\"endings\":\"[\\\"[title] Consider your vehicle's mounting ability. [step] Before buying a car rack, consider the stability and durability of your vehicle.\\\", \\\"Consult your owner's manual to find the best vehicle rack options for your type of vehicle. If your car may not meet the standards you desire, consult your owner's manual for more information about how to use your car rack.\\\", \\\"Consult your owner's manual or ask the salesperson for recommendations. [title] Decide if a front-loading machine or a heavy-duty vehicle already has racks built in.\\\", '[title] Select a car rack that will accommodate the equipment you want to transport. [step] Trunk racks, hitch racks and spare-tire racks transport bikes, but the number of bikes vary depending on the type of rack and model.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}