{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"541d648a-0fce-5202-891e-e743fd1c39ca","task_key":"default--validation--37751","task_revision_id":"1","upstream_id":"37751","short_description":"HellaSwag validation 37751","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to start a vending route [title] Choose the type and size of the locations you want for your vending route. [step] The places you put your machines will determine which vending machines you need to purchase. You should know : [substeps] Whether your machines will be inside or outside; the type of products people at the location would be interested in vending; and approximately how many people will use your machines.\",\"endings\":\"['This will help you decide what type of vending machine to have. For example, if you have gas carts with vending machines in them, you would want your machines to be able to use them.', 'How many machines can you build on your location. Generally , 20-30 machines, so models may be nearby, larger includes retail, craft, stationary, bridge, stationery, and colored plastic/wooden buildings.', '[title] Choose the type and size of machines that best meet your location needs. [step] Guesstimate the size of the equipment you need by assuming a daily average of one vend, per machine, per person.', 'Alternatively, you can purchase multiple vending machines. These might include : [substeps] Grocery stores and large grocery stores anywhere outdoors during cooler months.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}