{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a1718075-efe5-5a64-b4b2-73ed529d82d0","task_key":"default--validation--9561--variant-7ff776fec97a","task_revision_id":"1","upstream_id":"9561--variant-7ff776fec97a","short_description":"HellaSwag validation 9561","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to shop for boat insurance [title] Resist the impulse to simply call your current insurance provider and purchase boat insurance from them. [step] Many general insurance providers lack the experience with boats to make sure you get all the coverage you need. While you may ultimately find that you go with your original provider, you need to do some additional work to be an informed consumer.\",\"endings\":\"['If you feel sure that your current providers have a good opinion of you, then you may want to rely on your original providers anyway. [title] Examine your budget before considering boat insurance.', 'Simply call your current provider to buy new boat insurance and ask to be covered. Even if you know what type of boat insurance coverage you want, be aware you may have to pay for it later in life.', '[title] Decide whether you want an actual cash value policy, or an agreed amount policy. [substeps] Actual cash value policies pay for replacement costs * minus depreciation * at the time of the loss.', 'If you want to buy boat insurance, but you want to make sure you are sailing through waters, you must first visit a veterinarian and register to go. [title] Subscribe to the boat insurance website.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}