{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a95c01b6-b439-50ad-ad2c-4d38ba4b2e79","task_key":"default--validation--27745","task_revision_id":"1","upstream_id":"27745","short_description":"HellaSwag validation 27745","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to get renters insurance [title] Make a list of your items. [step] Renters insurance covers the cost of lost, stolen, or damaged items housed at an apartment or home you are renting. In order to evaluate what kind of policy you should buy, you should make a list of items you own.\",\"endings\":\"['Once you find the items you need to get renters insurance for, try checking online for reviews of current and past renters policies to see how much you already own. [title] Start by identifying items that you are willing to rent for a short fee.', 'At the bottom of the list, include what types of repairs you intend to make, which will attract renters, and which repairs can be made without renters insurance. [title] Create an account on your new policy.', \\\"You can add up about how much your total property costs so that, in the event of a total loss, you'll know how much you would need from insurance to replace your assets. [substeps] You can break down your items into categories to help with the process.\\\", 'Doing this will make your insurance plan very effective. [substeps] For example, if you own a used car, then it would be very necessary to make a list of all of the hours you drive each day.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}