{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fe1dd67d-752d-5cd4-bbc7-5a503b72bcee","task_key":"default--validation--33142","task_revision_id":"1","upstream_id":"33142","short_description":"HellaSwag validation 33142","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to plan a kitchen renovation [title] Decide how much you can spend on your kitchen renovation. [step] The amount of your budget will often dictate the rest of the planning process. [substeps] Establish a range instead of a fixed budget number.\",\"endings\":\"['A fixed budget allows you to control how much you spend on your renovation and what types of renovations you can do at a time. Factor in the cost of construction, ongoing construction costs, remodeling, and your expected renovation.', 'Decide which rooms will be rectangular and which will be partitioned into sections. Calculate portion sizes by multiplying the total number of sections into two x two.', 'For example, budgeting $7 , 500 to $8 , 500 is more reasonable than deciding to spend exactly $8 , 000. [title] Include all of the elements of a kitchen renovation when estimating your budget.', 'For example, a fixed budget of $100 , 000 or less is a good starting place. Find common resources in this range at the better business bureau (bbb) or online.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}