{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"352e968c-9ce1-5cce-a5a9-dde9fe69fec7","task_key":"default--validation--49817","task_revision_id":"1","upstream_id":"49817","short_description":"HellaSwag validation 49817","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to paint jeans [title] Cover your work area. [step] Protect your surfaces from the paint or it may stain them. Make sure to cover a large area.\",\"endings\":\"['You will be splattering the paint onto your jeans and it is likely to splatter all around your jeans as well. Lay out your tarp or newspapers to cover the area you will be working on.', 'You might want to put out a tarp or blanket for added protection. Do not make the area wet; any paint sitting around your work area could stain your floor.', 'A single spill can permanently stain denim if it gets onto your skin, so make sure to have everything else covered. [substeps] Get accumulation of lots of cleaning substances underneath your work surface.', \\\"Cover the top areas of your work area with paper towels or another material. Use painter's tape or duct tape.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}