{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b25c556f-5be5-5804-b781-e18963945227","task_key":"default--validation--26648","task_revision_id":"1","upstream_id":"26648","short_description":"HellaSwag validation 26648","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean up latex paint [title] Remove the paint while it is still wet. [step] Removing paint from carpet is easily removed when paint is still wet with a stain remover. Spot shot is a recommended brand of stain remover.\",\"endings\":\"['Using a white rag, scrub off paint. Clean with water then repeat the steps with the stain remover.', 'Grease-containing paint has a very high concentration of ascinol, which is very toxic to pets and is very messy and can cause vomiting. Products with less ascinol are, however, more effective.', 'This cleanser will be available at most paint repair stores. [title] Wash your carpet with mild dish detergent.', 'Remove small spills with a damp washcloth or rag. [substeps] Spray paint remover is also approved for use on carpet, and contains protective chemicals that can damage or permanently damage carpet fibers.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}