{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"22059c1c-1857-570a-a714-98dcc5bceb65","task_key":"default--validation--45022","task_revision_id":"1","upstream_id":"45022","short_description":"HellaSwag validation 45022","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to stain concrete floors [title] Smooth the surface. [step] Work across the floor using a sander, mechanical buffer, or floor grinder equipped with diamond bits. Focus on areas covered by visible imperfections.\",\"endings\":\"['Work in corner locations and near entryways. Work toward the edges of the room where you will be painting.', 'Dust, mop, or vacuum hard surfaces thoroughly to smooth them. [title] Paint the floor once a month with a primer or varnish.', '[title] Fill in spray paint with acrylic varnish, if necessary. [step] Spray paint is used to saturate the concrete and seal it in.', \\\"[substeps] This may not be necessary if the concrete floor is relatively new, but you'll need to follow this step if the concrete has any paint stains or similar spots on it. Likewise, if you removed tile flooring from the concrete before starting, you'll need to use a floor grinder to remove any stubborn bits of mortar from the floor.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}