{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4ae9221d-0341-5b3a-a5f4-69001437d186","task_key":"default--validation--15539","task_revision_id":"1","upstream_id":"15539","short_description":"HellaSwag validation 15539","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to remove a ceiling tile [title] Cover the floor with a protective material if desired. [step] Gypsum ceiling tiles tend to crumble around the edges, especially as they get older. If you want to keep this debris from getting onto your floors, lay down something to protect them, such as sheets of newspaper.\",\"endings\":\"[\\\"Usually, it won't come down on hard surfaces; instead, it will come down along the floor near the tiles. This can protect the surface but will also prevent tiles from sticking.\\\", '[substeps] If you are installing a ceiling tile over a concrete floor, make sure it is not 3 centimeters high, as it will be more difficult to get over the edge. [title] Consider using strips of kitchen aluminum sheet that are slightly larger than the ceiling tiles to prevent dust and build-up from building up.', 'A tarp is ideal if you are removing large amounts of ceiling tile. [title] Position a chair or stepladder underneath the tile.', '[title] Consider painting your ceiling white, if desired. [step] Many painters use this color in addition to primer to prevent blemishes and cuts in the tile.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}