{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"66f6f472-46a7-5d0b-9ab0-5af838186313","task_key":"default--validation--22043","task_revision_id":"1","upstream_id":"22043","short_description":"HellaSwag validation 22043","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to make your pavers level [title] Excavate. [step] Before you can install pavers you need to excavate the area 6 to 8 inches (15. 2 to 20. 3 cm). [title] Use a level.\",\"endings\":\"['[step] A level means that you start at the center point in the soil. When you excavate the site, you dig around the desired level.', '[step] Use a level to drill at the center of the raised paver. Drill slowly to prevent the paver from tipping over.', '[step] Take a large square rod and run it over the excavated area to ensure that it is flat and level. If you feel it rising and falling you will need to readjust the soil surface to make it even.', '[step] You can do this using a straightedge on the side of the tool bar, as long as it is big enough to hold the grit of your work and flush against the sharpened wood. Screw the level onto the top, or use a drawer, or use a vise, instead.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}