{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7d4f3db1-461d-5641-8589-6cd737623e8d","task_key":"default--validation--18984","task_revision_id":"1","upstream_id":"18984","short_description":"HellaSwag validation 18984","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to put in an above ground pool [title] Find what you need. [step] Before you start to build your pool, make sure you're equipped with all of the tools you'll need. Many of the tools you'll use can be rented from your local equipment rental outlet.\",\"endings\":\"[\\\"In order to get started, you'll need a deck, a sheet of paper, and a ball game. Another helpful tool is a pool racker, which is a floating board with a hole poked through the center.\\\", 'Also, if your pool is new, you may be able to rent other equipment. Check online until you have everything you need.', \\\"If you're planning on using the different tools, store them with you. Make sure to install a 24 hour auto service router when you first begin your first year of operation.\\\", '[substeps] Shovel tape measure phillips head screwdriver duct tape sand filter skimmer patio blocks (2 \\\" x 8 \\\" x 16 \\\") wrench (5/16 and ¼) level rake [title] Open the hardware packets. [step] Go through and separate the hardware so that it is easier to find what you need when you need it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}