{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"70defded-fb75-55a7-96ff-ee88798e38ee","task_key":"default--validation--49558","task_revision_id":"1","upstream_id":"49558","short_description":"HellaSwag validation 49558","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to repair a chip in tile [title] Find a tile to replace the broken tile with. [step] You may have extras on hand from when the floor was installed. You may need to go see a tile or flooring store to purchase another one of your tiles, or if that style is no longer available, you may need to settle on a complimenting style.\",\"endings\":\"['[title] Remove the plastic covering and repair the chip. [step] To patch the chip, you will need to trim off the plastic that covers the area where the chip was.', 'Maybe you prefer bold patterns for your chipped tile. Maybe you prefer geometric design for your linoleum floors.', '[title] Get some matching grout as well. [step] The grout is available at tile or flooring centers.', 'There are several different types of custom tile available, and when searching for your tiles, look for the one that works best for your project. [substeps] Decide what type of tile you want to replace the chip in and shop around for different options.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}