{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c161b6c-15dd-58c5-98ba-8677c42e1d61","task_key":"default--validation--37375","task_revision_id":"1","upstream_id":"37375","short_description":"HellaSwag validation 37375","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to apply clear coat [title] Choose a solvent and activator for your current weather conditions. [step] Think about size and area of the repair as well as the current temperature and humidity when making your choice. Some products are formulated for higher temperatures and extra moisture, so make sure to check the product labels.\",\"endings\":\"[\\\"[substeps] Popular solvents for hoses include ammonia and hydrogen peroxide, vinegar and ammonia. If you don't have access to the amounts you need per surface and chose a solvent, you can ask your vet for an estimate.\\\", '[substeps] Stackable acetone will work well for liquid stains like cola stains, sweat stains, or algae oil stains. Saline solutions are great for getting the job done and working on blue stains.', 'Applying a thinner coat during the dry season will cause humidity and moisture to build up, so opt for less heavy coats in the winter or check product details on the label. [substeps] You will pay a little extra for a thinner coat.', 'Keep in mind that you should also try to paint in optimal conditions. [title] Paint on a 75 ° f (24 ° c) day.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}