{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b3fae68c-621d-56b6-8f94-9bb9ab738fdc","task_key":"default--validation--50315","task_revision_id":"1","upstream_id":"50315","short_description":"HellaSwag validation 50315","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean granite countertops [title] Prepare countertops for proper cleaning. [step] Before you begin to actually clean the countertop, remove excess debris and move appliances or items so you can reach all areas. [substeps] Wipe up any spills or sticky residue on counter.\",\"endings\":\"['The best way to avoid staining your countertop is to remove liquids or sauces immediately from your countertop. If the liquid has set, use a hot wet rag to loosen and remove the caked on debris.', 'Wipe up any favorite color color or item. Also wipe up any spills in the cookware.', 'Wash work surfaces with soap and water and scrub surfaces which have accumulated dirt. Prepare a new countertop surface when it becomes chipped, broken or clogged.', 'Wipe up any spills with a damp kitchen sponge or rag. Use a garden hose to move cleaning supplies away from the countertop.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}