{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"afb787ac-1118-55b2-8e8d-e61410899186","task_key":"default--validation--1146","task_revision_id":"1","upstream_id":"1146","short_description":"HellaSwag validation 1146","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to select a headstone [title] Identify the cemetery's size limitations for head stones. [step] Each cemetery has its own set of rules and regulations for the size and type of headstones that can be erected. While large headstones might be accepted by some cemeteries, others have strict rules that limit a headstone's size.\",\"endings\":\"['[substeps] Call the cemetery where you will be interring your loved one and ask them if there are size limitations on headstones. [title] Consult the cemetery about regulations on headstone type.', \\\"Avoid choosing a headstone that's too big or hard to put together in some places. [substeps] If you're considering visiting a cemetery, check their size restrictions online before making a purchase.\\\", '[substeps] For example, kentucky news counties are laid on the hardy, mason, inner circle, and greater down roads, while illinois counties are laid on the modestly large. Across the country, either county or federal highway is laid on headstone.', 'If your grave has too many headstones and no longer meets these values, opt for larger ones. [substeps] Try building a monument for your cemetery to keep out archetypes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}