{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"09263251-3592-5f5a-b85b-6f953bc338c0","task_key":"default--validation--6166","task_revision_id":"1","upstream_id":"6166","short_description":"HellaSwag validation 6166","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to be cool on facebook [title] Have a life outside of facebook. [step] Facebook is a way to keep others in your life, not a way to create a life, so go out and have yours, in full dimension. Facebook isn't proof that you have friends--friends need real life interaction, not stacking up in a list.\",\"endings\":\"['Everyone is different, so make friends with those who are your real selves and others who aren\\\\'t. Have many \\\" friends \\\" outside of your walls.', \\\"Checking your facebook and keeping your profile up to date are one thing, so don't spend all your time on what should be a tool you use wisely. [title] Avoid typical updates and seek the quirky instead.\\\", \\\"If you've got an account, keep it. Likewise, if you have no followers, don't worry, that means you don't have anyone to talk to.\\\", 'In fact, most people need social interactions with their friends-not personal disposable status updates. [substeps] Try to minimize contact with people on facebook.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}