{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0858d9bb-cfdf-5bdf-8891-7f2876da0bbf","task_key":"default--validation--43230","task_revision_id":"1","upstream_id":"43230","short_description":"HellaSwag validation 43230","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to catch a liar [title] Look for a fake smile. [step] Real smiles and fake smiles look very different upon closer inspection. A fake smile is often limited to the person's mouth, while they genuinely smile with their eyes, cheeks and whole face.\",\"endings\":\"['This generally makes it hard for the person to keep their smile. If this is a regular front and center smile, the person may be faking it.', '[title] Pay attention to how their lips and teeth look. [step] See if they allow their mouth to expand slightly with their breath.', \\\"Also, look for ulterior motives: they might be trying to get you to pay them what they've promised or pay them back with something. [substeps] If you are someone a very confident and persistent liar, they will vanish.\\\", 'If the person seems to be fake smiling at you, they may have something to hide. The smile is an attempt to reassure you that all is okay.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}