{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1a313fdd-f700-5e51-b83a-5ddd1b43d5eb","task_key":"default--validation--25466","task_revision_id":"1","upstream_id":"25466","short_description":"HellaSwag validation 25466","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to hide your period supplies [title] Choose an opaque, boring-looking box. [step] Something made from plastic, metal, or cardboard will keep people from seeing what's inside. Its ordinary appearance will blend in with the background and lower the chances of people looking through it.\",\"endings\":\"[\\\"[substeps] Avoid boxes made from clear or frosted plastic. They may look pretty, but people will be able to see what's inside.\\\", 'Look for one that is clearly separate from any mail. Modern mailboxes usually have clear, opaque sides and doors so that potential people cannot see the contents inside.', '[substeps] Some boxes range from colorless to bright and jarring. Avoid packing your clothes and cosmetics in boxes that feature stains or smears.', \\\"[substeps] If you're going for a more natural look, opt for a plain cardboard box instead. Be warned: in a plastic box, people won't see what's inside.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}