{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f6b91be4-a319-5292-a18a-8a9b62c55864","task_key":"default--validation--35528","task_revision_id":"1","upstream_id":"35528","short_description":"HellaSwag validation 35528","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to get an adult male to use his manners [title] Consider what direct and indirect demands you have been making. [step] Reflect on the balance of personal demands you are making to your partner, and evaluate whether that balance is just. [title] Be prepared to evaluate your viewpoint of the situation as objectively as possible.\",\"endings\":\"['[title] First, determine whether your partner was accurate in his assessment of his sexual capabilities and responses. [title] Discuss there are a number of different ways that men in modern society can show discretion, professionalism, and safety in the absence of sexual restraint.', '[step] You have also made a choice that your partner must want and must follow. When you record and evaluate your demands, be sure you are or need to be able to speak with those who are making them.', '[title] Consider your own perspective, not personal. [step] Communicating your viewpoint will provide you with invaluable resources.', \\\"[step] Don't go into the impending talk convinced that anything he might say is either wrong or not heartfelt. That won't help to solve the problem.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}