{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9f4b528-a0ac-5853-85b2-68282b41cd90","task_key":"default--validation--46745","task_revision_id":"1","upstream_id":"46745","short_description":"HellaSwag validation 46745","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to convince your parents to get you something [title] Reflect on what you're asking for. [step] You should know what your parents usually say yes or no to. Compare your request to things your parents have gotten you in the past.\",\"endings\":\"['Try to see if you agree with them, or think they are being lenient. Does your request inspire them to give you something specific? [substeps] Perhaps you want to get your hair done or start a photography class.', \\\"[substeps] For example, if you're asking for a certain car, then factor in things like the expenses you'll have to cover. Your parents would probably be worried and want to spend more money on it when you're in fact asking for a new car.\\\", 'Does your current request seem larger than what you usually ask for? Take a look at your financial means and decide how long it would take you to pay for it yourself. Also consider if your request is something you want or something you really need.', \\\"[substeps] Keep in mind that you don't have to ask your parents if they'll buy anything you truly love and would love to bring home for the kids. However, you may want to ask if they'll allow you to babysit or if they'll let you drop by in advance of the house party.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}