{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"896ac3f3-fd37-56f2-b972-c9c017636117","task_key":"default--validation--32958","task_revision_id":"1","upstream_id":"32958","short_description":"HellaSwag validation 32958","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to get legal advice [title] Define your legal problem. [step] Before you can start looking for legal information and assistance, you need to be able to explain your problem. A family law issue is going to have a different answer than a problem with a contract or your landlord.\",\"endings\":\"['If you or someone you know has a medical problem, this should be what you are looking for. [substeps] Medical disorders are similar to relationship issues, so make sure you understand the specifics of those issues before you start your search.', 'You may also want to make sure your landlord understands where you are coming from or what your problem is. [substeps] Who pays your bills? If it is your landlord, is your \\\" who pay them \\\" your problem? Are you paying someone that is not supposed to be paying? [title] Get your landlord\\\\'s permission.', 'Make it clear from the outset why a guardianship matter will be the only course of action you take. Ask yourself what you want to do as a result.', '[substeps] Have you received legal papers from a court. If so, identify who is suing you and what they want.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}