{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a0c9cb70-c8ad-598b-9f1b-1ab112a2cf93","task_key":"default--validation--47861","task_revision_id":"1","upstream_id":"47861","short_description":"HellaSwag validation 47861","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to get an itemized list of costs from your attorney [title] Ask about itemized bills during your consultation. [step] Before hiring an attorney, you should schedule a consultation. At the consultation, you can ask a variety of questions, including about fees.\",\"endings\":\"[\\\"Also discuss the attorney's experience handling cases such as yours. [substeps] The attorney should be honest with you.\\\", 'For example, try to be as specific as possible when talking about premiums. [substeps] Before booking the consultation, make sure the attorney can accommodate your needs.', 'You should ask whether the lawyer will provide you with an itemized bill and whether it will increase your costs. [title] Make sure the lawyer agrees to provide itemized bills.', 'You should also ask about what services the attorney will charge. Some fees might include : [substeps] Client alimony other phone or customer service fees for hr a child custody and visitation schedule [title] Talk to the attorney about how to reach a settlement.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}