{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0316f31d-65f5-5552-aa1b-b34d825aaea7","task_key":"default--validation--12415","task_revision_id":"1","upstream_id":"12415","short_description":"HellaSwag validation 12415","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get married in italy [title] Plan ahead. [step] You will need at least six months to gather all of the appropriate documentation. Resist the temptation to just hop on a plane to italy with your fiancé.\",\"endings\":\"['This one is simply an impulse, because you may regret it later on. He/she will feel pressured and you are likely to come across as a spoiled jerk.', \\\"It is unlikely you will be able to have a legally binding ceremony if you don't have the proper documentation. [title] Consider the type of ceremony you want.\\\", \\\"You don't want to have to go there, though. If the marriage in italy is annulled, the plan will definitely crumble.\\\", 'Wait for your fiancé-to-be to arrive in italy, which can be embarrassing. As a result, you should plan to save at least six months to give yourselves time to find the perfect place.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}