{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"60191909-f473-5d2b-b0a8-5a5653353a14","task_key":"default--validation--5391","task_revision_id":"1","upstream_id":"5391","short_description":"HellaSwag validation 5391","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to recover from a vasectomy reversal [title] Wear tight-fitting underpants. [step] You should wear supportive underwear immediately after surgery. Relatively tight, close-fitting underwear will give you greater comfort and support.\",\"endings\":\"['Your doctor may suggest underwear with tender underwire, which may cause pain. Such underwear should not be forced over the incision site.', '[substeps] Your doctor may supply you with a different type of supportive undergarment. In this case, wear what your doctor gives you.', \\\"If the doctor determines you will want to wear tight underwear after the procedure, they will tell you how much to wear. How much underwear you should wear depends on the procedure's length.\\\", 'The underwear should not pull or dig into your vagina as this will put pressure on the area. [substeps] When this happens you will feel a slight pain but nothing significant.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}