{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c00d0c7-b19c-5cf3-bba9-c784bdd4fda1","task_key":"default--validation--33835","task_revision_id":"1","upstream_id":"33835","short_description":"HellaSwag validation 33835","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to treat cataracts with surgery [title] Consider surgery when cataracts begin to interfere with your everyday life. [step] Cataracts generally appear in people over the age of 55. They make it difficult to focus in brightly lit environments or to identify details on anything that isn't right in front of you.\",\"endings\":\"[\\\"[title] Try not to panic with the immediate danger while a surgeon performs a procedure. [step] You don't really expect to experience effective outcomes when surgery is performed by just one person.\\\", 'At first, cataracts may not interfere much with your vision. If your cataracts are mild, you can take some steps to ease the discomfort and/or inconvenience.', 'This is because they can often distract you from practical things like decorating the house. Many people suffering from cataracts also have issues with controlling their energy.', '[title] Look into alternative treatment if the problem persists. [step] Other treatments, such as seeing your doctor or getting surgery, are usually unnecessary for prompt recovery.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}