{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7520d6fe-8e37-50d2-a208-251ffe332fad","task_key":"default--validation--5311","task_revision_id":"1","upstream_id":"5311","short_description":"HellaSwag validation 5311","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to treat melasma [title] See your primary care physician. [step] Talk to your primary care physician about any hormonal medication changes and creams you can try for melasma before going to see a dermatologist. Treatment for melasma may be considered elective and it may not be covered by your insurance.\",\"endings\":\"['[substeps] Your primary care physician will be able to make specific recommendations as to the treatment that is best for you and for your baby. They may follow these recommendations to determine the best treatment plan for you.', '[substeps] Your doctor may suggest radiation therapy or deep thrombosis treatment. Melasma occurs when certain protein is injected into blood vessels through the nose.', 'Find out the costs of any treatments and procedures before you schedule them. [title] Stop taking medications that could be to blame.', '[title] Schedule an appointment with your primary care physician. [step] If you fear that you may have melasma, you should let your primary care physician know as soon as possible.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}