{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2f885f48-9ecf-583d-b9a4-00d1729abfc8","task_key":"default--validation--44419--variant-a8902d4268ae","task_revision_id":"1","upstream_id":"44419--variant-a8902d4268ae","short_description":"HellaSwag validation 44419","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to treat a sting from a stinging nettle [title] Avoid touching the area at first. [step] If possible, do not touch or rub the affected area for 10 minutes. Pour fresh water over the area without touching.\",\"endings\":\"['Even though the pain can be intense during the first few minutes, by avoiding any touching or rubbing, you may prevent the pain from lingering for days. [substeps] The chemical irritants from the plant can dry on the surface of the skin, then they can be removed by soap and water.', 'This will wash off any dirt that might be on the area so the nettle does not get wet again. [title] Apply the ice pack daily to reduce itching.', 'Then, submerge the area for 5 to 7 minutes. [title] Wash your face using a clean, clean face wash.', 'This will help soothe you and ensure that nothing will stick to the area. [substeps] You can now do whatever you want with the nettle leaves.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}