{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6133faa1-890c-51a0-a08b-648ade8d7c49","task_key":"default--validation--49811--variant-b3a2c20dee1f","task_revision_id":"1","upstream_id":"49811--variant-b3a2c20dee1f","short_description":"HellaSwag validation 49811","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to pluck your eyebrows [title] Soften the skin around your brows. [step] When the skin there is soft and supple, the hairs slide out much more easily. Plucking skin that is dry and stiff will make the experience more painful.\",\"endings\":\"['[substeps] Plan to pluck your brows right after you take a shower. The warm water and steam will have left your skin moist and supple.', '[title] Hold your eyebrow pencil firmly with the point facing upwards. [step] Using even pressure, lower the pencil when it is close to touching the brow bone.', 'Ideally, use an oil-free brow gel instead. Avoid any products containing alcohol, which can dry out the hairs and make plucking more painful.', 'Instead, use slightly looser hand controls to stop this from happening. [title] Stop plucking after four weeks.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}