{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c005b1e1-52b2-5d00-8de7-8cbb17dadbbe","task_key":"default--validation--40364","task_revision_id":"1","upstream_id":"40364","short_description":"HellaSwag validation 40364","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get ready for summer using earl grey tea [title] Wash your face. [step] Your face should be squeaky clean. [title] Prepare your earl grey teabags, and a glass with a little hot water in it.\",\"endings\":\"[\\\"[step] Earl grey tea is good for any skin, not everyone's dry skin, and it's not bad for those with acne. [title] Dissolve the ginger and lemon zest.\\\", \\\"[step] Use more than one teabag for optimal results. [title] Soak your teabags in the hot water to release the tea (or the color pigments you'll need).\\\", \\\"[step] Take your shampoo and hair dryer and start to blast the hot water down towards your head. Since earl grey tea floats, it doesn't evaporate as quickly as you think.\\\", '[step] Add a spoonful of earl grey to the glass and mix well with your thumb and fingers. [title] Add a few drops of rosewater and blend well.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}