{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"42900048-fe5d-59f9-8261-9aed77254280","task_key":"default--validation--36117","task_revision_id":"1","upstream_id":"36117","short_description":"HellaSwag validation 36117","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to contour your nose [title] Draw two lines down the sides of your nose. [step] Do this on top of your foundation. Use an angled eyeshadow brush to draw the contour lines.\",\"endings\":\"['At this point you should have two rounded, rectangle-like projections on the two lines. These should be bigger than the shape of your nose.', \\\"They should begin at the brow bone and go all the way to the end of your nose. Make sure to keep the lines light at first-it's easier to build product than to take it away.\\\", 'Start at the outer corners of your nose and complete your contour line at the top of your forehead. Begin below your nostrils near and up on the sides of your nose, making tiny arching lines.', \\\"Don't worry if you must use a thinner brush because your work will create only darkest lines. [title] Create a light line on your face.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}