{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"26df74e8-5e10-539e-a89d-9256621b73e5","task_key":"default--validation--2367","task_revision_id":"1","upstream_id":"2367","short_description":"HellaSwag validation 2367","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make your nose appear thinner with makeup [title] Start by putting on your foundation. [step] Before you contour your nose, you'll need to apple your normal foundation. Apply your foundation as your regularly would, including your nose.\",\"endings\":\"['An hour or two before makeup is due to start to make your nose appear smaller. If your foundation is too thin, it will not be as noticeable as it would be with foundation.', 'Use a crescent-shaped brush or an makeup sponge to apply your foundation as your usual would. [substeps] Never leave foundation on your nose, but for now continue using a makeup sponge.', \\\"You'll contour your nose on top of the foundation. This will help your makeup look more natural once you blend everything together.\\\", '[substeps] Line the section of your nose only there, if any. Before you go ahead and make your nose thin, put your regular foundation there.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}