{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7f1a5d8b-a08f-58bc-a501-68c702213f3a","task_key":"default--validation--36440","task_revision_id":"1","upstream_id":"36440","short_description":"HellaSwag validation 36440","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to form a beauty pageant [title] Make a schedule. [substeps] Pick the perfect year, month, date, and time of the day. Check the weather to see if the day will be good enough.\",\"endings\":\"['Make sure you schedule the pageant only on weekends, not the week before the pageant. Contest winners will have to go and compete for hours on end to try to win.', 'Keep a full face and body pack for the pageant. Pack : [substeps] Masks to wear headwear (optional) facial cleansers and foundation (optional ) [title] Make sure to get the right color clothes.', 'Check to see if you have something scheduled at that time. Make a special \\\" planner \\\" and write the time in it.', 'If the sky is dark, though, friday is the best time for a pageant. [title] Research beauty pageants; send your list to the beginning of the year.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}