{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"70796707-7aa7-5a54-bb8d-54b81c3ff158","task_key":"default--validation--40302","task_revision_id":"1","upstream_id":"40302","short_description":"HellaSwag validation 40302","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear the same suit all week [title] Avoid wearing the same shirt for more than one day. [step] Wearing a different shirt changes the whole look of the suit, which is what you want. It's also important to wear a different shirt because it soaks up your sweat during the day and will start to smell if you wear it for too long.\",\"endings\":\"[\\\"Try wearing a different shirt so that you don't have to wear the same outfit over and over again. [title] Change out the suit every two days.\\\", \\\"[substeps] If you have to wear the same shirt more than once, don't wear it on back-to-back days to give your look variety and let the shirt air out. [title] Wear an undershirt to keep your top shirt dry.\\\", 'It can end up looking like your undershirt is made of sweat. Additionally, it also smells like sweat because of the hassle of wearing the same shirt multiple days in a row.', \\\"Shirts always start to smell when you first wear them, so it's important to get rid of it. Wearing clean, pressed and proper clothing at work also shows that you care about your appearance.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}