{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0a6247ab-471b-5bce-953b-6ee8bc72f3f2","task_key":"default--validation--30681","task_revision_id":"1","upstream_id":"30681","short_description":"HellaSwag validation 30681","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to put together outfits [title] Organize your closet. [step] A lot of times when we don't know what to wear or how to put together an outfit it means that it is time to organize our closet. Organizing means it will be easier to locate the clothes you want for the outfits you'll be putting together and it will remind you what you own in way of clothing.\",\"endings\":\"['To organize your closet : [substeps] Figure out your weekly outfits best. Set a budget for all your outfits and put them together.', \\\"[title] Pick out clothes that you can throw away. [step] It is tempting to throw away clothes you no longer want to wear, but it isn't necessary.\\\", '[substeps] Collect clothing that is not too baggy, straight, and are not too loose. This puts your clothing to one side, and gives you a shoulder rest for occasional rearranging.', '[substeps] Get rid of anything you almost never wear. There is no point in keeping around something you wear only once or twice.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}