{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99021c0c-2166-5f2b-8552-a2fef63e839f","task_key":"default--validation--10008","task_revision_id":"1","upstream_id":"10008","short_description":"HellaSwag validation 10008","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose men's dress shoes [title] Look at and feel the leather. [step] Dress shoes made of leather are the ideal choice. Leather will be smooth but not too smooth, and may be shiny but not enough to see your reflection.\",\"endings\":\"['Watch out for shoes that seem too shiny and have a plastic feel to them. These are not likely to be genuine leather, so avoid them.', 'The leather should not have any large knots, dents, bumps, or scratches on the shoe-not over the laces. Leather would be crunchy and often needs to be trimmed during walks or gardening.', \\\"Looking at the leather in your shoe to make sure it isn't too tightly woven, or too loose, is always a good bet. [title] Make a list of all the appropriate styles.\\\", 'Leather that provides excellent visibility or perfect eye-reading can be ideal for business or professional occasions. However, leather can also be lumpy, lochy, or dangerous to wear.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}