{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"649c5ac6-ad41-5c03-a4bb-07b0704d3265","task_key":"default--validation--10278","task_revision_id":"1","upstream_id":"10278","short_description":"HellaSwag validation 10278","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to decorate shoes [title] Get some tennis shoes. [step] To decorate shoes with paint you'll need to use canvas tennis shoes. You can buy canvas shoes in white, black, and many other colors at department or drug stores.\",\"endings\":\"['Make sure to sand down the whole shoe before you paint it! [title] Apply the paint. [step] You can use oil paint or use powdered paint, but you will most likely need to allow the paint to dry overnight before you can use any real paint.', 'Get a few pairs to try out different designs, or just stick with one pair for your masterpiece. [substeps] You can choose canvas shoes with or without laces-both will work just fine.', 'To decorate your shoes, use paint primer to color them a solid color. [substeps] You can use as much or as little paint as your want.', \\\"You'll then need to paint them in your colors. Some paints will not work well with tennis shoes, so it's recommended to buy some to use.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}