{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"533b53c5-6021-5092-b104-48b6ecc45528","task_key":"default--validation--12005","task_revision_id":"1","upstream_id":"12005","short_description":"HellaSwag validation 12005","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a bracelet out of staples [title] Fill a stapler with the staples of choice. [step] You can go with the usual plain silver colored staples or be a little more funky and choose staples in different colors (these can be found in some stationery stores). [title] Punch enough staples for the bracelet.\",\"endings\":\"[\\\"[step] Do not punch the staples with force, as the loops will come out too small for you to work with. To know how many you'll probably need, estimate the width around your wrist as follows : [substeps] Measure your wrist with a measuring tape.\\\", \\\"[step] You may find it best to use a smaller amount depending on how many crochet patterns you want. If you can't find staples, or would rather make your own, you can purchase staples in many colors (would be a great choice if you are making your own really bold patterns).\\\", '[step] Simply glue the staple all the way around the perimeter of the bracelet, and staple it in place. Use enough for the bracelet to have a long, skinny, coil around something.', '[step] You can use a bit more fabric to add to it. [substeps] Arrange the staples that you chose into the same pattern (square, rectangular, straight, rectangle, etc.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}