{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c1e2f4f-317f-501b-bdc0-96d00e1c6690","task_key":"default--validation--8828","task_revision_id":"1","upstream_id":"8828","short_description":"HellaSwag validation 8828","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear leather leggings [title] Choose between real and faux leather. [step] Most leather leggings are actually made with faux leather instead of the real deal, but you can go with either option. [substeps] Most faux leather is made from stretchy fabric, like spandex, coated with some form of black polyurethane.\",\"endings\":\"['The black material forms a good barrier against excess heat and moisture, making it easy to wear. Add leather leggings to your wardrobe in order to keep them in style without leaving all or most of the ankle or thigh holes exposed.', 'This creates a cushioning effect, allowing you to get a good, durable fit. As such, faux leather leggings are very flexible and comfortable.', 'Faux leather is generally cheaper and easier to care for. You can usually wash faux leather in your washing machine without damaging the material, but real leather tends to be more temperamental.', 'This covering means that the leggings have a highly visible sheen and an ultra-reflective layer on them. This makes them a classic choice for casual situations.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}