{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"20eef2b3-00fb-5868-8b23-40d8ec96d9c5","task_key":"default--validation--3691","task_revision_id":"1","upstream_id":"3691","short_description":"HellaSwag validation 3691","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to put on suspenders [title] Fasten the suspenders to the back of your trousers. [step] Before you put on your trousers, secure the suspenders in place. Line up the suspenders with the center of your trousers.\",\"endings\":\"['Then, tie a stitch through the front of the suspenders. Sew or unfold the front of the suspenders to secure the front.', 'The back of the suspenders should be aligned with the center of your trousers. The suspenders should be connected through one of the front loops of your belt, located in the lower-right part of your trousers.', 'Clip or button them to the fabric, never the belt loops. [substeps] Fasten x-back suspenders halfway between your spine and your sides.', 'Make sure that they do not touch the bottom of your pants. Either fasten the suspenders or fasten the suspenders again.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}