{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"468ca185-28a1-52c6-8ac3-9b9158ac37c6","task_key":"default--validation--36665","task_revision_id":"1","upstream_id":"36665","short_description":"HellaSwag validation 36665","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a handkerchief [title] Purchase the fabric. [step] Check the fabric content to see what kind of maintenance it will require in order to stay looking good. For everyday use, you will probably want something that is washable and requires little ironing.\",\"endings\":\"['The photo cardstock, though, will work just fine for photos. [substeps] Select a photo width of 11 in.', 'Popular fabrics for handkerchief sleeves include corduroy, flannel and polyester. [substeps] When the material you are using has very little wear and tear, it is most visible if it is securely sewn into place.', '[substeps] Measure the inside width of the handkerchief. Measure how high you want the corners of the handkerchief to be.', '[substeps] Rather than purchase fabric, look around in your scraps. This is a great project for using up leftovers or recycling old fabrics.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}