{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"372ed8d1-ef47-5d95-a0c6-8e1c08eac348","task_key":"default--validation--1290","task_revision_id":"1","upstream_id":"1290","short_description":"HellaSwag validation 1290","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear mirror sunglasses [title] Determine the best frame for your face. [step] Some shapes look fantastic on round faces, while others look best on heart-shaped faces, and so on. The sunnies that your best friend pulls off flawlessly might not work for you.\",\"endings\":\"['In this case, finding and purchasing a perfect frame. [substeps] First, you need to consider what shape you want your face to be.', 'Consider making sure that the frames you choose are smooth and even when they are closed and you won\\\\'t have to worry about them taking the focus away from you. [title] Add yourself to the \\\" never mind the pair \\\" screen.', 'Make your sunglass shopping experience easiest by coming armed with the knowledge about which shapes best flatter your face. [substeps] If you have a heart-shaped face, look for square, cat eye, or sport sunglasses.', \\\"Instead, select a pair with a good frame. [substeps] If you're looking for a style that's somewhat blocky, go for a pair that will make a small area of your face flush.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}