{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"db754c16-ffb4-5643-acc2-c006d95e3a48","task_key":"default--validation--30023","task_revision_id":"1","upstream_id":"30023","short_description":"HellaSwag validation 30023","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to tell a girl you love her (teenagers ) [title] Practice what you are going to say. [step] Saying \\\" i love you \\\" for the first time is really scary. If you're nervous, planning what you are going to say can boost your confidence.\",\"endings\":\"[\\\"The awkwardness of saying it out loud is also untreatable. Set yourself up for success by practicing what you're going to say at different times so you feel less nervous when you've said it out loud.\\\", 'Think about what you want her to know, then rehearse your declaration of love. Instead of just saying \\\" i love you, \\\" consider going deeper.', 'When you start thinking you should say something, do it. Visualize it like a poem or song: the lyrics, your name, your favorite play, your favorite movies/tv shows, etc.', \\\"If you're shy, you can practice the terminology ahead of time to keep your nervousness in check. [substeps] Make sure that you're comfortable using the terminology in public as well.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}