{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0183eca9-decd-59dd-ba44-3cc8fc4559c4","task_key":"default--validation--3114","task_revision_id":"1","upstream_id":"3114","short_description":"HellaSwag validation 3114","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to plan a high school reunion [title] Begin at least one year in advance. [step] High school reunions involve a lot of coordination. There are venues to reserve, catering to organize, and dates to save.\",\"endings\":\"['Costs may change from time to time, but no matter what, keep your head in the game and find the perfect venue! [substeps] Knowing that you are doing something right will go a long way towards increasing your popularity. [title] Learn some about the things that your potential fiancé brings to your life.', \\\"If you're an outside liaison, you may want to boost up both time and effort to more efficiently structure your reunion. [title] Cut back on any reservations or commitments.\\\", 'To ensure that you leave yourself enough time to plan a great reunion, begin to organize your committee at least 12 months prior to your reunion date. You do not want to have to rush or slap together a reunion at the last minute.', \\\"At least one year will help you prioritize each occasion. Additionally, there's the possibility that some locations cannot hold a reunion.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}