{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"08469430-7ff2-5612-9842-88298a59601b","task_key":"default--validation--34063","task_revision_id":"1","upstream_id":"34063","short_description":"HellaSwag validation 34063","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to throw a graduation party [title] Decide on a date and time. [step] Because many people may be planning graduation parties around the same time of year, the logistics of choosing a time for your party can be difficult. [substeps] Consider having your party a few weeks after graduation, as many people choose to take trips or go on vacation directly after graduation day.\",\"endings\":\"['Being able to schedule your party should work if you have a huge family, and some parents will appreciate the extra effort and money into getting organized in advance. Consider focusing on having a big party with multiple people staying home all of the time.', \\\"Consider inviting your school's alumni or expatriate friends to your party to place them in charge of setting the party up. Consider filling out invitations as well, to avoid the need to update your invite list.\\\", 'Have your party during june or july, as many of your friends will start to leave in august and move into college campuses. Plan to have your party on a weekend day between 12 p.m.', 'At the same time, choosing a time of year that is not far from graduation day can save you considerable time and money later in the semester. For example, you may opt to have your party on june 15, or near new york city.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}