{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2ede648b-d747-55c6-ae4f-6a2d509932af","task_key":"default--validation--36010","task_revision_id":"1","upstream_id":"36010","short_description":"HellaSwag validation 36010","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to host an english style high tea [title] Pick a time. [step] Consider your guests when choosing a time. People often host tea during the afternoon.\",\"endings\":\"['Work and/or school may be a problem during the afternoon on weekdays, however. A weekend afternoon tea could be ideal if you let your guests know about the date you choose in enough time.', 'The most popular time is slightly below 10pm so it would work to host the group for one hour. However, if, at the end of that time, you want your group to be very close, then a mid afternoon tea can also work well.', 'Choose a time that is convenient for you and your guests. [substeps] For example, you might invite your friends and family over for tea at 8 pm.', 'If people you are inviting are already well entrenched in local politics, religious issues, or other matters, pick the right time to host the tea. [substeps] Consider setting up an informational meeting or meeting in the afternoon.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}