{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1733e0d3-eeca-59a7-81e7-8c56b0734a4d","task_key":"default--validation--36554","task_revision_id":"1","upstream_id":"36554","short_description":"HellaSwag validation 36554","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to make a personal wedding ceremony booklet [title] Provide basic information about the wedding. [step] The wedding ceremony booklet should include the full names of the bride and groom, the wedding date and time, and the city, state and venue where the wedding is being held. This information is usually on the cover of the booklet.\",\"endings\":\"['Just highlight this information by either typing it into the margins and double-clicking it. [title] Put in the wedding details as you normally would.', 'Be sure to include the date and location of the ceremony. The booklet guide is a popular job for a wedding wedding photographer.', '[title] List the elements and order of the wedding ceremony. [step] The ceremony booklet includes information about the actual ceremony so guests can follow along and participate.', \\\"Contact the venue directly to make sure you won't need to include more information. [title] Make a list of the things that are likely to happen at the ceremony, as well as the type of event expected.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}