{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"503e3e75-824d-5b85-a18a-d54beaae1aa9","task_key":"default--validation--25316","task_revision_id":"1","upstream_id":"25316","short_description":"HellaSwag validation 25316","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to determine when easter is [title] Mark the vernal equinox. [step] The date easter falls on is based on the ecclesiastical approximation of the vernal (spring) equinox. This approximation falls on the same date each year: march 21.\",\"endings\":\"['[substeps] For fall this equinox is ten years from february or april and in spring it is eight years from october. On july 1, april 15, the aquarius cycle has taken place.', 'Spring comes on december 31 and throws the weight of easter out in december. The first days of april are also the first days of the summer equinox.', '[substeps] In some eastern countries, easter is january 1 , 31st of december. In southern countries, easter may be april 1 , 12.', '[substeps] Note that this calculation relies on the ecclesiastical estimation of the vernal equinox, not the actual vernal equinox identified by the astronomical system of measurement. The actual moment of equinox can shift within 24 hours and may arrive a day before march 21.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}