{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"663bfadc-ddab-5edb-b44b-141358a71265","task_key":"default--validation--45067","task_revision_id":"1","upstream_id":"45067","short_description":"HellaSwag validation 45067","config":"default","split":"validation","body":"{\"activity_label\":\"Philosophy and Religion\",\"ctx\":\"[header] How to anoint oil [title] Check with your judicatory or a religious authority for specifics. [step] Each denomination has its own guidelines governing the way that oil is blessed for anointing purposes, as well as the way that anointing oil is used. [substeps] The most common restriction regards who may bless or anoint the oil.\",\"endings\":\"['Believers must purchase and learn the following mandates before they may apply. For example, in a specific denomination or field, someone can anoint the oil for religious purposes or for ritual purposes.', 'It is a guideline that will vary fairly widely in species. To find any non-non-buddhist group, you will likely need to look up nifty non-buddhist societies.', 'In some denominations, only a priest or similar clergyman can bless the oil. Some denominations even limit which clergymen are authorized to consecrate oil.', 'Anabolic physiology physiology, for example, prohibits shopping for oils at that exact minute. The consensus is that fertilizers and liquids must be consumed in order to make anabolic physiology.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}