{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"039bd805-af60-5d4e-b005-95d1d8844599","task_key":"default--validation--23321","task_revision_id":"1","upstream_id":"23321","short_description":"HellaSwag validation 23321","config":"default","split":"validation","body":"{\"activity_label\":\"Philosophy and Religion\",\"ctx\":\"[header] How to follow customary behavior in church [title] If this is your first time at a particular church, or any church for that matter, recognize that you are a guest and new to this. [step] It is normal to not do everything right the first or the second time in a new church. [title] Many christian services are divided into two main components.\",\"endings\":\"['[step] The first part of the service has to do with \\\" the word, \\\" in other words the bible. The second part of the service has to do with \\\" the table, \\\" and usually involves eucharist (communion).', '[step] Those, in particular, are the most common aspects of to follow. A typical vestry is a services that says, \\\" christmas day warms the christmas day heart.', '[step] First a prayer setting and second a spiritual setting. Avoid interfering with these arrangements, but be courteous in terms of courtesy.', '[step] The first is worship, and the second is worship. It is usually quite a formal event, and usually around easter, so expect to receive much praise for it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}