{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3852c12b-43c5-53e0-86ce-14bc9b2bae14","task_key":"default--validation--48334","task_revision_id":"1","upstream_id":"48334","short_description":"HellaSwag validation 48334","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to survive a bank robbery [title] Stay calm. [step] When the robbery starts, it's likely to be chaotic and confusing. Breathe deeply and try to stay calm.\",\"endings\":\"[\\\"Being in a heightened emotional state may cause you to do things that you normally wouldn't do, so it's important that you try to relax as much as possible throughout the duration of the robbery. [substeps] Fear may temporarily immobilize you, so try to keep a clear head so you can react appropriately to the robbery.\\\", 'That is, get easy, stay focused on the whole situation at hand. Notice everything that goes wrong-the robbery is not over yet so take a step back and take some deep breaths.', \\\"Remember that you're under no obligation to risk yourself by jumping around during the robbery and scaring everyone. The robber likely will become very angry if you are caught, so do not make the wait any longer.\\\", \\\"It's important to stay calm because stealing money you don't need is akin to being robbed. If you panic, you may get caught.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}