{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"21da0875-c48d-5e63-bb59-d067f7e2ec3d","task_key":"default--validation--45216","task_revision_id":"1","upstream_id":"45216","short_description":"HellaSwag validation 45216","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to have a party without your parents knowing [title] Research! [step] Find a time when your parents will be gone, and all your siblings and other family members (if you have any) are also gone. Most families have a calendar where they write down events, appointments, etc. Try to find a day and time where the house will be empty.\",\"endings\":\"[\\\"They won't know where the party will be. [substeps] Be cautious and keep your priorities straight.\\\", 'Avoid asking parents directly, because this may sound suspicious. Playing sick can also help you get the house to yourself if the whole family is going somewhere together, but this may backfire! The whole family may have to stay to \\\" take care \\\" of you.', '[title] Decide on invitations, directions, etc. [step] There are a number of websites that you can use where you can make invites.', \\\"Also try to schedule your party a few days before so you have plenty of time to prepare. This may also be a good time to sign up, if you're allowed to.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}