{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad2ff020-bd3d-5508-b94e-f93178b8a24d","task_key":"default--validation--18306","task_revision_id":"1","upstream_id":"18306","short_description":"HellaSwag validation 18306","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to have a successful parent teacher conference [title] Jot down what you want to talk about. [step] Write anything you'd like to bring up with your child's teacher beforehand. This ensures that you cover what you want to speak about and don't forget anything or get off track.\",\"endings\":\"[\\\"It also shows the teacher that you've put some thought into the meeting. [substeps] Write down any questions you have for the teacher.\\\", 'Writing down things you think will impress your child or help him or her recognize the importance of the center of attention. [substeps] When writing down everything you want to say, make sure you include it completely.', 'This also makes for more positive negotiating time. [substeps] It may help to keep a journal or a calendar handy to carry with you so that you can take breaks to write any issue.', '[substeps] If you are in a public place, write down the name of the person you will be speaking with (if you know him or her already). If you are talking to someone else, write down their name or email address.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}