{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dd0ab73b-b854-5ded-9c33-b6ad6564a4d0","task_key":"default--validation--26843","task_revision_id":"1","upstream_id":"26843","short_description":"HellaSwag validation 26843","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to discuss safe sex on a first date [title] Test the waters. [step] You may not be sure if you (or your date) want to have sex on the first date. Some people avoid having sex, or even talking about sex, until they've gotten to know the other person better.\",\"endings\":\"[\\\"Although it is scary and may scare others, it is absolutely okay. It's never ok to talk about sex on a first date or if you feel the pressure to discuss a topic.\\\", 'Before you actually get ready to have sex, try reading a book or magazine about oral contraceptives and make sure you follow their directions. [substeps] To be safe at first date, you should read book or magazine reviews for safe sex.', 'If you are both having a good time, the idea of sex may feel right for both of you. Look for signs that your date is interested.', \\\"For example, if you're hanging out with a group of friends and your date says they don't mind or want to have sex on a first date, then be sure to hold off until that date. [title] Discuss the benefits of having a new partner.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}