{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5f8d2f06-19b4-57a3-a72c-e67b006c1ea0","task_key":"default--validation--39900","task_revision_id":"1","upstream_id":"39900","short_description":"HellaSwag validation 39900","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to deal with sex problems in marriage [title] Choose a time when you are both alone and not distracted. [step] Having a conversation about sexual issues can be difficult and awkward, especially if you are both feeling the lack of sexual chemistry in the bedroom. Don't spring the conversation on your partner when you're having dinner at a restaurant or surrounded by friends.\",\"endings\":\"[\\\"Setting up a time that your partner is available and isn't distracted can help you take a break from pornography and get to know one another better. [substeps] If your partner is busy with work or has a busy day, this may be uncomfortable for you both.\\\", 'Look for a time when you are both alone, in a private space, to bring up the topic. [substeps] A good time may be when you are both getting ready for bed, or after you have dinner together at home.', \\\"This can come across as snooping in your partner's personal space, or not recognizing when something is wrong. [substeps] Try to find a quiet, distraction-free space where no one is around to tell you what's on your mind.\\\", '[substeps] If you know you might have a problem with religious or sexual ideas, then focus on the conversation in private. Ask your partner what they think is missing in the conversations you have before facing them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}