{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cf12257b-5ec2-52fa-99ee-98881f97c12d","task_key":"default--validation--40347","task_revision_id":"1","upstream_id":"40347","short_description":"HellaSwag validation 40347","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to catch a cheating partner [title] Notice if your partner spends less time with you. [step] Your partner may develop sudden new interests or friends if he or she is cheating. He or she may begin working longer hours.\",\"endings\":\"['He or she may have more commitments such as sports games and nights out. [substeps] Your partner may try to make as big of an impact on your life as possible.', 'If you ask your partner where he or she is going, your partner may give you a vague answer such as \\\" out. \\\" or \\\" meeting up with some people.', 'You may notice that your partner spends more time with you when you are not together. Someone who is cheating may spend more time than the other person.', '[substeps] Your partner may be dating other girls instead of spending time with you. He or she may not get extra money to spend with you.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}