{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"436d65a5-6c8d-55fe-821a-daee22ce239f","task_key":"default--validation--49758","task_revision_id":"1","upstream_id":"49758","short_description":"HellaSwag validation 49758","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to choose the right partner in life [title] Identify your deal-breakers. [step] Ask yourself which personality traits, habits, or circumstances you can't accept in a partner. Be prepared to stop seeing someone if any of your deal-breakers are present.\",\"endings\":\"[\\\"[substeps] Determine if you find any unappealing traits, like a bad-faith attitude, that you want out of a partner. [title] Consider whether you're dating if you're single.\\\", \\\"[substeps] Separate your true deal-breakers from traits that might be annoying, but aren't the end of the world. For instance, poor hygiene might be a true deal-breaker for you.\\\", 'If you\\\\'re having some difficult or complicated relationships, cut off ties and keep your sex life to yourself, and only interact with that person. [substeps] For example, you might say, \\\" we really aren\\\\'t as emotionally compatible as we used to be, and we\\\\'re not looking for more in-depth relationships.', \\\"Try to avoid people who are overly competitive, are always rude, or don't respect your boundaries. Also, think about whether or not you stand out to other people and people and reevaluate your goals.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}