{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"17884181-4009-5e6c-8e7c-79aef974611d","task_key":"default--validation--3503","task_revision_id":"1","upstream_id":"3503","short_description":"HellaSwag validation 3503","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to handle a mother in law [title] If your mil calls more than once a week and can't understand why this is inconvenient, start writing e-mails. [step] If they don't have an e-mail address, write letters. If she is calling just to see how you and your family are doing, that is a good way to keep her updated while keeping your space.\",\"endings\":\"[\\\"[title] Focus on a conversation not you both are having. [step] Out of respect for your mother-in-law, it's important that you talk to her in private whenever possible.\\\", 'Better still, keep your parents informed as to the situation. [title] In your letters, show her what time you normally wake them up.', 'Another way is to just let your husband talk to her. Listening to her voice all the time can be like she is living with you and you feel lack of space, therefore, let your husband talk.', '[title] Direct her to you if she wants to. [step] If your mil stops calling so often, then she needs time to get used to her new neighborhood.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}