{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a217c838-eb5c-53b8-91bd-30c735679077","task_key":"default--validation--15379","task_revision_id":"1","upstream_id":"15379","short_description":"HellaSwag validation 15379","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to have an open relationship [title] Ask yourself deeply why you want an open relationship. [step] Are you no longer satisfied with your partner? Do you simply want to try it? Do you think it will enhance your relationship with your current partner? These are all important considerations to make before trying to discuss the topic with your partner. [substeps] Be sure your motives are very strong before trying to discuss the topic with your partner.\",\"endings\":\"['For some people, the topic is incredibly taboo and negative. Unless your partner has mentioned it in the past, do not assume that your partner will accept it.', 'Telling your partner you would like to open the relationship is not the right way to start your relationship. Seek to open the relationship for the right reasons.', \\\"Your partner's reactions will appear to drive you away. [title] Examine whether you need to be honest with your partner.\\\", 'They are key to a successful, honest, and supportive relationship. Ideally, you want a partner who is good at showing initiative.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}