{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fcf43367-b0b9-544d-8204-246a618018fb","task_key":"default--validation--13434","task_revision_id":"1","upstream_id":"13434","short_description":"HellaSwag validation 13434","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to fall out of love with your best friend [title] Stay away from your best friend for a while. [step] Be civil and thoughtful in how you go about this. You don't want to completely reject the healthy relationship that you've established between each other, but you need to take care of your feelings.\",\"endings\":\"[\\\"[substeps] Tell your best friend that you aren't interested. On any occasion after inviting them out for dinner, you can mention that you're going to watch a movie and want to see what the other person has done.\\\", \\\"Don't make active efforts to see them on a daily basis. [substeps] There are countless ways to set up safe and respectful boundaries between you and this person.\\\", 'It is what relationships are made out of. [title] Unfollow your best friend.', \\\"Trying to give back without being too overbearing, especially in the long run, is not productive or effective. [substeps] Avoid overwhelming your friends by telling them outright that you're going to take them out on dates.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}