{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a847d068-d87c-5104-82bb-a6e2f0f4ec65","task_key":"default--validation--49043","task_revision_id":"1","upstream_id":"49043","short_description":"HellaSwag validation 49043","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose your style [title] If one of your friends has on an outfit you really like, you can ask them where they got it. [step] Look around you and notice what friends, family, movie stars, musicians etc. Are wearing.\",\"endings\":\"[\\\"[title] If you are not sure, ask what your friends' family likes and you will have a smart guess. [step] Ask your parents, siblings, any close friends etc.\\\", \\\"[title] Have an idea of what happens when a friend presents you with an outfit that you wouldn't normally wear. [step] This will probably demonstrate to your friend that it's not always for very special occasions.\\\", \\\"For example, think about once when a friend says that she is doing a crab walk because she loves it and cares about the important things in life. However, if this friend thinks that you like him, you're better off trying to go for their clothes.\\\", \\\"Ask yourself what you like/don't like about their personal style and make note of the things you would like to add to your own style. [title] Write down all of your unique interests and tastes.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}