{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"049fea6e-6c56-5e2c-901e-883faee19b72","task_key":"default--validation--14155","task_revision_id":"1","upstream_id":"14155","short_description":"HellaSwag validation 14155","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to be a successful pilgrim [title] Feel as if god wants you to undertake this journey and remain eager to see the temple. [step] Slowly, bring the shrine and the particular deity into your consciousness, right from the time you get the idea to go on the trip. Search the net for articles and experiences of people who visited the same place and your urge to make it happen will become stronger.\",\"endings\":\"['As that happens, the universe will pave the way for you to reach your destination, and all the details will fall into place. [title] Book a flight after checking the timings of the place of worship.', 'Go all over the world and look around. Give thanks and recognition to the divine, and unto god for your journey.', '[substeps] For more information on making a crucial pilgrimage with an unusual deity, see. [title] Accept your presence at the temple and be willing to welcome the people who go there with you.', \\\"Try to do all you can to be happy and hopeful whilst standing confidently to the end of your journey. It's the dream, not a hope! [title] Find the halls of worship.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}