{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80026411-2541-5778-a9e7-76ea5f523b9a","task_key":"default--validation--28631","task_revision_id":"1","upstream_id":"28631","short_description":"HellaSwag validation 28631","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to celebrate a second wedding anniversary [title] Pick up something for the bathroom. [step] There are a variety of gift options for the bathroom involving cotton. Try purchasing a nice bath towel or wash cloth for your loved one.\",\"endings\":\"['You could also pick up sandwiches, snacks, cookies, and cupcakes for guests. Celebrate a garter for a nice new garter or tie come with the gift.', 'Add a monogram to the items to differentiate them from your other towels as something special. [substeps] Accessorize your existing bathroom items with cotton! Make a cozy for your soap dispenser or toothbrush holder with cute designs or both of your initials.', 'The towels can be tailored or altered according to your personal tastes. On a day spa day, wrap cloths in take out bags, and wash them with shampoo and water.', 'Use a gift that shows your love for your loved one and tells her she can use the bathroom anywhere. See what other matching gifts they have to help them get to know each other better.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}