{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"46dab5b4-00b9-5b97-8a76-be33807fc748","task_key":"default--validation--5066","task_revision_id":"1","upstream_id":"5066","short_description":"HellaSwag validation 5066","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to celebrate your anniversary at home [title] Make dinner at home. [step] This is the age-old alternative for a candlelit dinner on the town: change the venue to your own home. It can be an extravagant, romantic affair, or just the two of your having fun eating your favorite foods together.\",\"endings\":\"[\\\"Surprise your significant other with a hot, home cooked meal for your anniversary celebration, or prepare the meal together for more of a feeling of togetherness. [substeps] If you're looking for a fresh, sophisticated take on the traditional home cooked dinner, plan a wine-and-cheese tasting for two.\\\", \\\"For example, if it's your anniversary on the beach, plan to cook a fancy meal with flower bouquets, candles and pet treats. You can also come up with vegetarian options to make the meal, too.\\\", \\\"Make sure your guest is comfortable with this type of dinner before you plan to do so; you don't want to ruin the meal you have planned. [substeps] You can do this part out several times throughout the year so that none of your guests come up with additional ideas that seem uninteresting.\\\", \\\"Be sure to include your own taste in pizza and burgers. In particular, research suggests that it's healthier to have some original pizza than you might at home.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}