{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a757360c-41dd-5cf5-9259-3bc77234c995","task_key":"default--validation--30846","task_revision_id":"1","upstream_id":"30846","short_description":"HellaSwag validation 30846","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to plan a beach wedding [title] Choose the right beach. [step] If there is a particular foreign or faraway beach that strikes your fancy, you can plan a destination wedding. Be aware, however, that attending this type of wedding may be inconvenient and expensive for your guests.\",\"endings\":\"['Romantic beach weddings are usually the time to share a meal with guests, get comfortable with one another, and participate in entertainment. [substeps] If the destination you chose is rustic, adventurous, or tropical, choose a warm beach wedding.', '[title] Make sure you are in a safe location. [step] When planning a beach wedding, make sure you are at least comfortably at least 10 minutes away from the beach.', 'Beaches are filled with expensive getaways, which will essentially make it impossible for guests to live comfortably comfortably. [substeps] Location is key; many beach weddings involve guests planning the trip and then having a pre-designated restaurant to host.', 'If you live near the beach, consider having your wedding close to home. Your guests will appreciate the convenience and lower cost, and you will probably have a bigger turnout for your event.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}