{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6573fe1e-7de2-54cb-bc18-54a7bba7923f","task_key":"default--validation--12504","task_revision_id":"1","upstream_id":"12504","short_description":"HellaSwag validation 12504","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to set up a food drive [title] Decide who you will be asking for donations. [step] Do you want to collect donations from your work or church? Or will you branch out to a general audience and advertise across your area? Making concrete decisions about this up front will help you stay organized as you go along. [substeps] If you're new to organizing food drives, stay small.\",\"endings\":\"['Don\\\\'t invite more than three people to your vehicle. A fun idea to explore is writing a \\\" bumper sticker \\\" for your car of the kind of donations you hope to get.', \\\"You will probably be able to tag along when other adults go to work. If you're new to making food drives, think about getting managers, employees, or even your own team to help (see step 10 below).\\\", \\\"Stick to your office, church, or civic organization and get a feel for the challenges of the process. Make note of what worked and didn't work for future food drives.\\\", \\\"First, make sure your target audience is small enough to get involved in the process. Next, don't take too much time deciding how many can donate.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}