{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eb34af2b-10d3-5e3b-b1aa-ef9df875e0de","task_key":"default--validation--35576","task_revision_id":"1","upstream_id":"35576","short_description":"HellaSwag validation 35576","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to find, interview, and hire employees for a small business [title] Apply for an employer identification number (ein). [step] This identification number is needed for the irs to collect all salary and tax information for your employees. Contact the irs and request the form ss-4, or find the downloadable version.\",\"endings\":\"['[substeps] This step only applies if you have yet to hire any employees. [title] Create a method to keep track of all employee salary and tax records.', \\\"[substeps] You cannot obtain an ein online through the irs's online tool. Rather, you can use your ein to send money or stock to the irs, by mail, by fax, or over the phone.\\\", 'Use this page to determine whether you want to apply for an ein. [substeps] Ein is made up of two halves, part for each person working on the job: a part for the person working on the job and part for the individual that does the work.', '[substeps] You cannot enroll in an ein through the irs website. Ein cannot be used to purchase, rent, or use your business name as an ein.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}