{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a23acbce-2ef2-5507-9b48-80f927d7a9c8","task_key":"default--validation--29373","task_revision_id":"1","upstream_id":"29373","short_description":"HellaSwag validation 29373","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to become a hair color specialist [title] Determine your state's requirements. [step] How long you need to be in cosmetology school depends mainly on how many hours are required by your individual u.s. state. You should determine how many hours of training and practice that you will need in order to qualify for a cosmetology certificate.\",\"endings\":\"['Once you have this information, you should start looking for a job, possibly a job search program. [substeps] Each state has a growing number of jobs available to those aspiring to become a beauty technician.', '[substeps] For example, montana requires 2000 hours, while new mexico requires 1600 hours. If you move to another state, you will need to transfer your license.', \\\"[substeps] To begin your program with the juice certification or the curls certification, you need to complete at least six months of practice before graduating. You'll need at least 40 hours of practice and at least 5 years of supervised instruction prior to going to the certification exam.\\\", '[substeps] The educational requirements in cosmetology schools vary widely from state to state. Some schools might require more than one degree to gain certification.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}