{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3071c92b-7919-5d54-b735-1bf370ea212e","task_key":"default--validation--49499","task_revision_id":"1","upstream_id":"49499","short_description":"HellaSwag validation 49499","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to work at a recording studio [title] Be friendly and adaptable. [step] Working at a recording studio requires a passion for music and the ability to adapt quickly to new technology. You should be able to get along well with others and work on a team.\",\"endings\":\"[\\\"You'll also need to be able to convert input from producers and musicians into results that are both technical and provoke an emotional response. [substeps] Working at a recording studio requires both technical and artistic skills.\\\", 'Also, you should be able to develop communication skills with others if necessary and share learning styles with the others who work with you. [title] Get experience before you move on to more complicated projects.', 'You may not be good at music management or used to working outside of a team. If you work with a band, perform a regular show at their studio and earn money in the process.', 'Consider pursuing an accredited degree, if you want to get certified as a director. [substeps] To get an accredited degree or certification, talk to an accredited music video or recording studio director.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}