{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"32e63eb0-2dbf-565f-b833-b5461d29cf52","task_key":"default--validation--1344","task_revision_id":"1","upstream_id":"1344","short_description":"HellaSwag validation 1344","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to become a body piercer [title] Make sure that you are a suitable candidate. [step] Before you go forward with your training, first consider whether piercing is the right option for you. [substeps] Body piercers must be at least 18 years old.\",\"endings\":\"['Pregnant women and those over 21 are at no additional risk of developing injuries or diseases. A figure greater than 33 is considered a well-qualified candidate.', \\\"Since piercing is a sport, you must be 16 years old to be a piercer. If you are over the age of 18, you'll need a detailed diagnosis from a pediatrician and a monthly medical history from a doctor.\\\", 'Body piercers also need to have exceptional hand-eye coordination. If you have unsteady or shaky hands, keep in mind that this could be a hurdle you may be unable to overcome.', \\\"As a responsible candidate, you'll be responsible for paying for the piercing so you know if it's appropriate for you. Generally, large or tightly open piercings are more acceptable than small or uneaten ones.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}