{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b8c7e35c-665e-50fd-8f25-ea5fac6250a5","task_key":"default--validation--20554","task_revision_id":"1","upstream_id":"20554","short_description":"HellaSwag validation 20554","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to treat rickets [title] Take the prescribed supplements. [step] If someone is diagnosed with rickets, the doctor will prescribe supplements to replace the vitamins your body lacks. In most cases, the doctor will prescribe vitamin d, calcium, and/or phosphorus supplements.\",\"endings\":\"['[substeps] Taking supplements is costly and rare; however, the doctor may provide you with supplements at various different intervals throughout the week. Make sure to ask your doctor before taking supplements.', '[substeps] If you are old enough to start consuming supplements, the doctor will usually measure the liquid medicines or any supplements that you take. The doctor may also measure the vitamin d in different formulas, and will ask you for specific directions on how you can ensure that the liquid medicine comes into the correct amount of water you need.', 'To ensure your body can best process these supplements, schedule an appointment with a registered dietitian. [title] Bring pads and/or tampons with you to work.', 'Make sure you or your child take the supplements as directed. [substeps] Vitamin d is best absorbed when taken with a meal, so plan to take your supplements with food.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}