{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6114b74-aa59-54bd-a299-b09b297b3339","task_key":"default--validation--48640","task_revision_id":"1","upstream_id":"48640","short_description":"HellaSwag validation 48640","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to decide if you should get bangs or not [title] Measure the length and width of your face. [step] This will help you rule out some face shapes and help you get a general sense of your look. Use a mirror and measuring tape if you need help.\",\"endings\":\"['[substeps] For the height, measure from your eyebrows to the part of your head closest to your nose. Add two inches if you want to do an up and down look.', '[substeps] If your face is about equally as long as it is wide, you most likely fall into the round, square, or heart shaped category. If your face is a bit longer than it is wide, you may fall into the oval, square, or heart shaped faces.', '[title] Tell the salesperson what width you want to have your hair slightly longer than shoulder length. [step] If you have your hair laying straight up in the center of your head, this is enough.', '[substeps] You might need to measure down from your cheekbone to the top of your head. And beyond that, you might need to measure over your cheekbone and somewhere somewhere underneath it (inside your ear, nose, jaw).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}