{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b92c6268-38f6-522e-9940-da92dd96f332","task_key":"default--validation--4142","task_revision_id":"1","upstream_id":"4142","short_description":"HellaSwag validation 4142","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a short hairstyle as an older woman [title] Decide how short you want to go. [step] Short can mean different things to different people, and the length of your hair can help you decide what cut to go for. For example, pixie cuts are great for women who want extremely short hairstyle while a bob works for those who want a chin-length cut.\",\"endings\":\"['What length you want has to do with either your hair or height : [substeps] Your height height. If you want to be younger, stick with half inch hair and half an inch length.', 'You might decide that you want to go : [substeps] Chin-length mouth-length cheek-length eye or ear length pixie cut [title] Define your hair goals. [step] Try to determine why you want a short haircut.', \\\"Shorter hair tends to look better with longer cuts, so don't buy a bob that looks too short. Stylists must know how to use or talk about shortest cuts.\\\", \\\"Here are some tips to help get you started : [substeps] It's not recommended that you go too long. Men over 40 can cut right above mid-length.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}