{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"65f50652-a298-5f61-bd13-7e2d56bac49b","task_key":"default--validation--17310--variant-a064d1ee8958","task_revision_id":"1","upstream_id":"17310--variant-a064d1ee8958","short_description":"HellaSwag validation 17310","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to cut knee holes in jeans [title] Put the jeans on and mark the knees where you want the holes. [step] Find a pair of jeans that fit you well, then put them on. Draw a horizontal line across your knee with a pen or a piece.\",\"endings\":\"['Your lines should be a bit larger than your actual knee holes! [title] Mark where you would like the hole to go. [step] Write down the distance that your line will go to make sure you will be able to keep track of your guidelines for the cutting distance and width.', 'Over the line, draw a permanent mark where you want the holes to be. Place the jeans on a blanket or set down your pants.', 'Use this line as a guideline to calculate your stitching distance. [substeps] Try not to forget your measurement after you cut the jeans.', 'You can do this on just 1 knee, or you can do it on both knees. [substeps] You can create more than 1 slit on each knee.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}