{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c12c63e-5cab-5a24-9be9-fe76113811a3","task_key":"default--validation--11319","task_revision_id":"1","upstream_id":"11319","short_description":"HellaSwag validation 11319","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to bake chicken breast [title] Prepare your chicken. [step] Remove your chicken from the package and pat it down with paper towels. Rub a little butter or olive oil onto the chicken to add moisture and flavor.\",\"endings\":\"[\\\"[title] Cut your chicken into 1-inch (2.5-cm) thick slices. [step] Add the chicken, using a 4-inch (10-cm) round knife to ensure the thin slices don't burn.\\\", '[title] Place about 2 ½ tablespoons of olive oil in a large bowl and mix together with a fork until it becomes glossy, which should take 1 minute. [title] Add your chicken seasoning and/or salt and pepper in a well ventilated large bowl.', \\\"If you haven't already, turn over your breasts in a slow cooker and after an hour or two, check to see if they're done. If they are, remove them and let stand for five minutes before cutting into next piece.\\\", \\\"[substeps] If you're using any seasoning, sprinkle it on both sides of the chicken now. For example, you could add something like garlic and dried basil or a cajun seasoning mix.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}