{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"290abde4-47fb-577b-8a81-5a7e6089dc82","task_key":"default--validation--25009","task_revision_id":"1","upstream_id":"25009","short_description":"HellaSwag validation 25009","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make buffalo chicken dip [title] Preheat your oven to 350 degrees. [step] You'll want to cook the entire dip in the oven once assembled to melt the cheese on top and ensure the flavors blend together. That said, the chicken needs to be pre-cooked, so this time in the oven is just to get everything hot.\",\"endings\":\"['[title] Warm the 8-oz package cream cheese up to room temperature. [step] This simply makes it easier to blend into the other ingredients, preventing chunks of cream cheese in some parts of the dip and none in the others.', '[substeps] Make sure that the oven is at temperature by flipping the oven dial and turning it on to 350ºf (180ºc). When the oven reaches 350 degrees f (178ºc), turn off the oven.', '[substeps] Alternatively, pre-cooked chicken dip can go the other route and be a little tougher by sticking to a lower oven temperature. Keep in mind that buffalo is a complex, high-calorie sauce.', \\\"[title] Pour the olive oil into a deep-fryer or pan. [step] To ensure that the oil cooks evenly, you'll need to make sure the oil is evenly distributed in the bottom of the pan as well.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}