{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51d054de-092d-5e16-b424-9230e812e0ea","task_key":"default--validation--23177","task_revision_id":"1","upstream_id":"23177","short_description":"HellaSwag validation 23177","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to cook chuck steak [title] Preheat the oven and season the steak. [step] Turn the oven on to 325 degrees f (162 c). Place 2 tablespoons of vegetable or canola oil in a large pot or dutch oven.\",\"endings\":\"[\\\"Place the pot on the stove and turn the heat to medium high. Stir the ingredients constantly to ensure that they're not clumped together in the pot and that they cook evenly.\\\", 'Turn on the burner to the highest setting possible and toss in the potatoes and onions. [substeps] To save time, put the pan upside down over a medium heat.', 'Let the oil heat up for about 15 minutes so that it only coats the steak. [substeps] The oil should be hot, but not overly hot.', 'Heat the oil over medium heat and sprinkle the chuck steak with salt and pepper. [substeps] You could use a big skillet if your chuck steaks are thin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}