{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d547162-0558-5e55-aac7-4ac63dc51718","task_key":"default--validation--5002","task_revision_id":"1","upstream_id":"5002","short_description":"HellaSwag validation 5002","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make healthy ramen [title] Bring two eggs to a boil. [step] Healthy chicken ramen has a classic chicken broth-based flavor that is reminiscent of instant ramen, but tastes fresher and is much healthier. Begin by adding two whole eggs in their shells, to a medium sized pot.\",\"endings\":\"['Add enough water to cover the eggs by 1 inch (2.54 cm), and heat them on medium heat until the water reaches a boil. [title] Let the eggs sit for 7 minutes in the water.', 'Add one tablespoon of olive oil or margarine, plus 1 teaspoon of lime juice, the juice of two lemons. Cover the pot with a lid and bring it to a boil over medium-high heat, which will boil the eggs.', 'Boil them over medium-high heat until the eggs begin to color and sizzle on the stove. [substeps] Once the eggs begin to sizzle, turn the heat down to medium-low and bring the chicken to a boil.', 'Cook the eggs in the boiling water for three to four minutes, until they become tender, and then lower the heat to medium and let stand until the eggs begin to simmer with the rest of the egg liquid. [substeps] Boil for five minutes, blending constantly to prevent the egg from cooking out.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}