{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f53af7ff-baa3-5c40-9a07-64a13b981a0f","task_key":"default--validation--16612","task_revision_id":"1","upstream_id":"16612","short_description":"HellaSwag validation 16612","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wash your face with rice water [title] Choose your rice. [step] You can prepare rice water with any type of rice, though white, brown, and jasmine rice are common options. If you already own rice, whatever kind of rice you have on hand will work.\",\"endings\":\"['[title] Put 1/2 cup (92.5 g) of rice into a bowl. [step] If you want to make large quantities of rice water, you can increase the amount of rice that you use, as long as you remember to also increase the water.', '[substeps] Soy rice works well, as will turmeric, cloves, and jalapeño. [title] Pour rice water into a mug or cup.', \\\"[substeps] New clients and those who have just started out might choose white rice, but it's less likely to taste good due to its other flavors. [title] Add a few drops of garlic and red pepper to your rice water.\\\", \\\"If you don't already own rice water, you can buy it at your local grocery store or online. [substeps] The rice doesn't need to be whole grains, which means it just needs to be dried.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}