{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ae335fb3-b118-5854-8b2f-908879628d16","task_key":"default--validation--15175","task_revision_id":"1","upstream_id":"15175","short_description":"HellaSwag validation 15175","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to cook rice in pressure cooker [title] Measure out rice and water. [step] Pour your desired measurement of rice into your pressure cooker. Add water according to the type and quantity of rice you are using.\",\"endings\":\"['[substeps] For example, you should add 15¾ cups (approximately 1.5 l) of water per cup of rice for a 7-8 pound (9.4-1.44 l) pressure cooker. If using a pressure cooker alone, add a pinch of salt to the rice before cooking.', 'One cup of white rice, for instance, needs 1.5 cups of water. [substeps] Substitute all or part of the water with a liquid for flavoring if you wish, such as chicken broth, vegetable broth, or prepared bouillion.', 'If you know the ingredients, use that to evaluate the amount of rice that the pressure cooker will have. [substeps] To be more precise, use 1 tbsp.', '[substeps] Add water according to the size of your rice (larger may mean larger grains). [title] Fill the pressure cooker with enough water to cover the rice.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}