{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b80f6c23-3c23-5f61-ab8f-5416ad093733","task_key":"default--validation--41094","task_revision_id":"1","upstream_id":"41094","short_description":"HellaSwag validation 41094","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to bathe a russian tortoise [title] Get a shallow pan or tray large enough to accommodate your tortoise. [step] Ideally, the container you choose should be made of porcelain or opaque plastic, so that your tortoise will not be tempted to climb out. [substeps] A clean, shallow kitty litter pan or a large porcelain baking dish may be ideal for this purpose.\",\"endings\":\"[\\\"Whatever you choose to use as a tub, only use it for bathing the tortoise. Once it's used to bathe the tortoise, it's no longer suitable for food preparation or other such purposes.\\\", 'Cold water works even better than lukewarm water, but not as well as hot. [title] Fill a pot or bucket with water.', 'Be sure that the pan has flushable walls, as well. Most pvc cookers require the lip of the pan to be at least ¼ \\\" (1/8 cm) below the mod to prevent the reptile from escaping.', '[title] Place the tortoise in the pan. [step] The tortoise will then act as his \\\" warm bath, \\\" but if you are planning to bathe him yourself, you should place him into it first.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}