{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4a3784f8-8f8c-5937-b5ac-3283d5ce0996","task_key":"default--validation--44535","task_revision_id":"1","upstream_id":"44535","short_description":"HellaSwag validation 44535","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to adjust a hot water heater [title] Consider whether your water temperature is in need of adjusting. [step] For safety reasons, most manufacturers recommend that the water you use in your home stay at around 120 ° f (49 ° c). Most water heaters will already be set to this temperature when they're installed.\",\"endings\":\"['[substeps] Newer faucets and toilets are generally less stable than older ones. [title] Try plugging your heating system back in if your water temperature is struggling.', 'If your water heater is in fact in need of adjusting, however, you may need to change it or add a key component to make it work. [substeps] Read the instructions on the cpu thermometer, or look online at a local retailer.', 'For added safety purposes, they may be adjusted any time of year with an extreme rate of warming. [substeps] You will need to adjust your water temperature using a thermometer and app that helps you monitor your water temperature from open windows and doors.', \\\"To limit the risk of injury, it's best to just leave it alone in most cases. [substeps] If your water seems unusually cold, the problem may be a broken heating element or poor insulation rather than the temperature of the water heater itself.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}