{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b58ed0f2-3b6f-5e51-a573-89e710274f12","task_key":"default--validation--5714","task_revision_id":"1","upstream_id":"5714","short_description":"HellaSwag validation 5714","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean a razor blade [title] Rinse off the razor blade. [step] Hold the head of the razor under a powerful stream of water. If it's necessary to remove the blade in order to clean it, do so very carefully.\",\"endings\":\"[\\\"Direct the water over the backside of the head, where it will help flush out hair, shaving cream residue and other gunk that's stuck on the blade (or blades). It may help to rotate the razor as you rinse to help the water hit it from different angles.\\\", 'Just rinsing it with water will usually stop the blades from sticking together, making cleaning easier. You can also use some dish soap or any other cleaning product you like to wash the razor blade clean.', 'As you rinse, make sure to use soap to rinse it. [title] Position the blade under a shallow dish or bowl.', 'Beginning at the nape of the hair near the shaft, gently flick the blade until the blade has been cleaned and completely rinsed. [substeps] Never rinse the blade under running water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}