{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"664c7c52-7e8d-5ffd-8816-b59d0e60155b","task_key":"default--validation--45863","task_revision_id":"1","upstream_id":"45863","short_description":"HellaSwag validation 45863","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to take out a monroe piercing [title] Wash your hands. [step] Before removing your monroe piercing, wash your hands thoroughly to remove any bacteria. Rinse your hands under warm water and then apply soap.\",\"endings\":\"['Scrub your hands for approximately 20 seconds and then rinse the soap off with warm running water. [substeps] Sing happy birthday while scrubbing your hands to ensure that you scrub for at least 20 seconds.', 'Scrub your piercing with a soft cloth or your fingers, rubbing away any dirt or gunk. [title] Remove the piercing with a washing device.', \\\"Lather your hands with warm water for at least 30 seconds, which will open up any pores that may be present, particularly if you have a french piercing. [substeps] You shouldn't touch the piercing or, worse, the piercer.\\\", 'Lather your hands with soap and lather up until lather is removed. [substeps] Any remaining soap may stain your hands.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}