{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c1715d14-4112-5b71-8e19-5b1c6a9a61e1","task_key":"default--validation--43458","task_revision_id":"1","upstream_id":"43458","short_description":"HellaSwag validation 43458","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to deep cleanse your face [title] Clean your face thoroughly. [step] Before you steam your face, clean it thoroughly. [substeps] Before washing your face, it's a good idea to wash your hands first.\",\"endings\":\"[\\\"The water will soak through your pores, which might cause irritating pimples. You don't need to scrub your face, but you might consider using a gentle soap or petroleum jelly to remove the residue.\\\", 'This removes dirt, oil, and germs from your face. Select a gentle cleanser to pick out with a gentle toothbrush.', \\\"Wash them thoroughly with warm water and soap, then gently pat your face dry with a towel. Cleansing doesn't rinse away filth or pollution, but it does take the rug out from under your mattress.\\\", 'Wet your hands and lather them in soap. Then, scrub for 20 seconds, making sure to wash between your fingers, under your nails, and on the back of your hands.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}