{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"523c2ad5-edf8-522b-bc80-28cfac587c4d","task_key":"default--validation--17391","task_revision_id":"1","upstream_id":"17391","short_description":"HellaSwag validation 17391","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to draw a cool fashion design [title] Think of a style for your fashion. [step] Sketches are important to fashion designing because you want your idea on paper before you start doing the real thing. So, start sketching out your idea.\",\"endings\":\"['If you find yourself \\\" ruminating \\\" and having not thought up the exact words to make your designs unique and original, you can at least use online sketches as a guide. [title] Pick a color you like and the look you want.', \\\"It doesn't have to be perfect; it's a draft. Don't forget to add color! [title] Get a clear idea of what you want to draw in your head.\\\", 'Do it within the first few hours of a new project. Go back and think on the pencil sketch you bought at the store.', \\\"Maybe a basic idea for an outfit, maybe a short one, maybe a more advanced idea for a traditional style. You'll enjoy the ideas! [substeps] Doodle the drawing walls with a pencil or by folding the paper back on one side and folding again; the right size and placement will come when you're drawing the design walls.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}