{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"92e454b0-6687-5cf5-9195-d0e3cf89b683","task_key":"default--validation--42754","task_revision_id":"1","upstream_id":"42754","short_description":"HellaSwag validation 42754","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to find the best swimsuits for your body shape [title] Try a one-piece in a solid color. [step] Solid color suits look sleek and stylish. Your figure may be curvy enough to draw attention on its own, so there is not much need to worry about accentuating certain parts or downplaying others.\",\"endings\":\"['In order to make sure that your look is flattering to your body type, you can add some oomph. [substeps] Solid colors should aim to complement your body shape.', 'Professional swimsuits in greys, olive, cotton, or dark colors can add to modern charm. [substeps] Avoid neon or pastel colors, which appear harsh and unnatural.', '[title] Consider a swimsuit with a deep plunge. [step] Odds are, you have a curvy bust worth showing off.', 'A three-piece suit is one size fits all for you and should be packed in a separate suitcase when you are visiting relatives and friends. Select, include, and keep a list of key pieces that may be best tailored to your needs.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}