{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75283e4a-5223-5141-9f58-df6cbc6063d4","task_key":"default--validation--20363","task_revision_id":"1","upstream_id":"20363","short_description":"HellaSwag validation 20363","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to identify a selkirk rex [title] Look at the cat's fur. [step] Easily the most identifiable characteristic of the selkirk rex is their thick, curly coat of hair. The selkirk's curly coat will have a soft, plush appearance, and will be soft to the touch as well.\",\"endings\":\"[\\\"If you look at an rex's coat it may also be slightly matted or thick. Sometimes this may be the result of skin burning, skin mite infestation, or some other deficiency of the hair.\\\", \\\"The curled hair will be most pronounced around the cat's tail and neck, where hair may bunch together. [substeps] The color and pattern of selkirk rex coats can vary from one cat to the next.\\\", 'However, they will often come from the mold, either in its natural form or manipulated as part of a decorative piece. They will also not shed fur so much that they appear damp or dirty.', 'This makes them easy to see or recognize. [substeps] Selkirk rex cats are extremely active, and the fact that they like to play will make them a great addition to your wardrobe.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}