{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a2d6bcbc-2fd4-545d-8631-0707237fefa2","task_key":"default--validation--5748","task_revision_id":"1","upstream_id":"5748","short_description":"HellaSwag validation 5748","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to play castanets [title] Choose the right size castanets. [step] This is largely a matter of preference, as the string connecting the shells of your castanets are adjustable. The size of your castanets will also influence the tone you create.\",\"endings\":\"['[substeps] Castanets that are too small will add to the melody. Larger sizes will place too much emphasis on tempo.', '[substeps] Stocks of castanets tend to be larger (often if not more), while stocks of castanets tend to be smaller (often even costing big bucks). Buying castanets too small can affect how your bass sound plays out.', '[substeps] After inserting the shells of your castanets into the shells you will now need to find a 3/4 \\\" wood or wooden base for the shells. Choose a base like the sand under which the shells break or clump.', \\\"Generally, you'll want a pair that fit comfortably in your hand and can be hidden easily from view in your palm. [substeps] Sizes range from three, which is usually for children or to achieve high pitches, up to nine for larger hands and deeper tones.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}