{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"597a421d-7406-52a0-bcfa-a2e03565a0dc","task_key":"default--validation--25017","task_revision_id":"1","upstream_id":"25017","short_description":"HellaSwag validation 25017","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to play left field in baseball [title] Get the gear. [step] A left fielder will need an outfielder's glove. The typical outfielder's glove is long with deep pockets.\",\"endings\":\"[\\\"The outfielder's glove can contain snaps, yarn, balls, or whatever, depending on the rules of your playing. You should wear no more than three fingers when you're playing left field.\\\", 'This helps with reaching fly balls on the run. Most are labeled appropriately and are available in a variety of styles and in a wide range of prices.', \\\"The infielder uses two pieces of aluminum with pockets. In general, players must have infielders' gloves that fit snugly, but do not pinch.\\\", \\\"While it usually feels unclean, it doesn't seem dirty or foul-smelling. If it doesn't feel dirty you can always use a different one.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}