# LongBench v2 / 66faa0f5bb02136c067c722c
task_id: 54cef6f7-f84e-5bb3-8876-6e72cff6c368
task_key: train--66faa0f5bb02136c067c722c
task_revision_id: 2
{"choice_A":"read_10x_mtx -> sc.external.pp.harmony_integrate-> scrublet-> pp.normalize_total -> pp.log1p -> tl.pca -> pp.neighbors-> tl.umap -> tl.leiden -> pl.umap","choice_B":"sc.external.pp.harmony_integrate-> read_10x_mtx -> scrublet-> pp.normalize_total -> pp.log1p -> tl.pca -> pl.umap -> pp.neighbors -> tl.leiden -> pl.umap","choice_C":"read_10x_h5 -> sc.external.pp.harmony_integrate -> scrublet-> pp.normalize_total -> pp.log1p -> tl.pca -> tl.umap -> tl.leiden -> pl.umap","choice_D":"sc.external.pp.harmony_integrate -> read_10x_h5 -> scrublet-> pp.normalize_total -> pp.log1p -> tl.pca -> pp.neighbors -> pl.tsne -> tl.tsne","context":"[](https://github.com/scverse/scanpy/stargazers)\n[](https://pypi.org/project/scanpy)\n[](https://pepy.tech/project/scanpy)\n[](https://anaconda.org/conda-forge/scanpy)\n[](https://scanpy.readthedocs.io)\n[](https://dev.azure.com/scverse/scanpy/_build)\n[](https://discourse.scverse.org/)\n[](https://scverse.zulipchat.com)\n[](https://numfocus.org/)\n\n# Scanpy – Single-Cell Analysis in Python\n\nScanpy is a scalable toolkit for analyzing single-cell gene expression data\nbuilt jointly with [anndata][]. It includes\npreprocessing, visualization, clustering, trajectory inference and differential\nexpression testing. The Python-based implementation efficiently deals with\ndatasets of more than one million cells.\n\nDiscuss usage on the scverse [Discourse][]. Read the [documentation][].\nIf you'd like to contribute by opening an issue or creating a pull request, please take a look at our [contribution guide][].\n\n[anndata]: https://anndata.readthedocs.io\n[discourse]: https://discourse.scverse.org/\n[documentation]: https://scanpy.readthedocs.io\n\n[//]: # (numfocus-fiscal-sponsor-attribution)\n\nscanpy is part of the scverse project ([website](https://scverse.org), [governance](https://scverse.org/about/roles)) and is fiscally sponsored by [NumFOCUS](https://numfocus.org/).\nIf you like scverse and want to support our mission, please consider making a [donation](https://numfocus.org/donate-to-scverse) to support our efforts.\n\n\n\n\n## Citation\n\nIf you use `scanpy` in your work, please cite the `scanpy` publication as follows:\n\n> **SCANPY: large-scale single-cell gene expression data analysis**\n>\n> F. Alexander Wolf, Philipp Angerer, Fabian J. Theis\n>\n> _Genome Biology_ 2018 Feb 06. doi: [10.1186/s13059-017-1382-0](https://doi.org/10.1186/s13059-017-1382-0).\n\nYou can cite the scverse publication as follows:\n\n> **The scverse project provides a computational ecosystem for single-cell omics data analysis**\n>\n> Isaac Virshup, Danila Bredikhin, Lukas Heumos, Giovanni Palla, Gregor Sturm, Adam Gayoso, Ilia Kats, Mikaela Koutrouli, Scverse Community, Bonnie Berger, Dana Pe’er, Aviv Regev, Sarah A. Teichmann, Francesca Finotello, F. Alexander Wolf, Nir Yosef, Oliver Stegle & Fabian J. Theis\n>\n> _Nat Biotechnol._ 2023 Apr 10. doi: [10.1038/s41587-023-01733-8](https://doi.org/10.1038/s41587-023-01733-8).\n\n\n[contribution guide]: CONTRIBUTING.md\n\n\nContributing\n============\n\nContributions to Scanpy are highly welcome!\n\nBefore filing an issue\n----------------------\n* Search the repository (also google) to see if someone has already reported the same issue.\n This allows contributors to spend less time responding to issues, and more time adding new features!\n* Please provide a minimal complete verifiable example for any bug.\n If you're not sure what this means, check out\n [this blog post](https://matthewrocklin.com/minimal-bug-reports)\n by Matthew Rocklin or [this definition](https://stackoverflow.com/help/mcve) from StackOverflow.\n* Let us know about your environment. Environment information is available via: `sc.logging.print_versions()`.\n\nContributing code\n-----------------\n\nWe love code contributions!\nIf you're interested in contributing code, please take a look over the [contribution guide](https://scanpy.readthedocs.io/en/latest/dev/index.html) in the main documentation.\n\n\n\n\n\n- [ ] Closes #\n- [ ] Tests included or not required because:\n\n- [ ] Release notes not necessary because:\n\n\n#!/usr/bin/env python3\nfrom __future__ import annotations\n\nimport argparse\nimport subprocess\nfrom typing import TYPE_CHECKING\n\nfrom packaging.version import Version\n\nif TYPE_CHECKING:\n from collections.abc import Sequence\n\n\nclass Args(argparse.Namespace):\n version: str\n dry_run: bool\n\n\ndef parse_args(argv: Sequence[str] | None = None) -> Args:\n parser = argparse.ArgumentParser(\n prog=\"towncrier-automation\",\n description=(\n \"This script runs towncrier for a given version, \"\n \"creates a branch off of the current one, \"\n \"and then creates a PR into the original branch with the changes. \"\n \"The PR will be backported to main if the current branch is not main.\"\n ),\n )\n parser.add_argument(\n \"version\",\n type=str,\n help=(\n \"The new version for the release must have at least three parts, like `major.minor.patch` and no `major.minor`. \"\n \"It can have a suffix like `major.minor.patch.dev0` or `major.minor.0rc1`.\"\n ),\n )\n parser.add_argument(\n \"--dry-run\",\n help=\"Whether or not to dry-run the actual creation of the pull request\",\n action=\"store_true\",\n )\n args = parser.parse_args(argv, Args())\n # validate the version\n if len(Version(args.version).release) != 3:\n msg = f\"Version argument {args.version} must contain major, minor, and patch version.\"\n raise ValueError(msg)\n return args\n\n\ndef main(argv: Sequence[str] | None = None) -> None:\n args = parse_args(argv)\n\n # Run towncrier\n subprocess.run(\n [\"towncrier\", \"build\", f\"--version={args.version}\", \"--yes\"], check=True\n )\n\n # Check if we are on the main branch to know if we need to backport\n base_branch = subprocess.run(\n [\"git\", \"rev-parse\", \"--abbrev-ref\", \"HEAD\"],\n capture_output=True,\n text=True,\n check=True,\n ).stdout.strip()\n pr_description = (\n \"\" if base_branch == \"main\" else \"@meeseeksmachine backport to main\"\n )\n branch_name = f\"release_notes_{args.version}\"\n\n # Create a new branch + commit\n subprocess.run([\"git\", \"switch\", \"-c\", branch_name], check=True)\n subprocess.run([\"git\", \"add\", \"docs/release-notes\"], check=True)\n pr_title = f\"(chore): generate {args.version} release notes\"\n subprocess.run([\"git\", \"commit\", \"-m\", pr_title], check=True)\n\n # push\n if not args.dry_run:\n subprocess.run(\n [\"git\", \"push\", \"--set-upstream\", \"origin\", branch_name], check=True\n )\n else:\n print(\"Dry run, not pushing\")\n\n # Create a PR\n subprocess.run(\n [\n \"gh\",\n \"pr\",\n \"create\",\n f\"--base={base_branch}\",\n f\"--title={pr_title}\",\n f\"--body={pr_description}\",\n *([\"--label=no milestone\"] if base_branch == \"main\" else []),\n *([\"--dry-run\"] if args.dry_run else []),\n ],\n check=True,\n )\n\n # Enable auto-merge\n if not args.dry_run:\n subprocess.run(\n [\"gh\", \"pr\", \"merge\", branch_name, \"--auto\", \"--squash\"], check=True\n )\n else:\n print(\"Dry run, not merging\")\n\n\nif __name__ == \"__main__\":\n main()\n\n\n#!/usr/bin/env python3\nfrom __future__ import annotations\n\nimport argparse\nimport sys\nfrom collections import deque\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nif sys.version_info >= (3, 11):\n import tomllib\nelse:\n import tomli as tomllib\n\nfrom packaging.requirements import Requirement\nfrom packaging.version import Version\n\nif TYPE_CHECKING:\n from collections.abc import Generator, Iterable\n\n\ndef min_dep(req: Requirement) -> Requirement:\n \"\"\"\n Given a requirement, return the minimum version specifier.\n\n Example\n -------\n\n >>> min_dep(Requirement(\"numpy>=1.0\"))\n \"numpy==1.0\"\n \"\"\"\n req_name = req.name\n if req.extras:\n req_name = f\"{req_name}[{','.join(req.extras)}]\"\n\n if not req.specifier:\n return Requirement(req_name)\n\n min_version = Version(\"0.0.0.a1\")\n for spec in req.specifier:\n if spec.operator in [\">\", \">=\", \"~=\"]:\n min_version = max(min_version, Version(spec.version))\n elif spec.operator == \"==\":\n min_version = Version(spec.version)\n\n return Requirement(f\"{req_name}=={min_version}.*\")\n\n\ndef extract_min_deps(\n dependencies: Iterable[Requirement], *, pyproject\n) -> Generator[Requirement, None, None]:\n dependencies = deque(dependencies) # We'll be mutating this\n project_name = pyproject[\"project\"][\"name\"]\n\n while len(dependencies) > 0:\n req = dependencies.pop()\n\n # If we are referring to other optional dependency lists, resolve them\n if req.name == project_name:\n assert req.extras, f\"Project included itself as dependency, without specifying extras: {req}\"\n for extra in req.extras:\n extra_deps = pyproject[\"project\"][\"optional-dependencies\"][extra]\n dependencies += map(Requirement, extra_deps)\n else:\n yield min_dep(req)\n\n\ndef main():\n parser = argparse.ArgumentParser(\n prog=\"min-deps\",\n description=\"\"\"Parse a pyproject.toml file and output a list of minimum dependencies.\n\n Output is directly passable to `pip install`.\"\"\",\n usage=\"pip install `python min-deps.py pyproject.toml`\",\n )\n parser.add_argument(\n \"path\", type=Path, help=\"pyproject.toml to parse minimum dependencies from\"\n )\n parser.add_argument(\n \"--extras\", type=str, nargs=\"*\", default=(), help=\"extras to install\"\n )\n\n args = parser.parse_args()\n\n pyproject = tomllib.loads(args.path.read_text())\n\n project_name = pyproject[\"project\"][\"name\"]\n deps = [\n *map(Requirement, pyproject[\"project\"][\"dependencies\"]),\n *(Requirement(f\"{project_name}[{extra}]\") for extra in args.extras),\n ]\n\n min_deps = extract_min_deps(deps, pyproject=pyproject)\n\n print(\" \".join(map(str, min_deps)))\n\n\nif __name__ == \"__main__\":\n main()\n\n\nfrom __future__ import annotations\n\nfrom typing import cast\n\nimport numpy as np\nimport pytest\nfrom anndata import AnnData\nfrom matplotlib import colormaps\nfrom matplotlib.colors import ListedColormap\n\nfrom scanpy.plotting._utils import _validate_palette\n\nviridis = cast(ListedColormap, colormaps[\"viridis\"])\n\n\n@pytest.mark.parametrize(\n \"palette\",\n [\n pytest.param(viridis.colors, id=\"viridis\"),\n pytest.param([\"b\", \"#cccccc\", \"r\", \"yellow\", \"lightblue\"], id=\"named\"),\n pytest.param([(1, 0, 0, 1), (0, 0, 1, 1)], id=\"rgba\"),\n ],\n)\n@pytest.mark.parametrize(\"typ\", [np.asarray, list])\ndef test_validate_palette_no_mod(palette, typ):\n palette = typ(palette)\n adata = AnnData(uns=dict(test_colors=palette))\n _validate_palette(adata, \"test\")\n assert palette is adata.uns[\"test_colors\"], \"Palette should not be modified\"\n\n\nfrom __future__ import annotations\n\nfrom functools import partial\n\nimport pytest\nfrom anndata import read_h5ad\n\nimport scanpy as sc\n\n\n@pytest.mark.parametrize(\n (\"name\", \"func\", \"msg\"),\n [\n pytest.param(\"PCA\", sc.pp.pca, \" with chunked as False\", id=\"pca\"),\n pytest.param(\n \"PCA\", partial(sc.pp.pca, layer=\"X_copy\"), \" from layers\", id=\"pca_layer\"\n ),\n pytest.param(\n \"regress_out\",\n partial(sc.pp.regress_out, keys=[\"n_counts\", \"percent_mito\"]),\n \"\",\n id=\"regress_out\",\n ),\n pytest.param(\n \"dendrogram\", partial(sc.tl.dendrogram, groupby=\"cat\"), \"\", id=\"dendrogram\"\n ),\n pytest.param(\"tsne\", sc.tl.tsne, \"\", id=\"tsne\"),\n pytest.param(\"scale\", sc.pp.scale, \"\", id=\"scale\"),\n pytest.param(\n \"downsample_counts\",\n partial(sc.pp.downsample_counts, counts_per_cell=1000),\n \"\",\n id=\"downsample_counts\",\n ),\n pytest.param(\n \"filter_genes\",\n partial(sc.pp.filter_genes, max_cells=1000),\n \"\",\n id=\"filter_genes\",\n ),\n pytest.param(\n \"filter_cells\",\n partial(sc.pp.filter_cells, max_genes=1000),\n \"\",\n id=\"filter_cells\",\n ),\n pytest.param(\n \"rank_genes_groups\",\n partial(sc.tl.rank_genes_groups, groupby=\"cat\"),\n \"\",\n id=\"rank_genes_groups\",\n ),\n pytest.param(\n \"score_genes\",\n partial(sc.tl.score_genes, gene_list=map(str, range(100))),\n \"\",\n id=\"score_genes\",\n ),\n ],\n)\ndef test_backed_error(backed_adata, name, func, msg):\n with pytest.raises(\n NotImplementedError,\n match=f\"{name} is not implemented for matrices of type {type(backed_adata.X)}{msg}\",\n ):\n func(backed_adata)\n\n\ndef test_log1p_backed_errors(backed_adata):\n with pytest.raises(\n NotImplementedError,\n match=\"log1p is not implemented for backed AnnData with backed mode not r+\",\n ):\n sc.pp.log1p(backed_adata, chunked=True)\n backed_adata.file.close()\n backed_adata = read_h5ad(backed_adata.filename, backed=\"r+\")\n with pytest.raises(\n NotImplementedError,\n match=f\"log1p is not implemented for matrices of type {type(backed_adata.X)} without `chunked=True`\",\n ):\n sc.pp.log1p(backed_adata)\n backed_adata.layers[\"X_copy\"] = backed_adata.X\n layer_type = type(backed_adata.layers[\"X_copy\"])\n with pytest.raises(\n NotImplementedError,\n match=f\"log1p is not implemented for matrices of type {layer_type} from layers\",\n ):\n sc.pp.log1p(backed_adata, layer=\"X_copy\")\n backed_adata.file.close()\n\n\ndef test_scatter_backed(backed_adata):\n sc.pp.pca(backed_adata, chunked=True)\n sc.pl.scatter(backed_adata, color=\"0\", basis=\"pca\")\n\n\ndef test_dotplot_backed(backed_adata):\n sc.pl.dotplot(backed_adata, [\"0\", \"1\", \"2\", \"3\"], groupby=\"cat\")\n\n\nfrom __future__ import annotations\n\nfrom functools import partial\nfrom itertools import chain, repeat\n\nimport numpy as np\nimport pandas as pd\nimport pytest\nfrom anndata import AnnData\nfrom scipy import sparse\n\nimport scanpy as sc\nfrom scanpy.datasets._utils import filter_oldformatwarning\nfrom testing.scanpy._helpers import anndata_v0_8_constructor_compat\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\n\n\n# Override so warning gets caught\ndef transpose_adata(adata: AnnData, *, expect_duplicates: bool = False) -> AnnData:\n if not expect_duplicates:\n return adata.T\n with pytest.warns(UserWarning, match=r\"Observation names are not unique\"):\n return adata.T\n\n\nTRANSPOSE_PARAMS = pytest.mark.parametrize(\n \"dim,transform,func\",\n [\n (\"obs\", lambda x, expect_duplicates=False: x, sc.get.obs_df),\n (\"var\", transpose_adata, sc.get.var_df),\n ],\n ids=[\"obs_df\", \"var_df\"],\n)\n\n\n@pytest.fixture\ndef adata():\n \"\"\"\n adata.X is np.ones((2, 2))\n adata.layers['double'] is sparse np.ones((2,2)) * 2 to also test sparse matrices\n \"\"\"\n return anndata_v0_8_constructor_compat(\n X=np.ones((2, 2), dtype=int),\n obs=pd.DataFrame(\n {\"obs1\": [0, 1], \"obs2\": [\"a\", \"b\"]}, index=[\"cell1\", \"cell2\"]\n ),\n var=pd.DataFrame(\n {\"gene_symbols\": [\"genesymbol1\", \"genesymbol2\"]}, index=[\"gene1\", \"gene2\"]\n ),\n layers={\"double\": sparse.csr_matrix(np.ones((2, 2)), dtype=int) * 2},\n )\n\n\n########################\n# obs_df, var_df tests #\n########################\n\n\ndef test_obs_df(adata):\n adata.obsm[\"eye\"] = np.eye(2, dtype=int)\n adata.obsm[\"sparse\"] = sparse.csr_matrix(np.eye(2), dtype=\"float64\")\n\n # make raw with different genes than adata\n adata.raw = anndata_v0_8_constructor_compat(\n X=np.array([[1, 2, 3], [2, 4, 6]], dtype=np.float64),\n var=pd.DataFrame(\n {\"gene_symbols\": [\"raw1\", \"raw2\", \"raw3\"]},\n index=[\"gene2\", \"gene3\", \"gene4\"],\n ),\n )\n pd.testing.assert_frame_equal(\n sc.get.obs_df(\n adata, keys=[\"gene2\", \"obs1\"], obsm_keys=[(\"eye\", 0), (\"sparse\", 1)]\n ),\n pd.DataFrame(\n {\"gene2\": [1, 1], \"obs1\": [0, 1], \"eye-0\": [1, 0], \"sparse-1\": [0.0, 1.0]},\n index=adata.obs_names,\n ),\n )\n pd.testing.assert_frame_equal(\n sc.get.obs_df(\n adata,\n keys=[\"genesymbol2\", \"obs1\"],\n obsm_keys=[(\"eye\", 0), (\"sparse\", 1)],\n gene_symbols=\"gene_symbols\",\n ),\n pd.DataFrame(\n {\n \"genesymbol2\": [1, 1],\n \"obs1\": [0, 1],\n \"eye-0\": [1, 0],\n \"sparse-1\": [0.0, 1.0],\n },\n index=adata.obs_names,\n ),\n )\n pd.testing.assert_frame_equal(\n sc.get.obs_df(adata, keys=[\"gene2\", \"obs1\"], layer=\"double\"),\n pd.DataFrame({\"gene2\": [2, 2], \"obs1\": [0, 1]}, index=adata.obs_names),\n )\n\n pd.testing.assert_frame_equal(\n sc.get.obs_df(\n adata,\n keys=[\"raw2\", \"raw3\", \"obs1\"],\n gene_symbols=\"gene_symbols\",\n use_raw=True,\n ),\n pd.DataFrame(\n {\"raw2\": [2.0, 4.0], \"raw3\": [3.0, 6.0], \"obs1\": [0, 1]},\n index=adata.obs_names,\n ),\n )\n # test only obs\n pd.testing.assert_frame_equal(\n sc.get.obs_df(adata, keys=[\"obs1\", \"obs2\"]),\n pd.DataFrame({\"obs1\": [0, 1], \"obs2\": [\"a\", \"b\"]}, index=[\"cell1\", \"cell2\"]),\n )\n # test only var\n pd.testing.assert_frame_equal(\n sc.get.obs_df(adata, keys=[\"gene1\", \"gene2\"]),\n pd.DataFrame({\"gene1\": [1, 1], \"gene2\": [1, 1]}, index=adata.obs_names),\n )\n pd.testing.assert_frame_equal(\n sc.get.obs_df(adata, keys=[\"gene1\", \"gene2\"]),\n pd.DataFrame({\"gene1\": [1, 1], \"gene2\": [1, 1]}, index=adata.obs_names),\n )\n # test handling of duplicated keys (in this case repeated gene names)\n pd.testing.assert_frame_equal(\n sc.get.obs_df(adata, keys=[\"gene1\", \"gene2\", \"gene1\", \"gene1\"]),\n pd.DataFrame(\n {\"gene1\": [1, 1], \"gene2\": [1, 1]},\n index=adata.obs_names,\n )[[\"gene1\", \"gene2\", \"gene1\", \"gene1\"]],\n )\n\n badkeys = [\"badkey1\", \"badkey2\"]\n with pytest.raises(KeyError) as badkey_err:\n sc.get.obs_df(adata, keys=badkeys)\n with pytest.raises(AssertionError):\n sc.get.obs_df(adata, keys=[\"gene1\"], use_raw=True, layer=\"double\")\n assert all(badkey_err.match(k) for k in badkeys)\n\n # test non unique index\n with pytest.warns(UserWarning, match=r\"Observation names are not unique\"):\n adata = sc.AnnData(\n np.arange(16).reshape(4, 4),\n obs=pd.DataFrame(index=[\"a\", \"a\", \"b\", \"c\"]),\n var=pd.DataFrame(index=[f\"gene{i}\" for i in range(4)]),\n )\n df = sc.get.obs_df(adata, [\"gene1\"])\n pd.testing.assert_index_equal(df.index, adata.obs_names)\n\n\ndef test_repeated_gene_symbols():\n \"\"\"\n Gene symbols column allows repeats, but we can't unambiguously get data for these values.\n \"\"\"\n gene_symbols = [f\"symbol_{i}\" for i in [\"a\", \"b\", \"b\", \"c\"]]\n var_names = pd.Index([f\"id_{i}\" for i in [\"a\", \"b.1\", \"b.2\", \"c\"]])\n adata = sc.AnnData(\n np.arange(3 * 4, dtype=np.float32).reshape((3, 4)),\n var=pd.DataFrame({\"gene_symbols\": gene_symbols}, index=var_names),\n )\n\n with pytest.raises(KeyError, match=\"symbol_b\"):\n sc.get.obs_df(adata, [\"symbol_b\"], gene_symbols=\"gene_symbols\")\n\n expected = pd.DataFrame(\n np.arange(3 * 4).reshape((3, 4))[:, [0, 3]].astype(np.float32),\n index=adata.obs_names,\n columns=[\"symbol_a\", \"symbol_c\"],\n )\n result = sc.get.obs_df(adata, [\"symbol_a\", \"symbol_c\"], gene_symbols=\"gene_symbols\")\n\n pd.testing.assert_frame_equal(expected, result)\n\n\n@filter_oldformatwarning\ndef test_backed_vs_memory():\n \"\"\"compares backed vs. memory\"\"\"\n from pathlib import Path\n\n # get location test h5ad file in datasets\n HERE = Path(sc.__file__).parent\n adata_file = HERE / \"datasets/10x_pbmc68k_reduced.h5ad\"\n adata_backed = sc.read(adata_file, backed=\"r\")\n adata = sc.read_h5ad(adata_file)\n\n # use non-sequential list of genes\n genes = list(adata.var_names[20::-2])\n obs_names = [\"bulk_labels\", \"n_genes\"]\n pd.testing.assert_frame_equal(\n sc.get.obs_df(adata, keys=genes + obs_names),\n sc.get.obs_df(adata_backed, keys=genes + obs_names),\n )\n\n # use non-sequential list of cell indices\n cell_indices = list(adata.obs_names[30::-2])\n pd.testing.assert_frame_equal(\n sc.get.var_df(adata, keys=cell_indices + [\"highly_variable\"]),\n sc.get.var_df(adata_backed, keys=cell_indices + [\"highly_variable\"]),\n )\n\n\ndef test_column_content():\n \"\"\"uses a larger dataset to test column order and content\"\"\"\n adata = pbmc68k_reduced()\n\n # test that columns content is correct for obs_df\n query = [\"CST3\", \"NKG7\", \"GNLY\", \"louvain\", \"n_counts\", \"n_genes\"]\n df = sc.get.obs_df(adata, query)\n for col in query:\n assert col in df\n np.testing.assert_array_equal(query, df.columns)\n np.testing.assert_array_equal(df[col].values, adata.obs_vector(col))\n\n # test that columns content is correct for var_df\n cell_ids = list(adata.obs.sample(5).index)\n query = cell_ids + [\"highly_variable\", \"dispersions_norm\", \"dispersions\"]\n df = sc.get.var_df(adata, query)\n np.testing.assert_array_equal(query, df.columns)\n for col in query:\n np.testing.assert_array_equal(df[col].values, adata.var_vector(col))\n\n\ndef test_var_df(adata):\n adata.varm[\"eye\"] = np.eye(2, dtype=int)\n adata.varm[\"sparse\"] = sparse.csr_matrix(np.eye(2), dtype=\"float64\")\n\n pd.testing.assert_frame_equal(\n sc.get.var_df(\n adata,\n keys=[\"cell2\", \"gene_symbols\"],\n varm_keys=[(\"eye\", 0), (\"sparse\", 1)],\n ),\n pd.DataFrame(\n {\n \"cell2\": [1, 1],\n \"gene_symbols\": [\"genesymbol1\", \"genesymbol2\"],\n \"eye-0\": [1, 0],\n \"sparse-1\": [0.0, 1.0],\n },\n index=adata.var_names,\n ),\n )\n pd.testing.assert_frame_equal(\n sc.get.var_df(adata, keys=[\"cell1\", \"gene_symbols\"], layer=\"double\"),\n pd.DataFrame(\n {\"cell1\": [2, 2], \"gene_symbols\": [\"genesymbol1\", \"genesymbol2\"]},\n index=adata.var_names,\n ),\n )\n # test only cells\n pd.testing.assert_frame_equal(\n sc.get.var_df(adata, keys=[\"cell1\", \"cell2\"]),\n pd.DataFrame(\n {\"cell1\": [1, 1], \"cell2\": [1, 1]},\n index=adata.var_names,\n ),\n )\n # test only var columns\n pd.testing.assert_frame_equal(\n sc.get.var_df(adata, keys=[\"gene_symbols\"]),\n pd.DataFrame(\n {\"gene_symbols\": [\"genesymbol1\", \"genesymbol2\"]},\n index=adata.var_names,\n ),\n )\n\n # test handling of duplicated keys (in this case repeated cell names)\n pd.testing.assert_frame_equal(\n sc.get.var_df(adata, keys=[\"cell1\", \"cell2\", \"cell2\", \"cell1\"]),\n pd.DataFrame(\n {\"cell1\": [1, 1], \"cell2\": [1, 1]},\n index=adata.var_names,\n )[[\"cell1\", \"cell2\", \"cell2\", \"cell1\"]],\n )\n\n badkeys = [\"badkey1\", \"badkey2\"]\n with pytest.raises(KeyError) as badkey_err:\n sc.get.var_df(adata, keys=badkeys)\n assert all(badkey_err.match(k) for k in badkeys)\n\n\n@TRANSPOSE_PARAMS\ndef test_just_mapping_keys(dim, transform, func):\n # https://github.com/scverse/scanpy/issues/1634\n # Test for error where just passing obsm_keys, but not keys, would cause error.\n mapping_attr = f\"{dim}m\"\n kwargs = {f\"{mapping_attr}_keys\": [(\"array\", 0), (\"array\", 1)]}\n\n adata = transform(\n sc.AnnData(\n X=np.zeros((5, 5)),\n obsm={\n \"array\": np.arange(10).reshape((5, 2)),\n },\n )\n )\n\n expected = pd.DataFrame(\n np.arange(10).reshape((5, 2)),\n index=getattr(adata, f\"{dim}_names\"),\n columns=[\"array-0\", \"array-1\"],\n )\n result = func(adata, **kwargs)\n\n pd.testing.assert_frame_equal(expected, result)\n\n\n##################################\n# Test errors for obs_df, var_df #\n##################################\n\n\ndef test_non_unique_cols_value_error():\n M, N = 5, 3\n adata = sc.AnnData(\n X=np.zeros((M, N)),\n obs=pd.DataFrame(\n np.arange(M * 2).reshape((M, 2)),\n columns=[\"repeated_col\", \"repeated_col\"],\n index=[f\"cell_{i}\" for i in range(M)],\n ),\n var=pd.DataFrame(\n index=[f\"gene_{i}\" for i in range(N)],\n ),\n )\n with pytest.raises(ValueError, match=r\"adata\\.obs contains duplicated columns\"):\n sc.get.obs_df(adata, [\"repeated_col\"])\n\n\ndef test_non_unique_var_index_value_error():\n adata = sc.AnnData(\n X=np.ones((2, 3)),\n obs=pd.DataFrame(index=[\"cell-0\", \"cell-1\"]),\n var=pd.DataFrame(index=[\"gene-0\", \"gene-0\", \"gene-1\"]),\n )\n with pytest.raises(ValueError, match=r\"adata\\.var_names contains duplicated items\"):\n sc.get.obs_df(adata, [\"gene-0\"])\n\n\ndef test_keys_in_both_obs_and_var_index_value_error():\n M, N = 5, 3\n adata = sc.AnnData(\n X=np.zeros((M, N)),\n obs=pd.DataFrame(\n np.arange(M),\n columns=[\"var_id\"],\n index=[f\"cell_{i}\" for i in range(M)],\n ),\n var=pd.DataFrame(\n index=[\"var_id\"] + [f\"gene_{i}\" for i in range(N - 1)],\n ),\n )\n with pytest.raises(KeyError, match=\"var_id\"):\n sc.get.obs_df(adata, [\"var_id\"])\n\n\n@TRANSPOSE_PARAMS\ndef test_repeated_cols(dim, transform, func):\n adata = transform(\n sc.AnnData(\n np.ones((5, 10)),\n obs=pd.DataFrame(\n np.ones((5, 2)), columns=[\"a_column_name\", \"a_column_name\"]\n ),\n var=pd.DataFrame(index=[f\"gene-{i}\" for i in range(10)]),\n )\n )\n # (?s) is inline re.DOTALL\n with pytest.raises(ValueError, match=rf\"(?s)^adata\\.{dim}.*a_column_name.*$\"):\n func(adata, [\"gene_5\"])\n\n\n@TRANSPOSE_PARAMS\ndef test_repeated_index_vals(dim, transform, func):\n # This one could be reverted, see:\n # https://github.com/scverse/scanpy/pull/1583#issuecomment-770641710\n alt_dim = [\"obs\", \"var\"][dim == \"obs\"]\n\n adata = transform(\n sc.AnnData(\n np.ones((5, 10)),\n var=pd.DataFrame(\n index=[\"repeated_id\"] * 2 + [f\"gene-{i}\" for i in range(8)]\n ),\n ),\n expect_duplicates=True,\n )\n\n with pytest.raises(\n ValueError,\n match=rf\"(?s)adata\\.{alt_dim}_names.*{alt_dim}_names_make_unique\",\n ):\n func(adata, \"gene_5\")\n\n\n@pytest.fixture(\n params=[\n \"obs_df\",\n \"var_df\",\n \"obs_df:use_raw\",\n \"obs_df:gene_symbols\",\n \"obs_df:gene_symbols,use_raw\",\n ]\n)\ndef shared_key_adata(request):\n kind = request.param\n adata = sc.AnnData(\n np.arange(50).reshape((5, 10)),\n obs=pd.DataFrame(np.zeros((5, 1)), columns=[\"var_id\"]),\n var=pd.DataFrame(index=[\"var_id\"] + [f\"gene_{i}\" for i in range(1, 10)]),\n )\n if kind == \"obs_df\":\n return (\n adata,\n sc.get.obs_df,\n r\"'var_id'.* adata\\.obs .* adata.var_names\",\n )\n elif kind == \"var_df\":\n return (\n adata.T,\n sc.get.var_df,\n r\"'var_id'.* adata\\.var .* adata.obs_names\",\n )\n elif kind == \"obs_df:use_raw\":\n adata.raw = adata\n adata.var_names = [f\"gene_{i}\" for i in range(10)]\n return (\n adata,\n partial(sc.get.obs_df, use_raw=True),\n r\"'var_id'.* adata\\.obs .* adata\\.raw\\.var_names\",\n )\n elif kind == \"obs_df:gene_symbols\":\n adata.var[\"gene_symbols\"] = adata.var_names\n adata.var_names = [f\"gene_{i}\" for i in range(10)]\n return (\n adata,\n partial(sc.get.obs_df, gene_symbols=\"gene_symbols\"),\n r\"'var_id'.* adata\\.obs .* adata\\.var\\['gene_symbols'\\]\",\n )\n elif kind == \"obs_df:gene_symbols,use_raw\":\n base = adata.copy()\n adata.var[\"gene_symbols\"] = adata.var_names\n adata.var_names = [f\"gene_{i}\" for i in range(10)]\n base.raw = adata\n return (\n base,\n partial(\n sc.get.obs_df,\n gene_symbols=\"gene_symbols\",\n use_raw=True,\n ),\n r\"'var_id'.* adata\\.obs .* adata\\.raw\\.var\\['gene_symbols'\\]\",\n )\n else:\n pytest.fail(\"add branch for new kind\")\n\n\ndef test_shared_key_errors(shared_key_adata):\n adata, func, regex = shared_key_adata\n\n # This should error\n with pytest.raises(KeyError, match=regex):\n func(adata, keys=[\"var_id\"])\n\n # This shouldn't error\n _ = func(adata, keys=[\"gene_2\"])\n\n\n##############################\n# rank_genes_groups_df tests #\n##############################\n\n\ndef test_rank_genes_groups_df():\n a = np.zeros((20, 3))\n a[:10, 0] = 5\n adata = AnnData(\n a,\n obs=pd.DataFrame(\n {\"celltype\": list(chain(repeat(\"a\", 10), repeat(\"b\", 10)))},\n index=[f\"cell{i}\" for i in range(a.shape[0])],\n ),\n var=pd.DataFrame(index=[f\"gene{i}\" for i in range(a.shape[1])]),\n )\n sc.tl.rank_genes_groups(adata, groupby=\"celltype\", method=\"wilcoxon\", pts=True)\n dedf = sc.get.rank_genes_groups_df(adata, \"a\")\n assert dedf[\"pvals\"].value_counts()[1.0] == 2\n assert sc.get.rank_genes_groups_df(adata, \"a\", log2fc_max=0.1).shape[0] == 2\n assert sc.get.rank_genes_groups_df(adata, \"a\", log2fc_min=0.1).shape[0] == 1\n assert sc.get.rank_genes_groups_df(adata, \"a\", pval_cutoff=0.9).shape[0] == 1\n del adata.uns[\"rank_genes_groups\"]\n sc.tl.rank_genes_groups(\n adata,\n groupby=\"celltype\",\n method=\"wilcoxon\",\n key_added=\"different_key\",\n pts=True,\n )\n with pytest.raises(KeyError):\n sc.get.rank_genes_groups_df(adata, \"a\")\n dedf2 = sc.get.rank_genes_groups_df(adata, \"a\", key=\"different_key\")\n pd.testing.assert_frame_equal(dedf, dedf2)\n assert \"pct_nz_group\" in dedf2.columns\n assert \"pct_nz_reference\" in dedf2.columns\n\n # get all groups\n dedf3 = sc.get.rank_genes_groups_df(adata, group=None, key=\"different_key\")\n assert \"a\" in dedf3[\"group\"].unique()\n assert \"b\" in dedf3[\"group\"].unique()\n adata.var_names.name = \"pr1388\"\n sc.get.rank_genes_groups_df(adata, group=None, key=\"different_key\")\n\n\n\"\"\"\nTests to make sure the example datasets load.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport subprocess\nimport warnings\nfrom collections import defaultdict\nfrom pathlib import Path\nfrom textwrap import dedent\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pytest\nfrom anndata.tests.helpers import assert_adata_equal\n\nimport scanpy as sc\nfrom testing.scanpy._pytest.marks import needs\n\nif TYPE_CHECKING:\n from collections.abc import Callable\n\n from anndata import AnnData\n\n\n@pytest.fixture(autouse=True)\ndef _tmp_dataset_dir(tmp_path: Path) -> None:\n \"\"\"Make sure that datasets are downloaded during the test run.\n\n The default test environment stores them in a cached location.\n \"\"\"\n sc.settings.datasetdir = tmp_path / \"scanpy_data\"\n\n\n@pytest.mark.internet\ndef test_burczynski06():\n with pytest.warns(UserWarning, match=r\"Variable names are not unique\"):\n adata = sc.datasets.burczynski06()\n assert adata.shape == (127, 22283)\n assert not (adata.X == 0).any()\n\n\n@pytest.mark.internet\n@needs.openpyxl\ndef test_moignard15():\n with warnings.catch_warnings():\n # https://foss.heptapod.net/openpyxl/openpyxl/-/issues/2051\n warnings.filterwarnings(\n \"ignore\",\n r\"datetime\\.datetime\\.utcnow\\(\\) is deprecated\",\n category=DeprecationWarning,\n module=\"openpyxl\",\n )\n adata = sc.datasets.moignard15()\n assert adata.shape == (3934, 42)\n\n\n@pytest.mark.internet\ndef test_paul15():\n sc.datasets.paul15()\n\n\n@pytest.mark.internet\ndef test_pbmc3k():\n adata = sc.datasets.pbmc3k()\n assert adata.shape == (2700, 32738)\n assert \"CD8A\" in adata.var_names\n\n\n@pytest.mark.internet\ndef test_pbmc3k_processed():\n with warnings.catch_warnings(record=True) as records:\n adata = sc.datasets.pbmc3k_processed()\n assert adata.shape == (2638, 1838)\n assert adata.raw.shape == (2638, 13714)\n\n assert len(records) == 0\n\n\n@pytest.mark.internet\ndef test_ebi_expression_atlas():\n adata = sc.datasets.ebi_expression_atlas(\"E-MTAB-4888\")\n # The shape changes sometimes\n assert 2261 <= adata.shape[0] <= 2315\n assert 23899 <= adata.shape[1] <= 24051\n\n\ndef test_krumsiek11():\n with pytest.warns(UserWarning, match=r\"Observation names are not unique\"):\n adata = sc.datasets.krumsiek11()\n assert adata.shape == (640, 11)\n assert set(adata.obs[\"cell_type\"]) == {\"Ery\", \"Mk\", \"Mo\", \"Neu\", \"progenitor\"}\n\n\ndef test_blobs():\n n_obs = np.random.randint(15, 30)\n n_var = np.random.randint(500, 600)\n adata = sc.datasets.blobs(n_variables=n_var, n_observations=n_obs)\n assert adata.shape == (n_obs, n_var)\n\n\ndef test_toggleswitch():\n with pytest.warns(UserWarning, match=r\"Observation names are not unique\"):\n sc.datasets.toggleswitch()\n\n\ndef test_pbmc68k_reduced():\n with warnings.catch_warnings():\n warnings.simplefilter(\"error\")\n sc.datasets.pbmc68k_reduced()\n\n\n@pytest.mark.internet\ndef test_visium_datasets():\n \"\"\"Tests that reading/ downloading works and is does not have global effects.\"\"\"\n with pytest.warns(UserWarning, match=r\"Variable names are not unique\"):\n hheart = sc.datasets.visium_sge(\"V1_Human_Heart\")\n with pytest.warns(UserWarning, match=r\"Variable names are not unique\"):\n hheart_again = sc.datasets.visium_sge(\"V1_Human_Heart\")\n assert_adata_equal(hheart, hheart_again)\n\n\n@pytest.mark.internet\ndef test_visium_datasets_dir_change(tmp_path: Path):\n \"\"\"Test that changing the dataset dir doesn't break reading.\"\"\"\n with pytest.warns(UserWarning, match=r\"Variable names are not unique\"):\n mbrain = sc.datasets.visium_sge(\"V1_Adult_Mouse_Brain\")\n sc.settings.datasetdir = tmp_path\n with pytest.warns(UserWarning, match=r\"Variable names are not unique\"):\n mbrain_again = sc.datasets.visium_sge(\"V1_Adult_Mouse_Brain\")\n assert_adata_equal(mbrain, mbrain_again)\n\n\n@pytest.mark.internet\ndef test_visium_datasets_images():\n \"\"\"Test that image download works and is does not have global effects.\"\"\"\n\n # Test that downloading tissue image works\n with pytest.warns(UserWarning, match=r\"Variable names are not unique\"):\n mbrain = sc.datasets.visium_sge(\"V1_Adult_Mouse_Brain\", include_hires_tiff=True)\n expected_image_path = sc.settings.datasetdir / \"V1_Adult_Mouse_Brain\" / \"image.tif\"\n image_path = Path(\n mbrain.uns[\"spatial\"][\"V1_Adult_Mouse_Brain\"][\"metadata\"][\"source_image_path\"]\n )\n assert image_path == expected_image_path\n\n # Test that tissue image exists and is a valid image file\n assert image_path.exists()\n\n # Test that tissue image is a tif image file (using `file`)\n process = subprocess.run(\n [\"file\", \"--mime-type\", image_path], stdout=subprocess.PIPE\n )\n output = process.stdout.strip().decode() # make process output string\n assert output == str(image_path) + \": image/tiff\"\n\n\ndef test_download_failure():\n from urllib.error import HTTPError\n\n with pytest.raises(HTTPError):\n sc.datasets.ebi_expression_atlas(\"not_a_real_accession\")\n\n\n# These are tested via doctest\nDS_INCLUDED = frozenset({\"krumsiek11\", \"toggleswitch\", \"pbmc68k_reduced\"})\n# These have parameters that affect shape and so on\nDS_DYNAMIC = frozenset({\"ebi_expression_atlas\"})\n# Additional marks for datasets besides “internet”\nDS_MARKS = defaultdict(list, moignard15=[needs.openpyxl])\n\n\n@pytest.mark.parametrize(\n \"ds_name\",\n [\n pytest.param(\n ds,\n id=ds,\n marks=[\n *(() if ds in DS_INCLUDED else [pytest.mark.internet]),\n *DS_MARKS[ds],\n ],\n )\n for ds in sorted(set(sc.datasets.__all__) - DS_DYNAMIC)\n ],\n)\ndef test_doc_shape(ds_name):\n dataset_fn: Callable[[], AnnData] = getattr(sc.datasets, ds_name)\n assert dataset_fn.__doc__, \"No docstring\"\n docstring = dedent(dataset_fn.__doc__)\n with warnings.catch_warnings():\n warnings.filterwarnings(\n \"ignore\",\n r\"(Observation|Variable) names are not unique\",\n category=UserWarning,\n )\n dataset = dataset_fn()\n assert repr(dataset) in docstring\n\n\nfrom __future__ import annotations\n\nfrom itertools import product\n\nimport numpy as np\nimport pandas as pd\nimport pytest\nfrom anndata import AnnData\nfrom anndata.tests.helpers import asarray, assert_equal\nfrom numpy.testing import assert_allclose\nfrom scipy import sparse as sp\nfrom scipy.sparse import issparse\n\nimport scanpy as sc\nfrom testing.scanpy._helpers import (\n anndata_v0_8_constructor_compat,\n check_rep_mutation,\n check_rep_results,\n)\nfrom testing.scanpy._helpers.data import pbmc3k, pbmc68k_reduced\nfrom testing.scanpy._pytest.params import ARRAY_TYPES\n\n\ndef test_log1p(tmp_path):\n A = np.random.rand(200, 10).astype(np.float32)\n A_l = np.log1p(A)\n ad = AnnData(A.copy())\n ad2 = AnnData(A.copy())\n ad3 = AnnData(A.copy())\n ad3.filename = tmp_path / \"test.h5ad\"\n sc.pp.log1p(ad)\n assert np.allclose(ad.X, A_l)\n sc.pp.log1p(ad2, chunked=True)\n assert np.allclose(ad2.X, ad.X)\n sc.pp.log1p(ad3, chunked=True)\n assert np.allclose(ad3.X, ad.X)\n\n # Test base\n ad4 = AnnData(A)\n sc.pp.log1p(ad4, base=2)\n assert np.allclose(ad4.X, A_l / np.log(2))\n\n\ndef test_log1p_deprecated_arg():\n A = np.random.rand(200, 10).astype(np.float32)\n with pytest.warns(FutureWarning, match=r\".*`X` was renamed to `data`\"):\n sc.pp.log1p(X=A)\n\n\n@pytest.fixture(params=[None, 2])\ndef base(request):\n return request.param\n\n\ndef test_log1p_rep(count_matrix_format, base, dtype):\n X = count_matrix_format(\n np.abs(sp.random(100, 200, density=0.3, dtype=dtype)).toarray()\n )\n check_rep_mutation(sc.pp.log1p, X, base=base)\n check_rep_results(sc.pp.log1p, X, base=base)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\ndef test_mean_var(array_type):\n pbmc = pbmc3k()\n pbmc.X = array_type(pbmc.X)\n\n true_mean = np.mean(asarray(pbmc.X), axis=0)\n true_var = np.var(asarray(pbmc.X), axis=0, dtype=np.float64, ddof=1)\n\n means, variances = sc.pp._utils._get_mean_var(pbmc.X)\n\n np.testing.assert_allclose(true_mean, means)\n np.testing.assert_allclose(true_var, variances)\n\n\ndef test_mean_var_sparse():\n from sklearn.utils.sparsefuncs import mean_variance_axis\n\n csr64 = sp.random(10000, 1000, format=\"csr\", dtype=np.float64)\n csc64 = csr64.tocsc()\n\n # Test that we're equivalent for 64 bit\n for mtx, ax in product((csr64, csc64), (0, 1)):\n scm, scv = sc.pp._utils._get_mean_var(mtx, axis=ax)\n skm, skv = mean_variance_axis(mtx, ax)\n skv *= mtx.shape[ax] / (mtx.shape[ax] - 1)\n\n assert np.allclose(scm, skm)\n assert np.allclose(scv, skv)\n\n csr32 = csr64.astype(np.float32)\n csc32 = csc64.astype(np.float32)\n\n # Test whether ours is more accurate for 32 bit\n for mtx32, mtx64 in [(csc32, csc64), (csr32, csr64)]:\n scm32, scv32 = sc.pp._utils._get_mean_var(mtx32)\n scm64, scv64 = sc.pp._utils._get_mean_var(mtx64)\n skm32, skv32 = mean_variance_axis(mtx32, 0)\n skm64, skv64 = mean_variance_axis(mtx64, 0)\n skv32 *= mtx.shape[0] / (mtx.shape[0] - 1)\n skv64 *= mtx.shape[0] / (mtx.shape[0] - 1)\n\n m_resid_sc = np.mean(np.abs(scm64 - scm32))\n m_resid_sk = np.mean(np.abs(skm64 - skm32))\n v_resid_sc = np.mean(np.abs(scv64 - scv32))\n v_resid_sk = np.mean(np.abs(skv64 - skv32))\n\n assert m_resid_sc < m_resid_sk\n assert v_resid_sc < v_resid_sk\n\n\ndef test_normalize_per_cell():\n A = np.array([[1, 0], [3, 0], [5, 6]], dtype=np.float32)\n adata = AnnData(A.copy())\n sc.pp.normalize_per_cell(adata, counts_per_cell_after=1, key_n_counts=\"n_counts2\")\n assert adata.X.sum(axis=1).tolist() == [1.0, 1.0, 1.0]\n # now with copy option\n adata = AnnData(A.copy())\n # note that sc.pp.normalize_per_cell is also used in\n # pl.highest_expr_genes with parameter counts_per_cell_after=100\n adata_copy = sc.pp.normalize_per_cell(adata, counts_per_cell_after=1, copy=True)\n assert adata_copy.X.sum(axis=1).tolist() == [1.0, 1.0, 1.0]\n # now sparse\n adata = AnnData(A.copy())\n adata_sparse = AnnData(sp.csr_matrix(A.copy()))\n sc.pp.normalize_per_cell(adata)\n sc.pp.normalize_per_cell(adata_sparse)\n assert adata.X.sum(axis=1).tolist() == adata_sparse.X.sum(axis=1).A1.tolist()\n\n\ndef test_subsample():\n adata = AnnData(np.ones((200, 10)))\n sc.pp.subsample(adata, n_obs=40)\n assert adata.n_obs == 40\n sc.pp.subsample(adata, fraction=0.1)\n assert adata.n_obs == 4\n\n\ndef test_subsample_copy():\n adata = AnnData(np.ones((200, 10)))\n assert sc.pp.subsample(adata, n_obs=40, copy=True).shape == (40, 10)\n assert sc.pp.subsample(adata, fraction=0.1, copy=True).shape == (20, 10)\n\n\ndef test_subsample_copy_backed(tmp_path):\n A = np.random.rand(200, 10).astype(np.float32)\n adata_m = AnnData(A.copy())\n adata_d = AnnData(A.copy())\n filename = tmp_path / \"test.h5ad\"\n adata_d.filename = filename\n # This should not throw an error\n assert sc.pp.subsample(adata_d, n_obs=40, copy=True).shape == (40, 10)\n np.testing.assert_array_equal(\n sc.pp.subsample(adata_m, n_obs=40, copy=True).X,\n sc.pp.subsample(adata_d, n_obs=40, copy=True).X,\n )\n with pytest.raises(NotImplementedError):\n sc.pp.subsample(adata_d, n_obs=40, copy=False)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\"zero_center\", [True, False])\n@pytest.mark.parametrize(\"max_value\", [None, 1.0])\ndef test_scale_matrix_types(array_type, zero_center, max_value):\n adata = pbmc68k_reduced()\n adata.X = adata.raw.X\n adata_casted = adata.copy()\n adata_casted.X = array_type(adata_casted.raw.X)\n sc.pp.scale(adata, zero_center=zero_center, max_value=max_value)\n sc.pp.scale(adata_casted, zero_center=zero_center, max_value=max_value)\n X = adata_casted.X\n if \"dask\" in array_type.__name__:\n X = X.compute()\n if issparse(X):\n X = X.todense()\n if issparse(adata.X):\n adata.X = adata.X.todense()\n assert_allclose(X, adata.X, rtol=1e-5, atol=1e-5)\n\n\nARRAY_TYPES_DASK_SPARSE = [\n a for a in ARRAY_TYPES if \"sparse\" in a.id and \"dask\" in a.id\n]\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES_DASK_SPARSE)\ndef test_scale_zero_center_warns_dask_sparse(array_type):\n adata = pbmc68k_reduced()\n adata.X = adata.raw.X\n adata_casted = adata.copy()\n adata_casted.X = array_type(adata_casted.raw.X)\n with pytest.warns(UserWarning, match=\"zero-center being used with `DaskArray`*\"):\n sc.pp.scale(adata_casted)\n sc.pp.scale(adata)\n assert_allclose(adata_casted.X, adata.X, rtol=1e-5, atol=1e-5)\n\n\ndef test_scale():\n adata = pbmc68k_reduced()\n adata.X = adata.raw.X\n v = adata[:, 0 : adata.shape[1] // 2]\n # Should turn view to copy https://github.com/scverse/anndata/issues/171#issuecomment-508689965\n assert v.is_view\n with pytest.warns(Warning, match=\"view\"):\n sc.pp.scale(v)\n assert not v.is_view\n assert_allclose(v.X.var(axis=0), np.ones(v.shape[1]), atol=0.01)\n assert_allclose(v.X.mean(axis=0), np.zeros(v.shape[1]), atol=0.00001)\n\n\n@pytest.fixture(params=[True, False])\ndef zero_center(request):\n return request.param\n\n\ndef test_scale_rep(count_matrix_format, zero_center):\n \"\"\"\n Test that it doesn't matter where the array being scaled is in the anndata object.\n \"\"\"\n X = count_matrix_format(sp.random(100, 200, density=0.3).toarray())\n check_rep_mutation(sc.pp.scale, X, zero_center=zero_center)\n check_rep_results(sc.pp.scale, X, zero_center=zero_center)\n\n\ndef test_scale_array(count_matrix_format, zero_center):\n \"\"\"\n Test that running sc.pp.scale on an anndata object and an array returns the same results.\n \"\"\"\n X = count_matrix_format(sp.random(100, 200, density=0.3).toarray())\n adata = anndata_v0_8_constructor_compat(X=X.copy())\n\n sc.pp.scale(adata, zero_center=zero_center)\n scaled_X = sc.pp.scale(X, zero_center=zero_center, copy=True)\n np.testing.assert_equal(asarray(scaled_X), asarray(adata.X))\n\n\ndef test_recipe_plotting():\n sc.settings.autoshow = False\n adata = AnnData(np.random.randint(0, 1000, (1000, 1000)))\n # These shouldn't throw an error\n sc.pp.recipe_seurat(adata.copy(), plot=True)\n sc.pp.recipe_zheng17(adata.copy(), plot=True)\n\n\ndef test_regress_out_ordinal():\n from scipy.sparse import random\n\n adata = AnnData(random(1000, 100, density=0.6, format=\"csr\"))\n adata.obs[\"percent_mito\"] = np.random.rand(adata.X.shape[0])\n adata.obs[\"n_counts\"] = adata.X.sum(axis=1)\n\n # results using only one processor\n single = sc.pp.regress_out(\n adata, keys=[\"n_counts\", \"percent_mito\"], n_jobs=1, copy=True\n )\n assert adata.X.shape == single.X.shape\n\n # results using 8 processors\n multi = sc.pp.regress_out(\n adata, keys=[\"n_counts\", \"percent_mito\"], n_jobs=8, copy=True\n )\n\n np.testing.assert_array_equal(single.X, multi.X)\n\n\ndef test_regress_out_layer():\n from scipy.sparse import random\n\n adata = AnnData(random(1000, 100, density=0.6, format=\"csr\"))\n adata.obs[\"percent_mito\"] = np.random.rand(adata.X.shape[0])\n adata.obs[\"n_counts\"] = adata.X.sum(axis=1)\n adata.layers[\"counts\"] = adata.X.copy()\n\n single = sc.pp.regress_out(\n adata, keys=[\"n_counts\", \"percent_mito\"], n_jobs=1, copy=True\n )\n assert adata.X.shape == single.X.shape\n\n layer = sc.pp.regress_out(\n adata, layer=\"counts\", keys=[\"n_counts\", \"percent_mito\"], n_jobs=1, copy=True\n )\n\n np.testing.assert_array_equal(single.X, layer.layers[\"counts\"])\n\n\ndef test_regress_out_view():\n from scipy.sparse import random\n\n adata = AnnData(random(500, 1100, density=0.2, format=\"csr\"))\n adata.obs[\"percent_mito\"] = np.random.rand(adata.X.shape[0])\n adata.obs[\"n_counts\"] = adata.X.sum(axis=1)\n subset_adata = adata[:, :1050]\n subset_adata_copy = subset_adata.copy()\n\n sc.pp.regress_out(subset_adata, keys=[\"n_counts\", \"percent_mito\"])\n sc.pp.regress_out(subset_adata_copy, keys=[\"n_counts\", \"percent_mito\"])\n assert_equal(subset_adata, subset_adata_copy)\n assert not subset_adata.is_view\n\n\ndef test_regress_out_categorical():\n import pandas as pd\n from scipy.sparse import random\n\n adata = AnnData(random(1000, 100, density=0.6, format=\"csr\"))\n # create a categorical column\n adata.obs[\"batch\"] = pd.Categorical(np.random.randint(1, 4, size=adata.X.shape[0]))\n\n multi = sc.pp.regress_out(adata, keys=\"batch\", n_jobs=8, copy=True)\n assert adata.X.shape == multi.X.shape\n\n\ndef test_regress_out_constants():\n adata = AnnData(np.hstack((np.full((10, 1), 0.0), np.full((10, 1), 1.0))))\n adata.obs[\"percent_mito\"] = np.random.rand(adata.X.shape[0])\n adata.obs[\"n_counts\"] = adata.X.sum(axis=1)\n adata_copy = adata.copy()\n\n sc.pp.regress_out(adata, keys=[\"n_counts\", \"percent_mito\"])\n assert_equal(adata, adata_copy)\n\n\ndef test_regress_out_constants_equivalent():\n # Tests that constant values don't change results\n # (since support for constant values is implemented by us)\n from sklearn.datasets import make_blobs\n\n X, cat = make_blobs(100, 20)\n a = sc.AnnData(np.hstack([X, np.zeros((100, 5))]), obs={\"cat\": pd.Categorical(cat)})\n b = sc.AnnData(X, obs={\"cat\": pd.Categorical(cat)})\n\n sc.pp.regress_out(a, \"cat\")\n sc.pp.regress_out(b, \"cat\")\n\n np.testing.assert_equal(a[:, b.var_names].X, b.X)\n\n\n@pytest.fixture(params=[lambda x: x.copy(), sp.csr_matrix, sp.csc_matrix])\ndef count_matrix_format(request):\n return request.param\n\n\n@pytest.fixture(params=[True, False])\ndef replace(request):\n return request.param\n\n\n@pytest.fixture(params=[np.int64, np.float32, np.float64])\ndef dtype(request):\n return request.param\n\n\ndef test_downsample_counts_per_cell(count_matrix_format, replace, dtype):\n TARGET = 1000\n X = np.random.randint(0, 100, (1000, 100)) * np.random.binomial(1, 0.3, (1000, 100))\n X = X.astype(dtype)\n adata = anndata_v0_8_constructor_compat(X=count_matrix_format(X).astype(dtype))\n with pytest.raises(ValueError, match=r\"Must specify exactly one\"):\n sc.pp.downsample_counts(\n adata, counts_per_cell=TARGET, total_counts=TARGET, replace=replace\n )\n with pytest.raises(ValueError, match=r\"Must specify exactly one\"):\n sc.pp.downsample_counts(adata, replace=replace)\n initial_totals = np.ravel(adata.X.sum(axis=1))\n adata = sc.pp.downsample_counts(\n adata, counts_per_cell=TARGET, replace=replace, copy=True\n )\n new_totals = np.ravel(adata.X.sum(axis=1))\n if sp.issparse(adata.X):\n assert all(adata.X.toarray()[X == 0] == 0)\n else:\n assert all(adata.X[X == 0] == 0)\n assert all(new_totals <= TARGET)\n assert all(initial_totals >= new_totals)\n assert all(\n initial_totals[initial_totals <= TARGET] == new_totals[initial_totals <= TARGET]\n )\n if not replace:\n assert np.all(X >= adata.X)\n assert X.dtype == adata.X.dtype\n\n\ndef test_downsample_counts_per_cell_multiple_targets(\n count_matrix_format, replace, dtype\n):\n TARGETS = np.random.randint(500, 1500, 1000)\n X = np.random.randint(0, 100, (1000, 100)) * np.random.binomial(1, 0.3, (1000, 100))\n X = X.astype(dtype)\n adata = anndata_v0_8_constructor_compat(X=count_matrix_format(X).astype(dtype))\n initial_totals = np.ravel(adata.X.sum(axis=1))\n with pytest.raises(ValueError, match=r\"counts_per_cell.*length as number of obs\"):\n sc.pp.downsample_counts(adata, counts_per_cell=[40, 10], replace=replace)\n adata = sc.pp.downsample_counts(\n adata, counts_per_cell=TARGETS, replace=replace, copy=True\n )\n new_totals = np.ravel(adata.X.sum(axis=1))\n if sp.issparse(adata.X):\n assert all(adata.X.toarray()[X == 0] == 0)\n else:\n assert all(adata.X[X == 0] == 0)\n assert all(new_totals <= TARGETS)\n assert all(initial_totals >= new_totals)\n assert all(\n initial_totals[initial_totals <= TARGETS]\n == new_totals[initial_totals <= TARGETS]\n )\n if not replace:\n assert np.all(X >= adata.X)\n assert X.dtype == adata.X.dtype\n\n\ndef test_downsample_total_counts(count_matrix_format, replace, dtype):\n X = np.random.randint(0, 100, (1000, 100)) * np.random.binomial(1, 0.3, (1000, 100))\n X = X.astype(dtype)\n adata_orig = anndata_v0_8_constructor_compat(X=count_matrix_format(X))\n total = X.sum()\n target = np.floor_divide(total, 10)\n initial_totals = np.ravel(adata_orig.X.sum(axis=1))\n adata = sc.pp.downsample_counts(\n adata_orig, total_counts=target, replace=replace, copy=True\n )\n new_totals = np.ravel(adata.X.sum(axis=1))\n if sp.issparse(adata.X):\n assert all(adata.X.toarray()[X == 0] == 0)\n else:\n assert all(adata.X[X == 0] == 0)\n assert adata.X.sum() == target\n assert all(initial_totals >= new_totals)\n if not replace:\n assert np.all(X >= adata.X)\n adata = sc.pp.downsample_counts(\n adata_orig, total_counts=total + 10, replace=False, copy=True\n )\n assert (adata.X == X).all()\n assert X.dtype == adata.X.dtype\n\n\ndef test_recipe_weinreb():\n # Just tests for failure for now\n adata = pbmc68k_reduced().raw.to_adata()\n adata.X = adata.X.toarray()\n\n orig = adata.copy()\n sc.pp.recipe_weinreb17(adata, log=False, copy=True)\n assert_equal(orig, adata)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\n (\"max_cells\", \"max_counts\", \"min_cells\", \"min_counts\"),\n [\n (100, None, None, None),\n (None, 100, None, None),\n (None, None, 20, None),\n (None, None, None, 20),\n ],\n)\ndef test_filter_genes(array_type, max_cells, max_counts, min_cells, min_counts):\n adata = pbmc68k_reduced()\n adata.X = adata.raw.X\n adata_casted = adata.copy()\n adata_casted.X = array_type(adata_casted.raw.X)\n sc.pp.filter_genes(\n adata,\n max_cells=max_cells,\n max_counts=max_counts,\n min_cells=min_cells,\n min_counts=min_counts,\n )\n sc.pp.filter_genes(\n adata_casted,\n max_cells=max_cells,\n max_counts=max_counts,\n min_cells=min_cells,\n min_counts=min_counts,\n )\n X = adata_casted.X\n if \"dask\" in array_type.__name__:\n X = X.compute()\n if issparse(X):\n X = X.todense()\n if issparse(adata.X):\n adata.X = adata.X.todense()\n assert_allclose(X, adata.X, rtol=1e-5, atol=1e-5)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\n (\"max_genes\", \"max_counts\", \"min_genes\", \"min_counts\"),\n [\n (100, None, None, None),\n (None, 100, None, None),\n (None, None, 20, None),\n (None, None, None, 20),\n ],\n)\ndef test_filter_cells(array_type, max_genes, max_counts, min_genes, min_counts):\n adata = pbmc68k_reduced()\n adata.X = adata.raw.X\n adata_casted = adata.copy()\n adata_casted.X = array_type(adata_casted.raw.X)\n sc.pp.filter_cells(\n adata,\n max_genes=max_genes,\n max_counts=max_counts,\n min_genes=min_genes,\n min_counts=min_counts,\n )\n sc.pp.filter_cells(\n adata_casted,\n max_genes=max_genes,\n max_counts=max_counts,\n min_genes=min_genes,\n min_counts=min_counts,\n )\n X = adata_casted.X\n if \"dask\" in array_type.__name__:\n X = X.compute()\n if issparse(X):\n X = X.todense()\n if issparse(adata.X):\n adata.X = adata.X.todense()\n assert_allclose(X, adata.X, rtol=1e-5, atol=1e-5)\n\n\nfrom __future__ import annotations\n\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nimport numpy.testing as npt\nimport pytest\nfrom anndata import read_zarr\n\nfrom scanpy._compat import DaskArray, ZappyArray\nfrom scanpy.datasets._utils import filter_oldformatwarning\nfrom scanpy.preprocessing import (\n filter_cells,\n filter_genes,\n log1p,\n normalize_per_cell,\n normalize_total,\n)\nfrom scanpy.preprocessing._distributed import materialize_as_ndarray\nfrom testing.scanpy._pytest.marks import needs\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\nHERE = Path(__file__).parent / Path(\"_data/\")\ninput_file = Path(HERE, \"10x-10k-subset.zarr\")\n\nDIST_TYPES = (DaskArray, ZappyArray)\n\n\npytestmark = [needs.zarr]\n\n\n@pytest.fixture\n@filter_oldformatwarning\ndef adata() -> AnnData:\n a = read_zarr(input_file)\n a.var_names_make_unique()\n a.X = a.X[:] # convert to numpy array\n return a\n\n\n@filter_oldformatwarning\n@pytest.fixture(\n params=[\n pytest.param(\"direct\", marks=[needs.zappy]),\n pytest.param(\"dask\", marks=[needs.dask, pytest.mark.anndata_dask_support]),\n ]\n)\ndef adata_dist(request: pytest.FixtureRequest) -> AnnData:\n # regular anndata except for X, which we replace on the next line\n a = read_zarr(input_file)\n a.var_names_make_unique()\n a.uns[\"dist-mode\"] = request.param\n input_file_X = f\"{input_file}/X\"\n if request.param == \"direct\":\n import zappy.direct\n\n a.X = zappy.direct.from_zarr(input_file_X)\n return a\n\n assert request.param == \"dask\"\n import dask.array as da\n\n a.X = da.from_zarr(input_file_X)\n return a\n\n\ndef test_log1p(adata: AnnData, adata_dist: AnnData):\n log1p(adata_dist)\n assert isinstance(adata_dist.X, DIST_TYPES)\n result = materialize_as_ndarray(adata_dist.X)\n log1p(adata)\n assert result.shape == adata.shape\n npt.assert_allclose(result, adata.X)\n\n\ndef test_normalize_per_cell(\n request: pytest.FixtureRequest, adata: AnnData, adata_dist: AnnData\n):\n if isinstance(adata_dist.X, DaskArray):\n reason = \"normalize_per_cell deprecated and broken for Dask\"\n request.applymarker(pytest.mark.xfail(reason=reason))\n normalize_per_cell(adata_dist)\n assert isinstance(adata_dist.X, DIST_TYPES)\n result = materialize_as_ndarray(adata_dist.X)\n normalize_per_cell(adata)\n assert result.shape == adata.shape\n npt.assert_allclose(result, adata.X)\n\n\ndef test_normalize_total(adata: AnnData, adata_dist: AnnData):\n normalize_total(adata_dist)\n assert isinstance(adata_dist.X, DIST_TYPES)\n result = materialize_as_ndarray(adata_dist.X)\n normalize_total(adata)\n assert result.shape == adata.shape\n npt.assert_allclose(result, adata.X)\n\n\ndef test_filter_cells_array(adata: AnnData, adata_dist: AnnData):\n cell_subset_dist, number_per_cell_dist = filter_cells(adata_dist.X, min_genes=3)\n assert isinstance(cell_subset_dist, DIST_TYPES)\n assert isinstance(number_per_cell_dist, DIST_TYPES)\n\n cell_subset, number_per_cell = filter_cells(adata.X, min_genes=3)\n npt.assert_allclose(materialize_as_ndarray(cell_subset_dist), cell_subset)\n npt.assert_allclose(materialize_as_ndarray(number_per_cell_dist), number_per_cell)\n\n\ndef test_filter_cells(adata: AnnData, adata_dist: AnnData):\n filter_cells(adata_dist, min_genes=3)\n assert isinstance(adata_dist.X, DIST_TYPES)\n result = materialize_as_ndarray(adata_dist.X)\n filter_cells(adata, min_genes=3)\n\n assert result.shape == adata.shape\n npt.assert_array_equal(adata_dist.obs[\"n_genes\"], adata.obs[\"n_genes\"])\n npt.assert_allclose(result, adata.X)\n\n\ndef test_filter_genes_array(adata: AnnData, adata_dist: AnnData):\n gene_subset_dist, number_per_gene_dist = filter_genes(adata_dist.X, min_cells=2)\n assert isinstance(gene_subset_dist, DIST_TYPES)\n assert isinstance(number_per_gene_dist, DIST_TYPES)\n\n gene_subset, number_per_gene = filter_genes(adata.X, min_cells=2)\n npt.assert_allclose(materialize_as_ndarray(gene_subset_dist), gene_subset)\n npt.assert_allclose(materialize_as_ndarray(number_per_gene_dist), number_per_gene)\n\n\ndef test_filter_genes(adata: AnnData, adata_dist: AnnData):\n filter_genes(adata_dist, min_cells=2)\n assert isinstance(adata_dist.X, DIST_TYPES)\n result = materialize_as_ndarray(adata_dist.X)\n filter_genes(adata, min_cells=2)\n assert result.shape == adata.shape\n npt.assert_allclose(result, adata.X)\n\n\n@filter_oldformatwarning\ndef test_write_zarr(adata: AnnData, adata_dist: AnnData):\n import zarr\n\n log1p(adata_dist)\n assert isinstance(adata_dist.X, DIST_TYPES)\n temp_store = zarr.TempStore()\n chunks = adata_dist.X.chunks\n if isinstance(chunks[0], tuple):\n chunks = (chunks[0][0],) + chunks[1]\n\n # write metadata using regular anndata\n adata.write_zarr(temp_store, chunks)\n if adata_dist.uns[\"dist-mode\"] == \"dask\":\n adata_dist.X.to_zarr(temp_store.dir_path(\"X\"), overwrite=True)\n elif adata_dist.uns[\"dist-mode\"] == \"direct\":\n adata_dist.X.to_zarr(temp_store.dir_path(\"X\"), chunks)\n else:\n pytest.fail(\"add branch for new dist-mode\")\n\n # read back as zarr directly and check it is the same as adata.X\n adata_log1p = read_zarr(temp_store)\n\n log1p(adata)\n npt.assert_allclose(adata_log1p.X, adata.X)\n\n\nfrom __future__ import annotations\n\nimport pickle\nfrom contextlib import nullcontext\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pytest\nfrom anndata import AnnData\nfrom scipy.sparse import csr_matrix\n\nimport scanpy as sc\nfrom testing.scanpy._helpers.data import paul15\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from numpy.typing import NDArray\n\n\nHERE = Path(__file__).parent\nDATA_PATH = HERE / \"_data\"\n\n\ndef _create_random_gene_names(n_genes, name_length) -> NDArray[np.str_]:\n \"\"\"\n creates a bunch of random gene names (just CAPS letters)\n \"\"\"\n return np.array(\n [\n \"\".join(map(chr, np.random.randint(65, 90, name_length)))\n for _ in range(n_genes)\n ]\n )\n\n\ndef _create_sparse_nan_matrix(rows, cols, percent_zero, percent_nan):\n \"\"\"\n creates a sparse matrix, with certain amounts of NaN and Zeros\n \"\"\"\n A = np.random.randint(0, 1000, rows * cols).reshape((rows, cols)).astype(\"float32\")\n maskzero = np.random.rand(rows, cols) < percent_zero\n masknan = np.random.rand(rows, cols) < percent_nan\n if np.any(maskzero):\n A[maskzero] = 0\n if np.any(masknan):\n A[masknan] = np.nan\n S = csr_matrix(A)\n return S\n\n\ndef _create_adata(n_obs, n_var, p_zero, p_nan):\n \"\"\"\n creates an AnnData with random data, sparseness and some NaN values\n \"\"\"\n X = _create_sparse_nan_matrix(n_obs, n_var, p_zero, p_nan)\n adata = AnnData(X)\n gene_names = _create_random_gene_names(n_var, name_length=6)\n adata.var_names = gene_names\n return adata\n\n\ndef test_score_with_reference():\n \"\"\"\n Checks if score_genes output agrees with pre-computed reference values.\n The reference values had been generated using the same code\n and stored as a pickle object in ./data\n \"\"\"\n\n adata = paul15()\n sc.pp.normalize_per_cell(adata, counts_per_cell_after=10000)\n sc.pp.scale(adata)\n\n sc.tl.score_genes(adata, gene_list=adata.var_names[:100], score_name=\"Test\")\n with (DATA_PATH / \"score_genes_reference_paul2015.pkl\").open(\"rb\") as file:\n reference = pickle.load(file)\n # np.testing.assert_allclose(reference, adata.obs[\"Test\"].to_numpy())\n np.testing.assert_array_equal(reference, adata.obs[\"Test\"].to_numpy())\n\n\ndef test_add_score():\n \"\"\"\n check the dtype of the scores\n check that non-existing genes get ignored\n \"\"\"\n # TODO: write a test that costs less resources and is more meaningful\n adata = _create_adata(100, 1000, p_zero=0, p_nan=0)\n\n sc.pp.normalize_per_cell(adata, counts_per_cell_after=1e4)\n sc.pp.log1p(adata)\n\n # the actual genes names are all 6letters\n # create some non-estinsting names with 7 letters:\n non_existing_genes = _create_random_gene_names(n_genes=3, name_length=7)\n some_genes = np.r_[\n np.unique(np.random.choice(adata.var_names, 10)), np.unique(non_existing_genes)\n ]\n sc.tl.score_genes(adata, some_genes, score_name=\"Test\")\n assert adata.obs[\"Test\"].dtype == \"float64\"\n\n\ndef test_sparse_nanmean():\n \"\"\"\n check that _sparse_nanmean() is equivalent to np.nanmean()\n \"\"\"\n from scanpy.tools._score_genes import _sparse_nanmean\n\n R, C = 60, 50\n\n # sparse matrix, no NaN\n S = _create_sparse_nan_matrix(R, C, percent_zero=0.3, percent_nan=0)\n # col/col sum\n np.testing.assert_allclose(\n S.toarray().mean(0), np.array(_sparse_nanmean(S, 0)).flatten()\n )\n np.testing.assert_allclose(\n S.toarray().mean(1), np.array(_sparse_nanmean(S, 1)).flatten()\n )\n\n # sparse matrix with nan\n S = _create_sparse_nan_matrix(R, C, percent_zero=0.3, percent_nan=0.3)\n np.testing.assert_allclose(\n np.nanmean(S.toarray(), 1), np.array(_sparse_nanmean(S, 1)).flatten()\n )\n np.testing.assert_allclose(\n np.nanmean(S.toarray(), 0), np.array(_sparse_nanmean(S, 0)).flatten()\n )\n\n # edge case of only NaNs per row\n A = np.full((10, 1), np.nan)\n\n meanA = np.array(_sparse_nanmean(csr_matrix(A), 0)).flatten()\n np.testing.assert_allclose(np.nanmean(A, 0), meanA)\n\n\ndef test_sparse_nanmean_on_dense_matrix():\n \"\"\"\n TypeError must be thrown when calling _sparse_nanmean with a dense matrix\n \"\"\"\n from scanpy.tools._score_genes import _sparse_nanmean\n\n with pytest.raises(TypeError):\n _sparse_nanmean(np.random.rand(4, 5), 0)\n\n\ndef test_score_genes_sparse_vs_dense():\n \"\"\"\n score_genes() should give the same result for dense and sparse matrices\n \"\"\"\n adata_sparse = _create_adata(100, 1000, p_zero=0.3, p_nan=0.3)\n\n adata_dense = adata_sparse.copy()\n adata_dense.X = adata_dense.X.toarray()\n\n gene_set = adata_dense.var_names[:10]\n\n sc.tl.score_genes(adata_sparse, gene_list=gene_set, score_name=\"Test\")\n sc.tl.score_genes(adata_dense, gene_list=gene_set, score_name=\"Test\")\n\n np.testing.assert_allclose(\n adata_sparse.obs[\"Test\"].values, adata_dense.obs[\"Test\"].values\n )\n\n\ndef test_score_genes_deplete():\n \"\"\"\n deplete some cells from a set of genes.\n their score should be <0 since the sum of markers is 0 and\n the sum of random genes is >=0\n\n check that for both sparse and dense matrices\n \"\"\"\n adata_sparse = _create_adata(100, 1000, p_zero=0.3, p_nan=0.3)\n\n adata_dense = adata_sparse.copy()\n adata_dense.X = adata_dense.X.toarray()\n\n # here's an arbitary gene set\n gene_set = adata_dense.var_names[:10]\n\n for adata in [adata_sparse, adata_dense]:\n # deplete these genes in 50 cells,\n ix_obs = np.random.choice(adata.shape[0], 50)\n adata[ix_obs][:, gene_set].X = 0\n\n sc.tl.score_genes(adata, gene_list=gene_set, score_name=\"Test\")\n scores = adata.obs[\"Test\"].values\n\n np.testing.assert_array_less(scores[ix_obs], 0)\n\n\ndef test_npnanmean_vs_sparsemean(monkeypatch):\n \"\"\"\n another check that _sparsemean behaves like np.nanmean!\n\n monkeypatch the _score_genes._sparse_nanmean function to np.nanmean\n and check that the result is the same as the non-patched (i.e. sparse_nanmean)\n function\n \"\"\"\n\n adata = _create_adata(100, 1000, p_zero=0.3, p_nan=0.3)\n gene_set = adata.var_names[:10]\n\n # the unpatched, i.e. _sparse_nanmean version\n sc.tl.score_genes(adata, gene_list=gene_set, score_name=\"Test\")\n sparse_scores = adata.obs[\"Test\"].values.tolist()\n\n # now patch _sparse_nanmean by np.nanmean inside sc.tools\n def mock_fn(x: csr_matrix, axis: Literal[0, 1]):\n return np.nanmean(x.toarray(), axis, dtype=\"float64\")\n\n monkeypatch.setattr(sc.tl._score_genes, \"_sparse_nanmean\", mock_fn)\n sc.tl.score_genes(adata, gene_list=gene_set, score_name=\"Test\")\n dense_scores = adata.obs[\"Test\"].values\n\n np.testing.assert_allclose(sparse_scores, dense_scores)\n\n\ndef test_missing_genes():\n adata = _create_adata(100, 1000, p_zero=0, p_nan=0)\n # These genes have a different length of name\n non_extant_genes = _create_random_gene_names(n_genes=3, name_length=7)\n\n with pytest.raises(ValueError, match=r\"No valid genes were passed for scoring\"):\n sc.tl.score_genes(adata, non_extant_genes)\n\n\ndef test_one_gene():\n # https://github.com/scverse/scanpy/issues/1395\n adata = _create_adata(100, 1000, p_zero=0, p_nan=0)\n sc.tl.score_genes(adata, [adata.var_names[0]])\n\n\ndef test_use_raw_None():\n adata = _create_adata(100, 1000, p_zero=0, p_nan=0)\n adata_raw = adata.copy()\n adata_raw.var_names = [str(i) for i in range(adata_raw.n_vars)]\n adata.raw = adata_raw\n\n sc.tl.score_genes(adata, adata_raw.var_names[:3], use_raw=None)\n\n\ndef test_layer():\n adata = _create_adata(100, 1000, p_zero=0, p_nan=0)\n\n sc.pp.normalize_per_cell(adata, counts_per_cell_after=1e4)\n sc.pp.log1p(adata)\n\n # score X\n gene_set = adata.var_names[:10]\n sc.tl.score_genes(adata, gene_set, score_name=\"X_score\")\n # score layer (`del` makes sure it actually uses the layer)\n adata.layers[\"test\"] = adata.X.copy()\n adata.raw = adata\n del adata.X\n sc.tl.score_genes(adata, gene_set, score_name=\"test_score\", layer=\"test\")\n\n np.testing.assert_array_equal(adata.obs[\"X_score\"], adata.obs[\"test_score\"])\n\n\n@pytest.mark.parametrize(\"gene_pool\", [[], [\"foo\", \"bar\"]])\ndef test_invalid_gene_pool(gene_pool):\n adata = _create_adata(100, 1000, p_zero=0, p_nan=0)\n\n with pytest.raises(ValueError, match=\"reference set\"):\n sc.tl.score_genes(adata, adata.var_names[:3], gene_pool=gene_pool)\n\n\ndef test_no_control_gene():\n np.random.seed(0)\n adata = _create_adata(100, 1, p_zero=0, p_nan=0)\n\n with pytest.raises(RuntimeError, match=\"No control genes found\"):\n sc.tl.score_genes(adata, adata.var_names[:1], ctrl_size=1)\n\n\n@pytest.mark.parametrize(\n \"ctrl_as_ref\", [True, False], ids=[\"ctrl_as_ref\", \"no_ctrl_as_ref\"]\n)\ndef test_gene_list_is_control(*, ctrl_as_ref: bool):\n np.random.seed(0)\n adata = sc.datasets.blobs(n_variables=10, n_observations=100, n_centers=20)\n adata.var_names = \"g\" + adata.var_names\n with (\n pytest.raises(RuntimeError, match=r\"No control genes found in any cut\")\n if ctrl_as_ref\n else nullcontext()\n ):\n sc.tl.score_genes(\n adata, gene_list=\"g3\", ctrl_size=1, n_bins=5, ctrl_as_ref=ctrl_as_ref\n )\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\nimport pytest\n\nimport scanpy as sc\n\n\n@pytest.mark.parametrize(\"method\", [\"t-test\", \"logreg\"])\ndef test_rank_genes_groups_with_renamed_categories(method):\n adata = sc.datasets.blobs(n_variables=4, n_centers=3, n_observations=200)\n assert np.allclose(adata.X[1], [9.214668, -2.6487126, 4.2020774, 0.51076424])\n\n # for method in ['logreg', 't-test']:\n\n sc.tl.rank_genes_groups(adata, \"blobs\", method=method)\n assert adata.uns[\"rank_genes_groups\"][\"names\"].dtype.names == (\"0\", \"1\", \"2\")\n assert adata.uns[\"rank_genes_groups\"][\"names\"][0].tolist() == (\"1\", \"3\", \"0\")\n\n adata.rename_categories(\"blobs\", [\"Zero\", \"One\", \"Two\"])\n assert adata.uns[\"rank_genes_groups\"][\"names\"][0].tolist() == (\"1\", \"3\", \"0\")\n\n sc.tl.rank_genes_groups(adata, \"blobs\", method=method)\n assert adata.uns[\"rank_genes_groups\"][\"names\"][0].tolist() == (\"1\", \"3\", \"0\")\n assert adata.uns[\"rank_genes_groups\"][\"names\"].dtype.names == (\"Zero\", \"One\", \"Two\")\n\n\ndef test_rank_genes_groups_with_renamed_categories_use_rep():\n adata = sc.datasets.blobs(n_variables=4, n_centers=3, n_observations=200)\n assert np.allclose(adata.X[1], [9.214668, -2.6487126, 4.2020774, 0.51076424])\n\n adata.layers[\"to_test\"] = adata.X.copy()\n adata.X = adata.X[::-1, :]\n\n sc.tl.rank_genes_groups(\n adata, \"blobs\", method=\"logreg\", layer=\"to_test\", use_raw=False\n )\n assert adata.uns[\"rank_genes_groups\"][\"names\"].dtype.names == (\"0\", \"1\", \"2\")\n assert adata.uns[\"rank_genes_groups\"][\"names\"][0].tolist() == (\"1\", \"3\", \"0\")\n\n sc.tl.rank_genes_groups(adata, \"blobs\", method=\"logreg\")\n assert adata.uns[\"rank_genes_groups\"][\"names\"][0].tolist() != (\"3\", \"1\", \"0\")\n\n\ndef test_rank_genes_groups_with_unsorted_groups():\n adata = sc.datasets.blobs(n_variables=10, n_centers=5, n_observations=200)\n adata._sanitize()\n adata.rename_categories(\"blobs\", [\"Zero\", \"One\", \"Two\", \"Three\", \"Four\"])\n bdata = adata.copy()\n sc.tl.rank_genes_groups(\n adata, \"blobs\", groups=[\"Zero\", \"One\", \"Three\"], method=\"logreg\"\n )\n sc.tl.rank_genes_groups(\n bdata, \"blobs\", groups=[\"One\", \"Three\", \"Zero\"], method=\"logreg\"\n )\n array_ad = pd.DataFrame(\n adata.uns[\"rank_genes_groups\"][\"scores\"][\"Three\"]\n ).to_numpy()\n array_bd = pd.DataFrame(\n bdata.uns[\"rank_genes_groups\"][\"scores\"][\"Three\"]\n ).to_numpy()\n np.testing.assert_equal(array_ad, array_bd)\n\n\nfrom __future__ import annotations\n\nfrom functools import partial\nfrom itertools import chain, combinations, repeat\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nimport matplotlib as mpl\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nimport pytest\nimport seaborn as sns\nfrom anndata import AnnData\nfrom matplotlib.testing.compare import compare_images\nfrom packaging.version import Version\n\nimport scanpy as sc\nfrom scanpy._compat import pkg_version\nfrom testing.scanpy._helpers.data import (\n krumsiek11,\n pbmc3k,\n pbmc3k_processed,\n pbmc68k_reduced,\n)\nfrom testing.scanpy._pytest.marks import needs\n\nif TYPE_CHECKING:\n from collections.abc import Callable\n\nHERE: Path = Path(__file__).parent\nROOT = HERE / \"_images\"\n\n\n# Test images are saved in the directory ./_images//\n# If test images need to be updated, simply copy actual.png to expected.png.\n\n\n@needs.leidenalg\ndef test_heatmap(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n adata = krumsiek11()\n sc.pl.heatmap(\n adata, adata.var_names, \"cell_type\", use_raw=False, show=False, dendrogram=True\n )\n save_and_compare_images(\"heatmap\")\n\n # test swap axes\n sc.pl.heatmap(\n adata,\n adata.var_names,\n \"cell_type\",\n use_raw=False,\n show=False,\n dendrogram=True,\n swap_axes=True,\n figsize=(10, 3),\n cmap=\"YlGnBu\",\n )\n save_and_compare_images(\"heatmap_swap_axes\")\n\n # test heatmap numeric column():\n\n # set as numeric column the vales for the first gene on the matrix\n adata.obs[\"numeric_value\"] = adata.X[:, 0]\n sc.pl.heatmap(\n adata,\n adata.var_names,\n \"numeric_value\",\n use_raw=False,\n num_categories=4,\n figsize=(4.5, 5),\n show=False,\n )\n save_and_compare_images(\"heatmap2\")\n\n # test var/obs standardization and layer\n adata.layers[\"test\"] = -1 * adata.X.copy()\n sc.pl.heatmap(\n adata,\n adata.var_names,\n \"cell_type\",\n use_raw=False,\n dendrogram=True,\n show=False,\n standard_scale=\"var\",\n layer=\"test\",\n )\n save_and_compare_images(\"heatmap_std_scale_var\")\n\n # test standard_scale_obs\n sc.pl.heatmap(\n adata,\n adata.var_names,\n \"cell_type\",\n use_raw=False,\n dendrogram=True,\n show=False,\n standard_scale=\"obs\",\n )\n save_and_compare_images(\"heatmap_std_scale_obs\")\n\n # test var_names as dict\n pbmc = pbmc68k_reduced()\n sc.tl.leiden(\n pbmc,\n key_added=\"clusters\",\n resolution=0.5,\n flavor=\"igraph\",\n n_iterations=2,\n directed=False,\n )\n # call umap to trigger colors for the clusters\n sc.pl.umap(pbmc, color=\"clusters\")\n marker_genes_dict = {\n \"3\": [\"GNLY\", \"NKG7\"],\n \"1\": [\"FCER1A\"],\n \"2\": [\"CD3D\"],\n \"0\": [\"FCGR3A\"],\n \"4\": [\"CD79A\", \"MS4A1\"],\n }\n sc.pl.heatmap(\n adata=pbmc,\n var_names=marker_genes_dict,\n groupby=\"clusters\",\n vmin=-2,\n vmax=2,\n cmap=\"RdBu_r\",\n dendrogram=True,\n swap_axes=True,\n )\n save_and_compare_images(\"heatmap_var_as_dict\")\n\n # test that plot elements are well aligned\n # small\n a = AnnData(\n np.array([[0, 0.3, 0.5], [1, 1.3, 1.5], [2, 2.3, 2.5]]),\n obs={\"foo\": \"a b c\".split()},\n var=pd.DataFrame({\"genes\": \"g1 g2 g3\".split()}).set_index(\"genes\"),\n )\n a.obs[\"foo\"] = a.obs[\"foo\"].astype(\"category\")\n sc.pl.heatmap(\n a, var_names=a.var_names, groupby=\"foo\", swap_axes=True, figsize=(4, 4)\n )\n save_and_compare_images(\"heatmap_small_swap_alignment\")\n\n sc.pl.heatmap(\n a, var_names=a.var_names, groupby=\"foo\", swap_axes=False, figsize=(4, 4)\n )\n save_and_compare_images(\"heatmap_small_alignment\")\n\n\n@pytest.mark.skipif(\n pkg_version(\"matplotlib\") < Version(\"3.1\"),\n reason=\"https://github.com/mwaskom/seaborn/issues/1953\",\n)\n@pytest.mark.parametrize(\n (\"obs_keys\", \"name\"),\n [(None, \"clustermap\"), (\"cell_type\", \"clustermap_withcolor\")],\n)\ndef test_clustermap(image_comparer, obs_keys, name):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n adata = krumsiek11()\n sc.pl.clustermap(adata, obs_keys)\n save_and_compare_images(name)\n\n\nparams_dotplot_matrixplot_stacked_violin = [\n pytest.param(id, fn, id=id)\n for id, fn in [\n (\n \"dotplot\",\n partial(\n sc.pl.dotplot, groupby=\"cell_type\", title=\"dotplot\", dendrogram=True\n ),\n ),\n (\n \"dotplot2\",\n partial(\n sc.pl.dotplot,\n groupby=\"numeric_column\",\n use_raw=False,\n num_categories=7,\n title=\"non categorical obs\",\n figsize=(7, 2.5),\n ),\n ),\n (\n \"dotplot3\",\n partial(\n sc.pl.dotplot,\n groupby=\"cell_type\",\n dot_max=0.7,\n dot_min=0.1,\n cmap=\"hot_r\",\n title=\"dot_max=0.7 dot_min=0.1, var_groups\",\n var_group_positions=[(0, 1), (9, 10)],\n var_group_labels=[\"A\", \"B\"],\n dendrogram=True,\n ),\n ),\n (\n \"dotplot_std_scale_group\",\n partial(\n sc.pl.dotplot,\n groupby=\"cell_type\",\n use_raw=False,\n dendrogram=True,\n layer=\"test\",\n swap_axes=True,\n title=\"swap_axes, layer=-1*X, scale=group\\nsmallest_dot=10\",\n standard_scale=\"group\",\n smallest_dot=10,\n ),\n ),\n (\n \"dotplot_dict\",\n partial(\n sc.pl.dotplot,\n groupby=\"cell_type\",\n dot_max=0.7,\n dot_min=0.1,\n color_map=\"winter\",\n title=\"var as dict\",\n dendrogram=True,\n ),\n ),\n (\n \"matrixplot\",\n partial(\n sc.pl.matrixplot,\n groupby=\"cell_type\",\n use_raw=False,\n title=\"matrixplot\",\n dendrogram=True,\n ),\n ),\n (\n \"matrixplot_std_scale_var_dict\",\n partial(\n sc.pl.matrixplot,\n groupby=\"cell_type\",\n dendrogram=True,\n standard_scale=\"var\",\n layer=\"test\",\n cmap=\"Blues_r\",\n title='scale var, custom colorbar_title, layer=\"test\"',\n colorbar_title=\"Scaled expression\",\n ),\n ),\n (\n \"matrixplot_std_scale_group\",\n partial(\n sc.pl.matrixplot,\n groupby=\"cell_type\",\n use_raw=False,\n standard_scale=\"group\",\n title=\"scale_group, swap_axes\",\n swap_axes=True,\n ),\n ),\n (\n \"matrixplot2\",\n partial(\n sc.pl.matrixplot,\n groupby=\"numeric_column\",\n use_raw=False,\n num_categories=4,\n title=\"non-categorical obs, custom figsize\",\n figsize=(8, 2.5),\n cmap=\"RdBu_r\",\n ),\n ),\n (\n \"stacked_violin\",\n partial(\n sc.pl.stacked_violin,\n groupby=\"cell_type\",\n use_raw=False,\n title=\"stacked_violin\",\n dendrogram=True,\n ),\n ),\n (\n \"stacked_violin_std_scale_var_dict\",\n partial(\n sc.pl.stacked_violin,\n groupby=\"cell_type\",\n dendrogram=True,\n standard_scale=\"var\",\n layer=\"test\",\n title='scale var, layer=\"test\"',\n ),\n ),\n (\n \"stacked_violin_std_scale_group\",\n partial(\n sc.pl.stacked_violin,\n groupby=\"cell_type\",\n use_raw=False,\n standard_scale=\"group\",\n title=\"scale_group\\nswap_axes\",\n swap_axes=True,\n cmap=\"Blues\",\n ),\n ),\n (\n \"stacked_violin_no_cat_obs\",\n partial(\n sc.pl.stacked_violin,\n groupby=\"numeric_column\",\n use_raw=False,\n num_categories=4,\n title=\"non-categorical obs, custom figsize\",\n figsize=(8, 2.5),\n ),\n ),\n ]\n]\n\n\n@pytest.mark.parametrize((\"id\", \"fn\"), params_dotplot_matrixplot_stacked_violin)\ndef test_dotplot_matrixplot_stacked_violin(image_comparer, id, fn):\n save_and_compare_images = partial(image_comparer, ROOT, tol=5)\n\n adata = krumsiek11()\n adata.obs[\"numeric_column\"] = adata.X[:, 0]\n adata.layers[\"test\"] = -1 * adata.X.copy()\n genes_dict = {\n \"group a\": [\"Gata2\", \"Gata1\"],\n \"group b\": [\"Fog1\", \"EKLF\", \"Fli1\", \"SCL\"],\n \"group c\": [\"Cebpa\", \"Pu.1\", \"cJun\", \"EgrNab\", \"Gfi1\"],\n }\n\n if id.endswith(\"dict\"):\n fn(adata, genes_dict, show=False)\n else:\n fn(adata, adata.var_names, show=False)\n save_and_compare_images(id)\n\n\ndef test_dotplot_obj(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n # test dotplot dot_min, dot_max, color_map, and var_groups\n pbmc = pbmc68k_reduced()\n genes = [\n *[\"CD79A\", \"MS4A1\", \"CD8A\", \"CD8B\", \"LYZ\", \"LGALS3\"],\n *[\"S100A8\", \"GNLY\", \"NKG7\", \"KLRB1\", \"FCGR3A\", \"FCER1A\", \"CST3\"],\n ]\n # test layer, var standardization, smallest_dot,\n # color title, size_title return_fig and dot_edge\n pbmc.layers[\"test\"] = pbmc.X * -1\n plot = sc.pl.dotplot(\n pbmc,\n genes,\n \"bulk_labels\",\n layer=\"test\",\n dendrogram=True,\n return_fig=True,\n standard_scale=\"var\",\n smallest_dot=40,\n colorbar_title=\"scaled column max\",\n size_title=\"Fraction of cells\",\n )\n plot.style(dot_edge_color=\"black\", dot_edge_lw=0.1, cmap=\"Reds\").show()\n\n save_and_compare_images(\"dotplot_std_scale_var\")\n\n\ndef test_dotplot_style_no_reset():\n pbmc = pbmc68k_reduced()\n plot = sc.pl.dotplot(pbmc, \"CD79A\", \"bulk_labels\", return_fig=True)\n assert isinstance(plot, sc.pl.DotPlot)\n assert plot.cmap == sc.pl.DotPlot.DEFAULT_COLORMAP\n plot.style(cmap=\"winter\")\n assert plot.cmap == \"winter\"\n plot.style(color_on=\"square\")\n assert plot.cmap == \"winter\", \"style() should not reset unspecified parameters\"\n\n\ndef test_dotplot_add_totals(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=5)\n\n pbmc = pbmc68k_reduced()\n markers = {\"T-cell\": \"CD3D\", \"B-cell\": \"CD79A\", \"myeloid\": \"CST3\"}\n sc.pl.dotplot(pbmc, markers, \"bulk_labels\", return_fig=True).add_totals().show()\n save_and_compare_images(\"dotplot_totals\")\n\n\ndef test_matrixplot_obj(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n adata = pbmc68k_reduced()\n marker_genes_dict = {\n \"3\": [\"GNLY\", \"NKG7\"],\n \"1\": [\"FCER1A\"],\n \"2\": [\"CD3D\"],\n \"0\": [\"FCGR3A\"],\n \"4\": [\"CD79A\", \"MS4A1\"],\n }\n\n plot = sc.pl.matrixplot(\n adata,\n marker_genes_dict,\n \"bulk_labels\",\n use_raw=False,\n title=\"added totals\",\n return_fig=True,\n )\n plot.add_totals(sort=\"descending\").style(edge_color=\"white\", edge_lw=0.5).show()\n save_and_compare_images(\"matrixplot_with_totals\")\n\n axes = plot.get_axes()\n assert \"mainplot_ax\" in axes, \"mainplot_ax not found in returned axes dict\"\n\n\ndef test_stacked_violin_obj(image_comparer, plt):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n pbmc = pbmc68k_reduced()\n markers = {\n \"T-cell\": [\"CD3D\", \"CD3E\", \"IL32\"],\n \"B-cell\": [\"CD79A\", \"CD79B\", \"MS4A1\"],\n \"myeloid\": [\"CST3\", \"LYZ\"],\n }\n plot = sc.pl.stacked_violin(\n pbmc,\n markers,\n \"bulk_labels\",\n use_raw=False,\n title=\"return_fig. add_totals\",\n return_fig=True,\n )\n plot.add_totals().style(row_palette=\"tab20\").show()\n save_and_compare_images(\"stacked_violin_return_fig\")\n\n\n# checking for https://github.com/scverse/scanpy/issues/3152\ndef test_stacked_violin_swap_axes_match(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=10)\n pbmc = pbmc68k_reduced()\n sc.tl.rank_genes_groups(\n pbmc,\n \"bulk_labels\",\n method=\"wilcoxon\",\n tie_correct=True,\n pts=True,\n key_added=\"wilcoxon\",\n )\n swapped_ax = sc.pl.rank_genes_groups_stacked_violin(\n pbmc,\n n_genes=2,\n key=\"wilcoxon\",\n groupby=\"bulk_labels\",\n swap_axes=True,\n return_fig=True,\n )\n swapped_ax.show()\n save_and_compare_images(\"stacked_violin_swap_axes_pbmc68k_reduced\")\n\n\ndef test_tracksplot(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n adata = krumsiek11()\n sc.pl.tracksplot(\n adata, adata.var_names, \"cell_type\", dendrogram=True, use_raw=False\n )\n save_and_compare_images(\"tracksplot\")\n\n\ndef test_multiple_plots(image_comparer):\n # only testing stacked_violin, matrixplot and dotplot\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n adata = pbmc68k_reduced()\n markers = {\n \"T-cell\": [\"CD3D\", \"CD3E\", \"IL32\"],\n \"B-cell\": [\"CD79A\", \"CD79B\", \"MS4A1\"],\n \"myeloid\": [\"CST3\", \"LYZ\"],\n }\n fig, (ax1, ax2, ax3) = plt.subplots(\n 1, 3, figsize=(20, 5), gridspec_kw={\"wspace\": 0.7}\n )\n _ = sc.pl.stacked_violin(\n adata,\n markers,\n groupby=\"bulk_labels\",\n ax=ax1,\n title=\"stacked_violin\",\n dendrogram=True,\n show=False,\n )\n _ = sc.pl.dotplot(\n adata,\n markers,\n groupby=\"bulk_labels\",\n ax=ax2,\n title=\"dotplot\",\n dendrogram=True,\n show=False,\n )\n _ = sc.pl.matrixplot(\n adata,\n markers,\n groupby=\"bulk_labels\",\n ax=ax3,\n title=\"matrixplot\",\n dendrogram=True,\n show=False,\n )\n save_and_compare_images(\"multiple_plots\")\n\n\ndef test_violin(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=40)\n\n with plt.rc_context():\n sc.pl.set_rcParams_defaults()\n sc.set_figure_params(dpi=50, color_map=\"viridis\")\n\n pbmc = pbmc68k_reduced()\n sc.pl.violin(\n pbmc,\n [\"n_genes\", \"percent_mito\", \"n_counts\"],\n stripplot=True,\n multi_panel=True,\n jitter=True,\n show=False,\n )\n save_and_compare_images(\"violin_multi_panel\")\n\n sc.pl.violin(\n pbmc,\n [\"n_genes\", \"percent_mito\", \"n_counts\"],\n ylabel=[\"foo\", \"bar\", \"baz\"],\n groupby=\"bulk_labels\",\n stripplot=True,\n multi_panel=True,\n jitter=True,\n show=False,\n rotation=90,\n )\n save_and_compare_images(\"violin_multi_panel_with_groupby\")\n\n # test use of layer\n pbmc.layers[\"negative\"] = pbmc.X * -1\n sc.pl.violin(\n pbmc,\n \"CST3\",\n groupby=\"bulk_labels\",\n stripplot=True,\n multi_panel=True,\n jitter=True,\n show=False,\n layer=\"negative\",\n use_raw=False,\n rotation=90,\n )\n save_and_compare_images(\"violin_multi_panel_with_layer\")\n\n\n# TODO: Generalize test to more plotting types\ndef test_violin_without_raw(tmp_path):\n # https://github.com/scverse/scanpy/issues/1546\n has_raw_pth = tmp_path / \"has_raw.png\"\n no_raw_pth = tmp_path / \"no_raw.png\"\n\n pbmc = pbmc68k_reduced()\n pbmc_no_raw = pbmc.raw.to_adata().copy()\n\n sc.pl.violin(pbmc, \"CST3\", groupby=\"bulk_labels\", show=False, jitter=False)\n plt.savefig(has_raw_pth)\n plt.close()\n\n sc.pl.violin(pbmc_no_raw, \"CST3\", groupby=\"bulk_labels\", show=False, jitter=False)\n plt.savefig(no_raw_pth)\n plt.close()\n\n assert compare_images(has_raw_pth, no_raw_pth, tol=5) is None\n\n\ndef test_dendrogram(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=10)\n\n pbmc = pbmc68k_reduced()\n sc.pl.dendrogram(pbmc, \"bulk_labels\")\n save_and_compare_images(\"dendrogram\")\n\n\ndef test_correlation(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n pbmc = pbmc68k_reduced()\n sc.pl.correlation_matrix(pbmc, \"bulk_labels\")\n save_and_compare_images(\"correlation\")\n\n\n_RANK_GENES_GROUPS_PARAMS = [\n (\n \"sharey\",\n partial(sc.pl.rank_genes_groups, n_genes=12, n_panels_per_row=3, show=False),\n ),\n (\n \"basic\",\n partial(\n sc.pl.rank_genes_groups,\n n_genes=12,\n n_panels_per_row=3,\n sharey=False,\n show=False,\n ),\n ),\n (\n \"heatmap\",\n partial(sc.pl.rank_genes_groups_heatmap, n_genes=4, cmap=\"YlGnBu\", show=False),\n ),\n (\n \"heatmap_swap_axes\",\n partial(\n sc.pl.rank_genes_groups_heatmap,\n n_genes=20,\n swap_axes=True,\n use_raw=False,\n show_gene_labels=False,\n show=False,\n vmin=-3,\n vmax=3,\n cmap=\"bwr\",\n ),\n ),\n (\n \"heatmap_swap_axes_vcenter\",\n partial(\n sc.pl.rank_genes_groups_heatmap,\n n_genes=20,\n swap_axes=True,\n use_raw=False,\n show_gene_labels=False,\n show=False,\n vmin=-3,\n vcenter=1,\n vmax=3,\n cmap=\"RdBu_r\",\n ),\n ),\n (\n \"stacked_violin\",\n partial(\n sc.pl.rank_genes_groups_stacked_violin,\n n_genes=3,\n show=False,\n groups=[\"3\", \"0\", \"5\"],\n ),\n ),\n (\n \"dotplot\",\n partial(sc.pl.rank_genes_groups_dotplot, n_genes=4, show=False),\n ),\n (\n \"dotplot_gene_names\",\n partial(\n sc.pl.rank_genes_groups_dotplot,\n var_names={\n \"T-cell\": [\"CD3D\", \"CD3E\", \"IL32\"],\n \"B-cell\": [\"CD79A\", \"CD79B\", \"MS4A1\"],\n \"myeloid\": [\"CST3\", \"LYZ\"],\n },\n values_to_plot=\"logfoldchanges\",\n cmap=\"bwr\",\n vmin=-3,\n vmax=3,\n show=False,\n ),\n ),\n (\n \"dotplot_logfoldchange\",\n partial(\n sc.pl.rank_genes_groups_dotplot,\n n_genes=4,\n values_to_plot=\"logfoldchanges\",\n vmin=-5,\n vmax=5,\n min_logfoldchange=3,\n cmap=\"RdBu_r\",\n swap_axes=True,\n title=\"log fold changes swap_axes\",\n show=False,\n ),\n ),\n (\n \"dotplot_logfoldchange_vcenter\",\n partial(\n sc.pl.rank_genes_groups_dotplot,\n n_genes=4,\n values_to_plot=\"logfoldchanges\",\n vmin=-5,\n vcenter=1,\n vmax=5,\n min_logfoldchange=3,\n cmap=\"RdBu_r\",\n swap_axes=True,\n title=\"log fold changes swap_axes\",\n show=False,\n ),\n ),\n (\n \"matrixplot\",\n partial(\n sc.pl.rank_genes_groups_matrixplot,\n n_genes=5,\n show=False,\n title=\"matrixplot\",\n gene_symbols=\"symbol\",\n use_raw=False,\n ),\n ),\n (\n \"matrixplot_gene_names_symbol\",\n partial(\n sc.pl.rank_genes_groups_matrixplot,\n var_names={\n \"T-cell\": [\"CD3D__\", \"CD3E__\", \"IL32__\"],\n \"B-cell\": [\"CD79A__\", \"CD79B__\", \"MS4A1__\"],\n \"myeloid\": [\"CST3__\", \"LYZ__\"],\n },\n values_to_plot=\"logfoldchanges\",\n cmap=\"bwr\",\n vmin=-3,\n vmax=3,\n gene_symbols=\"symbol\",\n use_raw=False,\n show=False,\n ),\n ),\n (\n \"matrixplot_n_genes_negative\",\n partial(\n sc.pl.rank_genes_groups_matrixplot,\n n_genes=-5,\n show=False,\n title=\"matrixplot n_genes=-5\",\n ),\n ),\n (\n \"matrixplot_swap_axes\",\n partial(\n sc.pl.rank_genes_groups_matrixplot,\n n_genes=5,\n show=False,\n swap_axes=True,\n values_to_plot=\"logfoldchanges\",\n vmin=-6,\n vmax=6,\n cmap=\"bwr\",\n title=\"log fold changes swap_axes\",\n ),\n ),\n (\n \"matrixplot_swap_axes_vcenter\",\n partial(\n sc.pl.rank_genes_groups_matrixplot,\n n_genes=5,\n show=False,\n swap_axes=True,\n values_to_plot=\"logfoldchanges\",\n vmin=-6,\n vcenter=1,\n vmax=6,\n cmap=\"bwr\",\n title=\"log fold changes swap_axes\",\n ),\n ),\n (\n \"tracksplot\",\n partial(\n sc.pl.rank_genes_groups_tracksplot,\n n_genes=3,\n show=False,\n groups=[\"3\", \"2\", \"1\"],\n ),\n ),\n (\n \"violin\",\n partial(\n sc.pl.rank_genes_groups_violin,\n groups=\"0\",\n n_genes=5,\n use_raw=True,\n jitter=False,\n strip=False,\n show=False,\n ),\n ),\n (\n \"violin_not_raw\",\n partial(\n sc.pl.rank_genes_groups_violin,\n groups=\"0\",\n n_genes=5,\n use_raw=False,\n jitter=False,\n strip=False,\n show=False,\n ),\n ),\n]\n\n\n@pytest.mark.parametrize(\n (\"name\", \"fn\"),\n [pytest.param(name, fn, id=name) for name, fn in _RANK_GENES_GROUPS_PARAMS],\n)\ndef test_rank_genes_groups(image_comparer, name, fn):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n pbmc = pbmc68k_reduced()\n sc.tl.rank_genes_groups(pbmc, \"louvain\", n_genes=pbmc.raw.shape[1])\n\n # add gene symbol\n pbmc.var[\"symbol\"] = pbmc.var.index + \"__\"\n\n with plt.rc_context({\"axes.grid\": True, \"figure.figsize\": (4, 4)}):\n fn(pbmc)\n key = \"ranked_genes\" if name == \"basic\" else f\"ranked_genes_{name}\"\n save_and_compare_images(key)\n plt.close()\n\n\n@pytest.fixture(scope=\"session\")\ndef gene_symbols_adatas_session() -> tuple[AnnData, AnnData]:\n \"\"\"Create two anndata objects which are equivalent except for var_names\n\n Both have ensembl ids and hgnc symbols as columns in var. The first has ensembl\n ids as var_names, the second has symbols.\n \"\"\"\n pbmc = pbmc3k_processed().raw.to_adata()\n pbmc_counts = pbmc3k()\n\n pbmc.layers[\"counts\"] = pbmc_counts[pbmc.obs_names, pbmc.var_names].X.copy()\n pbmc.var[\"gene_symbol\"] = pbmc.var_names\n pbmc.var[\"ensembl_id\"] = pbmc_counts.var[\"gene_ids\"].loc[pbmc.var_names]\n\n pbmc.var = pbmc.var.set_index(\"ensembl_id\", drop=False)\n\n # Cutting down on size for plotting, tracksplot and stacked_violin are slow\n pbmc = pbmc[pbmc.obs[\"louvain\"].isin(pbmc.obs[\"louvain\"].cat.categories[:4])]\n pbmc = pbmc[::3].copy()\n\n # Creating variations\n a = pbmc.copy()\n b = pbmc.copy()\n a.var = a.var.set_index(\"ensembl_id\")\n b.var = b.var.set_index(\"gene_symbol\")\n\n # Computing DE\n sc.tl.rank_genes_groups(a, groupby=\"louvain\")\n sc.tl.rank_genes_groups(b, groupby=\"louvain\")\n\n return a, b\n\n\n@pytest.fixture\ndef gene_symbols_adatas(gene_symbols_adatas_session) -> tuple[AnnData, AnnData]:\n a, b = gene_symbols_adatas_session\n return a.copy(), b.copy()\n\n\n@pytest.mark.parametrize(\n \"func\",\n [\n sc.pl.rank_genes_groups_dotplot,\n sc.pl.rank_genes_groups_heatmap,\n sc.pl.rank_genes_groups_matrixplot,\n sc.pl.rank_genes_groups_stacked_violin,\n sc.pl.rank_genes_groups_tracksplot,\n # TODO: add other rank_genes_groups plots here once they work\n ],\n)\ndef test_plot_rank_genes_groups_gene_symbols(\n gene_symbols_adatas, func, tmp_path, check_same_image\n):\n a, b = gene_symbols_adatas\n\n pth_1_a = tmp_path / f\"{func.__name__}_equivalent_gene_symbols_1_a.png\"\n pth_1_b = tmp_path / f\"{func.__name__}_equivalent_gene_symbols_1_b.png\"\n\n func(a, gene_symbols=\"gene_symbol\")\n plt.savefig(pth_1_a)\n plt.close()\n\n func(b)\n plt.savefig(pth_1_b)\n pass\n\n check_same_image(pth_1_a, pth_1_b, tol=1)\n\n pth_2_a = tmp_path / f\"{func.__name__}_equivalent_gene_symbols_2_a.png\"\n pth_2_b = tmp_path / f\"{func.__name__}_equivalent_gene_symbols_2_b.png\"\n\n func(a)\n plt.savefig(pth_2_a)\n plt.close()\n\n func(b, gene_symbols=\"ensembl_id\")\n plt.savefig(pth_2_b)\n plt.close()\n\n check_same_image(pth_2_a, pth_2_b, tol=1)\n\n\n@pytest.mark.parametrize(\n \"func\",\n [\n sc.pl.rank_genes_groups_dotplot,\n sc.pl.rank_genes_groups_heatmap,\n sc.pl.rank_genes_groups_matrixplot,\n sc.pl.rank_genes_groups_stacked_violin,\n sc.pl.rank_genes_groups_tracksplot,\n # TODO: add other rank_genes_groups plots here once they work\n ],\n)\ndef test_rank_genes_groups_plots_n_genes_vs_var_names(tmp_path, func, check_same_image):\n \"\"\"\\\n Checks that passing a negative value for n_genes works, and that passing\n var_names as a dict works.\n \"\"\"\n N = 3\n pbmc = pbmc68k_reduced().raw.to_adata()\n groups = pbmc.obs[\"louvain\"].cat.categories[:3]\n pbmc = pbmc[pbmc.obs[\"louvain\"].isin(groups)][::3].copy()\n\n sc.tl.rank_genes_groups(pbmc, groupby=\"louvain\")\n\n top_genes = {}\n bottom_genes = {}\n for g, subdf in sc.get.rank_genes_groups_df(pbmc, group=groups).groupby(\n \"group\", observed=True\n ):\n top_genes[g] = list(subdf[\"names\"].head(N))\n bottom_genes[g] = list(subdf[\"names\"].tail(N))\n\n positive_n_pth = tmp_path / f\"{func.__name__}_positive_n.png\"\n top_genes_pth = tmp_path / f\"{func.__name__}_top_genes.png\"\n negative_n_pth = tmp_path / f\"{func.__name__}_negative_n.png\"\n bottom_genes_pth = tmp_path / f\"{func.__name__}_bottom_genes.png\"\n\n def wrapped(pth, **kwargs):\n func(pbmc, groupby=\"louvain\", dendrogram=False, **kwargs)\n plt.savefig(pth)\n plt.close()\n\n wrapped(positive_n_pth, n_genes=N)\n wrapped(top_genes_pth, var_names=top_genes)\n\n check_same_image(positive_n_pth, top_genes_pth, tol=1)\n\n wrapped(negative_n_pth, n_genes=-N)\n wrapped(bottom_genes_pth, var_names=bottom_genes)\n\n check_same_image(negative_n_pth, bottom_genes_pth, tol=1)\n\n # Shouldn't be able to pass these together\n with pytest.raises(\n ValueError, match=\"n_genes and var_names are mutually exclusive\"\n ):\n wrapped(tmp_path / \"not_written.png\", n_genes=N, var_names=top_genes)\n\n\n@pytest.mark.parametrize(\n (\"id\", \"fn\"),\n [\n (\"heatmap\", sc.pl.heatmap),\n (\"dotplot\", sc.pl.dotplot),\n (\"matrixplot\", sc.pl.matrixplot),\n (\"stacked_violin\", sc.pl.stacked_violin),\n (\"tracksplot\", sc.pl.tracksplot),\n ],\n)\ndef test_genes_symbols(image_comparer, id, fn):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n adata = krumsiek11()\n\n # add a 'symbols' column\n adata.var[\"symbols\"] = adata.var.index.map(lambda x: f\"symbol_{x}\")\n symbols = [f\"symbol_{x}\" for x in adata.var_names]\n\n fn(adata, symbols, \"cell_type\", dendrogram=True, gene_symbols=\"symbols\", show=False)\n save_and_compare_images(f\"{id}_gene_symbols\")\n\n\n@pytest.fixture(scope=\"session\")\ndef pbmc_scatterplots_session() -> AnnData:\n # Wrapped in another fixture to avoid mutation\n pbmc = pbmc68k_reduced()\n pbmc.obs[\"mask\"] = pbmc.obs[\"louvain\"].isin([\"0\", \"1\", \"3\"])\n pbmc.layers[\"sparse\"] = pbmc.raw.X / 2\n pbmc.layers[\"test\"] = pbmc.X.copy() + 100\n pbmc.var[\"numbers\"] = [str(x) for x in range(pbmc.shape[1])]\n sc.pp.neighbors(pbmc)\n sc.tl.tsne(pbmc, random_state=0, n_pcs=30)\n sc.tl.diffmap(pbmc)\n return pbmc\n\n\n@pytest.fixture\ndef pbmc_scatterplots(pbmc_scatterplots_session) -> AnnData:\n return pbmc_scatterplots_session.copy()\n\n\n@pytest.mark.parametrize(\n (\"id\", \"fn\"),\n [\n (\"pca\", partial(sc.pl.pca, color=\"bulk_labels\")),\n (\n \"pca_with_fonts\",\n partial(\n sc.pl.pca,\n color=[\"bulk_labels\", \"louvain\"],\n legend_loc=\"on data\",\n legend_fontoutline=2,\n legend_fontweight=\"normal\",\n legend_fontsize=10,\n ),\n ),\n pytest.param(\n \"3dprojection\", partial(sc.pl.pca, color=\"bulk_labels\", projection=\"3d\")\n ),\n (\n \"multipanel\",\n partial(\n sc.pl.pca,\n color=[\"CD3D\", \"CD79A\"],\n components=[\"1,2\", \"1,3\"],\n vmax=5,\n use_raw=False,\n vmin=-5,\n cmap=\"seismic\",\n ),\n ),\n (\n \"multipanel_vcenter\",\n partial(\n sc.pl.pca,\n color=[\"CD3D\", \"CD79A\"],\n components=[\"1,2\", \"1,3\"],\n vmax=5,\n use_raw=False,\n vmin=-5,\n vcenter=1,\n cmap=\"seismic\",\n ),\n ),\n (\n \"pca_one_marker\",\n partial(sc.pl.pca, color=\"louvain\", marker=\"^\"),\n ),\n (\n \"pca_one_marker_multiple_colors\",\n partial(sc.pl.pca, color=[\"louvain\", \"bulk_labels\"], marker=\"^\"),\n ),\n (\n \"pca_multiple_markers_multiple_colors\",\n partial(sc.pl.pca, color=[\"louvain\", \"bulk_labels\"], marker=[\"^\", \"x\"]),\n ),\n (\n \"pca_marker_with_dimensions\",\n partial(\n sc.pl.pca, color=\"louvain\", marker=\"^\", dimensions=[(0, 1), (1, 2)]\n ),\n ),\n (\n \"pca_markers_with_dimensions\",\n partial(\n sc.pl.pca,\n color=\"louvain\",\n marker=[\"^\", \"x\"],\n dimensions=[(0, 1), (1, 2)],\n ),\n ),\n (\n \"pca_markers_colors_with_dimensions\",\n partial(\n sc.pl.pca,\n color=[\"louvain\", \"bulk_labels\"],\n marker=[\"^\", \"x\"],\n dimensions=[(0, 1), (1, 2)],\n ),\n ),\n (\n \"pca_sparse_layer\",\n partial(sc.pl.pca, color=[\"CD3D\", \"CD79A\"], layer=\"sparse\", cmap=\"viridis\"),\n ),\n pytest.param(\n \"tsne\",\n partial(sc.pl.tsne, color=[\"CD3D\", \"louvain\"]),\n marks=pytest.mark.xfail(\n reason=\"slight differences even after setting random_state.\"\n ),\n ),\n (\"umap_nocolor\", sc.pl.umap),\n (\n \"umap\",\n partial(\n sc.pl.umap,\n color=[\"louvain\"],\n palette=[\"b\", \"grey80\", \"r\", \"yellow\", \"black\", \"gray\", \"lightblue\"],\n frameon=False,\n ),\n ),\n (\n \"umap_gene_expr\",\n partial(\n sc.pl.umap,\n color=np.array([\"LYZ\", \"CD79A\"]),\n s=20,\n alpha=0.5,\n frameon=False,\n title=[\"gene1\", \"gene2\"],\n ),\n ),\n (\n \"umap_layer\",\n partial(\n sc.pl.umap,\n color=np.array([\"LYZ\", \"CD79A\"]),\n s=20,\n alpha=0.5,\n frameon=False,\n title=[\"gene1\", \"gene2\"],\n layer=\"test\",\n vmin=100,\n vcenter=101,\n ),\n ),\n (\n \"umap_with_edges\",\n partial(sc.pl.umap, color=\"louvain\", edges=True, edges_width=0.1, s=50),\n ),\n # ('diffmap', partial(sc.pl.diffmap, components='all', color=['CD3D'])),\n (\n \"umap_symbols\",\n partial(sc.pl.umap, color=[\"1\", \"2\", \"3\"], gene_symbols=\"numbers\"),\n ),\n (\n \"pca_mask\",\n partial(\n sc.pl.pca,\n color=[\"LYZ\", \"CD79A\", \"louvain\"],\n mask_obs=\"mask\",\n ),\n ),\n ],\n)\ndef test_scatterplots(image_comparer, pbmc_scatterplots, id, fn):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n fn(pbmc_scatterplots, show=False)\n save_and_compare_images(id)\n\n\ndef test_scatter_embedding_groups_and_size(image_comparer):\n # test that the 'groups' parameter sorts\n # cells, such that the cells belonging to the groups are\n # plotted on top. This new ordering requires that the size\n # vector is also ordered (if given).\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n pbmc = pbmc68k_reduced()\n sc.pl.embedding(\n pbmc,\n \"umap\",\n color=[\"bulk_labels\"],\n groups=[\"CD14+ Monocyte\", \"Dendritic\"],\n size=(np.arange(pbmc.shape[0]) / 40) ** 1.7,\n )\n save_and_compare_images(\"embedding_groups_size\")\n\n\ndef test_scatter_embedding_add_outline_vmin_vmax_norm(image_comparer, check_same_image):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n pbmc = pbmc68k_reduced()\n\n sc.pl.embedding(\n pbmc,\n \"X_umap\",\n color=[\"percent_mito\", \"n_counts\", \"bulk_labels\", \"percent_mito\"],\n s=200,\n frameon=False,\n add_outline=True,\n vmax=[\"p99.0\", partial(np.percentile, q=90), None, 0.03],\n vmin=0.01,\n vcenter=[0.015, None, None, 0.025],\n outline_color=(\"#555555\", \"0.9\"),\n outline_width=(0.5, 0.5),\n cmap=\"viridis_r\",\n alpha=0.9,\n wspace=0.5,\n )\n save_and_compare_images(\"embedding_outline_vmin_vmax\")\n\n\ndef test_scatter_embedding_add_outline_vmin_vmax_norm_ref(tmp_path, check_same_image):\n pbmc = pbmc68k_reduced()\n\n import matplotlib as mpl\n import matplotlib.pyplot as plt\n\n norm = mpl.colors.LogNorm()\n with pytest.raises(\n ValueError, match=\"Passing both norm and vmin/vmax/vcenter is not allowed.\"\n ):\n sc.pl.embedding(\n pbmc,\n \"X_umap\",\n color=[\"percent_mito\", \"n_counts\"],\n norm=norm,\n vmin=0,\n vmax=1,\n vcenter=0.5,\n cmap=\"RdBu_r\",\n )\n\n try:\n from matplotlib.colors import TwoSlopeNorm as DivNorm\n except ImportError:\n # matplotlib<3.2\n from matplotlib.colors import DivergingNorm as DivNorm\n\n from matplotlib.colors import Normalize\n\n norm = Normalize(0, 10000)\n divnorm = DivNorm(200, 150, 6000)\n\n # allowed\n sc.pl.umap(\n pbmc,\n color=[\"n_counts\", \"bulk_labels\", \"percent_mito\"],\n frameon=False,\n vmax=[\"p99.0\", None, None],\n vcenter=[0.015, None, None],\n norm=[None, norm, norm],\n wspace=0.5,\n )\n\n sc.pl.umap(\n pbmc,\n color=[\"n_counts\", \"bulk_labels\"],\n frameon=False,\n norm=norm,\n wspace=0.5,\n )\n plt.savefig(tmp_path / \"umap_norm_fig0.png\")\n plt.close()\n\n sc.pl.umap(\n pbmc,\n color=[\"n_counts\", \"bulk_labels\"],\n frameon=False,\n norm=divnorm,\n wspace=0.5,\n )\n plt.savefig(tmp_path / \"umap_norm_fig1.png\")\n plt.close()\n\n sc.pl.umap(\n pbmc,\n color=[\"n_counts\", \"bulk_labels\"],\n frameon=False,\n vcenter=200,\n vmin=150,\n vmax=6000,\n wspace=0.5,\n )\n plt.savefig(tmp_path / \"umap_norm_fig2.png\")\n plt.close()\n\n check_same_image(\n tmp_path / \"umap_norm_fig1.png\", tmp_path / \"umap_norm_fig2.png\", tol=1\n )\n\n with pytest.raises(AssertionError):\n check_same_image(\n tmp_path / \"umap_norm_fig1.png\", tmp_path / \"umap_norm_fig0.png\", tol=1\n )\n\n\ndef test_timeseries():\n adata = pbmc68k_reduced()\n sc.pp.neighbors(adata, n_neighbors=5, method=\"gauss\", knn=False)\n sc.tl.diffmap(adata)\n sc.tl.dpt(adata, n_branchings=1, n_dcs=10)\n sc.pl.dpt_timeseries(adata, as_heatmap=True)\n\n\ndef test_scatter_raw(tmp_path):\n pbmc = pbmc68k_reduced()[:100].copy()\n raw_pth = tmp_path / \"raw.png\"\n x_pth = tmp_path / \"X.png\"\n\n sc.pl.scatter(pbmc, color=\"HES4\", basis=\"umap\", use_raw=True)\n plt.savefig(raw_pth, dpi=60)\n plt.close()\n\n sc.pl.scatter(pbmc, color=\"HES4\", basis=\"umap\", use_raw=False)\n plt.savefig(x_pth, dpi=60)\n plt.close()\n\n comp = compare_images(str(raw_pth), str(x_pth), tol=5)\n assert \"Error\" in comp, \"Plots should change depending on use_raw.\"\n\n\ndef test_binary_scatter(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n data = AnnData(\n np.asarray([[-1, 2, 0], [3, 4, 0], [1, 2, 0]]).T,\n obs=dict(binary=np.asarray([False, True, True])),\n )\n sc.pp.pca(data)\n sc.pl.pca(data, color=\"binary\")\n if pkg_version(\"scikit-learn\") >= Version(\"1.5.0rc1\"):\n save_and_compare_images(\"binary_pca\")\n else:\n save_and_compare_images(\"binary_pca_old\")\n\n\ndef test_scatter_specify_layer_and_raw():\n pbmc = pbmc68k_reduced()\n pbmc.layers[\"layer\"] = pbmc.raw.X.copy()\n with pytest.raises(ValueError, match=r\"Cannot use both a layer and.*raw\"):\n sc.pl.umap(pbmc, color=\"HES4\", use_raw=True, layer=\"layer\")\n\n\n@pytest.mark.parametrize(\"color\", [\"n_genes\", \"bulk_labels\"])\ndef test_scatter_no_basis_per_obs(image_comparer, color):\n \"\"\"Test scatterplot of per-obs points with no basis\"\"\"\n\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n pbmc = pbmc68k_reduced()\n sc.pl.scatter(\n pbmc,\n x=\"HES4\",\n y=\"percent_mito\",\n color=color,\n use_raw=False,\n # palette only applies to categorical, i.e. color=='bulk_labels'\n palette=\"Set2\",\n )\n save_and_compare_images(f\"scatter_HES_percent_mito_{color}\")\n\n\ndef test_scatter_no_basis_per_var(image_comparer):\n \"\"\"Test scatterplot of per-var points with no basis\"\"\"\n\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n pbmc = pbmc68k_reduced()\n sc.pl.scatter(pbmc, x=\"AAAGCCTGGCTAAC-1\", y=\"AAATTCGATGCACA-1\", use_raw=False)\n save_and_compare_images(\"scatter_AAAGCCTGGCTAAC-1_vs_AAATTCGATGCACA-1\")\n\n\n@pytest.fixture\ndef pbmc_filtered() -> Callable[[], AnnData]:\n pbmc = pbmc68k_reduced()\n sc.pp.filter_genes(pbmc, min_cells=10)\n return pbmc.copy\n\n\ndef test_scatter_no_basis_raw(check_same_image, pbmc_filtered, tmpdir):\n adata = pbmc_filtered()\n\n \"\"\"Test scatterplots of raw layer with no basis.\"\"\"\n path1 = tmpdir / \"scatter_EGFL7_F12_FAM185A_rawNone.png\"\n path2 = tmpdir / \"scatter_EGFL7_F12_FAM185A_rawTrue.png\"\n path3 = tmpdir / \"scatter_EGFL7_F12_FAM185A_rawToAdata.png\"\n\n sc.pl.scatter(adata, x=\"EGFL7\", y=\"F12\", color=\"FAM185A\", use_raw=None)\n plt.savefig(path1)\n plt.close()\n\n # is equivalent to:\n sc.pl.scatter(adata, x=\"EGFL7\", y=\"F12\", color=\"FAM185A\", use_raw=True)\n plt.savefig(path2)\n plt.close()\n\n # and also to:\n sc.pl.scatter(adata.raw.to_adata(), x=\"EGFL7\", y=\"F12\", color=\"FAM185A\")\n plt.savefig(path3)\n\n check_same_image(path1, path2, tol=15)\n check_same_image(path1, path3, tol=15)\n\n\n@pytest.mark.parametrize(\n (\"x\", \"y\", \"color\", \"use_raw\"),\n [\n # test that plotting fails with a ValueError if trying to plot\n # var_names only found in raw and use_raw is False\n (\"EGFL7\", \"F12\", \"FAM185A\", False),\n # test that plotting fails if one axis is a per-var value and the\n # other is a per-obs value\n (\"HES4\", \"n_cells\", None, None),\n (\"percent_mito\", \"AAAGCCTGGCTAAC-1\", None, None),\n ],\n)\ndef test_scatter_no_basis_value_error(pbmc_filtered, x, y, color, use_raw):\n \"\"\"Test that `scatter()` raises `ValueError` where appropriate\n\n If `sc.pl.scatter()` receives variable labels that either cannot be\n found or are incompatible with one another, the function should\n raise a `ValueError`. This test checks that this happens as\n expected.\n \"\"\"\n with pytest.raises(\n ValueError, match=r\"inputs must all come from either `\\.obs` or `\\.var`\"\n ):\n sc.pl.scatter(pbmc_filtered(), x=x, y=y, color=color, use_raw=use_raw)\n\n\ndef test_rankings(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n pbmc = pbmc68k_reduced()\n sc.pp.pca(pbmc)\n sc.pl.pca_loadings(pbmc)\n save_and_compare_images(\"pca_loadings\")\n\n sc.pl.pca_loadings(pbmc, components=\"1,2,3\")\n save_and_compare_images(\"pca_loadings\")\n\n sc.pl.pca_loadings(pbmc, components=[1, 2, 3])\n save_and_compare_images(\"pca_loadings\")\n\n sc.pl.pca_loadings(pbmc, include_lowest=False)\n save_and_compare_images(\"pca_loadings_without_lowest\")\n\n sc.pl.pca_loadings(pbmc, n_points=10)\n save_and_compare_images(\"pca_loadings_10_points\")\n\n\n# TODO: Make more generic\ndef test_scatter_rep(tmpdir):\n \"\"\"\n Test to make sure I can predict when scatter reps should be the same\n \"\"\"\n TESTDIR = Path(tmpdir)\n rep_args = {\n \"raw\": {\"use_raw\": True},\n \"layer\": {\"layer\": \"layer\", \"use_raw\": False},\n \"X\": {\"use_raw\": False},\n }\n states = pd.DataFrame.from_records(\n zip(\n list(chain.from_iterable(repeat(x, 3) for x in [\"X\", \"raw\", \"layer\"])),\n list(chain.from_iterable(repeat(\"abc\", 3))),\n [1, 2, 3, 3, 1, 2, 2, 3, 1],\n ),\n columns=[\"rep\", \"gene\", \"result\"],\n )\n states[\"outpth\"] = [\n TESTDIR / f\"{state.gene}_{state.rep}_{state.result}.png\"\n for state in states.itertuples()\n ]\n pattern = np.array(list(chain.from_iterable(repeat(i, 5) for i in range(3))))\n coords = np.c_[np.arange(15) % 5, pattern]\n\n adata = AnnData(\n X=np.zeros((15, 3)),\n layers={\"layer\": np.zeros((15, 3))},\n obsm={\"X_pca\": coords},\n var=pd.DataFrame(index=[x for x in list(\"abc\")]),\n obs=pd.DataFrame(index=[f\"cell{i}\" for i in range(15)]),\n )\n adata.raw = adata.copy()\n adata.X[np.arange(15), pattern] = 1\n adata.raw.X[np.arange(15), (pattern + 1) % 3] = 1\n adata.layers[\"layer\"][np.arange(15), (pattern + 2) % 3] = 1\n\n for state in states.itertuples():\n sc.pl.pca(adata, color=state.gene, **rep_args[state.rep], show=False)\n plt.savefig(state.outpth, dpi=60)\n plt.close()\n\n for s1, s2 in combinations(states.itertuples(), 2):\n comp = compare_images(str(s1.outpth), str(s2.outpth), tol=5)\n if s1.result == s2.result:\n assert comp is None, comp\n else:\n assert \"Error\" in comp, f\"{s1.outpth}, {s2.outpth} aren't supposed to match\"\n\n\ndef test_no_copy():\n # https://github.com/scverse/scanpy/issues/1000\n # Tests that plotting functions don't make a copy from a view unless they\n # actually have to\n actual = pbmc68k_reduced()\n sc.pl.umap(actual, color=[\"bulk_labels\", \"louvain\"], show=False) # Set colors\n\n view = actual[np.random.choice(actual.obs_names, size=actual.shape[0] // 5), :]\n\n sc.pl.umap(view, color=[\"bulk_labels\", \"louvain\"], show=False)\n assert view.is_view\n\n rank_genes_groups_plotting_funcs = [\n sc.pl.rank_genes_groups,\n sc.pl.rank_genes_groups_dotplot,\n sc.pl.rank_genes_groups_heatmap,\n sc.pl.rank_genes_groups_matrixplot,\n sc.pl.rank_genes_groups_stacked_violin,\n # TODO: raises ValueError about empty distance matrix – investigate\n # sc.pl.rank_genes_groups_tracksplot,\n sc.pl.rank_genes_groups_violin,\n ]\n\n # the pbmc68k was generated using rank_genes_groups with method='logreg'\n # which does not generate 'logfoldchanges', although this field is\n # required by `sc.get.rank_genes_groups_df`.\n # After updating rank_genes_groups plots to use the latter function\n # an error appears. Re-running rank_genes_groups with default method\n # solves the problem.\n sc.tl.rank_genes_groups(actual, \"bulk_labels\")\n\n # Only plotting one group at a time to avoid generating dendrogram\n # TODO: Generating a dendrogram modifies the object, this should be\n # optional and also maybe not modify the object.\n for plotfunc in rank_genes_groups_plotting_funcs:\n view = actual[actual.obs[\"bulk_labels\"] == \"Dendritic\"]\n plotfunc(view, [\"Dendritic\"], show=False)\n assert view.is_view\n\n\ndef test_groupby_index(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n pbmc = pbmc68k_reduced()\n\n genes = [\n \"CD79A\",\n \"MS4A1\",\n \"CD8A\",\n \"CD8B\",\n \"LYZ\",\n \"LGALS3\",\n \"S100A8\",\n \"GNLY\",\n \"NKG7\",\n \"KLRB1\",\n \"FCGR3A\",\n \"FCER1A\",\n \"CST3\",\n ]\n pbmc_subset = pbmc[:10].copy()\n sc.pl.dotplot(pbmc_subset, genes, groupby=\"index\")\n save_and_compare_images(\"dotplot_groupby_index\")\n\n\n# test category order when groupby is a list (#1735)\ndef test_groupby_list(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=30)\n\n adata = krumsiek11()\n\n np.random.seed(1)\n\n cat_val = adata.obs.cell_type.tolist()\n np.random.shuffle(cat_val)\n cats = adata.obs.cell_type.cat.categories.tolist()\n np.random.shuffle(cats)\n adata.obs[\"rand_cat\"] = pd.Categorical(cat_val, categories=cats)\n\n with mpl.rc_context({\"figure.subplot.bottom\": 0.5}):\n sc.pl.dotplot(\n adata, [\"Gata1\", \"Gata2\"], groupby=[\"rand_cat\", \"cell_type\"], swap_axes=True\n )\n save_and_compare_images(\"dotplot_groupby_list_catorder\")\n\n\ndef test_color_cycler(caplog):\n # https://github.com/scverse/scanpy/issues/1885\n import logging\n\n pbmc = pbmc68k_reduced()\n colors = sns.color_palette(\"deep\")\n cyl = sns.rcmod.cycler(\"color\", sns.color_palette(\"deep\"))\n\n with (\n caplog.at_level(logging.WARNING),\n plt.rc_context({\"axes.prop_cycle\": cyl, \"patch.facecolor\": colors[0]}),\n ):\n sc.pl.umap(pbmc, color=\"phase\")\n plt.show()\n plt.close()\n\n assert caplog.text == \"\"\n\n\ndef test_repeated_colors_w_missing_value():\n # https://github.com/scverse/scanpy/issues/2133\n v = pd.Series(np.arange(10).astype(str))\n v[0] = np.nan\n v = v.astype(\"category\")\n\n ad = sc.AnnData(obs=pd.DataFrame(v, columns=[\"value\"]))\n ad.obsm[\"X_umap\"] = np.random.normal(size=(ad.n_obs, 2))\n\n sc.pl.umap(ad, color=\"value\")\n\n ad.uns[\"value_colors\"][1] = ad.uns[\"value_colors\"][0]\n\n sc.pl.umap(ad, color=\"value\")\n\n\n@pytest.mark.parametrize(\n \"plot\",\n [\n sc.pl.rank_genes_groups_dotplot,\n sc.pl.rank_genes_groups_heatmap,\n sc.pl.rank_genes_groups_matrixplot,\n sc.pl.rank_genes_groups_stacked_violin,\n sc.pl.rank_genes_groups_tracksplot,\n # TODO: add other rank_genes_groups plots here once they work\n ],\n)\ndef test_filter_rank_genes_groups_plots(tmp_path, plot, check_same_image):\n N_GENES = 4\n\n adata = pbmc68k_reduced()\n\n sc.tl.rank_genes_groups(adata, \"bulk_labels\", method=\"wilcoxon\", pts=True)\n\n sc.tl.filter_rank_genes_groups(\n adata,\n key_added=\"rank_genes_groups_filtered\",\n min_in_group_fraction=0.25,\n min_fold_change=1,\n max_out_group_fraction=0.5,\n )\n\n conditions = \"logfoldchanges >= 1 & pct_nz_group >= .25 & pct_nz_reference < .5\"\n df = sc.get.rank_genes_groups_df(adata, group=None, key=\"rank_genes_groups\")\n df = df.query(conditions)[[\"group\", \"names\"]]\n\n var_names = {\n k: v.head(N_GENES).tolist()\n for k, v in df.groupby(\"group\", observed=True)[\"names\"]\n }\n\n pth_a = tmp_path / f\"{plot.__name__}_filter_a.png\"\n pth_b = tmp_path / f\"{plot.__name__}_filter_b.png\"\n\n plot(adata, key=\"rank_genes_groups_filtered\", n_genes=N_GENES)\n plt.savefig(pth_a)\n plt.close()\n\n plot(adata, key=\"rank_genes_groups\", var_names=var_names)\n plt.savefig(pth_b)\n plt.close()\n\n check_same_image(pth_a, pth_b, tol=1)\n\n\n@needs.skmisc\n@pytest.mark.parametrize(\n (\"id\", \"params\"),\n [\n pytest.param(\"scrublet\", {}, id=\"scrublet\"),\n pytest.param(\"scrublet_no_threshold\", {}, id=\"scrublet_no_threshold\"),\n pytest.param(\n \"scrublet_with_batches\", dict(batch_key=\"batch\"), id=\"scrublet_with_batches\"\n ),\n ],\n)\ndef test_scrublet_plots(monkeypatch, image_comparer, id, params):\n save_and_compare_images = partial(image_comparer, ROOT, tol=10)\n\n adata = pbmc3k()[:200].copy()\n adata.obs[\"batch\"] = 100 * [\"a\"] + 100 * [\"b\"]\n\n with monkeypatch.context() as m:\n if id == \"scrublet_no_threshold\":\n m.setattr(\"skimage.filters.threshold_minimum\", None)\n sc.pp.scrublet(adata, use_approx_neighbors=False, **params)\n if id == \"scrublet_no_threshold\":\n assert \"threshold\" not in adata.uns[\"scrublet\"]\n\n sc.pl.scrublet_score_distribution(adata, return_fig=True, show=False)\n save_and_compare_images(id)\n\n\ndef test_umap_mask_equal(tmp_path, check_same_image):\n \"\"\"Check that all desired cells are coloured and masked cells gray\"\"\"\n pbmc = pbmc3k_processed()\n mask_obs = pbmc.obs[\"louvain\"].isin([\"B cells\", \"NK cells\"])\n\n ax = sc.pl.umap(pbmc, size=8.0, show=False)\n sc.pl.umap(pbmc[mask_obs], size=8.0, color=\"LDHB\", ax=ax)\n plt.savefig(p1 := tmp_path / \"umap_mask_fig1.png\")\n plt.close()\n\n sc.pl.umap(pbmc, size=8.0, color=\"LDHB\", mask_obs=mask_obs)\n plt.savefig(p2 := tmp_path / \"umap_mask_fig2.png\")\n plt.close()\n\n check_same_image(p1, p2, tol=1)\n\n\ndef test_umap_mask_mult_plots():\n \"\"\"Check that multiple images are plotted when color is a list.\"\"\"\n pbmc = pbmc3k_processed()\n color = [\"LDHB\", \"LYZ\", \"CD79A\"]\n mask_obs = pbmc.obs[\"louvain\"].isin([\"B cells\", \"NK cells\"])\n axes = sc.pl.umap(pbmc, color=color, mask_obs=mask_obs, show=False)\n assert isinstance(axes, list)\n assert len(axes) == len(color)\n\n\ndef test_string_mask(tmp_path, check_same_image):\n \"\"\"Check that the same mask given as string or bool array provides the same result\"\"\"\n pbmc = pbmc3k_processed()\n pbmc.obs[\"mask\"] = mask_obs = pbmc.obs[\"louvain\"].isin([\"B cells\", \"NK cells\"])\n\n sc.pl.umap(pbmc, mask_obs=mask_obs, color=\"LDHB\")\n plt.savefig(p1 := tmp_path / \"umap_mask_fig1.png\")\n plt.close()\n\n sc.pl.umap(pbmc, color=\"LDHB\", mask_obs=\"mask\")\n plt.savefig(p2 := tmp_path / \"umap_mask_fig2.png\")\n plt.close()\n\n check_same_image(p1, p2, tol=1)\n\n\ndef test_violin_scale_warning(monkeypatch):\n adata = pbmc3k_processed()\n monkeypatch.setattr(sc.pl.StackedViolin, \"DEFAULT_SCALE\", \"count\", raising=False)\n with pytest.warns(FutureWarning, match=\"Don’t set DEFAULT_SCALE\"):\n sc.pl.StackedViolin(adata, adata.var_names[:3], groupby=\"louvain\")\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pytest\nfrom anndata import AnnData\nfrom anndata.tests.helpers import assert_equal\nfrom scipy import sparse\nfrom scipy.sparse import csr_matrix, issparse\n\nimport scanpy as sc\nfrom scanpy._utils import axis_sum\nfrom testing.scanpy._helpers import (\n _check_check_values_warnings,\n check_rep_mutation,\n check_rep_results,\n)\n\n# TODO: Add support for sparse-in-dask\nfrom testing.scanpy._pytest.params import ARRAY_TYPES\n\nif TYPE_CHECKING:\n from collections.abc import Callable\n from typing import Any\n\nX_total = np.array([[1, 0], [3, 0], [5, 6]])\nX_frac = np.array([[1, 0, 1], [3, 0, 1], [5, 6, 1]])\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\"dtype\", [\"float32\", \"int64\"])\n@pytest.mark.parametrize(\"target_sum\", [None, 1.0])\n@pytest.mark.parametrize(\"exclude_highly_expressed\", [True, False])\ndef test_normalize_matrix_types(\n array_type, dtype, target_sum, exclude_highly_expressed\n):\n adata = sc.datasets.pbmc68k_reduced()\n adata.X = (adata.raw.X).astype(dtype)\n adata_casted = adata.copy()\n adata_casted.X = array_type(adata_casted.raw.X).astype(dtype)\n sc.pp.normalize_total(\n adata, target_sum=target_sum, exclude_highly_expressed=exclude_highly_expressed\n )\n sc.pp.normalize_total(\n adata_casted,\n target_sum=target_sum,\n exclude_highly_expressed=exclude_highly_expressed,\n )\n X = adata_casted.X\n if \"dask\" in array_type.__name__:\n X = X.compute()\n if issparse(X):\n X = X.todense()\n if issparse(adata.X):\n adata.X = adata.X.todense()\n np.testing.assert_allclose(X, adata.X, rtol=1e-5, atol=1e-5)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\"dtype\", [\"float32\", \"int64\"])\ndef test_normalize_total(array_type, dtype):\n adata = AnnData(array_type(X_total).astype(dtype))\n sc.pp.normalize_total(adata, key_added=\"n_counts\")\n assert np.allclose(np.ravel(axis_sum(adata.X, axis=1)), [3.0, 3.0, 3.0])\n sc.pp.normalize_total(adata, target_sum=1, key_added=\"n_counts2\")\n assert np.allclose(np.ravel(axis_sum(adata.X, axis=1)), [1.0, 1.0, 1.0])\n\n adata = AnnData(array_type(X_frac).astype(dtype))\n sc.pp.normalize_total(adata, exclude_highly_expressed=True, max_fraction=0.7)\n assert np.allclose(np.ravel(axis_sum(adata.X[:, 1:3], axis=1)), [1.0, 1.0, 1.0])\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\"dtype\", [\"float32\", \"int64\"])\ndef test_normalize_total_rep(array_type, dtype):\n # Test that layer kwarg works\n X = array_type(sparse.random(100, 50, format=\"csr\", density=0.2, dtype=dtype))\n check_rep_mutation(sc.pp.normalize_total, X, fields=[\"layer\"])\n check_rep_results(sc.pp.normalize_total, X, fields=[\"layer\"])\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\"dtype\", [\"float32\", \"int64\"])\ndef test_normalize_total_layers(array_type, dtype):\n adata = AnnData(array_type(X_total).astype(dtype))\n adata.layers[\"layer\"] = adata.X.copy()\n with pytest.warns(FutureWarning, match=r\".*layers.*deprecated\"):\n sc.pp.normalize_total(adata, layers=[\"layer\"])\n assert np.allclose(axis_sum(adata.layers[\"layer\"], axis=1), [3.0, 3.0, 3.0])\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\"dtype\", [\"float32\", \"int64\"])\ndef test_normalize_total_view(array_type, dtype):\n adata = AnnData(array_type(X_total).astype(dtype))\n v = adata[:, :]\n\n sc.pp.normalize_total(v)\n sc.pp.normalize_total(adata)\n\n assert not v.is_view\n assert_equal(adata, v)\n\n\ndef test_normalize_pearson_residuals_warnings(pbmc3k_parametrized):\n adata = pbmc3k_parametrized()\n\n if np.issubdtype(adata.X.dtype, np.integer):\n pytest.skip(\"Can’t store non-integral data with int dtype\")\n\n # depending on check_values, warnings should be raised for non-integer data\n adata_noninteger = adata.copy()\n x, y = np.nonzero(adata_noninteger.X)\n adata_noninteger.X[x[0], y[0]] = 0.5\n\n _check_check_values_warnings(\n function=sc.experimental.pp.normalize_pearson_residuals,\n adata=adata_noninteger,\n expected_warning=\"`normalize_pearson_residuals()` expects raw count data, but non-integers were found.\",\n )\n\n\n@pytest.mark.parametrize(\n (\"params\", \"match\"),\n [\n pytest.param(dict(theta=0), r\"Pearson residuals require theta > 0\", id=\"theta\"),\n pytest.param(\n dict(theta=-1), r\"Pearson residuals require theta > 0\", id=\"theta\"\n ),\n pytest.param(\n dict(clip=-1), r\"Pearson residuals require `clip>=0` or `clip=None`.\"\n ),\n ],\n)\ndef test_normalize_pearson_residuals_errors(pbmc3k_parametrized, params, match):\n adata = pbmc3k_parametrized()\n\n with pytest.raises(ValueError, match=match):\n sc.experimental.pp.normalize_pearson_residuals(adata, **params)\n\n\n@pytest.mark.parametrize(\n \"sparsity_func\", [np.array, csr_matrix], ids=lambda x: x.__name__\n)\n@pytest.mark.parametrize(\"dtype\", [\"float32\", \"int64\"])\n@pytest.mark.parametrize(\"theta\", [0.01, 1, 100, np.inf])\n@pytest.mark.parametrize(\"clip\", [None, 1, np.inf])\ndef test_normalize_pearson_residuals_values(sparsity_func, dtype, theta, clip):\n # toy data\n X = np.array([[3, 6], [2, 4], [1, 0]])\n ns = np.sum(X, axis=1)\n ps = np.sum(X, axis=0) / np.sum(X)\n mu = np.outer(ns, ps)\n\n # compute reference residuals\n if np.isinf(theta):\n # Poisson case\n residuals_reference = (X - mu) / np.sqrt(mu)\n else:\n # NB case\n residuals_reference = (X - mu) / np.sqrt(mu + mu**2 / theta)\n\n # compute output to test\n adata = AnnData(sparsity_func(X).astype(dtype))\n output = sc.experimental.pp.normalize_pearson_residuals(\n adata, theta=theta, clip=clip, inplace=False\n )\n output_X = output[\"X\"]\n sc.experimental.pp.normalize_pearson_residuals(\n adata, theta=theta, clip=clip, inplace=True\n )\n\n # check for correct new `adata.uns` keys\n assert {\"pearson_residuals_normalization\"} <= adata.uns.keys()\n assert {\"theta\", \"clip\", \"computed_on\"} <= adata.uns[\n \"pearson_residuals_normalization\"\n ].keys()\n # test against inplace\n np.testing.assert_array_equal(adata.X, output_X)\n\n if clip is None:\n # default clipping: compare to sqrt(n) threshold\n clipping_threshold = np.sqrt(adata.shape[0]).astype(np.float32)\n assert np.max(output_X) <= clipping_threshold\n assert np.min(output_X) >= -clipping_threshold\n elif np.isinf(clip):\n # no clipping: compare to raw residuals\n assert np.allclose(output_X, residuals_reference)\n else:\n # custom clipping: compare to custom threshold\n assert np.max(output_X) <= clip\n assert np.min(output_X) >= -clip\n\n\ndef _check_pearson_pca_fields(ad, n_cells, n_comps):\n assert {\"pearson_residuals_normalization\", \"pca\"} <= ad.uns.keys(), (\n \"Missing `.uns` keys. Expected `['pearson_residuals_normalization', 'pca']`, \"\n f\"but only {list(ad.uns.keys())} were found\"\n )\n assert (\n \"X_pca\" in ad.obsm\n ), f\"Missing `obsm` key `'X_pca'`, only {list(ad.obsm.keys())} were found\"\n assert (\n \"PCs\" in ad.varm\n ), f\"Missing `varm` key `'PCs'`, only {list(ad.varm.keys())} were found\"\n assert ad.obsm[\"X_pca\"].shape == (\n n_cells,\n n_comps,\n ), \"Wrong shape of PCA output in `X_pca`\"\n\n\n@pytest.mark.parametrize(\"n_hvgs\", [100, 200])\n@pytest.mark.parametrize(\"n_comps\", [30, 50])\n@pytest.mark.parametrize(\n (\"do_hvg\", \"params\", \"n_var_copy_name\"),\n [\n pytest.param(False, dict(), \"n_genes\", id=\"no_hvg\"),\n pytest.param(True, dict(), \"n_hvgs\", id=\"hvg_default\"),\n pytest.param(\n True, dict(use_highly_variable=False), \"n_genes\", id=\"hvg_opt_out\"\n ),\n pytest.param(False, dict(mask_var=\"test_mask\"), \"n_unmasked\", id=\"mask\"),\n ],\n)\ndef test_normalize_pearson_residuals_pca(\n *,\n pbmc3k_parametrized_small: Callable[[], AnnData],\n n_hvgs: int,\n n_comps: int,\n do_hvg: bool,\n params: dict[str, Any],\n n_var_copy_name: str, # number of variables in output if inplace=False\n):\n adata = pbmc3k_parametrized_small()\n n_cells, n_genes = adata.shape\n n_unmasked = n_genes - 5\n adata.var[\"test_mask\"] = np.r_[\n np.repeat(True, n_unmasked), np.repeat(False, n_genes - n_unmasked) # noqa: FBT003\n ]\n n_var_copy = locals()[n_var_copy_name]\n assert isinstance(n_var_copy, (int, np.integer))\n\n if do_hvg:\n sc.experimental.pp.highly_variable_genes(\n adata, flavor=\"pearson_residuals\", n_top_genes=n_hvgs\n )\n\n # inplace=False\n adata_pca = sc.experimental.pp.normalize_pearson_residuals_pca(\n adata.copy(), inplace=False, n_comps=n_comps, **params\n )\n # inplace=True modifies the input adata object\n sc.experimental.pp.normalize_pearson_residuals_pca(\n adata, inplace=True, n_comps=n_comps, **params\n )\n\n for ad, n_var_ret in (\n (adata_pca, n_var_copy),\n # inplace adatas should always retains original shape\n (adata, n_genes),\n ):\n _check_pearson_pca_fields(ad, n_cells, n_comps)\n\n # check adata shape to see if all genes or only HVGs are in the returned adata\n assert ad.shape == (n_cells, n_var_ret)\n\n # check PC shapes to see whether or not HVGs were used for PCA\n assert ad.varm[\"PCs\"].shape == (n_var_ret, n_comps)\n\n # check if there are columns of all-zeros in the PCs shapes\n # to see whether or not HVGs were used for PCA\n # either no all-zero-colums or all number corresponding to non-hvgs should exist\n assert sum(np.sum(np.abs(adata.varm[\"PCs\"]), axis=1) == 0) == (n_genes - n_var_copy)\n\n # compare PCA results beteen inplace / copied\n np.testing.assert_array_equal(adata.obsm[\"X_pca\"], adata_pca.obsm[\"X_pca\"])\n\n\n@pytest.mark.parametrize(\"n_hvgs\", [100, 200])\n@pytest.mark.parametrize(\"n_comps\", [30, 50])\ndef test_normalize_pearson_residuals_recipe(pbmc3k_parametrized_small, n_hvgs, n_comps):\n adata = pbmc3k_parametrized_small()\n n_cells, n_genes = adata.shape\n\n ### inplace = False ###\n # outputs the (potentially hvg-restricted) adata_pca object\n # PCA on all genes\n adata_pca, hvg = sc.experimental.pp.recipe_pearson_residuals(\n adata.copy(), inplace=False, n_comps=n_comps, n_top_genes=n_hvgs\n )\n\n # check PCA fields\n _check_pearson_pca_fields(adata_pca, n_cells, n_comps)\n # check adata output shape (only HVGs in output)\n assert adata_pca.shape == (n_cells, n_hvgs)\n # check PC shape (non-hvgs are removed, so only `n_hvgs` genes)\n assert adata_pca.varm[\"PCs\"].shape == (n_hvgs, n_comps)\n\n # check hvg df\n assert {\n \"means\",\n \"variances\",\n \"residual_variances\",\n \"highly_variable_rank\",\n \"highly_variable\",\n } <= set(hvg.columns)\n assert np.sum(hvg[\"highly_variable\"]) == n_hvgs\n assert hvg.shape[0] == n_genes\n\n ### inplace = True ###\n # modifies the input adata object\n # PCA on all genes\n sc.experimental.pp.recipe_pearson_residuals(\n adata, inplace=True, n_comps=n_comps, n_top_genes=n_hvgs\n )\n\n # check PCA fields and output shape\n _check_pearson_pca_fields(adata, n_cells, n_comps)\n # check adata shape (no change to input)\n assert adata.shape == (n_cells, n_genes)\n # check PC shape (non-hvgs are masked with 0s, so original number of genes)\n assert adata.varm[\"PCs\"].shape == (n_genes, n_comps)\n # number of all-zero-colums should be number of non-hvgs\n assert sum(np.sum(np.abs(adata.varm[\"PCs\"]), axis=1) == 0) == n_genes - n_hvgs\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nimport pytest\nimport scipy.sparse as sparse\nfrom anndata import AnnData, concat\nfrom anndata.tests.helpers import assert_equal\nfrom numpy.testing import assert_allclose, assert_array_equal\n\nimport scanpy as sc\nfrom testing.scanpy._pytest.marks import needs\n\nif TYPE_CHECKING:\n from collections.abc import Callable\n from typing import Any\n\npytestmark = [needs.skimage]\n\n\ndef pbmc200() -> AnnData:\n from testing.scanpy._helpers.data import _pbmc3k\n\n return _pbmc3k()[200:400].copy()\n\n\ndef paul500() -> AnnData:\n from testing.scanpy._helpers.data import _paul15\n\n return _paul15()[:500].copy()\n\n\n@pytest.mark.parametrize(\n (\"mk_data\", \"expected_idx\", \"expected_scores\"),\n [\n pytest.param(pbmc200, [13, 138], [0.149254] * 2, id=\"sparse\"),\n pytest.param(paul500, [180], [0.219178], id=\"dense\"),\n ],\n)\n@pytest.mark.parametrize(\"use_approx_neighbors\", [True, False, None])\ndef test_scrublet(\n mk_data: Callable[[], AnnData],\n expected_idx: list[int],\n expected_scores: list[float],\n use_approx_neighbors: bool | None,\n):\n \"\"\"Check that scrublet runs and detects some doublets.\"\"\"\n adata = mk_data()\n sc.pp.scrublet(adata, use_approx_neighbors=use_approx_neighbors)\n\n doublet_idx = np.flatnonzero(adata.obs[\"predicted_doublet\"]).tolist()\n assert doublet_idx == expected_idx\n assert_allclose(\n adata.obs[\"doublet_score\"].iloc[doublet_idx],\n expected_scores,\n atol=1e-5,\n rtol=1e-5,\n )\n\n\ndef test_scrublet_batched():\n \"\"\"Test that Scrublet run works with batched data.\"\"\"\n adata = pbmc200()\n adata.obs[\"batch\"] = 100 * [\"a\"] + 100 * [\"b\"]\n split = [adata[adata.obs[\"batch\"] == x].copy() for x in (\"a\", \"b\")]\n\n sc.pp.scrublet(adata, use_approx_neighbors=False, batch_key=\"batch\")\n\n doublet_idx = np.flatnonzero(adata.obs[\"predicted_doublet\"]).tolist()\n # only one in the first batch (<100)\n assert doublet_idx == [0, 2, 8, 15, 43, 88, 108, 113, 115, 132, 135, 175]\n assert_allclose(\n adata.obs[\"doublet_score\"].iloc[doublet_idx],\n np.array([0.109375, 0.164835])[([0] * 4 + [1] + [0] * 3 + [1] + [0] * 3)],\n atol=1e-5,\n rtol=1e-5,\n )\n assert adata.uns[\"scrublet\"][\"batches\"].keys() == {\"a\", \"b\"}\n\n # Check that results are independent\n for s in split:\n sc.pp.scrublet(s, use_approx_neighbors=False)\n merged = concat(split)\n\n pd.testing.assert_frame_equal(adata.obs[merged.obs.columns], merged.obs)\n\n\ndef _preprocess_for_scrublet(adata: AnnData) -> AnnData:\n adata_pp = adata.copy()\n sc.pp.filter_genes(adata_pp, min_cells=3)\n sc.pp.filter_cells(adata_pp, min_genes=3)\n adata_pp.layers[\"raw\"] = adata_pp.X.copy()\n sc.pp.normalize_total(adata_pp)\n logged = sc.pp.log1p(adata_pp, copy=True)\n sc.pp.highly_variable_genes(logged)\n return adata_pp[:, logged.var[\"highly_variable\"]].copy()\n\n\ndef _create_sim_from_parents(adata: AnnData, parents: np.ndarray) -> AnnData:\n \"\"\"Simulate doublets based on the randomly selected parents used previously.\"\"\"\n n_sim = parents.shape[0]\n I = sparse.coo_matrix(\n (\n np.ones(2 * n_sim),\n (np.repeat(np.arange(n_sim), 2), parents.flat),\n ),\n (n_sim, adata.n_obs),\n )\n X = I @ adata.layers[\"raw\"]\n return AnnData(\n X,\n var=pd.DataFrame(index=adata.var_names),\n obs={\"total_counts\": np.ravel(X.sum(axis=1))},\n obsm={\"doublet_parents\": parents.copy()},\n )\n\n\ndef test_scrublet_data(cache: pytest.Cache):\n \"\"\"\n Test that Scrublet processing is arranged correctly.\n\n Check that simulations run on raw data.\n \"\"\"\n random_state = 1234\n\n # Run Scrublet and let the main function run simulations\n adata_scrublet_auto_sim = sc.pp.scrublet(\n pbmc200(),\n use_approx_neighbors=False,\n copy=True,\n random_state=random_state,\n )\n\n # Now make our own simulated data so we can check the result from function\n # is the same, and by inference that the processing steps have not been\n # broken\n\n # Replicate the preprocessing steps used by the main function\n adata_obs = _preprocess_for_scrublet(pbmc200())\n # Simulate doublets using the same parents\n adata_sim = _create_sim_from_parents(\n adata_obs, adata_scrublet_auto_sim.uns[\"scrublet\"][\"doublet_parents\"]\n )\n\n # Apply the same post-normalisation the Scrublet function would\n sc.pp.normalize_total(adata_obs, target_sum=1e6)\n sc.pp.normalize_total(adata_sim, target_sum=1e6)\n\n adata_scrublet_manual_sim = sc.pp.scrublet(\n adata_obs,\n adata_sim=adata_sim,\n use_approx_neighbors=False,\n copy=True,\n random_state=random_state,\n )\n\n try:\n # Require that the doublet scores are the same whether simulation is via\n # the main function or manually provided\n assert_allclose(\n adata_scrublet_manual_sim.obs[\"doublet_score\"],\n adata_scrublet_auto_sim.obs[\"doublet_score\"],\n atol=1e-15,\n rtol=1e-15,\n )\n except AssertionError:\n import zarr\n\n # try debugging https://github.com/scverse/scanpy/issues/3068\n cache_path = cache.mkdir(\"debug\")\n store_manual = zarr.ZipStore(cache_path / \"scrublet-manual.zip\", mode=\"w\")\n store_auto = zarr.ZipStore(cache_path / \"scrublet-auto.zip\", mode=\"w\")\n z_manual = zarr.zeros(\n adata_scrublet_manual_sim.shape[0], chunks=10, store=store_manual\n )\n z_auto = zarr.zeros(\n adata_scrublet_auto_sim.shape[0], chunks=10, store=store_auto\n )\n z_manual[...] = adata_scrublet_manual_sim.obs[\"doublet_score\"].values\n z_auto[...] = adata_scrublet_auto_sim.obs[\"doublet_score\"].values\n\n raise\n\n\n@pytest.fixture(scope=\"module\")\ndef scrub_small_sess() -> AnnData:\n # Reduce size of input for faster test\n adata = pbmc200()\n sc.pp.filter_genes(adata, min_counts=100)\n\n sc.pp.scrublet(adata, use_approx_neighbors=False)\n return adata\n\n\n@pytest.fixture\ndef scrub_small(scrub_small_sess: AnnData):\n return scrub_small_sess.copy()\n\n\ntest_params = {\n \"expected_doublet_rate\": 0.1,\n \"synthetic_doublet_umi_subsampling\": 0.8,\n \"knn_dist_metric\": \"manhattan\",\n \"normalize_variance\": False,\n \"log_transform\": True,\n \"mean_center\": False,\n \"n_prin_comps\": 10,\n \"n_neighbors\": 2,\n \"threshold\": 0.1,\n}\n\n\n@pytest.mark.parametrize((\"param\", \"value\"), test_params.items())\ndef test_scrublet_params(scrub_small: AnnData, param: str, value: Any):\n \"\"\"\n Test that Scrublet args are passed.\n\n Check that changes to parameters change scrublet results.\n \"\"\"\n curr = sc.pp.scrublet(\n adata=scrub_small, use_approx_neighbors=False, copy=True, **{param: value}\n )\n with pytest.raises(AssertionError):\n assert_equal(scrub_small, curr)\n\n\ndef test_scrublet_simulate_doublets():\n \"\"\"Check that doublet simulation runs and simulates some doublets.\"\"\"\n adata_obs = pbmc200()\n sc.pp.filter_genes(adata_obs, min_cells=3)\n sc.pp.filter_cells(adata_obs, min_genes=3)\n adata_obs.layers[\"raw\"] = adata_obs.X\n sc.pp.normalize_total(adata_obs)\n logged = sc.pp.log1p(adata_obs, copy=True)\n\n _ = sc.pp.highly_variable_genes(logged)\n adata_obs = adata_obs[:, logged.var[\"highly_variable\"]]\n\n adata_sim = sc.pp.scrublet_simulate_doublets(\n adata_obs, sim_doublet_ratio=0.02, layer=\"raw\"\n )\n\n assert_array_equal(\n adata_sim.obsm[\"doublet_parents\"],\n np.array([[13, 132], [106, 43], [152, 3], [160, 103]]),\n )\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pytest\nfrom sklearn.neighbors import KNeighborsTransformer\n\nfrom scanpy._utils.compute.is_constant import is_constant\nfrom scanpy.neighbors._common import (\n _get_sparse_matrix_from_indices_distances,\n _has_self_column,\n _ind_dist_shortcut,\n)\n\nif TYPE_CHECKING:\n from collections.abc import Callable\n from typing import Literal\n\n from scipy import sparse\n\n\ndef mk_knn_matrix(\n n_obs: int,\n n_neighbors: int,\n *,\n style: Literal[\"basic\", \"rapids\", \"sklearn\"],\n duplicates: bool = False,\n) -> sparse.csr_matrix:\n n_col = n_neighbors + (1 if style == \"sklearn\" else 0)\n dists = np.abs(np.random.randn(n_obs, n_col)) + 1e-8\n idxs = np.arange(n_obs * n_col).reshape((n_col, n_obs)).T\n if style == \"rapids\":\n idxs[:, 0] += 1 # does not include cell itself\n else:\n dists[:, 0] = 0.0 # includes cell itself\n if duplicates:\n # Don’t use the first column, as that might be the cell itself\n dists[n_obs // 4 : n_obs, 2] = 0.0\n # keep self column to simulate output from kNN transformers\n mat = _get_sparse_matrix_from_indices_distances(idxs, dists, keep_self=True)\n\n # check if out helper here works as expected\n assert _has_self_column(idxs, dists) == (style != \"rapids\")\n if duplicates:\n # Make sure the actual matrix has a regular sparsity pattern\n assert is_constant(mat.getnnz(axis=1))\n # Make sure implicit zeros for duplicates would change the sparsity pattern\n mat_sparsified = mat.copy()\n mat_sparsified.eliminate_zeros()\n assert not is_constant(mat_sparsified.getnnz(axis=1))\n\n return mat\n\n\n@pytest.mark.parametrize(\"n_neighbors\", [3, pytest.param(None, id=\"all\")])\n@pytest.mark.parametrize(\"style\", [\"basic\", \"rapids\", \"sklearn\"])\n@pytest.mark.parametrize(\"duplicates\", [True, False], ids=[\"duplicates\", \"unique\"])\ndef test_ind_dist_shortcut_manual(\n *,\n n_neighbors: int | None,\n style: Literal[\"basic\", \"rapids\", \"sklearn\"],\n duplicates: bool,\n):\n n_obs = 10\n if n_neighbors is None:\n n_neighbors = n_obs\n mat = mk_knn_matrix(n_obs, n_neighbors, style=style, duplicates=duplicates)\n\n assert (mat.nnz / n_obs) == n_neighbors + (1 if style == \"sklearn\" else 0)\n assert _ind_dist_shortcut(mat) is not None\n\n\n@pytest.mark.parametrize(\"n_neighbors\", [3, pytest.param(None, id=\"all\")])\n@pytest.mark.parametrize(\n \"mk_mat\",\n [\n pytest.param(\n lambda n_obs, n_neighbors: KNeighborsTransformer(\n n_neighbors=n_neighbors\n ).fit_transform(np.random.randn(n_obs, n_obs // 4)),\n id=\"sklearn_auto\",\n )\n ],\n)\ndef test_ind_dist_shortcut_premade(\n n_neighbors: int | None, mk_mat: Callable[[int, int], sparse.csr_matrix]\n):\n n_obs = 10\n if n_neighbors is None:\n # KNeighborsTransformer interprets this as “number of neighbors excluding cell itself”\n # so it can be at most n_obs - 1\n n_neighbors = n_obs - 1\n mat = mk_mat(n_obs, n_neighbors)\n\n assert (mat.nnz / n_obs) == n_neighbors + 1\n assert _ind_dist_shortcut(mat) is not None\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nfrom anndata import AnnData\n\nimport scanpy as sc\n\n\ndef generate_test_data():\n # Create an artificial data set\n test_data = AnnData(X=np.ones((9, 10)))\n test_data.uns[\"rank_genes_groups\"] = dict()\n test_data.uns[\"rank_genes_groups\"][\"names\"] = np.rec.fromarrays(\n [[\"a\", \"b\", \"c\", \"d\", \"e\"], [\"a\", \"f\", \"g\", \"h\", \"i\"]], names=\"c0,c1\"\n )\n test_data.uns[\"rank_genes_groups\"][\"pvals_adj\"] = np.rec.fromarrays(\n [[0.001, 0.01, 0.02, 0.05, 0.6], [0.001, 0.01, 0.02, 0.05, 0.6]], names=\"c0,c1\"\n )\n\n marker_genes = {\"type 1\": {\"a\", \"b\", \"c\"}, \"type 2\": {\"a\", \"f\", \"g\"}}\n\n return test_data, marker_genes\n\n\ndef test_marker_overlap_base():\n # Test all overlap calculations on artificial data\n test_data, marker_genes = generate_test_data()\n\n t1 = sc.tl.marker_gene_overlap(test_data, marker_genes)\n\n assert t1[\"c0\"][\"type 1\"] == 3.0\n assert t1[\"c1\"][\"type 2\"] == 3.0\n\n\ndef test_marker_overlap_normalization():\n test_data, marker_genes = generate_test_data()\n\n t2 = sc.tl.marker_gene_overlap(test_data, marker_genes, normalize=\"reference\")\n t3 = sc.tl.marker_gene_overlap(test_data, marker_genes, normalize=\"data\")\n\n assert t2[\"c0\"][\"type 1\"] == 1.0\n assert t3[\"c1\"][\"type 2\"] == 0.6\n\n\ndef test_marker_overlap_methods():\n test_data, marker_genes = generate_test_data()\n\n t4 = sc.tl.marker_gene_overlap(test_data, marker_genes, method=\"overlap_coef\")\n t5 = sc.tl.marker_gene_overlap(test_data, marker_genes, method=\"jaccard\")\n\n assert t4[\"c0\"][\"type 1\"] == 1.0\n assert t5[\"c0\"][\"type 1\"] == 0.6\n\n\ndef test_marker_overlap_subsetting():\n test_data, marker_genes = generate_test_data()\n\n t6 = sc.tl.marker_gene_overlap(test_data, marker_genes, top_n_markers=2)\n t7 = sc.tl.marker_gene_overlap(test_data, marker_genes, adj_pval_threshold=0.01)\n\n assert t6[\"c0\"][\"type 1\"] == 2.0\n assert t7[\"c0\"][\"type 1\"] == 1.0\n\n\nfrom __future__ import annotations\n\nfrom functools import partial\nfrom pathlib import Path\n\nimport numpy as np\nimport pytest\nfrom matplotlib import cm\n\nimport scanpy as sc\nfrom testing.scanpy._helpers.data import pbmc3k_processed, pbmc68k_reduced\nfrom testing.scanpy._pytest.marks import needs\n\nHERE: Path = Path(__file__).parent\nROOT = HERE / \"_images\"\n\n\npytestmark = [needs.igraph]\n\n\n@pytest.fixture(scope=\"module\")\ndef pbmc_session():\n pbmc = pbmc68k_reduced()\n sc.tl.paga(pbmc, groups=\"bulk_labels\")\n pbmc.obs[\"cool_feature\"] = pbmc[:, \"CST3\"].X.squeeze().copy()\n assert not pbmc.obs[\"cool_feature\"].isna().all()\n return pbmc\n\n\n@pytest.fixture\ndef pbmc(pbmc_session):\n return pbmc_session.copy()\n\n\n@pytest.mark.parametrize(\n (\"test_id\", \"func\"),\n [\n (\"\", sc.pl.paga),\n (\"continuous\", partial(sc.pl.paga, color=\"CST3\")),\n (\"continuous_obs\", partial(sc.pl.paga, color=\"cool_feature\")),\n (\"continuous_multiple\", partial(sc.pl.paga, color=[\"CST3\", \"GATA2\"])),\n (\"compare\", partial(sc.pl.paga_compare, legend_fontoutline=2)),\n pytest.param(\n \"compare_continuous\",\n partial(sc.pl.paga_compare, color=\"CST3\", legend_fontsize=5),\n marks=pytest.mark.xfail(reason=\"expects .uns['paga']['pos']\"),\n ),\n (\n \"compare_pca\",\n partial(sc.pl.paga_compare, basis=\"X_pca\", legend_fontweight=\"normal\"),\n ),\n ],\n)\ndef test_paga_plots(image_comparer, pbmc, test_id, func):\n save_and_compare_images = partial(image_comparer, ROOT, tol=30)\n\n common = dict(threshold=0.5, max_edge_width=1.0, random_state=0, show=False)\n\n func(pbmc, **common)\n save_and_compare_images(f\"paga_{test_id}\" if test_id else \"paga\")\n\n\ndef test_paga_pie(image_comparer, pbmc):\n save_and_compare_images = partial(image_comparer, ROOT, tol=30)\n\n colors = {\n c: {cm.Set1(_): 0.33 for _ in range(3)}\n for c in pbmc.obs[\"bulk_labels\"].cat.categories\n }\n colors[\"Dendritic\"] = {cm.Set2(_): 0.25 for _ in range(4)}\n\n sc.pl.paga(pbmc, color=colors, colorbar=False)\n save_and_compare_images(\"paga_pie\")\n\n\ndef test_paga_path(image_comparer, pbmc):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n pbmc.uns[\"iroot\"] = 0\n sc.tl.dpt(pbmc)\n sc.pl.paga_path(\n pbmc,\n nodes=[\"Dendritic\"],\n keys=[\"HES4\", \"SRM\", \"CSTB\"],\n show=False,\n )\n save_and_compare_images(\"paga_path\")\n\n\ndef test_paga_compare(image_comparer):\n # Tests that https://github.com/scverse/scanpy/issues/1887 is fixed\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n pbmc = pbmc3k_processed()\n sc.tl.paga(pbmc, groups=\"louvain\")\n\n sc.pl.paga_compare(pbmc, basis=\"umap\", show=False)\n\n save_and_compare_images(\"paga_compare_pbmc3k\")\n\n\ndef test_paga_positions_reproducible():\n \"\"\"Check exact reproducibility and effect of random_state on paga positions\"\"\"\n # https://github.com/scverse/scanpy/issues/1859\n pbmc = pbmc68k_reduced()\n sc.tl.paga(pbmc, \"bulk_labels\")\n\n a = pbmc.copy()\n b = pbmc.copy()\n c = pbmc.copy()\n\n sc.pl.paga(a, show=False, random_state=42)\n sc.pl.paga(b, show=False, random_state=42)\n sc.pl.paga(c, show=False, random_state=13)\n\n np.testing.assert_array_equal(a.uns[\"paga\"][\"pos\"], b.uns[\"paga\"][\"pos\"])\n assert a.uns[\"paga\"][\"pos\"].tolist() != c.uns[\"paga\"][\"pos\"].tolist()\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom functools import partial\nfrom operator import eq\nfrom string import ascii_letters\n\nimport numba\nimport numpy as np\nimport pandas as pd\nimport pytest\nimport threadpoolctl\nfrom packaging.version import Version\nfrom scipy import sparse\n\nimport scanpy as sc\nfrom scanpy._compat import DaskArray\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\nfrom testing.scanpy._pytest.params import ARRAY_TYPES\n\nmark_flaky = pytest.mark.xfail(\n strict=False,\n reason=\"This used to work reliably, but doesn’t anymore\",\n)\n\n\n@pytest.fixture(scope=\"session\", params=[sc.metrics.gearys_c, sc.metrics.morans_i])\ndef metric(request: pytest.FixtureRequest):\n return request.param\n\n\n@pytest.fixture(\n scope=\"session\",\n params=[\n # pytest.param(eq, marks=[mark_flaky]),\n pytest.param(eq),\n pytest.param(partial(np.testing.assert_allclose, rtol=1e-14), id=\"allclose\"),\n ],\n)\ndef assert_equal(request: pytest.FixtureRequest):\n return request.param\n\n\n@pytest.fixture(params=[\"single-threaded\", \"multi-threaded\"])\ndef threading(request):\n if request.param == \"single-threaded\":\n with threadpoolctl.threadpool_limits(limits=1):\n yield None\n elif request.param == \"multi-threaded\":\n yield None\n\n\ndef test_consistency(metric, threading):\n pbmc = pbmc68k_reduced()\n pbmc.layers[\"raw\"] = pbmc.raw.X.copy()\n g = pbmc.obsp[\"connectivities\"]\n equality_check = partial(np.testing.assert_allclose, atol=1e-11)\n\n # This can fail\n equality_check(\n metric(g, pbmc.obs[\"percent_mito\"]),\n metric(g, pbmc.obs[\"percent_mito\"]),\n )\n equality_check(\n metric(g, pbmc.obs[\"percent_mito\"]),\n metric(pbmc, vals=pbmc.obs[\"percent_mito\"]),\n )\n\n equality_check( # Test that series and vectors return same value\n metric(g, pbmc.obs[\"percent_mito\"]),\n metric(g, pbmc.obs[\"percent_mito\"].values),\n )\n\n equality_check(\n metric(pbmc, obsm=\"X_pca\"),\n metric(g, pbmc.obsm[\"X_pca\"].T),\n )\n\n all_genes = metric(pbmc, layer=\"raw\")\n first_gene = metric(pbmc, vals=pbmc.obs_vector(pbmc.var_names[0], layer=\"raw\"))\n\n if Version(numba.__version__) < Version(\"0.57\"):\n np.testing.assert_allclose(all_genes[0], first_gene, rtol=1e-5)\n else:\n np.testing.assert_allclose(all_genes[0], first_gene, rtol=1e-9)\n\n # Test that results are similar for sparse and dense reps of same data\n equality_check(\n metric(pbmc, layer=\"raw\"),\n metric(pbmc, vals=pbmc.layers[\"raw\"].T.toarray()),\n )\n\n\n@pytest.mark.parametrize(\n (\"metric\", \"size\", \"expected\"),\n [\n pytest.param(sc.metrics.gearys_c, 30, 0.0, id=\"gearys_c\"),\n pytest.param(sc.metrics.morans_i, 50, 1.0, id=\"morans_i\"),\n ],\n)\ndef test_correctness(metric, size, expected):\n # Test case with perfectly seperated groups\n connected = np.zeros(100)\n connected[np.random.choice(100, size=size, replace=False)] = 1\n graph = np.zeros((100, 100))\n graph[np.ix_(connected.astype(bool), connected.astype(bool))] = 1\n graph[np.ix_(~connected.astype(bool), ~connected.astype(bool))] = 1\n graph = sparse.csr_matrix(graph)\n\n np.testing.assert_equal(metric(graph, connected), expected)\n np.testing.assert_equal(\n metric(graph, connected),\n metric(graph, sparse.csr_matrix(connected)),\n )\n # Checking that obsp works\n adata = sc.AnnData(sparse.csr_matrix((100, 100)), obsp={\"connectivities\": graph})\n np.testing.assert_equal(metric(adata, vals=connected), expected)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\ndef test_graph_metrics_w_constant_values(metric, array_type, threading):\n # https://github.com/scverse/scanpy/issues/1806\n pbmc = pbmc68k_reduced()\n XT = array_type(pbmc.raw.X.T.copy())\n g = pbmc.obsp[\"connectivities\"].copy()\n equality_check = partial(np.testing.assert_allclose, atol=1e-11)\n\n if isinstance(XT, DaskArray):\n pytest.skip(\"DaskArray yet not supported\")\n\n const_inds = np.random.choice(XT.shape[0], 10, replace=False)\n with warnings.catch_warnings():\n warnings.simplefilter(\"ignore\", sparse.SparseEfficiencyWarning)\n XT_zero_vals = XT.copy()\n XT_zero_vals[const_inds, :] = 0\n XT_const_vals = XT.copy()\n XT_const_vals[const_inds, :] = 42\n\n results_full = metric(g, XT)\n # TODO: Check for warnings\n with pytest.warns(\n UserWarning, match=r\"10 variables were constant, will return nan for these\"\n ):\n results_const_zeros = metric(g, XT_zero_vals)\n with pytest.warns(\n UserWarning, match=r\"10 variables were constant, will return nan for these\"\n ):\n results_const_vals = metric(g, XT_const_vals)\n\n assert not np.isnan(results_full).any()\n equality_check(results_const_zeros, results_const_vals)\n np.testing.assert_array_equal(np.nan, results_const_zeros[const_inds])\n np.testing.assert_array_equal(np.nan, results_const_vals[const_inds])\n\n non_const_mask = ~np.isin(np.arange(XT.shape[0]), const_inds)\n equality_check(results_full[non_const_mask], results_const_zeros[non_const_mask])\n\n\ndef test_confusion_matrix():\n mtx = sc.metrics.confusion_matrix([\"a\", \"b\"], [\"c\", \"d\"], normalize=False)\n assert mtx.loc[\"a\", \"c\"] == 1\n assert mtx.loc[\"a\", \"d\"] == 0\n assert mtx.loc[\"b\", \"d\"] == 1\n assert mtx.loc[\"b\", \"c\"] == 0\n\n mtx = sc.metrics.confusion_matrix([\"a\", \"b\"], [\"c\", \"d\"], normalize=True)\n assert mtx.loc[\"a\", \"c\"] == 1.0\n assert mtx.loc[\"a\", \"d\"] == 0.0\n assert mtx.loc[\"b\", \"d\"] == 1.0\n assert mtx.loc[\"b\", \"c\"] == 0.0\n\n mtx = sc.metrics.confusion_matrix(\n [\"a\", \"a\", \"b\", \"b\"], [\"c\", \"d\", \"c\", \"d\"], normalize=True\n )\n assert np.all(mtx == 0.5)\n\n\ndef test_confusion_matrix_randomized():\n chars = np.array(list(ascii_letters))\n pos = np.random.choice(len(chars), size=np.random.randint(50, 150))\n a = chars[pos]\n b = np.random.permutation(chars)[pos]\n df = pd.DataFrame({\"a\": a, \"b\": b})\n\n pd.testing.assert_frame_equal(\n sc.metrics.confusion_matrix(\"a\", \"b\", df),\n sc.metrics.confusion_matrix(df[\"a\"], df[\"b\"]),\n )\n pd.testing.assert_frame_equal(\n sc.metrics.confusion_matrix(df[\"a\"].values, df[\"b\"].values),\n sc.metrics.confusion_matrix(a, b),\n )\n\n\ndef test_confusion_matrix_api():\n data = pd.DataFrame(\n {\"a\": np.random.randint(5, size=100), \"b\": np.random.randint(5, size=100)}\n )\n expected = sc.metrics.confusion_matrix(data[\"a\"], data[\"b\"])\n\n pd.testing.assert_frame_equal(expected, sc.metrics.confusion_matrix(\"a\", \"b\", data))\n\n pd.testing.assert_frame_equal(\n expected, sc.metrics.confusion_matrix(\"a\", data[\"b\"], data)\n )\n\n pd.testing.assert_frame_equal(\n expected, sc.metrics.confusion_matrix(data[\"a\"], \"b\", data)\n )\n\n\nfrom __future__ import annotations\n\nimport anndata\nimport numpy as np\nimport pytest\nfrom sklearn.neighbors import KDTree\nfrom umap import UMAP\n\nimport scanpy as sc\nfrom scanpy import settings\nfrom scanpy._compat import pkg_version\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\n\nX = np.array(\n [\n [1.0, 2.5, 3.0, 5.0, 8.7],\n [4.2, 7.0, 9.0, 11.0, 7.0],\n [5.1, 2.0, 9.0, 4.0, 9.0],\n [7.0, 9.4, 6.8, 9.1, 8.0],\n [8.9, 8.6, 9.6, 1.0, 2.0],\n [6.5, 8.9, 2.2, 4.5, 8.9],\n ],\n dtype=np.float32,\n)\n\nT = np.array([[2.0, 3.5, 4.0, 1.0, 4.7], [3.2, 2.0, 5.0, 5.0, 8.0]], dtype=np.float32)\n\n\n@pytest.fixture\ndef adatas():\n pbmc = pbmc68k_reduced()\n n_split = 500\n adata_ref = sc.AnnData(pbmc.X[:n_split, :], obs=pbmc.obs.iloc[:n_split])\n adata_new = sc.AnnData(pbmc.X[n_split:, :])\n\n sc.pp.pca(adata_ref)\n sc.pp.neighbors(adata_ref)\n sc.tl.umap(adata_ref)\n\n return adata_ref, adata_new\n\n\ndef test_representation(adatas):\n adata_ref = adatas[0].copy()\n adata_new = adatas[1].copy()\n\n ing = sc.tl.Ingest(adata_ref)\n ing.fit(adata_new)\n\n assert ing._use_rep == \"X_pca\"\n assert ing._obsm[\"rep\"].shape == (adata_new.n_obs, settings.N_PCS)\n assert ing._pca_centered\n\n sc.pp.pca(adata_ref, n_comps=30, zero_center=False)\n sc.pp.neighbors(adata_ref)\n\n ing = sc.tl.Ingest(adata_ref)\n ing.fit(adata_new)\n\n assert ing._use_rep == \"X_pca\"\n assert ing._obsm[\"rep\"].shape == (adata_new.n_obs, 30)\n assert not ing._pca_centered\n\n sc.pp.neighbors(adata_ref, use_rep=\"X\")\n\n ing = sc.tl.Ingest(adata_ref)\n ing.fit(adata_new)\n\n assert ing._use_rep == \"X\"\n assert ing._obsm[\"rep\"] is adata_new.X\n\n\ndef test_neighbors(adatas):\n adata_ref = adatas[0].copy()\n adata_new = adatas[1].copy()\n\n ing = sc.tl.Ingest(adata_ref)\n ing.fit(adata_new)\n ing.neighbors(k=10)\n indices = ing._indices\n\n tree = KDTree(adata_ref.obsm[\"X_pca\"])\n true_indices = tree.query(ing._obsm[\"rep\"], 10, return_distance=False)\n\n num_correct = 0.0\n for i in range(adata_new.n_obs):\n num_correct += np.sum(np.in1d(true_indices[i], indices[i]))\n percent_correct = num_correct / (adata_new.n_obs * 10)\n\n assert percent_correct > 0.99\n\n\n@pytest.mark.parametrize(\"n\", [3, 4])\ndef test_neighbors_defaults(adatas, n):\n adata_ref = adatas[0].copy()\n adata_new = adatas[1].copy()\n\n sc.pp.neighbors(adata_ref, n_neighbors=n)\n\n ing = sc.tl.Ingest(adata_ref)\n ing.fit(adata_new)\n ing.neighbors()\n assert ing._indices.shape[1] == n\n\n\n@pytest.mark.skipif(\n pkg_version(\"anndata\") < sc.tl._ingest.ANNDATA_MIN_VERSION,\n reason=\"`AnnData.concatenate` does not concatenate `.obsm` in old anndata versions\",\n)\ndef test_ingest_function(adatas):\n adata_ref = adatas[0].copy()\n adata_new = adatas[1].copy()\n\n sc.tl.ingest(\n adata_new,\n adata_ref,\n obs=\"bulk_labels\",\n embedding_method=[\"umap\", \"pca\"],\n inplace=True,\n )\n\n assert \"bulk_labels\" in adata_new.obs\n assert \"X_umap\" in adata_new.obsm\n assert \"X_pca\" in adata_new.obsm\n\n ad = sc.tl.ingest(\n adata_new,\n adata_ref,\n obs=\"bulk_labels\",\n embedding_method=[\"umap\", \"pca\"],\n inplace=False,\n )\n\n assert \"bulk_labels\" in ad.obs\n assert \"X_umap\" in ad.obsm\n assert \"X_pca\" in ad.obsm\n\n\ndef test_ingest_map_embedding_umap():\n adata_ref = sc.AnnData(X)\n adata_new = sc.AnnData(T)\n\n sc.pp.neighbors(\n adata_ref, method=\"umap\", use_rep=\"X\", n_neighbors=4, random_state=0\n )\n sc.tl.umap(adata_ref, random_state=0)\n\n ing = sc.tl.Ingest(adata_ref)\n ing.fit(adata_new)\n ing.map_embedding(method=\"umap\")\n\n reducer = UMAP(min_dist=0.5, random_state=0, n_neighbors=4)\n reducer.fit(X)\n umap_transformed_t = reducer.transform(T)\n\n assert np.allclose(ing._obsm[\"X_umap\"], umap_transformed_t)\n\n\ndef test_ingest_backed(adatas, tmp_path):\n adata_ref = adatas[0].copy()\n adata_new = adatas[1].copy()\n\n adata_new.write_h5ad(f\"{tmp_path}/new.h5ad\")\n\n adata_new = anndata.read_h5ad(f\"{tmp_path}/new.h5ad\", backed=\"r\")\n\n ing = sc.tl.Ingest(adata_ref)\n with pytest.raises(\n NotImplementedError,\n match=f\"Ingest.fit is not implemented for matrices of type {type(adata_new.X)}\",\n ):\n ing.fit(adata_new)\n\n\nfrom __future__ import annotations\n\nimport os\nfrom collections import defaultdict\nfrom inspect import Parameter, signature\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING, TypedDict\n\nimport pytest\nfrom anndata import AnnData\n\n# CLI is locally not imported by default but on travis it is?\nimport scanpy.cli\nfrom scanpy._utils import _import_name, descend_classes_and_funcs\n\nif TYPE_CHECKING:\n from types import FunctionType\n from typing import Any\n\nmod_dir = Path(scanpy.__file__).parent\nproj_dir = mod_dir.parent\n\n\napi_module_names = [\n \"sc\",\n \"sc.pp\",\n \"sc.tl\",\n \"sc.pl\",\n \"sc.experimental.pp\",\n \"sc.external.pp\",\n \"sc.external.tl\",\n \"sc.external.pl\",\n \"sc.external.exporting\",\n \"sc.get\",\n \"sc.logging\",\n # \"sc.neighbors\", # Not documented\n \"sc.datasets\",\n \"sc.queries\",\n \"sc.metrics\",\n]\napi_modules = {\n mod_name: _import_name(f\"scanpy{mod_name.removeprefix('sc')}\")\n for mod_name in api_module_names\n}\n\n\n# get all exported functions that aren’t re-exports from anndata\napi_functions = [\n pytest.param(func, f\"{mod_name}.{name}\", id=f\"{mod_name}.{name}\")\n for mod_name, mod in api_modules.items()\n for name in sorted(mod.__all__)\n if callable(func := getattr(mod, name)) and func.__module__.startswith(\"scanpy.\")\n]\n\n\n@pytest.fixture\ndef in_project_dir():\n wd_orig = Path.cwd()\n os.chdir(proj_dir)\n try:\n yield proj_dir\n finally:\n os.chdir(wd_orig)\n\n\n@pytest.mark.xfail(reason=\"TODO: unclear if we want this to totally match, let’s see\")\ndef test_descend_classes_and_funcs():\n funcs = set(descend_classes_and_funcs(scanpy, \"scanpy\"))\n assert {p.values[0] for p in api_functions} == funcs\n\n\n@pytest.mark.parametrize((\"f\", \"qualname\"), api_functions)\ndef test_function_headers(f, qualname):\n filename = getsourcefile(f)\n lines, lineno = getsourcelines(f)\n if f.__doc__ is None:\n msg = f\"Function `{qualname}` has no docstring\"\n text = lines[0]\n else:\n lines = getattr(f, \"__orig_doc__\", f.__doc__).split(\"\\n\")\n broken = [\n i for i, l in enumerate(lines) if l.strip() and not l.startswith(\" \")\n ]\n if not any(broken):\n return\n msg = f'''\\\nHeader of function `{qualname}`’s docstring should start with one-line description\nand be consistently indented like this:\n\n␣␣␣␣\"\"\"\\\\\n␣␣␣␣My one-line␣description.\n\n␣␣␣␣…\n␣␣␣␣\"\"\"\n\nThe displayed line is under-indented.\n'''\n text = f\">{lines[broken[0]]}<\"\n raise SyntaxError(msg, (filename, lineno, 2, text))\n\n\ndef param_is_pos(p: Parameter) -> bool:\n return p.kind in {\n Parameter.POSITIONAL_ONLY,\n Parameter.POSITIONAL_OR_KEYWORD,\n }\n\n\ndef is_deprecated(f: FunctionType) -> bool:\n # TODO: use deprecated decorator instead\n # https://github.com/scverse/scanpy/issues/2505\n return f.__name__ in {\n \"normalize_per_cell\",\n \"filter_genes_dispersion\",\n }\n\n\nclass ExpectedSig(TypedDict):\n first_name: str\n copy_default: Any\n return_ann: str | None\n\n\ncopy_sigs: defaultdict[str, ExpectedSig | None] = defaultdict(\n lambda: ExpectedSig(first_name=\"adata\", copy_default=False, return_ann=None)\n)\n# full exceptions\ncopy_sigs[\"sc.external.tl.phenograph\"] = None # external\ncopy_sigs[\"sc.pp.filter_genes_dispersion\"] = None # deprecated\ncopy_sigs[\"sc.pp.filter_cells\"] = None # unclear `inplace` situation\ncopy_sigs[\"sc.pp.filter_genes\"] = None # unclear `inplace` situation\ncopy_sigs[\"sc.pp.subsample\"] = None # returns indices along matrix\n# partial exceptions: “data” instead of “adata”\ncopy_sigs[\"sc.pp.log1p\"][\"first_name\"] = \"data\"\ncopy_sigs[\"sc.pp.normalize_per_cell\"][\"first_name\"] = \"data\"\ncopy_sigs[\"sc.pp.pca\"][\"first_name\"] = \"data\"\ncopy_sigs[\"sc.pp.scale\"][\"first_name\"] = \"data\"\ncopy_sigs[\"sc.pp.sqrt\"][\"first_name\"] = \"data\"\n# other partial exceptions\ncopy_sigs[\"sc.pp.normalize_total\"][\"return_ann\"] = copy_sigs[\n \"sc.experimental.pp.normalize_pearson_residuals\"\n][\"return_ann\"] = \"AnnData | dict[str, np.ndarray] | None\"\ncopy_sigs[\"sc.external.pp.magic\"][\"copy_default\"] = None\n\n\n@pytest.mark.parametrize((\"f\", \"qualname\"), api_functions)\ndef test_sig_conventions(f, qualname):\n sig = signature(f)\n\n # TODO: replace the following check with lint rule for all funtions eventually\n if not is_deprecated(f):\n n_pos = sum(1 for p in sig.parameters.values() if param_is_pos(p))\n assert n_pos <= 3, \"Public functions should have <= 3 positional parameters\"\n\n first_param = next(iter(sig.parameters.values()), None)\n if first_param is None:\n return\n\n if first_param.name == \"adata\":\n assert first_param.annotation in {\"AnnData\", AnnData}\n elif first_param.name == \"data\":\n assert first_param.annotation.startswith(\"AnnData |\")\n elif first_param.name in {\"filename\", \"path\"}:\n assert first_param.annotation == \"Path | str\"\n\n # Test if functions with `copy` follow conventions\n if (copy_param := sig.parameters.get(\"copy\")) is not None and (\n expected_sig := copy_sigs[qualname]\n ) is not None:\n s = ExpectedSig(\n first_name=first_param.name,\n copy_default=copy_param.default,\n return_ann=sig.return_annotation,\n )\n expected_sig = expected_sig.copy()\n if expected_sig[\"return_ann\"] is None:\n expected_sig[\"return_ann\"] = f\"{first_param.annotation} | None\"\n assert s == expected_sig\n if not is_deprecated(f):\n assert not param_is_pos(copy_param)\n\n\ndef getsourcefile(obj):\n \"\"\"inspect.getsourcefile, but supports singledispatch\"\"\"\n from inspect import getsourcefile\n\n if wrapped := getattr(obj, \"__wrapped__\", None):\n return getsourcefile(wrapped)\n\n return getsourcefile(obj)\n\n\ndef getsourcelines(obj):\n \"\"\"inspect.getsourcelines, but supports singledispatch\"\"\"\n from inspect import getsourcelines\n\n if wrapped := getattr(obj, \"__wrapped__\", None):\n return getsourcelines(wrapped)\n\n return getsourcelines(obj)\n\n\nfrom __future__ import annotations\n\nimport shutil\nfrom pathlib import Path\nfrom unittest.mock import patch\n\nimport h5py\nimport numpy as np\nimport pytest\n\nimport scanpy as sc\n\nROOT = Path(__file__).parent\nROOT = ROOT / \"_data\" / \"10x_data\"\nVISIUM_ROOT = Path(__file__).parent / \"_data\" / \"visium_data\"\n\n\ndef assert_anndata_equal(a1, a2):\n assert a1.shape == a2.shape\n assert (a1.obs == a2.obs).all(axis=None)\n assert (a1.var == a2.var).all(axis=None)\n assert np.allclose(a1.X.todense(), a2.X.todense())\n\n\n@pytest.mark.parametrize(\n (\"mtx_path\", \"h5_path\"),\n [\n pytest.param(\n ROOT / \"1.2.0\" / \"filtered_gene_bc_matrices\" / \"hg19_chr21\",\n ROOT / \"1.2.0\" / \"filtered_gene_bc_matrices_h5.h5\",\n ),\n pytest.param(\n ROOT / \"3.0.0\" / \"filtered_feature_bc_matrix\",\n ROOT / \"3.0.0\" / \"filtered_feature_bc_matrix.h5\",\n ),\n ],\n)\n@pytest.mark.parametrize(\"prefix\", [None, \"prefix_\"])\ndef test_read_10x(tmp_path, mtx_path, h5_path, prefix):\n if prefix is not None:\n # Build files named \"prefix_XXX.xxx\" in a temporary directory.\n mtx_path_orig = mtx_path\n mtx_path = tmp_path / \"filtered_gene_bc_matrices_prefix\"\n mtx_path.mkdir()\n for item in mtx_path_orig.iterdir():\n if item.is_file():\n shutil.copyfile(item, mtx_path / f\"{prefix}{item.name}\")\n\n mtx = sc.read_10x_mtx(mtx_path, var_names=\"gene_symbols\", prefix=prefix)\n h5 = sc.read_10x_h5(h5_path)\n\n # Drop genome column for comparing v3\n if \"3.0.0\" in str(h5_path):\n h5.var.drop(columns=\"genome\", inplace=True)\n\n # Check equivalence\n assert_anndata_equal(mtx, h5)\n\n # Test that it can be written:\n from_mtx_pth = tmp_path / \"from_mtx.h5ad\"\n from_h5_pth = tmp_path / \"from_h5.h5ad\"\n\n mtx.write(from_mtx_pth)\n h5.write(from_h5_pth)\n\n assert_anndata_equal(sc.read_h5ad(from_mtx_pth), sc.read_h5ad(from_h5_pth))\n\n\ndef test_read_10x_h5_v1():\n spec_genome_v1 = sc.read_10x_h5(\n ROOT / \"1.2.0\" / \"filtered_gene_bc_matrices_h5.h5\",\n genome=\"hg19_chr21\",\n )\n nospec_genome_v1 = sc.read_10x_h5(\n ROOT / \"1.2.0\" / \"filtered_gene_bc_matrices_h5.h5\"\n )\n assert_anndata_equal(spec_genome_v1, nospec_genome_v1)\n\n\ndef test_read_10x_h5_v2_multiple_genomes():\n genome1_v1 = sc.read_10x_h5(\n ROOT / \"1.2.0\" / \"multiple_genomes.h5\",\n genome=\"hg19_chr21\",\n )\n genome2_v1 = sc.read_10x_h5(\n ROOT / \"1.2.0\" / \"multiple_genomes.h5\",\n genome=\"another_genome\",\n )\n # the test data are such that X is the same shape for both \"genomes\",\n # but the values are different\n assert (genome1_v1.X != genome2_v1.X).sum() > 0, (\n \"loading data from two different genomes in 10x v2 format. \"\n \"should be different, but is the same. \"\n )\n\n\ndef test_read_10x_h5():\n spec_genome_v3 = sc.read_10x_h5(\n ROOT / \"3.0.0\" / \"filtered_feature_bc_matrix.h5\",\n genome=\"GRCh38_chr21\",\n )\n nospec_genome_v3 = sc.read_10x_h5(ROOT / \"3.0.0\" / \"filtered_feature_bc_matrix.h5\")\n assert_anndata_equal(spec_genome_v3, nospec_genome_v3)\n\n\ndef test_error_10x_h5_legacy(tmp_path):\n onepth = ROOT / \"1.2.0\" / \"filtered_gene_bc_matrices_h5.h5\"\n twopth = tmp_path / \"two_genomes.h5\"\n with h5py.File(onepth, \"r\") as one, h5py.File(twopth, \"w\") as two:\n one.copy(\"hg19_chr21\", two)\n one.copy(\"hg19_chr21\", two, name=\"hg19_chr21_copy\")\n with pytest.raises(ValueError, match=r\"contains more than one genome\"):\n sc.read_10x_h5(twopth)\n sc.read_10x_h5(twopth, genome=\"hg19_chr21_copy\")\n\n\ndef test_error_missing_genome():\n legacy_pth = ROOT / \"1.2.0\" / \"filtered_gene_bc_matrices_h5.h5\"\n v3_pth = ROOT / \"3.0.0\" / \"filtered_feature_bc_matrix.h5\"\n with pytest.raises(ValueError, match=r\".*hg19_chr21.*\"):\n sc.read_10x_h5(legacy_pth, genome=\"not a genome\")\n with pytest.raises(ValueError, match=r\".*GRCh38_chr21.*\"):\n sc.read_10x_h5(v3_pth, genome=\"not a genome\")\n\n\n@pytest.fixture(params=[1, 2])\ndef visium_pth(request, tmp_path) -> Path:\n visium1_pth = VISIUM_ROOT / \"1.0.0\"\n if request.param == 1:\n return visium1_pth\n elif request.param == 2:\n visium2_pth = tmp_path / \"visium2\"\n with patch.object(shutil, \"copystat\"):\n # copy only data, not file metadata\n shutil.copytree(visium1_pth, visium2_pth)\n header = \"barcode,in_tissue,array_row,array_col,pxl_row_in_fullres,pxl_col_in_fullres\"\n orig = visium2_pth / \"spatial\" / \"tissue_positions_list.csv\"\n csv = f\"{header}\\n{orig.read_text()}\"\n orig.unlink()\n (orig.parent / \"tissue_positions.csv\").write_text(csv)\n return visium2_pth\n else:\n pytest.fail(\"add branch for new visium version\")\n\n\ndef test_read_visium_counts(visium_pth):\n \"\"\"Test checking that read_visium reads the right genome\"\"\"\n spec_genome_v3 = sc.read_visium(visium_pth, genome=\"GRCh38\")\n nospec_genome_v3 = sc.read_visium(visium_pth)\n assert_anndata_equal(spec_genome_v3, nospec_genome_v3)\n\n\ndef test_10x_h5_gex():\n # Tests that gex option doesn't, say, make the function return None\n h5_pth = ROOT / \"3.0.0\" / \"filtered_feature_bc_matrix.h5\"\n assert_anndata_equal(\n sc.read_10x_h5(h5_pth, gex_only=True), sc.read_10x_h5(h5_pth, gex_only=False)\n )\n\n\ndef test_10x_probe_barcode_read():\n # Tests the 10x probe barcode matrix is read correctly\n h5_pth = VISIUM_ROOT / \"2.1.0\" / \"raw_probe_bc_matrix.h5\"\n probe_anndata = sc.read_10x_h5(h5_pth)\n assert set(probe_anndata.var.columns) == {\n \"feature_types\",\n \"filtered_probes\",\n \"gene_ids\",\n \"gene_name\",\n \"genome\",\n \"probe_ids\",\n \"probe_region\",\n }\n assert set(probe_anndata.obs.columns) == {\"filtered_barcodes\"}\n assert probe_anndata.shape == (4987, 1000)\n assert probe_anndata.X.nnz == 858\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pytest\n\nimport scanpy as sc\n\n\ndef test_sim_toggleswitch():\n with pytest.warns(UserWarning, match=r\"Observation names are not unique\"):\n adata = sc.tl.sim(\"toggleswitch\")\n np.allclose(adata.X, sc.datasets.toggleswitch().X, np.finfo(np.float32).eps)\n\n\nfrom __future__ import annotations\n\nfrom typing import get_args\n\nimport anndata as ad\nimport numpy as np\nimport pandas as pd\nimport pytest\nfrom packaging.version import Version\nfrom scipy import sparse\n\nimport scanpy as sc\nfrom scanpy._utils import _resolve_axis\nfrom scanpy.get._aggregated import AggType\nfrom testing.scanpy._helpers import assert_equal\nfrom testing.scanpy._helpers.data import pbmc3k_processed\nfrom testing.scanpy._pytest.params import ARRAY_TYPES_MEM\n\n\n@pytest.fixture(params=get_args(AggType))\ndef metric(request: pytest.FixtureRequest) -> AggType:\n return request.param\n\n\n@pytest.fixture\ndef df_base():\n ax_base = [\"A\", \"B\"]\n return pd.DataFrame(index=ax_base)\n\n\n@pytest.fixture\ndef df_groupby():\n ax_groupby = [\n *[\"v0\", \"v1\", \"v2\"],\n *[\"w0\", \"w1\"],\n *[\"a1\", \"a2\", \"a3\"],\n *[\"b1\", \"b2\"],\n *[\"c1\", \"c2\"],\n \"d0\",\n ]\n\n df_groupby = pd.DataFrame(index=pd.Index(ax_groupby, name=\"cell\"))\n df_groupby[\"key\"] = pd.Categorical([c[0] for c in ax_groupby])\n df_groupby[\"key_superset\"] = pd.Categorical([c[0] for c in ax_groupby]).map(\n {\"v\": \"v\", \"w\": \"v\", \"a\": \"a\", \"b\": \"a\", \"c\": \"a\", \"d\": \"a\"}\n )\n df_groupby[\"key_subset\"] = pd.Categorical([c[1] for c in ax_groupby])\n df_groupby[\"weight\"] = 2.0\n return df_groupby\n\n\n@pytest.fixture\ndef X():\n data = [\n *[[0, -2], [1, 13], [2, 1]], # v\n *[[3, 12], [4, 2]], # w\n *[[5, 11], [6, 3], [7, 10]], # a\n *[[8, 4], [9, 9]], # b\n *[[10, 5], [11, 8]], # c\n [12, 6], # d\n ]\n return np.array(data, dtype=np.float32)\n\n\ndef gen_adata(data_key, dim, df_base, df_groupby, X):\n if (data_key == \"varm\" and dim == \"obs\") or (data_key == \"obsm\" and dim == \"var\"):\n pytest.skip(\"invalid parameter combination\")\n\n obs_df, var_df = (df_groupby, df_base) if dim == \"obs\" else (df_base, df_groupby)\n data = X.T if dim == \"var\" and data_key != \"varm\" else X\n if data_key != \"X\":\n data_dict_sparse = {data_key: {\"test\": sparse.csr_matrix(data)}}\n data_dict_dense = {data_key: {\"test\": data}}\n else:\n data_dict_sparse = {data_key: sparse.csr_matrix(data)}\n data_dict_dense = {data_key: data}\n\n adata_sparse = ad.AnnData(obs=obs_df, var=var_df, **data_dict_sparse)\n adata_dense = ad.AnnData(obs=obs_df, var=var_df, **data_dict_dense)\n return adata_sparse, adata_dense\n\n\n@pytest.mark.parametrize(\"axis\", [0, 1])\ndef test_mask(axis):\n blobs = sc.datasets.blobs()\n mask = blobs.obs[\"blobs\"] == 0\n blobs.obs[\"mask_col\"] = mask\n if axis == 1:\n blobs = blobs.T\n by_name = sc.get.aggregate(blobs, \"blobs\", \"sum\", axis=axis, mask=\"mask_col\")\n by_value = sc.get.aggregate(blobs, \"blobs\", \"sum\", axis=axis, mask=mask)\n\n assert_equal(by_name, by_value)\n\n assert np.all(by_name[\"0\"].layers[\"sum\"] == 0)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES_MEM)\ndef test_aggregate_vs_pandas(metric, array_type):\n adata = pbmc3k_processed().raw.to_adata()\n adata = adata[\n adata.obs[\"louvain\"].isin(adata.obs[\"louvain\"].cat.categories[:5]), :1_000\n ].copy()\n adata.X = array_type(adata.X)\n adata.obs[\"percent_mito_binned\"] = pd.cut(adata.obs[\"percent_mito\"], bins=5)\n result = sc.get.aggregate(adata, [\"louvain\", \"percent_mito_binned\"], metric)\n\n if metric == \"count_nonzero\":\n expected = (\n (adata.to_df() != 0)\n .astype(np.float64)\n .join(adata.obs[[\"louvain\", \"percent_mito_binned\"]])\n .groupby([\"louvain\", \"percent_mito_binned\"], observed=True)\n .agg(\"sum\")\n )\n else:\n expected = (\n adata.to_df()\n .astype(np.float64)\n .join(adata.obs[[\"louvain\", \"percent_mito_binned\"]])\n .groupby([\"louvain\", \"percent_mito_binned\"], observed=True)\n .agg(metric)\n )\n expected.index = expected.index.to_frame().apply(\n lambda x: \"_\".join(map(str, x)), axis=1\n )\n expected.index.name = None\n expected.columns.name = None\n\n result_df = result.to_df(layer=metric)\n result_df.index.name = None\n result_df.columns.name = None\n\n if Version(pd.__version__) < Version(\"2\"):\n # Order of results returned by groupby changed in pandas 2\n assert expected.shape == result_df.shape\n assert expected.index.isin(result_df.index).all()\n\n expected = expected.loc[result_df.index]\n\n pd.testing.assert_frame_equal(result_df, expected, check_dtype=False, atol=1e-5)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES_MEM)\ndef test_aggregate_axis(array_type, metric):\n adata = pbmc3k_processed().raw.to_adata()\n adata = adata[\n adata.obs[\"louvain\"].isin(adata.obs[\"louvain\"].cat.categories[:5]), :1_000\n ].copy()\n adata.X = array_type(adata.X)\n expected = sc.get.aggregate(adata, [\"louvain\"], metric)\n actual = sc.get.aggregate(adata.T, [\"louvain\"], metric, axis=1).T\n\n assert_equal(expected, actual)\n\n\ndef test_aggregate_entry():\n args = (\"blobs\", [\"mean\", \"var\", \"count_nonzero\"])\n\n adata = sc.datasets.blobs()\n X_result = sc.get.aggregate(adata, *args)\n # layer adata\n layer_adata = ad.AnnData(\n obs=adata.obs,\n var=adata.var,\n layers={\"test\": adata.X.copy()},\n )\n layer_result = sc.get.aggregate(layer_adata, *args, layer=\"test\")\n obsm_adata = ad.AnnData(\n obs=adata.obs,\n var=adata.var,\n obsm={\"test\": adata.X.copy()},\n )\n obsm_result = sc.get.aggregate(obsm_adata, *args, obsm=\"test\")\n varm_adata = ad.AnnData(\n obs=adata.var,\n var=adata.obs,\n varm={\"test\": adata.X.copy()},\n )\n varm_result = sc.get.aggregate(varm_adata, *args, varm=\"test\")\n\n X_result_min = X_result.copy()\n del X_result_min.var\n X_result_min.var_names = [str(x) for x in np.arange(X_result_min.n_vars)]\n\n assert_equal(X_result, layer_result)\n assert_equal(X_result_min, obsm_result)\n assert_equal(X_result.layers, obsm_result.layers)\n assert_equal(X_result.layers, varm_result.T.layers)\n\n\ndef test_aggregate_incorrect_dim():\n adata = pbmc3k_processed().raw.to_adata()\n\n with pytest.raises(ValueError, match=\"was 'foo'\"):\n sc.get.aggregate(adata, [\"louvain\"], \"sum\", axis=\"foo\")\n\n\n@pytest.mark.parametrize(\"axis_name\", [\"obs\", \"var\"])\ndef test_aggregate_axis_specification(axis_name):\n axis, axis_name = _resolve_axis(axis_name)\n by = \"blobs\" if axis == 0 else \"labels\"\n\n adata = sc.datasets.blobs()\n adata.var[\"labels\"] = np.tile([\"a\", \"b\"], adata.shape[1])[: adata.shape[1]]\n\n agg_index = sc.get.aggregate(adata, by=by, func=\"mean\", axis=axis)\n agg_name = sc.get.aggregate(adata, by=by, func=\"mean\", axis=axis_name)\n\n np.testing.assert_equal(agg_index.layers[\"mean\"], agg_name.layers[\"mean\"])\n\n if axis_name == \"obs\":\n agg_unspecified = sc.get.aggregate(adata, by=by, func=\"mean\")\n np.testing.assert_equal(agg_name.layers[\"mean\"], agg_unspecified.layers[\"mean\"])\n\n\n@pytest.mark.parametrize(\n (\"matrix\", \"df\", \"keys\", \"metrics\", \"expected\"),\n [\n pytest.param(\n np.block(\n [\n [np.ones((2, 2)), np.zeros((2, 2))],\n [np.zeros((2, 2)), np.ones((2, 2))],\n ]\n ),\n pd.DataFrame(\n {\n \"a\": [\"a\", \"a\", \"b\", \"b\"],\n \"b\": [\"c\", \"d\", \"d\", \"d\"],\n },\n index=[\"a_c\", \"a_d\", \"b_d1\", \"b_d2\"],\n ),\n [\"a\", \"b\"],\n [\"count_nonzero\"], # , \"sum\", \"mean\"],\n ad.AnnData(\n obs=pd.DataFrame(\n {\"a\": [\"a\", \"a\", \"b\"], \"b\": [\"c\", \"d\", \"d\"]},\n index=[\"a_c\", \"a_d\", \"b_d\"],\n ).astype(\"category\"),\n var=pd.DataFrame(index=[f\"gene_{i}\" for i in range(4)]),\n layers={\n \"count_nonzero\": np.array(\n [[1, 1, 0, 0], [1, 1, 0, 0], [0, 0, 2, 2]]\n ),\n # \"sum\": np.array([[2, 0], [0, 2]]),\n # \"mean\": np.array([[1, 0], [0, 1]]),\n },\n ),\n id=\"count_nonzero\",\n ),\n pytest.param(\n np.block(\n [\n [np.ones((2, 2)), np.zeros((2, 2))],\n [np.zeros((2, 2)), np.ones((2, 2))],\n ]\n ),\n pd.DataFrame(\n {\n \"a\": [\"a\", \"a\", \"b\", \"b\"],\n \"b\": [\"c\", \"d\", \"d\", \"d\"],\n },\n index=[\"a_c\", \"a_d\", \"b_d1\", \"b_d2\"],\n ),\n [\"a\", \"b\"],\n [\"sum\", \"mean\", \"count_nonzero\"],\n ad.AnnData(\n obs=pd.DataFrame(\n {\"a\": [\"a\", \"a\", \"b\"], \"b\": [\"c\", \"d\", \"d\"]},\n index=[\"a_c\", \"a_d\", \"b_d\"],\n ).astype(\"category\"),\n var=pd.DataFrame(index=[f\"gene_{i}\" for i in range(4)]),\n layers={\n \"sum\": np.array([[1, 1, 0, 0], [1, 1, 0, 0], [0, 0, 2, 2]]),\n \"mean\": np.array([[1, 1, 0, 0], [1, 1, 0, 0], [0, 0, 1, 1]]),\n \"count_nonzero\": np.array(\n [[1, 1, 0, 0], [1, 1, 0, 0], [0, 0, 2, 2]]\n ),\n },\n ),\n id=\"sum-mean-count_nonzero\",\n ),\n pytest.param(\n np.block(\n [\n [np.ones((2, 2)), np.zeros((2, 2))],\n [np.zeros((2, 2)), np.ones((2, 2))],\n ]\n ),\n pd.DataFrame(\n {\n \"a\": [\"a\", \"a\", \"b\", \"b\"],\n \"b\": [\"c\", \"d\", \"d\", \"d\"],\n },\n index=[\"a_c\", \"a_d\", \"b_d1\", \"b_d2\"],\n ),\n [\"a\", \"b\"],\n [\"mean\"],\n ad.AnnData(\n obs=pd.DataFrame(\n {\"a\": [\"a\", \"a\", \"b\"], \"b\": [\"c\", \"d\", \"d\"]},\n index=[\"a_c\", \"a_d\", \"b_d\"],\n ).astype(\"category\"),\n var=pd.DataFrame(index=[f\"gene_{i}\" for i in range(4)]),\n layers={\n \"mean\": np.array([[1, 1, 0, 0], [1, 1, 0, 0], [0, 0, 1, 1]]),\n },\n ),\n id=\"mean\",\n ),\n ],\n)\ndef test_aggregate_examples(matrix, df, keys, metrics, expected):\n adata = ad.AnnData(\n X=matrix,\n obs=df,\n var=pd.DataFrame(index=[f\"gene_{i}\" for i in range(matrix.shape[1])]),\n )\n result = sc.get.aggregate(adata, by=keys, func=metrics)\n\n print(result)\n print(expected)\n\n assert_equal(expected, result)\n\n\n@pytest.mark.parametrize(\n (\"label_cols\", \"cols\", \"expected\"),\n [\n pytest.param(\n dict(\n a=pd.Categorical([\"a\", \"b\", \"c\"]),\n b=pd.Categorical([\"d\", \"d\", \"f\"]),\n ),\n [\"a\", \"b\"],\n pd.Categorical([\"a_d\", \"b_d\", \"c_f\"]),\n id=\"two_of_two\",\n ),\n pytest.param(\n dict(\n a=pd.Categorical([\"a\", \"b\", \"c\"]),\n b=pd.Categorical([\"d\", \"d\", \"f\"]),\n c=pd.Categorical([\"g\", \"h\", \"h\"]),\n ),\n [\"a\", \"b\", \"c\"],\n pd.Categorical([\"a_d_g\", \"b_d_h\", \"c_f_h\"]),\n id=\"three_of_three\",\n ),\n pytest.param(\n dict(\n a=pd.Categorical([\"a\", \"b\", \"c\"]),\n b=pd.Categorical([\"d\", \"d\", \"f\"]),\n c=pd.Categorical([\"g\", \"h\", \"h\"]),\n ),\n [\"a\", \"c\"],\n pd.Categorical([\"a_g\", \"b_h\", \"c_h\"]),\n id=\"two_of_three-1\",\n ),\n pytest.param(\n dict(\n a=pd.Categorical([\"a\", \"b\", \"c\"]),\n b=pd.Categorical([\"d\", \"d\", \"f\"]),\n c=pd.Categorical([\"g\", \"h\", \"h\"]),\n ),\n [\"b\", \"c\"],\n pd.Categorical([\"d_g\", \"d_h\", \"f_h\"]),\n id=\"two_of_three-2\",\n ),\n ],\n)\ndef test_combine_categories(label_cols, cols, expected):\n from scanpy.get._aggregated import _combine_categories\n\n label_df = pd.DataFrame(label_cols)\n result, result_label_df = _combine_categories(label_df, cols)\n\n assert isinstance(result, pd.Categorical)\n\n pd.testing.assert_extension_array_equal(result, expected)\n\n pd.testing.assert_index_equal(\n pd.Index(result), result_label_df.index.astype(\"category\")\n )\n\n reconstructed_df = pd.DataFrame(\n [x.split(\"_\") for x in result], columns=cols, index=result.astype(str)\n ).astype(\"category\")\n pd.testing.assert_frame_equal(reconstructed_df, result_label_df)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES_MEM)\ndef test_aggregate_arraytype(array_type, metric):\n adata = pbmc3k_processed().raw.to_adata()\n adata = adata[\n adata.obs[\"louvain\"].isin(adata.obs[\"louvain\"].cat.categories[:5]), :1_000\n ].copy()\n adata.X = array_type(adata.X)\n aggregate = sc.get.aggregate(adata, [\"louvain\"], metric)\n assert isinstance(aggregate.layers[metric], np.ndarray)\n\n\ndef test_aggregate_obsm_varm():\n adata_obsm = sc.datasets.blobs()\n adata_obsm.obs[\"blobs\"] = adata_obsm.obs[\"blobs\"].astype(str)\n adata_obsm.obsm[\"test\"] = adata_obsm.X[:, ::2].copy()\n adata_varm = adata_obsm.T.copy()\n\n result_obsm = sc.get.aggregate(adata_obsm, \"blobs\", [\"sum\", \"mean\"], obsm=\"test\")\n result_varm = sc.get.aggregate(adata_varm, \"blobs\", [\"sum\", \"mean\"], varm=\"test\")\n\n assert_equal(result_obsm, result_varm.T)\n\n expected_sum = (\n pd.DataFrame(adata_obsm.obsm[\"test\"], index=adata_obsm.obs_names)\n .groupby(adata_obsm.obs[\"blobs\"], observed=True)\n .sum()\n )\n expected_mean = (\n pd.DataFrame(adata_obsm.obsm[\"test\"], index=adata_obsm.obs_names)\n .groupby(adata_obsm.obs[\"blobs\"], observed=True)\n .mean()\n )\n\n assert_equal(expected_sum.values, result_obsm.layers[\"sum\"])\n assert_equal(expected_mean.values, result_obsm.layers[\"mean\"])\n\n\ndef test_aggregate_obsm_labels():\n from itertools import chain, repeat\n\n label_counts = [(\"a\", 5), (\"b\", 3), (\"c\", 4)]\n blocks = [np.ones((n, 1)) for _, n in label_counts]\n obs_names = pd.Index(\n [f\"cell_{i:02d}\" for i in range(sum(b.shape[0] for b in blocks))]\n )\n entry = pd.DataFrame(\n sparse.block_diag(blocks).toarray(),\n columns=[f\"dim_{i}\" for i in range(len(label_counts))],\n index=obs_names,\n )\n\n adata = ad.AnnData(\n obs=pd.DataFrame(\n {\n \"labels\": list(\n chain.from_iterable(repeat(l, n) for (l, n) in label_counts)\n )\n },\n index=obs_names,\n ),\n var=pd.DataFrame(index=[\"gene_0\"]),\n obsm={\"entry\": entry},\n )\n\n expected = ad.AnnData(\n obs=pd.DataFrame({\"labels\": pd.Categorical(list(\"abc\"))}, index=list(\"abc\")),\n var=pd.DataFrame(index=[f\"dim_{i}\" for i in range(3)]),\n layers={\n \"sum\": np.diag([n for _, n in label_counts]),\n },\n )\n result = sc.get.aggregate(adata, by=\"labels\", func=\"sum\", obsm=\"entry\")\n assert_equal(expected, result)\n\n\ndef test_dispatch_not_implemented():\n adata = sc.datasets.blobs()\n with pytest.raises(NotImplementedError):\n sc.get.aggregate(adata.X, adata.obs[\"blobs\"], \"sum\")\n\n\ndef test_factors():\n from itertools import product\n\n obs = pd.DataFrame(\n product(range(5), range(5), range(5), range(5)), columns=list(\"abcd\")\n )\n obs.index = [f\"cell_{i:04d}\" for i in range(obs.shape[0])]\n adata = ad.AnnData(\n X=np.arange(obs.shape[0]).reshape(-1, 1),\n obs=obs,\n )\n\n res = sc.get.aggregate(adata, by=[\"a\", \"b\", \"c\", \"d\"], func=\"sum\")\n np.testing.assert_equal(res.layers[\"sum\"], adata.X)\n\n\nfrom __future__ import annotations\n\nimport numpy as np\n\nfrom scanpy.tools import filter_rank_genes_groups, rank_genes_groups\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\n\nnames_no_reference = np.array(\n [\n [\"CD3D\", \"ITM2A\", \"CD3D\", \"CCL5\", \"CD7\", \"nan\", \"CD79A\", \"nan\", \"NKG7\", \"LYZ\"],\n [\"CD3E\", \"CD3D\", \"nan\", \"NKG7\", \"CD3D\", \"AIF1\", \"CD79B\", \"nan\", \"GNLY\", \"CST3\"],\n [\"IL32\", \"RPL39\", \"nan\", \"CST7\", \"nan\", \"nan\", \"nan\", \"SNHG7\", \"CD7\", \"nan\"],\n [\"nan\", \"SRSF7\", \"IL32\", \"GZMA\", \"nan\", \"LST1\", \"IGJ\", \"nan\", \"CTSW\", \"nan\"],\n [\n \"nan\",\n \"nan\",\n \"CD2\",\n \"CTSW\",\n \"CD8B\",\n \"TYROBP\",\n \"ISG20\",\n \"SNHG8\",\n \"GZMB\",\n \"nan\",\n ],\n ]\n)\n\nnames_reference = np.array(\n [\n [\"CD3D\", \"ITM2A\", \"CD3D\", \"nan\", \"CD3D\", \"nan\", \"CD79A\", \"nan\", \"CD7\"],\n [\"nan\", \"nan\", \"nan\", \"CD3D\", \"nan\", \"AIF1\", \"nan\", \"nan\", \"NKG7\"],\n [\"nan\", \"nan\", \"nan\", \"NKG7\", \"nan\", \"FCGR3A\", \"ISG20\", \"SNHG7\", \"CTSW\"],\n [\"nan\", \"CD3D\", \"nan\", \"CCL5\", \"CD7\", \"nan\", \"CD79B\", \"nan\", \"GNLY\"],\n [\"CD3E\", \"IL32\", \"nan\", \"IL32\", \"CD27\", \"FCER1G\", \"nan\", \"nan\", \"nan\"],\n ]\n)\n\nnames_compare_abs = np.array(\n [\n [\n \"CD3D\",\n \"ITM2A\",\n \"HLA-DRB1\",\n \"CCL5\",\n \"HLA-DPA1\",\n \"nan\",\n \"CD79A\",\n \"nan\",\n \"NKG7\",\n \"LYZ\",\n ],\n [\n \"HLA-DPA1\",\n \"nan\",\n \"CD3D\",\n \"NKG7\",\n \"HLA-DRB1\",\n \"AIF1\",\n \"CD79B\",\n \"nan\",\n \"GNLY\",\n \"CST3\",\n ],\n [\n \"nan\",\n \"PSAP\",\n \"CD74\",\n \"CST7\",\n \"CD74\",\n \"PSAP\",\n \"FCER1G\",\n \"SNHG7\",\n \"CD7\",\n \"HLA-DRA\",\n ],\n [\n \"IL32\",\n \"nan\",\n \"HLA-DRB5\",\n \"GZMA\",\n \"HLA-DRB5\",\n \"LST1\",\n \"nan\",\n \"nan\",\n \"CTSW\",\n \"HLA-DRB1\",\n ],\n [\n \"nan\",\n \"FCER1G\",\n \"HLA-DPB1\",\n \"CTSW\",\n \"HLA-DPB1\",\n \"TYROBP\",\n \"TYROBP\",\n \"S100A10\",\n \"GZMB\",\n \"HLA-DPA1\",\n ],\n ]\n)\n\n\ndef test_filter_rank_genes_groups():\n adata = pbmc68k_reduced()\n\n # fix filter defaults\n args = {\n \"adata\": adata,\n \"key_added\": \"rank_genes_groups_filtered\",\n \"min_in_group_fraction\": 0.25,\n \"min_fold_change\": 1,\n \"max_out_group_fraction\": 0.5,\n }\n\n rank_genes_groups(\n adata, \"bulk_labels\", reference=\"Dendritic\", method=\"wilcoxon\", n_genes=5\n )\n filter_rank_genes_groups(**args)\n\n assert np.array_equal(\n names_reference,\n np.array(adata.uns[\"rank_genes_groups_filtered\"][\"names\"].tolist()),\n )\n\n rank_genes_groups(adata, \"bulk_labels\", method=\"wilcoxon\", n_genes=5)\n filter_rank_genes_groups(**args)\n\n assert np.array_equal(\n names_no_reference,\n np.array(adata.uns[\"rank_genes_groups_filtered\"][\"names\"].tolist()),\n )\n\n rank_genes_groups(adata, \"bulk_labels\", method=\"wilcoxon\", pts=True, n_genes=5)\n filter_rank_genes_groups(**args)\n\n assert np.array_equal(\n names_no_reference,\n np.array(adata.uns[\"rank_genes_groups_filtered\"][\"names\"].tolist()),\n )\n\n # test compare_abs\n rank_genes_groups(\n adata, \"bulk_labels\", method=\"wilcoxon\", pts=True, rankby_abs=True, n_genes=5\n )\n\n filter_rank_genes_groups(\n adata,\n compare_abs=True,\n min_in_group_fraction=-1,\n max_out_group_fraction=1,\n min_fold_change=3.1,\n )\n\n assert np.array_equal(\n names_compare_abs,\n np.array(adata.uns[\"rank_genes_groups_filtered\"][\"names\"].tolist()),\n )\n\n\nfrom __future__ import annotations\n\nimport os\nimport re\nfrom contextlib import nullcontext\nfrom pathlib import Path\nfrom subprocess import PIPE\nfrom typing import TYPE_CHECKING\n\nimport pytest\n\nimport scanpy\nfrom scanpy.cli import main\n\nif TYPE_CHECKING:\n from _pytest.capture import CaptureFixture\n from _pytest.monkeypatch import MonkeyPatch\n\nHERE = Path(__file__).parent\n\n\n@pytest.fixture\ndef _set_path(monkeypatch: MonkeyPatch) -> None:\n monkeypatch.setenv(\"PATH\", str(HERE / \"_scripts\"), prepend=os.pathsep)\n\n\ndef test_builtin_settings(capsys: CaptureFixture):\n main([\"settings\"])\n captured = capsys.readouterr()\n assert captured.out == f\"{scanpy.settings}\\n\"\n\n\n@pytest.mark.parametrize(\"args\", [[], [\"-h\"]])\ndef test_help_displayed(args: list[str], capsys: CaptureFixture):\n # -h raises it, no args doesn’t. Maybe not ideal but meh.\n ctx = pytest.raises(SystemExit) if args else nullcontext()\n with ctx as se:\n main(args)\n if se is not None:\n assert se.value.code == 0\n captured = capsys.readouterr()\n assert captured.out.startswith(\"usage: \")\n\n\n@pytest.mark.usefixtures(\"_set_path\")\ndef test_help_output(capsys: CaptureFixture):\n with pytest.raises(SystemExit, match=\"^0$\"):\n main([\"-h\"])\n captured = capsys.readouterr()\n assert re.search(\n r\"^positional arguments:\\n\\s+\\{settings,[\\w,-]*testbin[\\w,-]*\\}$\",\n captured.out,\n re.MULTILINE,\n )\n\n\n@pytest.mark.usefixtures(\"_set_path\")\ndef test_external():\n # We need to capture the output manually, since subprocesses don’t write to sys.stderr\n cmdline = [\"testbin\", \"-t\", \"--testarg\", \"testpos\"]\n cmd = main(cmdline, stdout=PIPE, encoding=\"utf-8\", check=True)\n assert cmd.stdout == \"test -t --testarg testpos\\n\"\n\n\ndef test_error_wrong_command(capsys: CaptureFixture):\n with pytest.raises(SystemExit, match=\"^2$\"):\n main([\"idonotexist--\"])\n captured = capsys.readouterr()\n assert \"invalid choice: 'idonotexist--' (choose from\" in captured.err\n\n\nfrom __future__ import annotations\n\nimport sys\nfrom contextlib import redirect_stdout\nfrom datetime import datetime\nfrom io import StringIO\nfrom typing import TYPE_CHECKING\n\nimport pytest\n\nimport scanpy as sc\nfrom scanpy import Verbosity\nfrom scanpy import logging as log\nfrom scanpy import settings as s\n\nif TYPE_CHECKING:\n from pathlib import Path\n\n\ndef test_defaults():\n assert s.logpath is None\n\n\ndef test_records(caplog: pytest.LogCaptureFixture):\n s.verbosity = Verbosity.debug\n log.error(\"0\")\n log.warning(\"1\")\n log.info(\"2\")\n log.hint(\"3\")\n log.debug(\"4\")\n assert caplog.record_tuples == [\n (\"root\", 40, \"0\"),\n (\"root\", 30, \"1\"),\n (\"root\", 20, \"2\"),\n (\"root\", 15, \"3\"),\n (\"root\", 10, \"4\"),\n ]\n\n\ndef test_formats(capsys: pytest.CaptureFixture):\n s.logfile = sys.stderr\n s.verbosity = Verbosity.debug\n log.error(\"0\")\n assert capsys.readouterr().err == \"ERROR: 0\\n\"\n log.warning(\"1\")\n assert capsys.readouterr().err == \"WARNING: 1\\n\"\n log.info(\"2\")\n assert capsys.readouterr().err == \"2\\n\"\n log.hint(\"3\")\n assert capsys.readouterr().err == \"--> 3\\n\"\n log.debug(\"4\")\n assert capsys.readouterr().err == \" 4\\n\"\n\n\ndef test_deep(capsys: pytest.CaptureFixture):\n s.logfile = sys.stderr\n s.verbosity = Verbosity.hint\n log.hint(\"0\")\n assert capsys.readouterr().err == \"--> 0\\n\"\n log.hint(\"1\", deep=\"1!\")\n assert capsys.readouterr().err == \"--> 1\\n\"\n s.verbosity = Verbosity.debug\n log.hint(\"2\")\n assert capsys.readouterr().err == \"--> 2\\n\"\n log.hint(\"3\", deep=\"3!\")\n assert capsys.readouterr().err == \"--> 3: 3!\\n\"\n\n\ndef test_logfile(tmp_path: Path, caplog: pytest.LogCaptureFixture):\n s.verbosity = Verbosity.hint\n\n io = StringIO()\n s.logfile = io\n assert s.logfile is io\n assert s.logpath is None\n log.error(\"test!\")\n assert io.getvalue() == \"ERROR: test!\\n\"\n\n # setting a logfile removes all handlers\n assert not caplog.records\n\n p = tmp_path / \"test.log\"\n s.logpath = p\n assert s.logpath == p\n assert s.logfile.name == str(p)\n log.hint(\"test2\")\n log.debug(\"invisible\")\n assert s.logpath.read_text() == \"--> test2\\n\"\n\n # setting a logfile removes all handlers\n assert not caplog.records\n\n\ndef test_timing(monkeypatch, capsys: pytest.CaptureFixture):\n counter = 0\n\n class IncTime:\n @staticmethod\n def now(tz):\n nonlocal counter\n counter += 1\n return datetime(2000, 1, 1, second=counter, microsecond=counter, tzinfo=tz)\n\n monkeypatch.setattr(log, \"datetime\", IncTime)\n s.logfile = sys.stderr\n s.verbosity = Verbosity.debug\n\n log.hint(\"1\")\n assert counter == 1\n assert capsys.readouterr().err == \"--> 1\\n\"\n\n start = log.info(\"2\")\n assert counter == 2\n assert capsys.readouterr().err == \"2\\n\"\n\n log.hint(\"3\")\n assert counter == 3\n assert capsys.readouterr().err == \"--> 3\\n\"\n\n log.info(\"4\", time=start)\n assert counter == 4\n assert capsys.readouterr().err == \"4 (0:00:02)\\n\"\n\n log.info(\"5 {time_passed}\", time=start)\n assert counter == 5\n assert capsys.readouterr().err == \"5 0:00:03\\n\"\n\n\n@pytest.mark.parametrize(\n \"func\",\n [\n sc.logging.print_header,\n sc.logging.print_versions,\n sc.logging.print_version_and_date,\n ],\n)\ndef test_call_outputs(func):\n \"\"\"\n Tests that these functions print to stdout and don't error.\n\n Checks that https://github.com/scverse/scanpy/issues/1437 is fixed.\n \"\"\"\n output_io = StringIO()\n with redirect_stdout(output_io):\n func()\n output = output_io.getvalue()\n assert output != \"\"\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pytest\nfrom anndata import AnnData\nfrom scipy.sparse import csr_matrix, issparse\nfrom sklearn.neighbors import KNeighborsTransformer\n\nimport scanpy as sc\nfrom scanpy import Neighbors\nfrom testing.scanpy._helpers import anndata_v0_8_constructor_compat\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from pytest_mock import MockerFixture\n\n# the input data\nX = [[1, 0], [3, 0], [5, 6], [0, 4]]\nn_neighbors = 3 # includes data points themselves\n\n# distances\ndistances_euclidean = [\n [0.0, 2.0, 0.0, 4.123105525970459],\n [2.0, 0.0, 0.0, 5.0],\n [0.0, 6.324555397033691, 0.0, 5.385164737701416],\n [4.123105525970459, 5.0, 0.0, 0.0],\n]\n\ndistances_euclidean_all = [\n [0.0, 2.0, 7.211102485656738, 4.123105525970459],\n [2.0, 0.0, 6.324555397033691, 5.0],\n [7.211102485656738, 6.324555397033691, 0.0, 5.385164737701416],\n [4.123105525970459, 5.0, 5.385164737701416, 0.0],\n]\n\n\n# umap \"kernel\" – only knn results\nconnectivities_umap = [\n [0.0, 1.0, 0.0, 1.0],\n [1.0, 0.0, 0.5849691143165735, 0.8277419907567016],\n [0.0, 0.5849691143165735, 0.0, 1.0],\n [1.0, 0.8277419907567016, 1.0, 0.0],\n]\n\ntransitions_sym_umap = [\n [0.0, 0.4818987107873648, 0.0, 0.3951883393150153],\n [0.48189871078736474, 0.0, 0.3594582764005241, 0.24216345431293487],\n [0.0, 0.3594582764005241, 0.0, 0.5039226836320637],\n [0.39518833931501524, 0.24216345431293487, 0.5039226836320637, 0.0],\n]\n\ntransitions_umap = [\n [0.0, 0.5395987596963403, 0.0, 0.4604012403036599],\n [0.430368608684738, 0.0, 0.3176747629691457, 0.2519566283461165],\n [0.0, 0.40673754271561435, 0.0, 0.5932624572843856],\n [0.33921243006981133, 0.23275092618009624, 0.42803664375009237, 0.0],\n]\n\n\n# gauss kernel [diffmap, dpt] – knn and dense results\nconnectivities_gauss_knn = [\n [0.0, 0.8466368913650513, 0.0, 0.5660185813903809],\n [0.8466368913650513, 0.0, 0.4223647117614746, 0.4902938902378082],\n [0.0, 0.4223647117614746, 0.0, 0.5840492248535156],\n [0.5660185813903809, 0.4902938902378082, 0.5840492248535156, 0.0],\n]\n\nconnectivities_gauss_noknn = [\n [1.0, 0.676927387714386, 0.024883469566702843, 0.1962655782699585],\n [0.676927387714386, 1.0, 0.08414449542760849, 0.1353352814912796],\n [0.024883469566702843, 0.08414449542760849, 1.0, 0.16558068990707397],\n [0.1962655782699585, 0.1353352814912796, 0.16558068990707397, 1.0],\n]\n\ntransitions_sym_gauss_knn = [\n [0.0, 0.5146393179893494, 0.0, 0.36445462703704834],\n [0.5146393179893494, 0.0, 0.3581143319606781, 0.2239987552165985],\n [0.0, 0.3581143319606781, 0.0, 0.5245543718338013],\n [0.36445462703704834, 0.2239987552165985, 0.5245543718338013, 0.0],\n]\n\ntransitions_sym_gauss_noknn = [\n [\n 0.5093212127685547,\n 0.34393802285194397,\n 0.016115963459014893,\n 0.11607448011636734,\n ],\n [0.34393805265426636, 0.506855845451355, 0.054364752024412155, 0.07984541356563568],\n [\n 0.016115965321660042,\n 0.054364752024412155,\n 0.8235670328140259,\n 0.12452481687068939,\n ],\n [0.11607448011636734, 0.07984541356563568, 0.1245248094201088, 0.6867417693138123],\n]\n\ntransitions_gauss_knn = [\n [0.0, 0.5824036598205566, 0.0, 0.4175964295864105],\n [0.4547595679759979, 0.0, 0.3184431493282318, 0.22679725289344788],\n [0.0, 0.4027276933193207, 0.0, 0.5972723364830017],\n [0.3180755078792572, 0.22123482823371887, 0.46068981289863586, 0.0],\n]\n\ntransitions_gauss_noknn = [\n [0.5093212127685547, 0.3450769782066345, 0.01887294091284275, 0.12672874331474304],\n [0.34280285239219666, 0.506855845451355, 0.06345486640930176, 0.08688655495643616],\n [0.01376173086464405, 0.04657683148980141, 0.8235670328140259, 0.11609435081481934],\n [0.10631592571735382, 0.07337487488985062, 0.13356748223304749, 0.6867417693138123],\n]\n\n\ndef get_neighbors() -> Neighbors:\n return Neighbors(anndata_v0_8_constructor_compat(np.array(X)))\n\n\n@pytest.fixture\ndef neigh() -> Neighbors:\n return get_neighbors()\n\n\n@pytest.mark.parametrize(\"method\", [\"umap\", \"gauss\"])\ndef test_distances_euclidean(\n mocker: MockerFixture, neigh: Neighbors, method: Literal[\"umap\", \"gauss\"]\n):\n \"\"\"umap and gauss behave the same for distances.\n\n They call pynndescent for large data.\n \"\"\"\n from pynndescent import NNDescent\n\n # When trying to compress a too-small index, pynndescent complains\n mocker.patch.object(NNDescent, \"compress_index\", return_val=None)\n\n neigh.compute_neighbors(n_neighbors, method=method)\n np.testing.assert_allclose(neigh.distances.toarray(), distances_euclidean)\n\n\n@pytest.mark.parametrize(\n (\"transformer\", \"knn\"),\n [\n # knn=False trivially returns all distances\n pytest.param(None, False, id=\"knn=False\"),\n # pynndescent returns all distances when data is so small\n pytest.param(\"pynndescent\", True, id=\"pynndescent\"),\n # Explicit brute force also returns all distances\n pytest.param(\n KNeighborsTransformer(n_neighbors=n_neighbors, algorithm=\"brute\"),\n True,\n id=\"sklearn\",\n ),\n ],\n)\ndef test_distances_all(neigh: Neighbors, transformer, knn):\n neigh.compute_neighbors(\n n_neighbors, transformer=transformer, method=\"gauss\", knn=knn\n )\n dists = neigh.distances.toarray() if issparse(neigh.distances) else neigh.distances\n np.testing.assert_allclose(dists, distances_euclidean_all)\n\n\n@pytest.mark.parametrize(\n (\"method\", \"conn\", \"trans\", \"trans_sym\"),\n [\n pytest.param(\n \"umap\",\n connectivities_umap,\n transitions_umap,\n transitions_sym_umap,\n id=\"umap\",\n ),\n pytest.param(\n \"gauss\",\n connectivities_gauss_knn,\n transitions_gauss_knn,\n transitions_sym_gauss_knn,\n id=\"gauss\",\n ),\n ],\n)\ndef test_connectivities_euclidean(neigh: Neighbors, method, conn, trans, trans_sym):\n neigh.compute_neighbors(n_neighbors, method=method)\n np.testing.assert_allclose(neigh.connectivities.toarray(), conn)\n neigh.compute_transitions()\n np.testing.assert_allclose(neigh.transitions_sym.toarray(), trans_sym, rtol=1e-5)\n np.testing.assert_allclose(neigh.transitions.toarray(), trans, rtol=1e-5)\n\n\ndef test_gauss_noknn_connectivities_euclidean(neigh):\n neigh.compute_neighbors(n_neighbors, method=\"gauss\", knn=False)\n np.testing.assert_allclose(neigh.connectivities, connectivities_gauss_noknn)\n neigh.compute_transitions()\n np.testing.assert_allclose(\n neigh.transitions_sym, transitions_sym_gauss_noknn, rtol=1e-5\n )\n np.testing.assert_allclose(neigh.transitions, transitions_gauss_noknn, rtol=1e-5)\n\n\ndef test_metrics_argument():\n no_knn_euclidean = get_neighbors()\n no_knn_euclidean.compute_neighbors(\n n_neighbors, method=\"gauss\", knn=False, metric=\"euclidean\"\n )\n no_knn_manhattan = get_neighbors()\n no_knn_manhattan.compute_neighbors(\n n_neighbors, method=\"gauss\", knn=False, metric=\"manhattan\"\n )\n assert not np.allclose(no_knn_euclidean.distances, no_knn_manhattan.distances)\n\n\ndef test_use_rep_argument():\n adata = AnnData(np.random.randn(30, 300))\n sc.pp.pca(adata)\n neigh_pca = Neighbors(adata)\n neigh_pca.compute_neighbors(n_pcs=5, use_rep=\"X_pca\")\n neigh_none = Neighbors(adata)\n neigh_none.compute_neighbors(n_pcs=5, use_rep=None)\n np.testing.assert_allclose(\n neigh_pca.distances.toarray(), neigh_none.distances.toarray()\n )\n\n\n@pytest.mark.parametrize(\"conv\", [csr_matrix.toarray, csr_matrix])\ndef test_restore_n_neighbors(neigh, conv):\n neigh.compute_neighbors(n_neighbors, method=\"gauss\")\n\n ad = AnnData(np.array(X))\n # Allow deprecated usage for now\n with warnings.catch_warnings():\n warnings.filterwarnings(\"ignore\", category=FutureWarning, module=\"anndata\")\n ad.uns[\"neighbors\"] = dict(connectivities=conv(neigh.connectivities))\n neigh_restored = Neighbors(ad)\n assert neigh_restored.n_neighbors == 1\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\nimport pytest\nfrom anndata import AnnData\nfrom scipy import sparse\n\nimport scanpy as sc\nfrom scanpy.preprocessing._qc import (\n describe_obs,\n describe_var,\n top_proportions,\n top_segment_proportions,\n)\n\n\n@pytest.fixture\ndef anndata():\n a = np.random.binomial(100, 0.005, (1000, 1000))\n adata = AnnData(\n sparse.csr_matrix(a),\n obs=pd.DataFrame(index=[f\"cell{i}\" for i in range(a.shape[0])]),\n var=pd.DataFrame(index=[f\"gene{i}\" for i in range(a.shape[1])]),\n )\n return adata\n\n\n@pytest.mark.parametrize(\n \"a\",\n [np.ones((100, 100)), sparse.csr_matrix(np.ones((100, 100)))],\n ids=[\"dense\", \"sparse\"],\n)\ndef test_proportions(a):\n prop = top_proportions(a, 100)\n assert (prop[:, -1] == 1).all()\n assert np.array_equal(np.sort(prop, axis=1), prop)\n assert np.apply_along_axis(lambda x: len(np.unique(x)) == 1, 0, prop).all()\n assert (prop[:, 49] == 0.5).all()\n\n\ndef test_segments_binary():\n a = np.concatenate([np.zeros((300, 50)), np.ones((300, 50))], 1)\n a = np.apply_along_axis(np.random.permutation, 1, a)\n seg = top_segment_proportions(a, [25, 50, 100])\n assert (seg[:, 0] == 0.5).all()\n assert (top_segment_proportions(a, [25]) == 0.5).all()\n assert (seg[:, 1] == 1.0).all()\n assert (seg[:, 2] == 1.0).all()\n segfull = top_segment_proportions(a, np.arange(100) + 1)\n propfull = top_proportions(a, 100)\n assert (segfull == propfull).all()\n\n\n@pytest.mark.parametrize(\n \"cls\", [np.asarray, sparse.csr_matrix, sparse.csc_matrix, sparse.coo_matrix]\n)\ndef test_top_segments(cls):\n a = cls(np.ones((300, 100)))\n seg = top_segment_proportions(a, [50, 100])\n assert (seg[:, 0] == 0.5).all()\n assert (seg[:, 1] == 1.0).all()\n segfull = top_segment_proportions(a, np.arange(100) + 1)\n propfull = top_proportions(a, 100)\n assert (segfull == propfull).all()\n\n\n# While many of these are trivial,\n# they’re also just making sure the metrics are there\ndef test_qc_metrics():\n adata = AnnData(X=sparse.csr_matrix(np.random.binomial(100, 0.005, (1000, 1000))))\n adata.var[\"mito\"] = np.concatenate(\n (np.ones(100, dtype=bool), np.zeros(900, dtype=bool))\n )\n adata.var[\"negative\"] = False\n sc.pp.calculate_qc_metrics(adata, qc_vars=[\"mito\", \"negative\"], inplace=True)\n assert (adata.obs[\"n_genes_by_counts\"] < adata.shape[1]).all()\n assert (\n adata.obs[\"n_genes_by_counts\"] >= adata.obs[\"log1p_n_genes_by_counts\"]\n ).all()\n assert (adata.obs[\"total_counts\"] == np.ravel(adata.X.sum(axis=1))).all()\n assert (adata.obs[\"total_counts\"] >= adata.obs[\"log1p_total_counts\"]).all()\n assert (\n adata.obs[\"total_counts_mito\"] >= adata.obs[\"log1p_total_counts_mito\"]\n ).all()\n assert (adata.obs[\"total_counts_negative\"] == 0).all()\n assert (\n adata.obs[\"pct_counts_in_top_50_genes\"]\n <= adata.obs[\"pct_counts_in_top_100_genes\"]\n ).all()\n for col in filter(lambda x: \"negative\" not in x, adata.obs.columns):\n assert (adata.obs[col] >= 0).all() # Values should be positive or zero\n assert (adata.obs[col] != 0).any().all() # Nothing should be all zeros\n if col.startswith(\"pct_counts_in_top\"):\n assert (adata.obs[col] <= 100).all()\n assert (adata.obs[col] >= 0).all()\n for col in adata.var.columns:\n assert (adata.var[col] >= 0).all()\n assert (adata.var[\"mean_counts\"] < np.ravel(adata.X.max(axis=0).todense())).all()\n assert (adata.var[\"mean_counts\"] >= adata.var[\"log1p_mean_counts\"]).all()\n assert (adata.var[\"total_counts\"] >= adata.var[\"log1p_total_counts\"]).all()\n # Should return the same thing if run again\n old_obs, old_var = adata.obs.copy(), adata.var.copy()\n sc.pp.calculate_qc_metrics(adata, qc_vars=[\"mito\", \"negative\"], inplace=True)\n assert set(adata.obs.columns) == set(old_obs.columns)\n assert set(adata.var.columns) == set(old_var.columns)\n for col in adata.obs:\n assert np.allclose(adata.obs[col], old_obs[col])\n for col in adata.var:\n assert np.allclose(adata.var[col], old_var[col])\n # with log1p=False\n adata = AnnData(X=sparse.csr_matrix(np.random.binomial(100, 0.005, (1000, 1000))))\n adata.var[\"mito\"] = np.concatenate(\n (np.ones(100, dtype=bool), np.zeros(900, dtype=bool))\n )\n adata.var[\"negative\"] = False\n sc.pp.calculate_qc_metrics(\n adata, qc_vars=[\"mito\", \"negative\"], log1p=False, inplace=True\n )\n assert not np.any(adata.obs.columns.str.startswith(\"log1p_\"))\n assert not np.any(adata.var.columns.str.startswith(\"log1p_\"))\n\n\ndef adata_mito():\n a = np.random.binomial(100, 0.005, (1000, 1000))\n init_var = pd.DataFrame(\n dict(mito=np.concatenate((np.ones(100, dtype=bool), np.zeros(900, dtype=bool))))\n )\n adata_dense = AnnData(X=a, var=init_var.copy())\n return adata_dense, init_var\n\n\n@pytest.mark.parametrize(\n \"cls\", [np.asarray, sparse.csr_matrix, sparse.csc_matrix, sparse.coo_matrix]\n)\ndef test_qc_metrics_format(cls):\n adata_dense, init_var = adata_mito()\n sc.pp.calculate_qc_metrics(adata_dense, qc_vars=[\"mito\"], inplace=True)\n adata = AnnData(X=cls(adata_dense.X), var=init_var.copy())\n sc.pp.calculate_qc_metrics(adata, qc_vars=[\"mito\"], inplace=True)\n assert np.allclose(adata.obs, adata_dense.obs)\n for col in adata.var: # np.allclose doesn't like mix of types\n assert np.allclose(adata.var[col], adata_dense.var[col])\n\n\ndef test_qc_metrics_format_str_qc_vars():\n adata_dense, init_var = adata_mito()\n sc.pp.calculate_qc_metrics(adata_dense, qc_vars=\"mito\", inplace=True)\n adata = AnnData(X=adata_dense.X, var=init_var.copy())\n sc.pp.calculate_qc_metrics(adata, qc_vars=\"mito\", inplace=True)\n assert np.allclose(adata.obs, adata_dense.obs)\n for col in adata.var: # np.allclose doesn't like mix of types\n assert np.allclose(adata.var[col], adata_dense.var[col])\n\n\ndef test_qc_metrics_percentage(): # In response to #421\n adata_dense, init_var = adata_mito()\n sc.pp.calculate_qc_metrics(adata_dense, percent_top=[])\n sc.pp.calculate_qc_metrics(adata_dense, percent_top=())\n sc.pp.calculate_qc_metrics(adata_dense, percent_top=None)\n sc.pp.calculate_qc_metrics(adata_dense, percent_top=[1, 2, 3, 10])\n sc.pp.calculate_qc_metrics(adata_dense, percent_top=[1])\n with pytest.raises(IndexError):\n sc.pp.calculate_qc_metrics(adata_dense, percent_top=[1, 2, 3, -5])\n with pytest.raises(IndexError):\n sc.pp.calculate_qc_metrics(adata_dense, percent_top=[20, 30, 1001])\n\n\ndef test_layer_raw(anndata):\n adata = anndata.copy()\n adata.raw = adata.copy()\n adata.layers[\"counts\"] = adata.X.copy()\n obs_orig, var_orig = sc.pp.calculate_qc_metrics(adata)\n sc.pp.log1p(adata) # To be sure they aren't reusing it\n obs_layer, var_layer = sc.pp.calculate_qc_metrics(adata, layer=\"counts\")\n obs_raw, var_raw = sc.pp.calculate_qc_metrics(adata, use_raw=True)\n assert np.allclose(obs_orig, obs_layer)\n assert np.allclose(obs_orig, obs_raw)\n assert np.allclose(var_orig, var_layer)\n assert np.allclose(var_orig, var_raw)\n\n\ndef test_inner_methods(anndata):\n adata = anndata.copy()\n full_inplace = adata.copy()\n partial_inplace = adata.copy()\n obs_orig, var_orig = sc.pp.calculate_qc_metrics(adata)\n assert np.all(obs_orig == describe_obs(adata))\n assert np.all(var_orig == describe_var(adata))\n sc.pp.calculate_qc_metrics(full_inplace, inplace=True)\n describe_obs(partial_inplace, inplace=True)\n describe_var(partial_inplace, inplace=True)\n assert np.all(full_inplace.obs == partial_inplace.obs)\n assert np.all(full_inplace.var == partial_inplace.var)\n assert np.all(partial_inplace.obs[obs_orig.columns] == obs_orig)\n assert np.all(partial_inplace.var[var_orig.columns] == var_orig)\n\n\nfrom __future__ import annotations\n\nimport pandas as pd\nimport pytest\n\nimport scanpy as sc\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\nfrom testing.scanpy._pytest.marks import needs\n\n\n@pytest.mark.internet\n@needs.gprofiler\ndef test_enrich():\n pbmc = pbmc68k_reduced()\n sc.tl.rank_genes_groups(pbmc, \"louvain\", n_genes=pbmc.shape[1])\n enrich_anndata = sc.queries.enrich(pbmc, \"1\")\n de = pd.DataFrame()\n for k in [\"pvals_adj\", \"names\"]:\n de[k] = pbmc.uns[\"rank_genes_groups\"][k][\"1\"]\n de_genes = de.loc[lambda x: x[\"pvals_adj\"] < 0.05, \"names\"]\n enrich_list = sc.queries.enrich(list(de_genes))\n assert (enrich_anndata == enrich_list).all().all()\n\n # scverse/scanpy/#1043\n sc.tl.filter_rank_genes_groups(pbmc, min_fold_change=1)\n sc.queries.enrich(pbmc, \"1\")\n\n gene_dict = {\"set1\": [\"KLF4\", \"PAX5\"], \"set2\": [\"SOX2\", \"NANOG\"]}\n enrich_list = sc.queries.enrich(\n gene_dict, org=\"hsapiens\", gprofiler_kwargs=dict(sources=[\"GO:BP\"])\n )\n assert \"set1\" in enrich_list[\"query\"].unique()\n assert \"set2\" in enrich_list[\"query\"].unique()\n\n\n@pytest.mark.internet\n@needs.pybiomart\ndef test_mito_genes():\n pbmc = pbmc68k_reduced()\n mt_genes = sc.queries.mitochondrial_genes(\"hsapiens\")\n assert (\n pbmc.var_names.isin(mt_genes[\"external_gene_name\"]).sum() == 1\n ) # Should only be MT-ND3\n\n\nfrom __future__ import annotations\n\nfrom operator import mul, truediv\nfrom types import ModuleType\n\nimport numpy as np\nimport pytest\nfrom anndata.tests.helpers import asarray\nfrom scipy.sparse import csr_matrix, issparse\n\nfrom scanpy._compat import DaskArray\nfrom scanpy._utils import (\n axis_mul_or_truediv,\n axis_sum,\n check_nonnegative_integers,\n descend_classes_and_funcs,\n elem_mul,\n is_constant,\n)\nfrom testing.scanpy._pytest.marks import needs\nfrom testing.scanpy._pytest.params import (\n ARRAY_TYPES,\n ARRAY_TYPES_DASK,\n ARRAY_TYPES_SPARSE,\n ARRAY_TYPES_SPARSE_DASK_UNSUPPORTED,\n)\n\n\ndef test_descend_classes_and_funcs():\n # create module hierarchy\n a = ModuleType(\"a\")\n a.b = ModuleType(\"a.b\")\n\n # populate with classes\n a.A = type(\"A\", (), {})\n a.A.__module__ = a.__name__\n a.b.B = type(\"B\", (), {})\n a.b.B.__module__ = a.b.__name__\n\n # create a loop to check if that gets caught\n a.b.a = a\n\n assert {a.A, a.b.B} == set(descend_classes_and_funcs(a, \"a\"))\n\n\ndef test_axis_mul_or_truediv_badop():\n dividend = np.array([[0, 1.0, 1.0], [1.0, 0, 1.0]])\n divisor = np.array([0.1, 0.2])\n with pytest.raises(ValueError, match=\".*not one of truediv or mul\"):\n axis_mul_or_truediv(dividend, divisor, op=np.add, axis=0)\n\n\ndef test_axis_mul_or_truediv_bad_out():\n dividend = csr_matrix(np.array([[0, 1.0, 1.0], [1.0, 0, 1.0]]))\n divisor = np.array([0.1, 0.2])\n with pytest.raises(ValueError, match=\"`out` argument provided but not equal to X\"):\n axis_mul_or_truediv(dividend, divisor, op=truediv, out=dividend.copy(), axis=0)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\"op\", [truediv, mul])\ndef test_scale_row(array_type, op):\n dividend = array_type(asarray([[0, 1.0, 1.0], [1.0, 0, 1.0]]))\n divisor = np.array([0.1, 0.2])\n if op is mul:\n divisor = 1 / divisor\n expd = np.array([[0, 10.0, 10.0], [5.0, 0, 5.0]])\n out = dividend if issparse(dividend) or isinstance(dividend, np.ndarray) else None\n res = asarray(axis_mul_or_truediv(dividend, divisor, op=op, axis=0, out=out))\n np.testing.assert_array_equal(res, expd)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\"op\", [truediv, mul])\ndef test_scale_column(array_type, op):\n dividend = array_type(asarray([[0, 1.0, 2.0], [3.0, 0, 4.0]]))\n divisor = np.array([0.1, 0.2, 0.5])\n if op is mul:\n divisor = 1 / divisor\n expd = np.array([[0, 5.0, 4.0], [30.0, 0, 8.0]])\n out = dividend if issparse(dividend) or isinstance(dividend, np.ndarray) else None\n res = asarray(axis_mul_or_truediv(dividend, divisor, op=op, axis=1, out=out))\n np.testing.assert_array_equal(res, expd)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\ndef test_divide_by_zero(array_type):\n dividend = array_type(asarray([[0, 1.0, 2.0], [3.0, 0, 4.0]]))\n divisor = np.array([0.1, 0.2, 0.0])\n expd = np.array([[0, 5.0, 2.0], [30.0, 0, 4.0]])\n res = asarray(\n axis_mul_or_truediv(\n dividend, divisor, op=truediv, axis=1, allow_divide_by_zero=False\n )\n )\n np.testing.assert_array_equal(res, expd)\n res = asarray(\n axis_mul_or_truediv(\n dividend, divisor, op=truediv, axis=1, allow_divide_by_zero=True\n )\n )\n expd = np.array([[0, 5.0, np.inf], [30.0, 0, np.inf]])\n np.testing.assert_array_equal(res, expd)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES_SPARSE)\ndef test_scale_out_with_dask_or_sparse_raises(array_type):\n dividend = array_type(asarray([[0, 1.0, 2.0], [3.0, 0, 4.0]]))\n divisor = np.array([0.1, 0.2, 0.5])\n if isinstance(dividend, DaskArray):\n with pytest.raises(\n TypeError if \"dask\" in array_type.__name__ else ValueError,\n match=\"`out`*\",\n ):\n axis_mul_or_truediv(dividend, divisor, op=truediv, axis=1, out=dividend)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES_DASK)\n@pytest.mark.parametrize(\"axis\", [0, 1])\n@pytest.mark.parametrize(\"op\", [truediv, mul])\ndef test_scale_rechunk(array_type, axis, op):\n import dask.array as da\n\n dividend = array_type(\n asarray([[0, 1.0, 2.0], [3.0, 0, 4.0], [3.0, 0, 4.0]])\n ).rechunk(((3,), (3,)))\n divisor = da.from_array(np.array([0.1, 0.2, 0.5]), chunks=(1,))\n if op is mul:\n divisor = 1 / divisor\n if axis == 1:\n expd = np.array([[0, 5.0, 4.0], [30.0, 0, 8.0], [30.0, 0, 8.0]])\n else:\n expd = np.array([[0, 10.0, 20.0], [15.0, 0, 20.0], [6.0, 0, 8.0]])\n out = dividend if issparse(dividend) or isinstance(dividend, np.ndarray) else None\n with pytest.warns(UserWarning, match=\"Rechunking scaling_array*\"):\n res = asarray(axis_mul_or_truediv(dividend, divisor, op=op, axis=axis, out=out))\n np.testing.assert_array_equal(res, expd)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\ndef test_elem_mul(array_type):\n m1 = array_type(asarray([[0, 1, 1], [1, 0, 1]]))\n m2 = array_type(asarray([[2, 2, 1], [3, 2, 0]]))\n expd = np.array([[0, 2, 1], [3, 0, 0]])\n res = asarray(elem_mul(m1, m2))\n np.testing.assert_array_equal(res, expd)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\ndef test_axis_sum(array_type):\n m1 = array_type(asarray([[0, 1, 1], [1, 0, 1]]))\n expd_0 = np.array([1, 1, 2])\n expd_1 = np.array([2, 2])\n res_0 = asarray(axis_sum(m1, axis=0))\n res_1 = asarray(axis_sum(m1, axis=1))\n if \"matrix\" in array_type.__name__: # for sparse since dimension is kept\n res_0 = res_0.ravel()\n res_1 = res_1.ravel()\n np.testing.assert_array_equal(res_0, expd_0)\n np.testing.assert_array_equal(res_1, expd_1)\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\n (\"array_value\", \"expected\"),\n [\n pytest.param(\n np.random.poisson(size=(100, 100)).astype(np.float64),\n True,\n id=\"poisson-float64\",\n ),\n pytest.param(\n np.random.poisson(size=(100, 100)).astype(np.uint32),\n True,\n id=\"poisson-uint32\",\n ),\n pytest.param(np.random.normal(size=(100, 100)), False, id=\"normal\"),\n pytest.param(np.array([[0, 0, 0], [0, -1, 0], [0, 0, 0]]), False, id=\"middle\"),\n ],\n)\ndef test_check_nonnegative_integers(array_type, array_value, expected):\n X = array_type(array_value)\n\n received = check_nonnegative_integers(X)\n if isinstance(X, DaskArray):\n assert isinstance(received, DaskArray)\n # compute\n received = received.compute()\n assert not isinstance(received, DaskArray)\n if isinstance(received, np.bool_):\n # convert to python bool\n received = received.item()\n assert received is expected\n\n\n# TODO: Make it work for sparse-in-dask\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES_SPARSE_DASK_UNSUPPORTED)\ndef test_is_constant(array_type):\n constant_inds = [1, 3]\n A = np.arange(20).reshape(5, 4)\n A[constant_inds, :] = 10\n A = array_type(A)\n AT = array_type(A.T)\n\n assert not is_constant(A)\n assert not np.any(is_constant(A, axis=0))\n np.testing.assert_array_equal(\n [False, True, False, True, False], is_constant(A, axis=1)\n )\n\n assert not is_constant(AT)\n assert not np.any(is_constant(AT, axis=1))\n np.testing.assert_array_equal(\n [False, True, False, True, False], is_constant(AT, axis=0)\n )\n\n\n@needs.dask\n@pytest.mark.parametrize(\n (\"axis\", \"expected\"),\n [\n pytest.param(None, False, id=\"None\"),\n pytest.param(0, [True, True, False, False], id=\"0\"),\n pytest.param(1, [False, False, True, True, False, True], id=\"1\"),\n ],\n)\n@pytest.mark.parametrize(\"block_type\", [np.array, csr_matrix])\ndef test_is_constant_dask(axis, expected, block_type):\n import dask.array as da\n\n if (axis is None) and block_type is csr_matrix:\n pytest.skip(\"Dask has weak support for scipy sparse matrices\")\n\n x_data = [\n [0, 0, 1, 1],\n [0, 0, 1, 1],\n [0, 0, 0, 0],\n [0, 0, 0, 0],\n [0, 0, 1, 0],\n [0, 0, 0, 0],\n ]\n x = da.from_array(np.array(x_data), chunks=2).map_blocks(block_type)\n result = is_constant(x, axis=axis).compute()\n np.testing.assert_array_equal(expected, result)\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nfrom anndata import AnnData\n\nimport scanpy as sc\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\n\n\ndef test_embedding_density():\n # Test that density values are scaled\n # Test that the highest value is in the middle for a grid layout\n test_data = AnnData(X=np.ones((9, 10)))\n test_data.obsm[\"X_test\"] = np.array([[x, y] for x in range(3) for y in range(3)])\n sc.tl.embedding_density(test_data, \"test\")\n\n max_dens = np.max(test_data.obs[\"test_density\"])\n min_dens = np.min(test_data.obs[\"test_density\"])\n max_idx = test_data.obs[\"test_density\"].idxmax()\n\n assert max_idx == \"4\"\n assert max_dens == 1\n assert min_dens == 0\n\n\ndef test_embedding_density_plot():\n # Test that sc.pl.embedding_density() runs without error\n adata = pbmc68k_reduced()\n sc.tl.embedding_density(adata, \"umap\")\n sc.pl.embedding_density(adata, \"umap\", key=\"umap_density\", show=False)\n\n\nfrom __future__ import annotations\n\nfrom functools import partial\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nimport matplotlib as mpl\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nimport pytest\nimport seaborn as sns\nfrom matplotlib.colors import Normalize\nfrom matplotlib.testing.compare import compare_images\n\nimport scanpy as sc\nfrom testing.scanpy._helpers.data import pbmc3k_processed\n\nif TYPE_CHECKING:\n from scanpy.plotting._utils import _LegendLoc\n\n\nHERE: Path = Path(__file__).parent\nROOT = HERE / \"_images\"\n\nMISSING_VALUES_ROOT = ROOT / \"embedding-missing-values\"\n\n\ndef check_images(pth1, pth2, *, tol):\n result = compare_images(pth1, pth2, tol=tol)\n assert result is None, result\n\n\n@pytest.fixture(scope=\"module\")\ndef adata():\n \"\"\"A bit cute.\"\"\"\n from matplotlib.image import imread\n from sklearn.cluster import DBSCAN\n from sklearn.datasets import make_blobs\n\n empty_pixel = np.array([1.0, 1.0, 1.0, 0]).reshape(1, 1, -1)\n image = imread(HERE.parent / \"docs/_static/img/Scanpy_Logo_RGB.png\")\n x, y = np.where(np.logical_and.reduce(~np.equal(image, empty_pixel), axis=2))\n\n # Just using to calculate the hex coords\n hexes = plt.hexbin(x, y, gridsize=(44, 100))\n counts = hexes.get_array()\n pixels = hexes.get_offsets()[counts != 0]\n plt.close()\n\n labels = DBSCAN(eps=20, min_samples=2).fit(pixels).labels_\n order = np.argsort(labels)\n adata = sc.AnnData(\n make_blobs(\n pd.Series(labels[order]).value_counts().values,\n n_features=20,\n shuffle=False,\n random_state=42,\n )[0],\n obs={\"label\": pd.Categorical(labels[order].astype(str))},\n obsm={\"spatial\": pixels[order, ::-1]},\n uns={\n \"spatial\": {\n \"scanpy_img\": {\n \"images\": {\"hires\": image},\n \"scalefactors\": {\n \"tissue_hires_scalef\": 1,\n \"spot_diameter_fullres\": 10,\n },\n }\n }\n },\n )\n sc.pp.pca(adata)\n\n # Adding some missing values\n adata.obs[\"label_missing\"] = adata.obs[\"label\"].copy()\n adata.obs[\"label_missing\"][::2] = np.nan\n\n adata.obs[\"1_missing\"] = adata.obs_vector(\"1\")\n adata.obs.loc[\n adata.obsm[\"spatial\"][:, 0] < adata.obsm[\"spatial\"][:, 0].mean(), \"1_missing\"\n ] = np.nan\n\n return adata\n\n\n@pytest.fixture\ndef fixture_request(request):\n \"\"\"Returns a Request object.\n\n Allows you to access names of parameterized tests from within a test.\n \"\"\"\n return request\n\n\n@pytest.fixture(\n params=[(0, 0, 0, 1), None],\n ids=[\"na_color.black_tup\", \"na_color.default\"],\n)\ndef na_color(request):\n return request.param\n\n\n@pytest.fixture(params=[True, False], ids=[\"na_in_legend.True\", \"na_in_legend.False\"])\ndef na_in_legend(request):\n return request.param\n\n\n@pytest.fixture(\n params=[partial(sc.pl.pca, show=False), partial(sc.pl.spatial, show=False)],\n ids=[\"pca\", \"spatial\"],\n)\ndef plotfunc(request):\n return request.param\n\n\n@pytest.fixture(\n params=[\"on data\", \"right margin\", \"lower center\", None],\n ids=[\"legend.on_data\", \"legend.on_right\", \"legend.on_bottom\", \"legend.off\"],\n)\ndef legend_loc(request) -> _LegendLoc | None:\n return request.param\n\n\n@pytest.fixture(\n params=[lambda x: list(x.cat.categories[:3]), lambda x: []],\n ids=[\"groups.3\", \"groups.all\"],\n)\ndef groupsfunc(request):\n return request.param\n\n\n@pytest.fixture(\n params=[\n pytest.param(\n {\"vmin\": None, \"vmax\": None, \"vcenter\": None, \"norm\": None},\n id=\"vbounds.default\",\n ),\n pytest.param(\n {\"vmin\": 0, \"vmax\": 5, \"vcenter\": None, \"norm\": None}, id=\"vbounds.numbers\"\n ),\n pytest.param(\n {\"vmin\": \"p15\", \"vmax\": \"p90\", \"vcenter\": None, \"norm\": None},\n id=\"vbounds.percentile\",\n ),\n pytest.param(\n {\"vmin\": 0, \"vmax\": \"p99\", \"vcenter\": 0.1, \"norm\": None},\n id=\"vbounds.vcenter\",\n ),\n pytest.param(\n {\"vmin\": None, \"vmax\": None, \"vcenter\": None, \"norm\": Normalize(0, 5)},\n id=\"vbounds.norm\",\n ),\n ]\n)\ndef vbounds(request):\n return request.param\n\n\ndef test_missing_values_categorical(\n *,\n fixture_request: pytest.FixtureRequest,\n image_comparer,\n adata,\n plotfunc,\n na_color,\n na_in_legend,\n legend_loc,\n groupsfunc,\n):\n save_and_compare_images = partial(image_comparer, MISSING_VALUES_ROOT, tol=15)\n\n base_name = fixture_request.node.name\n\n # Passing through a dict so it's easier to use default values\n kwargs = {}\n kwargs[\"legend_loc\"] = legend_loc\n kwargs[\"groups\"] = groupsfunc(adata.obs[\"label\"])\n if na_color is not None:\n kwargs[\"na_color\"] = na_color\n kwargs[\"na_in_legend\"] = na_in_legend\n\n plotfunc(adata, color=[\"label\", \"label_missing\"], **kwargs)\n\n save_and_compare_images(base_name)\n\n\ndef test_missing_values_continuous(\n *,\n fixture_request: pytest.FixtureRequest,\n image_comparer,\n adata,\n plotfunc,\n na_color,\n vbounds,\n):\n save_and_compare_images = partial(image_comparer, MISSING_VALUES_ROOT, tol=15)\n\n base_name = fixture_request.node.name\n\n # Passing through a dict so it's easier to use default values\n kwargs = {}\n kwargs.update(vbounds)\n if na_color is not None:\n kwargs[\"na_color\"] = na_color\n\n plotfunc(adata, color=[\"1\", \"1_missing\"], **kwargs)\n\n save_and_compare_images(base_name)\n\n\ndef test_enumerated_palettes(fixture_request, adata, tmpdir, plotfunc):\n tmpdir = Path(tmpdir)\n base_name = fixture_request.node.name\n\n categories = adata.obs[\"label\"].cat.categories\n colors_rgb = dict(zip(categories, sns.color_palette(n_colors=12)))\n\n dict_pth = tmpdir / f\"rgbdict_{base_name}.png\"\n list_pth = tmpdir / f\"rgblist_{base_name}.png\"\n\n # making a copy so colors aren't saved\n plotfunc(adata.copy(), color=\"label\", palette=colors_rgb)\n plt.savefig(dict_pth, dpi=40)\n plt.close()\n plotfunc(adata.copy(), color=\"label\", palette=[colors_rgb[c] for c in categories])\n plt.savefig(list_pth, dpi=40)\n plt.close()\n\n check_images(dict_pth, list_pth, tol=15)\n\n\ndef test_dimension_broadcasting(adata, tmpdir, check_same_image):\n tmpdir = Path(tmpdir)\n\n with pytest.raises(\n ValueError,\n match=r\"Could not broadcast together arguments with shapes: \\[2, 3, 1\\]\",\n ):\n sc.pl.pca(\n adata, color=[\"label\", \"1_missing\"], dimensions=[(0, 1), (1, 2), (2, 3)]\n )\n\n dims_pth = tmpdir / \"broadcast_dims.png\"\n color_pth = tmpdir / \"broadcast_colors.png\"\n\n sc.pl.pca(adata, color=[\"label\", \"label\", \"label\"], dimensions=(2, 3), show=False)\n plt.savefig(dims_pth, dpi=40)\n plt.close()\n sc.pl.pca(adata, color=\"label\", dimensions=[(2, 3), (2, 3), (2, 3)], show=False)\n plt.savefig(color_pth, dpi=40)\n plt.close()\n\n check_same_image(dims_pth, color_pth, tol=5)\n\n\ndef test_marker_broadcasting(adata, tmpdir, check_same_image):\n tmpdir = Path(tmpdir)\n\n with pytest.raises(\n ValueError,\n match=r\"Could not broadcast together arguments with shapes: \\[2, 1, 3\\]\",\n ):\n sc.pl.pca(adata, color=[\"label\", \"1_missing\"], marker=[\".\", \"^\", \"x\"])\n\n dims_pth = tmpdir / \"broadcast_markers.png\"\n color_pth = tmpdir / \"broadcast_colors_for_markers.png\"\n\n sc.pl.pca(adata, color=[\"label\", \"label\", \"label\"], marker=\"^\", show=False)\n plt.savefig(dims_pth, dpi=40)\n plt.close()\n sc.pl.pca(adata, color=\"label\", marker=[\"^\", \"^\", \"^\"], show=False)\n plt.savefig(color_pth, dpi=40)\n plt.close()\n\n check_same_image(dims_pth, color_pth, tol=5)\n\n\ndef test_dimensions_same_as_components(adata, tmpdir, check_same_image):\n tmpdir = Path(tmpdir)\n adata = adata.copy()\n adata.obs[\"mean\"] = np.ravel(adata.X.mean(axis=1))\n\n comp_pth = tmpdir / \"components_plot.png\"\n dims_pth = tmpdir / \"dimension_plot.png\"\n\n # TODO: Deprecate components kwarg\n # with pytest.warns(FutureWarning, match=r\"components .* deprecated\"):\n sc.pl.pca(\n adata,\n color=[\"mean\", \"label\"],\n components=[\"1,2\", \"2,3\"],\n show=False,\n )\n plt.savefig(comp_pth, dpi=40)\n plt.close()\n\n sc.pl.pca(\n adata,\n color=[\"mean\", \"mean\", \"label\", \"label\"],\n dimensions=[(0, 1), (1, 2), (0, 1), (1, 2)],\n show=False,\n )\n plt.savefig(dims_pth, dpi=40)\n plt.close()\n\n check_same_image(dims_pth, comp_pth, tol=5)\n\n\ndef test_embedding_colorbar_location(image_comparer):\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n adata = pbmc3k_processed().raw.to_adata()\n\n sc.pl.pca(adata, color=\"LDHB\", colorbar_loc=None)\n\n save_and_compare_images(\"no_colorbar\")\n\n\n# Spatial specific\n\n\ndef test_visium_circles(image_comparer): # standard visium data\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n adata = sc.read_visium(HERE / \"_data\" / \"visium_data\" / \"1.0.0\")\n adata.obs = adata.obs.astype({\"array_row\": \"str\"})\n\n sc.pl.spatial(\n adata,\n color=\"array_row\",\n groups=[\"24\", \"33\"],\n crop_coord=(100, 400, 400, 100),\n alpha=0.5,\n size=1.3,\n show=False,\n )\n\n save_and_compare_images(\"spatial_visium\")\n\n\ndef test_visium_default(image_comparer): # default values\n from packaging.version import parse as parse_version\n\n if parse_version(mpl.__version__) < parse_version(\"3.7.0\"):\n pytest.xfail(\"Matplotlib 3.7.0+ required for this test\")\n\n save_and_compare_images = partial(image_comparer, ROOT, tol=5)\n\n adata = sc.read_visium(HERE / \"_data\" / \"visium_data\" / \"1.0.0\")\n adata.obs = adata.obs.astype({\"array_row\": \"str\"})\n\n # Points default to transparent if an image is included\n sc.pl.spatial(adata, show=False)\n\n save_and_compare_images(\"spatial_visium_default\")\n\n\ndef test_visium_empty_img_key(image_comparer): # visium coordinates but image empty\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n adata = sc.read_visium(HERE / \"_data\" / \"visium_data\" / \"1.0.0\")\n adata.obs = adata.obs.astype({\"array_row\": \"str\"})\n\n sc.pl.spatial(adata, img_key=None, color=\"array_row\", show=False)\n\n save_and_compare_images(\"spatial_visium_empty_image\")\n\n sc.pl.embedding(adata, basis=\"spatial\", color=\"array_row\", show=False)\n save_and_compare_images(\"spatial_visium_embedding\")\n\n\ndef test_spatial_general(image_comparer): # general coordinates\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n adata = sc.read_visium(HERE / \"_data\" / \"visium_data\" / \"1.0.0\")\n adata.obs = adata.obs.astype({\"array_row\": \"str\"})\n spatial_metadata = adata.uns.pop(\n \"spatial\"\n ) # spatial data don't have imgs, so remove entry from uns\n # Required argument for now\n spot_size = list(spatial_metadata.values())[0][\"scalefactors\"][\n \"spot_diameter_fullres\"\n ]\n\n sc.pl.spatial(adata, show=False, spot_size=spot_size)\n save_and_compare_images(\"spatial_general_nocol\")\n\n # category\n sc.pl.spatial(adata, show=False, spot_size=spot_size, color=\"array_row\")\n save_and_compare_images(\"spatial_general_cat\")\n\n # continuous\n sc.pl.spatial(adata, show=False, spot_size=spot_size, color=\"array_col\")\n save_and_compare_images(\"spatial_general_cont\")\n\n\ndef test_spatial_external_img(image_comparer): # external image\n save_and_compare_images = partial(image_comparer, ROOT, tol=15)\n\n adata = sc.read_visium(HERE / \"_data\" / \"visium_data\" / \"1.0.0\")\n adata.obs = adata.obs.astype({\"array_row\": \"str\"})\n\n img = adata.uns[\"spatial\"][\"custom\"][\"images\"][\"hires\"]\n scalef = adata.uns[\"spatial\"][\"custom\"][\"scalefactors\"][\"tissue_hires_scalef\"]\n sc.pl.spatial(\n adata,\n color=\"array_row\",\n scale_factor=scalef,\n img=img,\n basis=\"spatial\",\n show=False,\n )\n save_and_compare_images(\"spatial_external_img\")\n\n\n@pytest.fixture(scope=\"module\")\ndef equivalent_spatial_plotters(adata):\n no_spatial = adata.copy()\n del no_spatial.uns[\"spatial\"]\n\n img_key = \"hires\"\n library_id = list(adata.uns[\"spatial\"])[0]\n spatial_data = adata.uns[\"spatial\"][library_id]\n img = spatial_data[\"images\"][img_key]\n scale_factor = spatial_data[\"scalefactors\"][f\"tissue_{img_key}_scalef\"]\n spot_size = spatial_data[\"scalefactors\"][\"spot_diameter_fullres\"]\n\n orig_plotter = partial(sc.pl.spatial, adata, color=\"1\", show=False)\n removed_plotter = partial(\n sc.pl.spatial,\n no_spatial,\n color=\"1\",\n img=img,\n scale_factor=scale_factor,\n spot_size=spot_size,\n show=False,\n )\n\n return (orig_plotter, removed_plotter)\n\n\n@pytest.fixture(scope=\"module\")\ndef equivalent_spatial_plotters_no_img(equivalent_spatial_plotters):\n orig, removed = equivalent_spatial_plotters\n return (partial(orig, img_key=None), partial(removed, img=None, scale_factor=None))\n\n\n@pytest.fixture(\n params=[\n pytest.param({\"crop_coord\": (50, 200, 0, 500)}, id=\"crop\"),\n pytest.param({\"size\": 0.5}, id=\"size:.5\"),\n pytest.param({\"size\": 2}, id=\"size:2\"),\n pytest.param({\"spot_size\": 5}, id=\"spotsize\"),\n pytest.param({\"bw\": True}, id=\"bw\"),\n # Shape of the image for particular fixture, should not be hardcoded like this\n pytest.param({\"img\": np.ones((774, 1755, 4)), \"scale_factor\": 1.0}, id=\"img\"),\n pytest.param(\n {\"na_color\": (0, 0, 0, 0), \"color\": \"1_missing\"}, id=\"na_color.transparent\"\n ),\n pytest.param(\n {\"na_color\": \"lightgray\", \"color\": \"1_missing\"}, id=\"na_color.lightgray\"\n ),\n ]\n)\ndef spatial_kwargs(request):\n return request.param\n\n\ndef test_manual_equivalency(equivalent_spatial_plotters, tmpdir, spatial_kwargs):\n \"\"\"\n Tests that manually passing values to sc.pl.spatial is similar to automatic extraction.\n \"\"\"\n orig, removed = equivalent_spatial_plotters\n\n TESTDIR = Path(tmpdir)\n orig_pth = TESTDIR / \"orig.png\"\n removed_pth = TESTDIR / \"removed.png\"\n\n orig(**spatial_kwargs)\n plt.savefig(orig_pth, dpi=40)\n plt.close()\n removed(**spatial_kwargs)\n plt.savefig(removed_pth, dpi=40)\n plt.close()\n\n check_images(orig_pth, removed_pth, tol=1)\n\n\ndef test_manual_equivalency_no_img(\n equivalent_spatial_plotters_no_img, tmpdir, spatial_kwargs\n):\n if \"bw\" in spatial_kwargs:\n # Has no meaning when there is no image\n pytest.skip()\n orig, removed = equivalent_spatial_plotters_no_img\n\n TESTDIR = Path(tmpdir)\n orig_pth = TESTDIR / \"orig.png\"\n removed_pth = TESTDIR / \"removed.png\"\n\n orig(**spatial_kwargs)\n plt.savefig(orig_pth, dpi=40)\n plt.close()\n removed(**spatial_kwargs)\n plt.savefig(removed_pth, dpi=40)\n plt.close()\n\n check_images(orig_pth, removed_pth, tol=1)\n\n\ndef test_white_background_vs_no_img(adata, tmpdir, spatial_kwargs):\n if {\"bw\", \"img\", \"img_key\", \"na_color\"}.intersection(spatial_kwargs):\n # These arguments don't make sense for this check\n pytest.skip()\n\n white_background = np.ones_like(\n adata.uns[\"spatial\"][\"scanpy_img\"][\"images\"][\"hires\"]\n )\n TESTDIR = Path(tmpdir)\n white_pth = TESTDIR / \"white_background.png\"\n noimg_pth = TESTDIR / \"no_img.png\"\n\n sc.pl.spatial(\n adata,\n color=\"2\",\n img=white_background,\n scale_factor=1.0,\n show=False,\n **spatial_kwargs,\n )\n plt.savefig(white_pth)\n sc.pl.spatial(adata, color=\"2\", img_key=None, show=False, **spatial_kwargs)\n plt.savefig(noimg_pth)\n\n check_images(white_pth, noimg_pth, tol=1)\n\n\ndef test_spatial_na_color(adata, tmpdir):\n \"\"\"\n Check that na_color defaults to transparent when an image is present, light gray when not.\n \"\"\"\n white_background = np.ones_like(\n adata.uns[\"spatial\"][\"scanpy_img\"][\"images\"][\"hires\"]\n )\n TESTDIR = Path(tmpdir)\n lightgray_pth = TESTDIR / \"lightgray.png\"\n transparent_pth = TESTDIR / \"transparent.png\"\n noimg_pth = TESTDIR / \"noimg.png\"\n whiteimg_pth = TESTDIR / \"whiteimg.png\"\n\n def plot(pth, **kwargs):\n sc.pl.spatial(adata, color=\"1_missing\", show=False, **kwargs)\n plt.savefig(pth, dpi=40)\n plt.close()\n\n plot(lightgray_pth, na_color=\"lightgray\", img_key=None)\n plot(transparent_pth, na_color=(0.0, 0.0, 0.0, 0.0), img_key=None)\n plot(noimg_pth, img_key=None)\n plot(whiteimg_pth, img=white_background, scale_factor=1.0)\n\n check_images(lightgray_pth, noimg_pth, tol=1)\n check_images(transparent_pth, whiteimg_pth, tol=1)\n with pytest.raises(AssertionError):\n check_images(lightgray_pth, transparent_pth, tol=1)\n\n\nfrom __future__ import annotations\n\nimport sys\nfrom pathlib import Path\nfrom textwrap import dedent\nfrom typing import TYPE_CHECKING, TypedDict, Union, cast\n\nimport pytest\n\n# just import for the IMPORTED check\nimport scanpy as _sc # noqa: F401\n\nif TYPE_CHECKING: # So editors understand that we’re using those fixtures\n import os\n from collections.abc import Generator\n\n from testing.scanpy._pytest.fixtures import * # noqa: F403\n\n# define this after importing scanpy but before running tests\nIMPORTED = frozenset(sys.modules.keys())\n\n\n@pytest.fixture(scope=\"session\", autouse=True)\ndef _manage_log_handlers() -> Generator[None, None, None]:\n \"\"\"Remove handlers from all loggers on session teardown.\n\n Fixes .\n See also .\n \"\"\"\n import logging\n\n import scanpy as sc\n\n yield\n\n loggers = [\n sc.settings._root_logger,\n logging.getLogger(),\n *logging.Logger.manager.loggerDict.values(),\n ]\n for logger in loggers:\n if not isinstance(logger, logging.Logger):\n continue # loggerDict can contain `logging.Placeholder`s\n for handler in logger.handlers[:]:\n if isinstance(handler, logging.StreamHandler):\n logger.removeHandler(handler)\n\n\n@pytest.fixture(autouse=True)\ndef _caplog_adapter(caplog: pytest.LogCaptureFixture) -> Generator[None, None, None]:\n \"\"\"Allow use of scanpy’s logger with caplog\"\"\"\n import scanpy as sc\n\n sc.settings._root_logger.addHandler(caplog.handler)\n yield\n sc.settings._root_logger.removeHandler(caplog.handler)\n\n\n@pytest.fixture\ndef imported_modules():\n return IMPORTED\n\n\nclass CompareResult(TypedDict):\n rms: float\n expected: str\n actual: str\n diff: str\n tol: int\n\n\n@pytest.fixture\ndef check_same_image(add_nunit_attachment):\n from urllib.parse import quote\n\n from matplotlib.testing.compare import compare_images\n\n def check_same_image(\n expected: Path | os.PathLike,\n actual: Path | os.PathLike,\n *,\n tol: int,\n basename: str = \"\",\n ) -> None:\n __tracebackhide__ = True\n\n def fmt_descr(descr):\n return f\"{descr} ({basename})\" if basename else descr\n\n result = cast(\n Union[CompareResult, None],\n compare_images(str(expected), str(actual), tol=tol, in_decorator=True),\n )\n if result is None:\n return\n\n add_nunit_attachment(result[\"expected\"], fmt_descr(\"Expected\"))\n add_nunit_attachment(result[\"actual\"], fmt_descr(\"Result\"))\n add_nunit_attachment(result[\"diff\"], fmt_descr(\"Difference\"))\n\n result_urls = {\n k: f\"file://{quote(v)}\" if isinstance(v, str) else v\n for k, v in result.items()\n }\n msg = dedent(\n \"\"\"\\\n Image files did not match.\n RMS Value: {rms}\n Expected: {expected}\n Actual: {actual}\n Difference: {diff}\n Tolerance: {tol}\n \"\"\"\n ).format_map(result_urls)\n raise AssertionError(msg)\n\n return check_same_image\n\n\n@pytest.fixture\ndef image_comparer(check_same_image):\n from matplotlib import pyplot as plt\n\n def save_and_compare(*path_parts: Path | os.PathLike, tol: int):\n __tracebackhide__ = True\n\n base_pth = Path(*path_parts)\n\n if not base_pth.is_dir():\n base_pth.mkdir()\n expected_pth = base_pth / \"expected.png\"\n actual_pth = base_pth / \"actual.png\"\n plt.savefig(actual_pth, dpi=40)\n plt.close()\n if not expected_pth.is_file():\n raise OSError(f\"No expected output found at {expected_pth}.\")\n check_same_image(expected_pth, actual_pth, tol=tol)\n\n return save_and_compare\n\n\n@pytest.fixture\ndef plt():\n from matplotlib import pyplot as plt\n\n return plt\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pytest\nfrom anndata import AnnData\nfrom scipy.sparse import csc_matrix, csr_matrix\n\nimport scanpy as sc\n\n# test \"data\" for 3 cells * 4 genes\nX_original = [\n [-1, 2, 0, 0],\n [1, 2, 4, 0],\n [0, 2, 2, 0],\n] # with gene std 1,0,2,0 and center 0,2,2,0\nX_scaled_original = [\n [-1, 2, 0, 0],\n [1, 2, 2, 0],\n [0, 2, 1, 0],\n] # with gene std 1,0,1,0 and center 0,2,1,0\nX_centered_original = [\n [-1, 0, -1, 0],\n [1, 0, 1, 0],\n [0, 0, 0, 0],\n] # with gene std 1,0,1,0 and center 0,0,0,0\nX_scaled_original_clipped = [\n [-1, 1, 0, 0],\n [1, 1, 1, 0],\n [0, 1, 1, 0],\n] # with gene std 1,0,1,0 and center 0,2,1,0\n\n\nX_for_mask = [\n [27, 27, 27, 27],\n [27, 27, 27, 27],\n [-1, 2, 0, 0],\n [1, 2, 4, 0],\n [0, 2, 2, 0],\n [27, 27, 27, 27],\n [27, 27, 27, 27],\n]\nX_scaled_for_mask = [\n [27, 27, 27, 27],\n [27, 27, 27, 27],\n [-1, 2, 0, 0],\n [1, 2, 2, 0],\n [0, 2, 1, 0],\n [27, 27, 27, 27],\n [27, 27, 27, 27],\n]\nX_centered_for_mask = [\n [27, 27, 27, 27],\n [27, 27, 27, 27],\n [-1, 0, -1, 0],\n [1, 0, 1, 0],\n [0, 0, 0, 0],\n [27, 27, 27, 27],\n [27, 27, 27, 27],\n]\nX_scaled_for_mask_clipped = [\n [27, 27, 27, 27],\n [27, 27, 27, 27],\n [-1, 1, 0, 0],\n [1, 1, 1, 0],\n [0, 1, 1, 0],\n [27, 27, 27, 27],\n [27, 27, 27, 27],\n]\n\n\n@pytest.mark.parametrize(\n \"typ\", [np.array, csr_matrix, csc_matrix], ids=lambda x: x.__name__\n)\n@pytest.mark.parametrize(\"dtype\", [\"float32\", \"int64\"])\n@pytest.mark.parametrize(\n (\"mask_obs\", \"X\", \"X_centered\", \"X_scaled\"),\n [\n (None, X_original, X_centered_original, X_scaled_original),\n (\n np.array((0, 0, 1, 1, 1, 0, 0), dtype=bool),\n X_for_mask,\n X_centered_for_mask,\n X_scaled_for_mask,\n ),\n ],\n)\ndef test_scale(*, typ, dtype, mask_obs, X, X_centered, X_scaled):\n # test AnnData arguments\n # test scaling with default zero_center == True\n adata0 = AnnData(typ(X).astype(dtype))\n sc.pp.scale(adata0, mask_obs=mask_obs)\n assert np.allclose(csr_matrix(adata0.X).toarray(), X_centered)\n # test scaling with explicit zero_center == True\n adata1 = AnnData(typ(X).astype(dtype))\n sc.pp.scale(adata1, zero_center=True, mask_obs=mask_obs)\n assert np.allclose(csr_matrix(adata1.X).toarray(), X_centered)\n # test scaling with explicit zero_center == False\n adata2 = AnnData(typ(X).astype(dtype))\n sc.pp.scale(adata2, zero_center=False, mask_obs=mask_obs)\n assert np.allclose(csr_matrix(adata2.X).toarray(), X_scaled)\n # test bare count arguments, for simplicity only with explicit copy=True\n # test scaling with default zero_center == True\n data0 = typ(X, dtype=dtype)\n cdata0 = sc.pp.scale(data0, copy=True, mask_obs=mask_obs)\n assert np.allclose(csr_matrix(cdata0).toarray(), X_centered)\n # test scaling with explicit zero_center == True\n data1 = typ(X, dtype=dtype)\n cdata1 = sc.pp.scale(data1, zero_center=True, copy=True, mask_obs=mask_obs)\n assert np.allclose(csr_matrix(cdata1).toarray(), X_centered)\n # test scaling with explicit zero_center == False\n data2 = typ(X, dtype=dtype)\n cdata2 = sc.pp.scale(data2, zero_center=False, copy=True, mask_obs=mask_obs)\n assert np.allclose(csr_matrix(cdata2).toarray(), X_scaled)\n\n\ndef test_mask_string():\n with pytest.raises(ValueError, match=r\"Cannot refer to mask.* without.*anndata\"):\n sc.pp.scale(np.array(X_original), mask_obs=\"mask\")\n adata = AnnData(np.array(X_for_mask, dtype=\"float32\"))\n adata.obs[\"some cells\"] = np.array((0, 0, 1, 1, 1, 0, 0), dtype=bool)\n sc.pp.scale(adata, mask_obs=\"some cells\")\n assert np.array_equal(adata.X, X_centered_for_mask)\n assert \"mean of some cells\" in adata.var.columns\n\n\n@pytest.mark.parametrize(\"zero_center\", [True, False])\ndef test_clip(zero_center):\n adata = sc.datasets.pbmc3k()\n sc.pp.scale(adata, max_value=1, zero_center=zero_center)\n if zero_center:\n assert adata.X.min() >= -1\n assert adata.X.max() <= 1\n\n\n@pytest.mark.parametrize(\n (\"mask_obs\", \"X\", \"X_scaled\", \"X_clipped\"),\n [\n (None, X_original, X_scaled_original, X_scaled_original_clipped),\n (\n np.array((0, 0, 1, 1, 1, 0, 0), dtype=bool),\n X_for_mask,\n X_scaled_for_mask,\n X_scaled_for_mask_clipped,\n ),\n ],\n)\ndef test_scale_sparse(*, mask_obs, X, X_scaled, X_clipped):\n adata0 = AnnData(csr_matrix(X).astype(np.float32))\n sc.pp.scale(adata0, mask_obs=mask_obs, zero_center=False)\n assert np.allclose(csr_matrix(adata0.X).toarray(), X_scaled)\n # test scaling with explicit zero_center == True\n adata1 = AnnData(csr_matrix(X).astype(np.float32))\n sc.pp.scale(adata1, zero_center=False, mask_obs=mask_obs, max_value=1)\n assert np.allclose(csr_matrix(adata1.X).toarray(), X_clipped)\n\n\nfrom __future__ import annotations\n\nimport pytest\nfrom sklearn.metrics.cluster import normalized_mutual_info_score\n\nimport scanpy as sc\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\nfrom testing.scanpy._pytest.marks import needs\n\n\n@pytest.fixture\ndef adata_neighbors():\n return pbmc68k_reduced()\n\n\nFLAVORS = [\n pytest.param(\"igraph\", marks=needs.igraph),\n pytest.param(\"leidenalg\", marks=needs.leidenalg),\n]\n\n\n@needs.leidenalg\n@needs.igraph\n@pytest.mark.parametrize(\"flavor\", FLAVORS)\n@pytest.mark.parametrize(\"resolution\", [1, 2])\n@pytest.mark.parametrize(\"n_iterations\", [-1, 3])\ndef test_leiden_basic(adata_neighbors, flavor, resolution, n_iterations):\n sc.tl.leiden(\n adata_neighbors,\n flavor=flavor,\n resolution=resolution,\n n_iterations=n_iterations,\n directed=(flavor == \"leidenalg\"),\n key_added=\"leiden_custom\",\n )\n assert adata_neighbors.uns[\"leiden_custom\"][\"params\"][\"resolution\"] == resolution\n assert (\n adata_neighbors.uns[\"leiden_custom\"][\"params\"][\"n_iterations\"] == n_iterations\n )\n\n\n@needs.leidenalg\n@needs.igraph\n@pytest.mark.parametrize(\"flavor\", FLAVORS)\ndef test_leiden_random_state(adata_neighbors, flavor):\n is_leiden_alg = flavor == \"leidenalg\"\n n_iterations = 2 if is_leiden_alg else -1\n adata_1 = sc.tl.leiden(\n adata_neighbors,\n flavor=flavor,\n random_state=1,\n copy=True,\n directed=is_leiden_alg,\n n_iterations=n_iterations,\n )\n adata_1_again = sc.tl.leiden(\n adata_neighbors,\n flavor=flavor,\n random_state=1,\n copy=True,\n directed=is_leiden_alg,\n n_iterations=n_iterations,\n )\n adata_2 = sc.tl.leiden(\n adata_neighbors,\n flavor=flavor,\n random_state=2,\n copy=True,\n directed=is_leiden_alg,\n n_iterations=n_iterations,\n )\n assert (adata_1.obs[\"leiden\"] == adata_1_again.obs[\"leiden\"]).all()\n assert (adata_2.obs[\"leiden\"] != adata_1_again.obs[\"leiden\"]).any()\n\n\n@needs.igraph\ndef test_leiden_igraph_directed(adata_neighbors):\n with pytest.raises(ValueError, match=r\"Cannot use igraph’s leiden.*directed\"):\n sc.tl.leiden(adata_neighbors, flavor=\"igraph\", directed=True)\n\n\n@needs.igraph\ndef test_leiden_wrong_flavor(adata_neighbors):\n with pytest.raises(ValueError, match=r\"flavor must be.*'igraph'.*'leidenalg'.*but\"):\n sc.tl.leiden(adata_neighbors, flavor=\"foo\")\n\n\n@needs.igraph\n@needs.leidenalg\ndef test_leiden_igraph_partition_type(adata_neighbors):\n import leidenalg\n\n with pytest.raises(ValueError, match=r\"Do not pass in partition_type\"):\n sc.tl.leiden(\n adata_neighbors,\n flavor=\"igraph\",\n partition_type=leidenalg.RBConfigurationVertexPartition,\n )\n\n\n@needs.leidenalg\n@needs.igraph\ndef test_leiden_equal_defaults_same_args(adata_neighbors):\n \"\"\"Ensure the two implementations are the same for the same args.\"\"\"\n leiden_alg_clustered = sc.tl.leiden(\n adata_neighbors, flavor=\"leidenalg\", copy=True, n_iterations=2\n )\n igraph_clustered = sc.tl.leiden(\n adata_neighbors, flavor=\"igraph\", copy=True, directed=False, n_iterations=2\n )\n assert (\n normalized_mutual_info_score(\n leiden_alg_clustered.obs[\"leiden\"], igraph_clustered.obs[\"leiden\"]\n )\n > 0.9\n )\n\n\n@needs.leidenalg\n@needs.igraph\ndef test_leiden_equal_defaults(adata_neighbors):\n \"\"\"Ensure that the old leidenalg defaults are close enough to the current default outputs.\"\"\"\n leiden_alg_clustered = sc.tl.leiden(\n adata_neighbors, flavor=\"leidenalg\", directed=True, copy=True\n )\n igraph_clustered = sc.tl.leiden(\n adata_neighbors, copy=True, n_iterations=2, directed=False\n )\n assert (\n normalized_mutual_info_score(\n leiden_alg_clustered.obs[\"leiden\"], igraph_clustered.obs[\"leiden\"]\n )\n > 0.9\n )\n\n\n@needs.igraph\ndef test_leiden_objective_function(adata_neighbors):\n \"\"\"Ensure that popping this as a `clustering_kwargs` and using it does not error out.\"\"\"\n sc.tl.leiden(\n adata_neighbors,\n objective_function=\"modularity\",\n flavor=\"igraph\",\n directed=False,\n )\n\n\n@needs.igraph\n@pytest.mark.parametrize(\n (\"clustering\", \"key\"),\n [\n pytest.param(sc.tl.louvain, \"louvain\", marks=needs.louvain),\n pytest.param(sc.tl.leiden, \"leiden\", marks=needs.leidenalg),\n ],\n)\ndef test_clustering_subset(adata_neighbors, clustering, key):\n clustering(adata_neighbors, key_added=key)\n\n for c in adata_neighbors.obs[key].unique():\n print(\"Analyzing cluster \", c)\n cells_in_c = adata_neighbors.obs[key] == c\n ncells_in_c = adata_neighbors.obs[key].value_counts().loc[c]\n key_sub = str(key) + \"_sub\"\n clustering(\n adata_neighbors,\n restrict_to=(key, [c]),\n key_added=key_sub,\n )\n # Get new clustering labels\n new_partition = adata_neighbors.obs[key_sub]\n\n cat_counts = new_partition[cells_in_c].value_counts()\n\n # Only original cluster's cells assigned to new categories\n assert cat_counts.sum() == ncells_in_c\n\n # Original category's cells assigned only to new categories\n nonzero_cat = cat_counts[cat_counts > 0].index\n common_cat = nonzero_cat.intersection(adata_neighbors.obs[key].cat.categories)\n assert len(common_cat) == 0\n\n\n@needs.louvain\n@needs.igraph\ndef test_louvain_basic(adata_neighbors):\n sc.tl.louvain(adata_neighbors)\n sc.tl.louvain(adata_neighbors, use_weights=True)\n sc.tl.louvain(adata_neighbors, use_weights=True, flavor=\"igraph\")\n sc.tl.louvain(adata_neighbors, flavor=\"igraph\")\n\n\n@needs.louvain\n@pytest.mark.parametrize(\"random_state\", [10, 999])\n@pytest.mark.parametrize(\"resolution\", [0.9, 1.1])\ndef test_louvain_custom_key(adata_neighbors, resolution, random_state):\n sc.tl.louvain(\n adata_neighbors,\n key_added=\"louvain_custom\",\n random_state=random_state,\n resolution=resolution,\n )\n assert (\n adata_neighbors.uns[\"louvain_custom\"][\"params\"][\"random_state\"] == random_state\n )\n assert adata_neighbors.uns[\"louvain_custom\"][\"params\"][\"resolution\"] == resolution\n\n\n@needs.louvain\n@needs.igraph\ndef test_partition_type(adata_neighbors):\n import louvain\n\n sc.tl.louvain(adata_neighbors, partition_type=louvain.RBERVertexPartition)\n sc.tl.louvain(adata_neighbors, partition_type=louvain.SurpriseVertexPartition)\n\n\n@pytest.mark.parametrize(\n (\"clustering\", \"default_key\", \"default_res\", \"custom_resolutions\"),\n [\n pytest.param(sc.tl.leiden, \"leiden\", 0.8, [0.9, 1.1], marks=needs.leidenalg),\n pytest.param(sc.tl.louvain, \"louvain\", 0.8, [0.9, 1.1], marks=needs.louvain),\n ],\n)\ndef test_clustering_custom_key(\n adata_neighbors, clustering, default_key, default_res, custom_resolutions\n):\n custom_keys = [f\"{default_key}_{res}\" for res in custom_resolutions]\n\n # Run clustering with default key, then custom keys\n clustering(adata_neighbors, resolution=default_res)\n for key, res in zip(custom_keys, custom_resolutions):\n clustering(adata_neighbors, resolution=res, key_added=key)\n\n # ensure that all clustering parameters are added to user provided keys and not overwritten\n assert adata_neighbors.uns[default_key][\"params\"][\"resolution\"] == default_res\n for key, res in zip(custom_keys, custom_resolutions):\n assert adata_neighbors.uns[key][\"params\"][\"resolution\"] == res\n\n\nfrom __future__ import annotations\n\nimport json\nimport sys\nfrom subprocess import run\n\n\ndef descend(profimp_data, modules, path):\n module = profimp_data[\"module\"]\n path = [*path, module]\n if module in modules:\n yield \" → \".join(e for e in path if e is not None)\n modules.remove(module)\n for child in profimp_data[\"children\"]:\n yield from descend(child, modules, path)\n\n\ndef get_import_paths(modules):\n proc = run(\n [sys.executable, \"-m\", \"profimp.main\", \"import scanpy\"],\n capture_output=True,\n check=True,\n )\n data = json.loads(proc.stdout)\n return descend(data, set(modules), [])\n\n\ndef test_deferred_imports(imported_modules):\n slow_to_import = {\n \"umap\", # neighbors, tl.umap\n \"seaborn\", # plotting\n \"sklearn.metrics\", # neighbors\n \"pynndescent\", # neighbors\n \"networkx\", # diffmap, paga, plotting._utils\n # TODO: 'matplotlib.pyplot',\n # TODO (maybe): 'numba',\n }\n falsely_imported = slow_to_import & imported_modules\n\n assert not falsely_imported, \"\\n\".join(get_import_paths(falsely_imported))\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\nimport pytest\nfrom anndata.tests.helpers import assert_equal\nfrom sklearn.metrics import silhouette_score\n\nimport scanpy as sc\nfrom scanpy.preprocessing._combat import _design_matrix, _standardize_data\n\n\ndef test_norm():\n # this test trivially checks whether mean normalisation worked\n\n # load in data\n adata = sc.datasets.blobs()\n key = \"blobs\"\n data = pd.DataFrame(data=adata.X.T, index=adata.var_names, columns=adata.obs_names)\n\n # construct a pandas series of the batch annotation\n batch = pd.Series(adata.obs[key])\n model = pd.DataFrame({\"batch\": batch})\n\n # standardize the data\n s_data, design, var_pooled, stand_mean = _standardize_data(model, data, \"batch\")\n\n assert np.allclose(s_data.mean(axis=1), np.zeros(s_data.shape[0]))\n\n\ndef test_covariates():\n adata = sc.datasets.blobs()\n key = \"blobs\"\n\n X1 = sc.pp.combat(adata, key=key, inplace=False)\n\n np.random.seed(0)\n adata.obs[\"cat1\"] = np.random.binomial(3, 0.5, size=(adata.n_obs))\n adata.obs[\"cat2\"] = np.random.binomial(2, 0.1, size=(adata.n_obs))\n adata.obs[\"num1\"] = np.random.normal(size=(adata.n_obs))\n\n X2 = sc.pp.combat(\n adata, key=key, covariates=[\"cat1\", \"cat2\", \"num1\"], inplace=False\n )\n sc.pp.combat(adata, key=key, covariates=[\"cat1\", \"cat2\", \"num1\"], inplace=True)\n\n assert X1.shape == X2.shape\n\n df = adata.obs[[\"cat1\", \"cat2\", \"num1\", key]]\n batch_cats = adata.obs[key].cat.categories\n design = _design_matrix(df, key, batch_cats)\n\n assert len(design.columns) == 4 + len(batch_cats) - 1\n\n\ndef test_combat_obs_names():\n # Test for fix to #1170\n X = np.random.random((200, 100))\n obs = pd.DataFrame(\n {\"batch\": pd.Categorical(np.random.randint(0, 2, 200))},\n index=np.repeat(np.arange(100), 2).astype(str), # Non-unique index\n )\n with pytest.warns(UserWarning, match=\"Observation names are not unique\"):\n a = sc.AnnData(X, obs)\n b = a.copy()\n b.obs_names_make_unique()\n\n sc.pp.combat(a, \"batch\")\n sc.pp.combat(b, \"batch\")\n\n assert_equal(a.X, b.X)\n\n a.obs_names_make_unique()\n assert_equal(a, b)\n\n\ndef test_silhouette():\n # this test checks wether combat can align data from several gaussians\n # it checks this by computing the silhouette coefficient in a pca embedding\n\n # load in data\n adata = sc.datasets.blobs()\n\n # apply combat\n sc.pp.combat(adata, \"blobs\")\n\n # compute pca\n sc.pp.pca(adata)\n X_pca = adata.obsm[\"X_pca\"]\n\n # compute silhouette coefficient in pca\n sh = silhouette_score(X_pca[:, :2], adata.obs[\"blobs\"].values)\n\n assert sh < 0.1\n\n\nfrom __future__ import annotations\n\nimport itertools\nfrom pathlib import Path\nfrom string import ascii_letters\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nimport pytest\nfrom anndata import AnnData\nfrom pandas.testing import assert_frame_equal, assert_index_equal\nfrom scipy import sparse\n\nimport scanpy as sc\nfrom scanpy.preprocessing._utils import _get_mean_var\nfrom testing.scanpy._helpers import _check_check_values_warnings\nfrom testing.scanpy._helpers.data import pbmc3k, pbmc68k_reduced\nfrom testing.scanpy._pytest.marks import needs\nfrom testing.scanpy._pytest.params import ARRAY_TYPES\n\nif TYPE_CHECKING:\n from typing import Callable, Literal\n\nFILE = Path(__file__).parent / Path(\"_scripts/seurat_hvg.csv\")\nFILE_V3 = Path(__file__).parent / Path(\"_scripts/seurat_hvg_v3.csv.gz\")\nFILE_V3_BATCH = Path(__file__).parent / Path(\"_scripts/seurat_hvg_v3_batch.csv\")\nFILE_CELL_RANGER = Path(__file__).parent / \"_scripts/cell_ranger_hvg.csv\"\n\n\n@pytest.fixture(scope=\"session\")\ndef adata_sess() -> AnnData:\n adata = sc.datasets.blobs()\n rng = np.random.default_rng(0)\n adata.var_names = rng.choice(list(ascii_letters), adata.n_vars, replace=False)\n return adata\n\n\n@pytest.fixture\ndef adata(adata_sess: AnnData) -> AnnData:\n return adata_sess.copy()\n\n\ndef test_runs(adata):\n sc.pp.highly_variable_genes(adata)\n\n\ndef test_supports_batch(adata):\n gen = np.random.default_rng(0)\n adata.obs[\"batch\"] = pd.array(\n gen.binomial(3, 0.5, size=adata.n_obs), dtype=\"category\"\n )\n sc.pp.highly_variable_genes(adata, batch_key=\"batch\")\n assert \"highly_variable_nbatches\" in adata.var.columns\n assert \"highly_variable_intersection\" in adata.var.columns\n\n\ndef test_supports_layers(adata_sess):\n def execute(layer: str | None) -> AnnData:\n gen = np.random.default_rng(0)\n adata = adata_sess.copy()\n assert isinstance(adata.X, np.ndarray)\n if layer:\n adata.X, adata.layers[layer] = None, adata.X.copy()\n gen.shuffle(adata.layers[layer])\n adata.obs[\"batch\"] = pd.array(\n gen.binomial(4, 0.5, size=adata.n_obs), dtype=\"category\"\n )\n sc.pp.highly_variable_genes(\n adata, batch_key=\"batch\", n_top_genes=3, layer=layer\n )\n assert \"highly_variable_nbatches\" in adata.var.columns\n assert adata.var[\"highly_variable\"].sum() == 3\n return adata\n\n adata1, adata2 = map(execute, [None, \"test_layer\"])\n assert (adata1.var[\"highly_variable\"] != adata2.var[\"highly_variable\"]).any()\n\n\ndef test_no_batch_matches_batch(adata):\n sc.pp.highly_variable_genes(adata)\n no_batch_hvg = adata.var[\"highly_variable\"].copy()\n assert no_batch_hvg.any()\n adata.obs[\"batch\"] = pd.array([\"batch\"], dtype=\"category\").repeat(len(adata))\n sc.pp.highly_variable_genes(adata, batch_key=\"batch\")\n assert np.all(no_batch_hvg == adata.var[\"highly_variable\"])\n assert np.all(\n adata.var[\"highly_variable_intersection\"] == adata.var[\"highly_variable\"]\n )\n\n\n@pytest.mark.parametrize(\"batch_key\", [None, \"batch\"], ids=[\"single\", \"batched\"])\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\ndef test_no_inplace(adata, array_type, batch_key):\n \"\"\"Tests that, with `n_top_genes=None` the returned dataframe has the expected columns.\"\"\"\n adata.X = array_type(adata.X)\n if batch_key:\n adata.obs[batch_key] = np.tile([\"a\", \"b\"], adata.shape[0] // 2)\n sc.pp.highly_variable_genes(adata, batch_key=batch_key, n_bins=3)\n assert adata.var[\"highly_variable\"].any()\n\n colnames = {\"means\", \"dispersions\", \"dispersions_norm\", \"highly_variable\"} | (\n {\"mean_bin\"}\n if batch_key is None\n else {\"highly_variable_nbatches\", \"highly_variable_intersection\"}\n )\n hvg_df = sc.pp.highly_variable_genes(\n adata, batch_key=batch_key, n_bins=3, inplace=False\n )\n assert isinstance(hvg_df, pd.DataFrame)\n assert colnames == set(hvg_df.columns)\n\n\n@pytest.mark.parametrize(\"base\", [None, 10])\n@pytest.mark.parametrize(\"flavor\", [\"seurat\", \"cell_ranger\"])\ndef test_keep_layer(base, flavor):\n adata = pbmc3k()\n # cell_ranger flavor can raise error if many 0 genes\n sc.pp.filter_genes(adata, min_counts=1)\n\n sc.pp.log1p(adata, base=base)\n assert isinstance(adata.X, sparse.csr_matrix)\n X_orig = adata.X.copy()\n\n if flavor == \"seurat\":\n sc.pp.highly_variable_genes(adata, n_top_genes=50, flavor=flavor)\n elif flavor == \"cell_ranger\":\n sc.pp.highly_variable_genes(adata, flavor=flavor)\n else:\n pytest.fail(f\"Unknown {flavor=}\")\n\n assert np.allclose(X_orig.toarray(), adata.X.toarray())\n\n\n@pytest.mark.parametrize(\n \"flavor\",\n [\n \"seurat\",\n pytest.param(\n \"cell_ranger\",\n marks=pytest.mark.xfail(reason=\"can’t deal with duplicate bin edges\"),\n ),\n ],\n)\ndef test_no_filter_genes(flavor):\n \"\"\"Test that even with columns containing all-zeros in the data, n_top_genes is respected.\"\"\"\n adata = sc.datasets.pbmc3k()\n means, _ = _get_mean_var(adata.X)\n assert (means == 0).any()\n sc.pp.normalize_total(adata, target_sum=10000)\n sc.pp.log1p(adata)\n sc.pp.highly_variable_genes(adata, flavor=flavor, n_top_genes=10000)\n assert adata.var[\"highly_variable\"].sum() == 10000\n\n\ndef _check_pearson_hvg_columns(output_df: pd.DataFrame, n_top_genes: int):\n assert pd.api.types.is_float_dtype(output_df[\"residual_variances\"].dtype)\n\n assert output_df[\"highly_variable\"].to_numpy().dtype is np.dtype(\"bool\")\n assert np.sum(output_df[\"highly_variable\"]) == n_top_genes\n\n assert np.nanmax(output_df[\"highly_variable_rank\"].to_numpy()) <= n_top_genes - 1\n\n\ndef test_pearson_residuals_inputchecks(pbmc3k_parametrized_small):\n adata = pbmc3k_parametrized_small()\n\n # depending on check_values, warnings should be raised for non-integer data\n if adata.X.dtype == \"float32\":\n adata_noninteger = adata.copy()\n x, y = np.nonzero(adata_noninteger.X)\n adata_noninteger.X[x[0], y[0]] = 0.5\n\n _check_check_values_warnings(\n function=sc.experimental.pp.highly_variable_genes,\n adata=adata_noninteger,\n expected_warning=\"`flavor='pearson_residuals'` expects raw count data, but non-integers were found.\",\n kwargs=dict(\n flavor=\"pearson_residuals\",\n n_top_genes=100,\n ),\n )\n\n # errors should be raised for invalid theta values\n for theta in [0, -1]:\n with pytest.raises(ValueError, match=\"Pearson residuals require theta > 0\"):\n sc.experimental.pp.highly_variable_genes(\n adata.copy(), theta=theta, flavor=\"pearson_residuals\", n_top_genes=100\n )\n\n with pytest.raises(\n ValueError, match=\"Pearson residuals require `clip>=0` or `clip=None`.\"\n ):\n sc.experimental.pp.highly_variable_genes(\n adata.copy(), clip=-1, flavor=\"pearson_residuals\", n_top_genes=100\n )\n\n\n@pytest.mark.parametrize(\"subset\", [True, False], ids=[\"subset\", \"full\"])\n@pytest.mark.parametrize(\n \"clip\", [None, np.inf, 30], ids=[\"noclip\", \"infclip\", \"30clip\"]\n)\n@pytest.mark.parametrize(\"theta\", [100, np.inf], ids=[\"100theta\", \"inftheta\"])\n@pytest.mark.parametrize(\"n_top_genes\", [100, 200], ids=[\"100n\", \"200n\"])\ndef test_pearson_residuals_general(\n pbmc3k_parametrized_small, subset, clip, theta, n_top_genes\n):\n adata = pbmc3k_parametrized_small()\n # cleanup var\n del adata.var\n\n # compute reference output\n residuals_res = sc.experimental.pp.normalize_pearson_residuals(\n adata, clip=clip, theta=theta, inplace=False\n )\n assert isinstance(residuals_res, dict)\n residual_variances_reference = np.var(residuals_res[\"X\"], axis=0)\n\n if subset:\n # lazyly sort by residual variance and take top N\n top_n_idx = np.argsort(-residual_variances_reference)[:n_top_genes]\n # (results in sorted \"gene order\" in reference)\n residual_variances_reference = residual_variances_reference[top_n_idx]\n\n # compute output to be tested\n output_df = sc.experimental.pp.highly_variable_genes(\n adata,\n flavor=\"pearson_residuals\",\n n_top_genes=n_top_genes,\n subset=subset,\n inplace=False,\n clip=clip,\n theta=theta,\n )\n assert output_df is not None\n\n sc.experimental.pp.highly_variable_genes(\n adata,\n flavor=\"pearson_residuals\",\n n_top_genes=n_top_genes,\n subset=subset,\n inplace=True,\n clip=clip,\n theta=theta,\n )\n\n # compare inplace=True and inplace=False output\n pd.testing.assert_frame_equal(output_df, adata.var)\n\n # check output is complete\n for key in [\n \"highly_variable\",\n \"means\",\n \"variances\",\n \"residual_variances\",\n \"highly_variable_rank\",\n ]:\n assert key in output_df.columns\n\n # check consistency with normalization method\n if subset:\n # sort values before comparing as reference is sorted as well for subset case\n sort_output_idx = np.argsort(-output_df[\"residual_variances\"].to_numpy())\n assert np.allclose(\n output_df[\"residual_variances\"].to_numpy()[sort_output_idx],\n residual_variances_reference,\n )\n else:\n assert np.allclose(\n output_df[\"residual_variances\"].to_numpy(), residual_variances_reference\n )\n\n # check hvg flag\n hvg_idx = np.where(output_df[\"highly_variable\"])[0]\n topn_idx = np.sort(\n np.argsort(-output_df[\"residual_variances\"].to_numpy())[:n_top_genes]\n )\n assert np.all(hvg_idx == topn_idx)\n\n # check ranks\n assert np.nanmin(output_df[\"highly_variable_rank\"].to_numpy()) == 0\n\n # more general checks on ranks, hvg flag and residual variance\n _check_pearson_hvg_columns(output_df, n_top_genes)\n\n\n@pytest.mark.parametrize(\"subset\", [True, False], ids=[\"subset\", \"full\"])\n@pytest.mark.parametrize(\"n_top_genes\", [100, 200], ids=[\"100n\", \"200n\"])\ndef test_pearson_residuals_batch(pbmc3k_parametrized_small, subset, n_top_genes):\n adata = pbmc3k_parametrized_small()\n # cleanup var\n del adata.var\n n_genes = adata.shape[1]\n\n output_df = sc.experimental.pp.highly_variable_genes(\n adata,\n flavor=\"pearson_residuals\",\n n_top_genes=n_top_genes,\n batch_key=\"batch\",\n subset=subset,\n inplace=False,\n )\n assert output_df is not None\n\n sc.experimental.pp.highly_variable_genes(\n adata,\n flavor=\"pearson_residuals\",\n n_top_genes=n_top_genes,\n batch_key=\"batch\",\n subset=subset,\n inplace=True,\n )\n\n # compare inplace=True and inplace=False output\n pd.testing.assert_frame_equal(output_df, adata.var)\n\n # check output is complete\n for key in [\n \"highly_variable\",\n \"means\",\n \"variances\",\n \"residual_variances\",\n \"highly_variable_rank\",\n \"highly_variable_nbatches\",\n \"highly_variable_intersection\",\n ]:\n assert key in output_df.columns\n\n # general checks on ranks, hvg flag and residual variance\n _check_pearson_hvg_columns(output_df, n_top_genes)\n\n # check intersection flag\n nbatches = len(np.unique(adata.obs[\"batch\"]))\n assert output_df[\"highly_variable_intersection\"].to_numpy().dtype is np.dtype(\n \"bool\"\n )\n assert np.sum(output_df[\"highly_variable_intersection\"]) <= n_top_genes * nbatches\n assert np.all(output_df[\"highly_variable\"][output_df.highly_variable_intersection])\n\n # check ranks (with batch_key these are the median of within-batch ranks)\n assert pd.api.types.is_float_dtype(output_df[\"highly_variable_rank\"].dtype)\n\n # check nbatches\n assert output_df[\"highly_variable_nbatches\"].to_numpy().dtype is np.dtype(\"int\")\n assert np.min(output_df[\"highly_variable_nbatches\"].to_numpy()) >= 0\n assert np.max(output_df[\"highly_variable_nbatches\"].to_numpy()) <= nbatches\n\n # check subsetting\n if subset:\n assert len(output_df) == n_top_genes\n else:\n assert len(output_df) == n_genes\n\n\n@pytest.mark.parametrize(\"func\", [\"hvg\", \"fgd\"])\n@pytest.mark.parametrize(\n (\"flavor\", \"params\", \"ref_path\"),\n [\n pytest.param(\n \"seurat\", dict(min_mean=0.0125, max_mean=3, min_disp=0.5), FILE, id=\"seurat\"\n ),\n pytest.param(\n \"cell_ranger\", dict(n_top_genes=100), FILE_CELL_RANGER, id=\"cell_ranger\"\n ),\n ],\n)\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\ndef test_compare_to_upstream( # noqa: PLR0917\n request: pytest.FixtureRequest,\n func: Literal[\"hvg\", \"fgd\"],\n flavor: Literal[\"seurat\", \"cell_ranger\"],\n params: dict[str, float | int],\n ref_path: Path,\n array_type: Callable,\n):\n if func == \"fgd\" and flavor == \"cell_ranger\":\n reason = \"The deprecated filter_genes_dispersion behaves differently with cell_ranger\"\n request.applymarker(pytest.mark.xfail(reason=reason))\n hvg_info = pd.read_csv(ref_path)\n\n pbmc = pbmc68k_reduced()\n pbmc.X = pbmc.raw.X\n pbmc.X = array_type(pbmc.X)\n pbmc.var_names_make_unique()\n sc.pp.filter_cells(pbmc, min_counts=1)\n sc.pp.normalize_total(pbmc, target_sum=1e4)\n\n if func == \"hvg\":\n sc.pp.log1p(pbmc)\n sc.pp.highly_variable_genes(pbmc, flavor=flavor, **params, inplace=True)\n elif func == \"fgd\":\n sc.pp.filter_genes_dispersion(\n pbmc, flavor=flavor, **params, log=True, subset=False\n )\n else:\n raise AssertionError()\n\n np.testing.assert_array_equal(\n hvg_info[\"highly_variable\"], pbmc.var[\"highly_variable\"]\n )\n\n # (still) Not equal to tolerance rtol=2e-05, atol=2e-05\n # np.testing.assert_allclose(4, 3.9999, rtol=2e-05, atol=2e-05)\n np.testing.assert_allclose(\n hvg_info[\"means\"],\n pbmc.var[\"means\"],\n rtol=2e-05,\n atol=2e-05,\n )\n np.testing.assert_allclose(\n hvg_info[\"dispersions\"],\n pbmc.var[\"dispersions\"],\n rtol=2e-05,\n atol=2e-05,\n )\n np.testing.assert_allclose(\n hvg_info[\"dispersions_norm\"],\n pbmc.var[\"dispersions_norm\"],\n rtol=2e-05 if \"dask\" not in array_type.__name__ else 1e-4,\n atol=2e-05 if \"dask\" not in array_type.__name__ else 1e-4,\n )\n\n\n@needs.skmisc\ndef test_compare_to_seurat_v3():\n ### test without batch\n seurat_hvg_info = pd.read_csv(FILE_V3)\n\n pbmc = pbmc3k()\n sc.pp.filter_cells(pbmc, min_genes=200) # this doesnt do anything btw\n sc.pp.filter_genes(pbmc, min_cells=3)\n\n pbmc_dense = pbmc.copy()\n pbmc_dense.X = pbmc_dense.X.toarray()\n\n sc.pp.highly_variable_genes(pbmc, n_top_genes=1000, flavor=\"seurat_v3\")\n sc.pp.highly_variable_genes(pbmc_dense, n_top_genes=1000, flavor=\"seurat_v3\")\n\n np.testing.assert_allclose(\n seurat_hvg_info[\"variance\"],\n pbmc.var[\"variances\"],\n rtol=2e-05,\n atol=2e-05,\n )\n np.testing.assert_allclose(\n seurat_hvg_info[\"variance.standardized\"],\n pbmc.var[\"variances_norm\"],\n rtol=2e-05,\n atol=2e-05,\n )\n np.testing.assert_allclose(\n pbmc_dense.var[\"variances_norm\"],\n pbmc.var[\"variances_norm\"],\n rtol=2e-05,\n atol=2e-05,\n )\n\n ### test with batch\n # introduce a dummy \"technical covariate\"; this is used in Seurat's SelectIntegrationFeatures\n pbmc.obs[\"dummy_tech\"] = (\n \"source_\" + pd.array([*range(1, 6), 5]).repeat(500).astype(\"string\")\n )[: pbmc.n_obs]\n\n seurat_v3_paper = sc.pp.highly_variable_genes(\n pbmc,\n n_top_genes=2000,\n flavor=\"seurat_v3_paper\",\n batch_key=\"dummy_tech\",\n inplace=False,\n )\n\n seurat_v3 = sc.pp.highly_variable_genes(\n pbmc,\n n_top_genes=2000,\n flavor=\"seurat_v3\",\n batch_key=\"dummy_tech\",\n inplace=False,\n )\n\n seurat_hvg_info_batch = pd.read_csv(FILE_V3_BATCH)\n seu = pd.Index(seurat_hvg_info_batch[\"x\"].to_numpy())\n\n gene_intersection_paper = seu.intersection(\n seurat_v3_paper[seurat_v3_paper[\"highly_variable\"]].index\n )\n gene_intersection_impl = seu.intersection(\n seurat_v3[seurat_v3[\"highly_variable\"]].index\n )\n assert len(gene_intersection_paper) / 2000 > 0.95\n assert len(gene_intersection_impl) / 2000 < 0.95\n\n\n@needs.skmisc\ndef test_seurat_v3_warning():\n pbmc = pbmc3k()[:200].copy()\n sc.pp.log1p(pbmc)\n with pytest.warns(\n UserWarning,\n match=\"`flavor='seurat_v3'` expects raw count data, but non-integers were found.\",\n ):\n sc.pp.highly_variable_genes(pbmc, flavor=\"seurat_v3\")\n\n\ndef test_batches():\n adata = pbmc68k_reduced()\n adata[:100, :100].X = np.zeros((100, 100))\n\n adata.obs[\"batch\"] = [\"0\" if i < 100 else \"1\" for i in range(adata.n_obs)]\n adata_1 = adata[adata.obs[\"batch\"] == \"0\"].copy()\n adata_2 = adata[adata.obs[\"batch\"] == \"1\"].copy()\n\n sc.pp.highly_variable_genes(\n adata,\n batch_key=\"batch\",\n flavor=\"cell_ranger\",\n n_top_genes=200,\n )\n\n sc.pp.filter_genes(adata_1, min_cells=1)\n sc.pp.filter_genes(adata_2, min_cells=1)\n hvg1 = sc.pp.highly_variable_genes(\n adata_1, flavor=\"cell_ranger\", n_top_genes=200, inplace=False\n )\n assert hvg1 is not None\n hvg2 = sc.pp.highly_variable_genes(\n adata_2, flavor=\"cell_ranger\", n_top_genes=200, inplace=False\n )\n assert hvg2 is not None\n\n np.testing.assert_allclose(\n adata.var[\"dispersions_norm\"].iat[100],\n 0.5 * hvg1[\"dispersions_norm\"].iat[0] + 0.5 * hvg2[\"dispersions_norm\"].iat[100],\n rtol=1.0e-7,\n atol=1.0e-7,\n )\n np.testing.assert_allclose(\n adata.var[\"dispersions_norm\"].iat[101],\n 0.5 * hvg1[\"dispersions_norm\"].iat[1] + 0.5 * hvg2[\"dispersions_norm\"].iat[101],\n rtol=1.0e-7,\n atol=1.0e-7,\n )\n np.testing.assert_allclose(\n adata.var[\"dispersions_norm\"].iat[0],\n 0.5 * hvg2[\"dispersions_norm\"].iat[0],\n rtol=1.0e-7,\n atol=1.0e-7,\n )\n\n colnames = [\n \"means\",\n \"dispersions\",\n \"dispersions_norm\",\n \"highly_variable\",\n ]\n\n assert np.all(np.isin(colnames, hvg1.columns))\n\n\n@needs.skmisc\ndef test_seurat_v3_mean_var_output_with_batchkey():\n pbmc = pbmc3k()\n pbmc.var_names_make_unique()\n n_cells = pbmc.shape[0]\n batch = np.zeros((n_cells), dtype=int)\n batch[1500:] = 1\n pbmc.obs[\"batch\"] = batch\n\n # true_mean, true_var = _get_mean_var(pbmc.X)\n true_mean = np.mean(pbmc.X.toarray(), axis=0)\n true_var = np.var(pbmc.X.toarray(), axis=0, dtype=np.float64, ddof=1)\n\n result_df = sc.pp.highly_variable_genes(\n pbmc, batch_key=\"batch\", flavor=\"seurat_v3\", n_top_genes=4000, inplace=False\n )\n np.testing.assert_allclose(true_mean, result_df[\"means\"], rtol=2e-05, atol=2e-05)\n np.testing.assert_allclose(true_var, result_df[\"variances\"], rtol=2e-05, atol=2e-05)\n\n\ndef test_cellranger_n_top_genes_warning():\n X = np.random.poisson(2, (100, 30))\n adata = AnnData(X)\n sc.pp.normalize_total(adata)\n sc.pp.log1p(adata)\n\n with pytest.warns(\n UserWarning,\n match=\"`n_top_genes` > number of normalized dispersions, returning all genes with normalized dispersions.\",\n ):\n sc.pp.highly_variable_genes(adata, n_top_genes=1000, flavor=\"cell_ranger\")\n\n\ndef test_cutoff_info():\n adata = pbmc3k()[:200].copy()\n sc.pp.normalize_total(adata)\n sc.pp.log1p(adata)\n with pytest.warns(UserWarning, match=\"pass `n_top_genes`, all cutoffs are ignored\"):\n sc.pp.highly_variable_genes(adata, n_top_genes=10, max_mean=3.1)\n\n\n@pytest.mark.parametrize(\"flavor\", [\"seurat\", \"cell_ranger\"])\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\n@pytest.mark.parametrize(\"batch_key\", [None, \"batch\"])\ndef test_subset_inplace_consistency(flavor, array_type, batch_key):\n \"\"\"Tests that, with `n_top_genes=n`\n - `inplace` and `subset` interact correctly\n - for both the `seurat` and `cell_ranger` flavors\n - for dask arrays and non-dask arrays\n - for both with and without batch_key\n \"\"\"\n adata = sc.datasets.blobs(n_observations=20, n_variables=80, random_state=0)\n rng = np.random.default_rng(0)\n adata.obs[\"batch\"] = rng.choice([\"a\", \"b\"], adata.shape[0])\n adata.X = array_type(np.abs(adata.X).astype(int))\n\n if flavor == \"seurat\" or flavor == \"cell_ranger\":\n sc.pp.normalize_total(adata, target_sum=1e4)\n sc.pp.log1p(adata)\n\n elif flavor == \"seurat_v3\":\n pass\n\n else:\n raise ValueError(f\"Unknown flavor {flavor}\")\n\n n_genes = adata.shape[1]\n\n adatas: dict[bool, AnnData] = {}\n dfs: dict[bool, pd.DataFrame] = {}\n # for loops instead of parametrization to compare between settings\n for subset, inplace in itertools.product([True, False], repeat=2):\n adata_copy = adata.copy()\n\n output_df = sc.pp.highly_variable_genes(\n adata_copy,\n flavor=flavor,\n n_top_genes=15,\n batch_key=batch_key,\n subset=subset,\n inplace=inplace,\n )\n\n assert (output_df is None) == inplace\n assert len(adata_copy.var if inplace else output_df) == (\n 15 if subset else n_genes\n )\n assert sum((adata_copy.var if inplace else output_df)[\"highly_variable\"]) == 15\n\n if not inplace:\n assert isinstance(output_df, pd.DataFrame)\n\n if inplace:\n assert subset not in adatas\n adatas[subset] = adata_copy\n else:\n assert subset not in dfs\n dfs[subset] = output_df\n\n # check that the results are consistent for subset True/False: inplace True\n adata_subset = adatas[False][:, adatas[False].var[\"highly_variable\"]]\n assert adata_subset.var_names.equals(adatas[True].var_names)\n\n # check that the results are consistent for subset True/False: inplace False\n df_subset = dfs[False][dfs[False][\"highly_variable\"]]\n assert df_subset.index.equals(dfs[True].index)\n\n # check that the results are consistent for inplace True/False: subset True\n assert adatas[True].var_names.equals(dfs[True].index)\n\n\n@pytest.mark.parametrize(\"flavor\", [\"seurat\", \"cell_ranger\"])\n@pytest.mark.parametrize(\"batch_key\", [None, \"batch\"], ids=[\"single\", \"batched\"])\n@pytest.mark.parametrize(\n \"to_dask\", [p for p in ARRAY_TYPES if \"dask\" in p.values[0].__name__]\n)\ndef test_dask_consistency(adata: AnnData, flavor, batch_key, to_dask):\n adata.X = np.abs(adata.X).astype(int)\n if batch_key is not None:\n adata.obs[batch_key] = np.tile([\"a\", \"b\"], adata.shape[0] // 2)\n sc.pp.normalize_total(adata, target_sum=1e4)\n sc.pp.log1p(adata)\n\n adata_dask = adata.copy()\n adata_dask.X = to_dask(adata_dask.X)\n\n output_mem, output_dask = (\n sc.pp.highly_variable_genes(ad, flavor=flavor, n_top_genes=15, inplace=False)\n for ad in [adata, adata_dask]\n )\n\n assert isinstance(output_mem, pd.DataFrame)\n assert isinstance(output_dask, pd.DataFrame)\n\n assert_index_equal(adata.var_names, output_mem.index, check_names=False)\n assert_index_equal(adata.var_names, output_dask.index, check_names=False)\n\n assert_frame_equal(output_mem, output_dask, atol=1e-4)\n\n\nfrom __future__ import annotations\n\nimport pytest\n\nimport scanpy as sc\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\n\n\ndef test_deprecate_multicore_tsne():\n pbmc = pbmc68k_reduced()\n\n with pytest.warns(\n UserWarning, match=\"calling tsne with n_jobs > 1 would use MulticoreTSNE\"\n ):\n sc.tl.tsne(pbmc, n_jobs=2)\n\n with pytest.warns(FutureWarning, match=\"Argument `use_fast_tsne` is deprecated\"):\n sc.tl.tsne(pbmc, use_fast_tsne=True)\n\n with pytest.warns(UserWarning, match=\"Falling back to scikit-learn\"):\n sc.tl.tsne(pbmc, use_fast_tsne=True)\n\n\ndef test_deprecate_use_highly_variable_genes():\n pbmc = pbmc68k_reduced()\n\n with pytest.warns(\n FutureWarning, match=\"Argument `use_highly_variable` is deprecated\"\n ):\n sc.pp.pca(pbmc, use_highly_variable=True)\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pytest\nfrom numpy.testing import assert_array_almost_equal, assert_array_equal, assert_raises\n\nimport scanpy as sc\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\nfrom testing.scanpy._pytest.marks import needs\n\n\n@pytest.mark.parametrize(\n (\"key_added\", \"key_obsm\", \"key_uns\"),\n [\n pytest.param(None, \"X_tsne\", \"tsne\", id=\"None\"),\n pytest.param(\"custom_key\", \"custom_key\", \"custom_key\", id=\"custom_key\"),\n ],\n)\ndef test_tsne(key_added: str | None, key_obsm: str, key_uns: str):\n pbmc = pbmc68k_reduced()[:200].copy()\n\n euclidean1 = sc.tl.tsne(pbmc, metric=\"euclidean\", copy=True)\n with pytest.warns(UserWarning, match=\"In previous versions of scanpy\"):\n euclidean2 = sc.tl.tsne(\n pbmc, metric=\"euclidean\", n_jobs=2, key_added=key_added, copy=True\n )\n cosine = sc.tl.tsne(pbmc, metric=\"cosine\", copy=True)\n\n # Reproducibility\n np.testing.assert_equal(euclidean1.obsm[\"X_tsne\"], euclidean2.obsm[key_obsm])\n # Metric has some effect\n assert not np.array_equal(euclidean1.obsm[\"X_tsne\"], cosine.obsm[\"X_tsne\"])\n\n # Params are recorded\n assert euclidean1.uns[\"tsne\"][\"params\"][\"n_jobs\"] == 1\n assert euclidean2.uns[key_uns][\"params\"][\"n_jobs\"] == 2\n assert cosine.uns[\"tsne\"][\"params\"][\"n_jobs\"] == 1\n assert euclidean1.uns[\"tsne\"][\"params\"][\"metric\"] == \"euclidean\"\n assert euclidean2.uns[key_uns][\"params\"][\"metric\"] == \"euclidean\"\n assert cosine.uns[\"tsne\"][\"params\"][\"metric\"] == \"cosine\"\n\n\n@pytest.mark.parametrize(\n (\"key_added\", \"key_obsm\", \"key_uns\"),\n [\n pytest.param(None, \"X_umap\", \"umap\", id=\"None\"),\n pytest.param(\"custom_key\", \"custom_key\", \"custom_key\", id=\"custom_key\"),\n ],\n)\ndef test_umap_init_dtype(key_added: str | None, key_obsm: str, key_uns: str):\n pbmc1 = pbmc68k_reduced()[:100, :].copy()\n pbmc2 = pbmc1.copy()\n for pbmc, dtype, k in [(pbmc1, np.float32, None), (pbmc2, np.float64, key_added)]:\n sc.tl.umap(pbmc, init_pos=pbmc.obsm[\"X_pca\"][:, :2].astype(dtype), key_added=k)\n\n # check that embeddings are close for different dtypes\n assert_array_almost_equal(pbmc1.obsm[\"X_umap\"], pbmc2.obsm[key_obsm])\n\n # check that params are recorded\n assert pbmc1.uns[\"umap\"][\"params\"][\"a\"] == pbmc2.uns[key_uns][\"params\"][\"a\"]\n assert pbmc1.uns[\"umap\"][\"params\"][\"b\"] == pbmc2.uns[key_uns][\"params\"][\"b\"]\n\n\n@pytest.mark.parametrize(\n \"layout\",\n [\n pytest.param(\"fa\", marks=needs.fa2),\n pytest.param(\"fr\", marks=needs.igraph),\n ],\n)\ndef test_umap_init_paga(layout):\n pbmc = pbmc68k_reduced()[:100, :].copy()\n sc.tl.paga(pbmc)\n sc.pl.paga(pbmc, layout=layout, show=False)\n sc.tl.umap(pbmc, init_pos=\"paga\")\n\n\ndef test_diffmap():\n pbmc = pbmc68k_reduced()\n\n sc.tl.diffmap(pbmc)\n d1 = pbmc.obsm[\"X_diffmap\"].copy()\n sc.tl.diffmap(pbmc)\n d2 = pbmc.obsm[\"X_diffmap\"].copy()\n assert_array_equal(d1, d2)\n\n # Checking if specifying random_state works, arrays shouldn't be equal\n sc.tl.diffmap(pbmc, random_state=1234)\n d3 = pbmc.obsm[\"X_diffmap\"].copy()\n assert_raises(AssertionError, assert_array_equal, d1, d3)\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom functools import wraps\nfrom typing import TYPE_CHECKING\n\nimport anndata as ad\nimport numpy as np\nimport pytest\nfrom anndata import AnnData\nfrom anndata.tests.helpers import (\n asarray,\n assert_equal,\n)\nfrom packaging.version import Version\nfrom scipy import sparse\nfrom scipy.sparse import issparse\n\nimport scanpy as sc\nfrom testing.scanpy import _helpers\nfrom testing.scanpy._helpers.data import pbmc3k_normalized\nfrom testing.scanpy._pytest.marks import needs\nfrom testing.scanpy._pytest.params import (\n ARRAY_TYPES,\n ARRAY_TYPES_SPARSE_DASK_UNSUPPORTED,\n param_with,\n)\n\nif TYPE_CHECKING:\n from collections.abc import Callable\n from typing import Literal\n\n from scanpy._compat import DaskArray\n\nA_list = np.array(\n [\n [0, 0, 7, 0, 0],\n [8, 5, 0, 2, 0],\n [6, 0, 0, 2, 5],\n [0, 0, 0, 1, 0],\n [8, 8, 2, 1, 0],\n [0, 0, 0, 4, 5],\n ]\n)\n\nA_pca = np.array(\n [\n [-4.4783009, 5.55508466, 1.73111572, -0.06029139, 0.17292555],\n [5.4855141, -0.42651191, -0.74776055, -0.74532146, 0.74633582],\n [0.01161428, -4.0156662, 2.37252748, -1.33122372, -0.29044446],\n [-3.61934397, 0.48525412, -2.96861931, -1.16312545, -0.33230607],\n [7.14050048, 1.86330409, -0.05786325, 1.25045782, -0.50213107],\n [-4.53998399, -3.46146476, -0.32940009, 2.04950419, 0.20562023],\n ]\n)\n\nA_svd = np.array(\n [\n [-0.77034038, -2.00750922, 6.64603489, -0.39669256, -0.22212097],\n [-9.47135856, -0.6326006, -1.33787112, -0.24894361, -1.02044665],\n [-5.90007339, 4.99658727, 0.70712592, -2.15188849, 0.30430008],\n [-0.19132409, 0.42172251, 0.11169531, 0.50977966, -0.71637566],\n [-11.1286238, -2.73045559, 0.08040596, 1.06850585, 0.74173764],\n [-1.50180389, 5.56886849, 1.64034442, 2.24476032, -0.05109001],\n ]\n)\n\n\ndef _chunked_1d(\n f: Callable[[np.ndarray], DaskArray],\n) -> Callable[[np.ndarray], DaskArray]:\n @wraps(f)\n def wrapper(a: np.ndarray) -> DaskArray:\n da = f(a)\n return da.rechunk((da.chunksize[0], -1))\n\n return wrapper\n\n\nDASK_CONVERTERS = {\n f: _chunked_1d(f)\n for f in (_helpers.as_dense_dask_array, _helpers.as_sparse_dask_array)\n}\n\n\n@pytest.fixture(\n params=[\n param_with(at, marks=[needs.dask_ml]) if \"dask\" in at.id else at\n for at in ARRAY_TYPES_SPARSE_DASK_UNSUPPORTED\n ]\n)\ndef array_type(request: pytest.FixtureRequest):\n # If one uses dask for PCA it will always require dask-ml.\n # dask-ml can’t do 2D-chunked arrays, so rechunk them.\n if as_dask_array := DASK_CONVERTERS.get(request.param):\n return as_dask_array\n\n # When not using dask, just return the array type\n assert \"dask\" not in request.param.__name__, \"add more branches or refactor\"\n return request.param\n\n\n@pytest.fixture(params=[None, \"valid\", \"invalid\"])\ndef svd_solver_type(request: pytest.FixtureRequest):\n return request.param\n\n\n@pytest.fixture(params=[True, False], ids=[\"zero_center\", \"no_zero_center\"])\ndef zero_center(request: pytest.FixtureRequest):\n return request.param\n\n\n@pytest.fixture\ndef pca_params(\n array_type, svd_solver_type: Literal[None, \"valid\", \"invalid\"], zero_center\n):\n all_svd_solvers = {\"auto\", \"full\", \"arpack\", \"randomized\", \"tsqr\", \"lobpcg\"}\n\n expected_warning = None\n svd_solver = None\n if svd_solver_type is not None:\n if array_type in DASK_CONVERTERS.values():\n svd_solver = (\n {\"auto\", \"full\", \"tsqr\", \"randomized\"}\n if zero_center\n else {\"tsqr\", \"randomized\"}\n )\n elif array_type in {sparse.csr_matrix, sparse.csc_matrix}:\n svd_solver = (\n {\"lobpcg\", \"arpack\"} if zero_center else {\"arpack\", \"randomized\"}\n )\n elif array_type is asarray:\n svd_solver = (\n {\"auto\", \"full\", \"arpack\", \"randomized\"}\n if zero_center\n else {\"arpack\", \"randomized\"}\n )\n else:\n pytest.fail(f\"Unknown array type {array_type}\")\n if svd_solver_type == \"invalid\":\n svd_solver = all_svd_solvers - svd_solver\n expected_warning = \"Ignoring\"\n\n svd_solver = np.random.choice(list(svd_solver))\n # explicit check for special case\n if (\n svd_solver == \"randomized\"\n and zero_center\n and array_type in [sparse.csr_matrix, sparse.csc_matrix]\n ):\n expected_warning = \"not work with sparse input\"\n\n return (svd_solver, expected_warning)\n\n\ndef test_pca_warnings(array_type, zero_center, pca_params):\n svd_solver, expected_warning = pca_params\n A = array_type(A_list).astype(\"float32\")\n adata = AnnData(A)\n\n if expected_warning is not None:\n with pytest.warns(UserWarning, match=expected_warning):\n sc.pp.pca(adata, svd_solver=svd_solver, zero_center=zero_center)\n return\n\n try:\n with warnings.catch_warnings():\n warnings.simplefilter(\"error\")\n warnings.filterwarnings(\n \"ignore\",\n \"pkg_resources is deprecated as an API\",\n DeprecationWarning,\n )\n sc.pp.pca(adata, svd_solver=svd_solver, zero_center=zero_center)\n except UserWarning:\n # TODO: Fix this case, maybe by increasing test data size.\n # https://github.com/scverse/scanpy/issues/2744\n if svd_solver == \"lobpcg\":\n pytest.xfail(reason=\"lobpcg doesn’t work with this small test data\")\n raise\n\n\n# This warning test is out of the fixture because it is a special case in the logic of the function\ndef test_pca_warnings_sparse():\n for array_type in (sparse.csr_matrix, sparse.csc_matrix):\n A = array_type(A_list).astype(\"float32\")\n adata = AnnData(A)\n with pytest.warns(UserWarning, match=\"not work with sparse input\"):\n sc.pp.pca(adata, svd_solver=\"randomized\", zero_center=True)\n\n\ndef test_pca_transform(array_type):\n A = array_type(A_list).astype(\"float32\")\n A_pca_abs = np.abs(A_pca)\n A_svd_abs = np.abs(A_svd)\n\n adata = AnnData(A)\n\n with warnings.catch_warnings(record=True) as record:\n sc.pp.pca(adata, n_comps=4, zero_center=True, dtype=\"float64\")\n assert len(record) == 0, record\n\n assert np.linalg.norm(A_pca_abs[:, :4] - np.abs(adata.obsm[\"X_pca\"])) < 2e-05\n\n with warnings.catch_warnings(record=True) as record:\n sc.pp.pca(\n adata,\n n_comps=5,\n zero_center=True,\n svd_solver=\"randomized\",\n dtype=\"float64\",\n random_state=14,\n )\n if sparse.issparse(A):\n assert any(\n isinstance(r.message, UserWarning)\n and \"svd_solver 'randomized' does not work with sparse input\"\n in str(r.message)\n for r in record\n )\n else:\n assert len(record) == 0\n\n assert np.linalg.norm(A_pca_abs - np.abs(adata.obsm[\"X_pca\"])) < 2e-05\n\n with warnings.catch_warnings(record=True) as record:\n sc.pp.pca(adata, n_comps=4, zero_center=False, dtype=\"float64\", random_state=14)\n assert len(record) == 0\n\n assert np.linalg.norm(A_svd_abs[:, :4] - np.abs(adata.obsm[\"X_pca\"])) < 2e-05\n\n\ndef test_pca_shapes():\n \"\"\"\n Tests that n_comps behaves correctly\n See https://github.com/scverse/scanpy/issues/1051\n \"\"\"\n adata = AnnData(np.random.randn(30, 20))\n sc.pp.pca(adata)\n assert adata.obsm[\"X_pca\"].shape == (30, 19)\n\n adata = AnnData(np.random.randn(20, 30))\n sc.pp.pca(adata)\n assert adata.obsm[\"X_pca\"].shape == (20, 19)\n\n with pytest.raises(\n ValueError,\n match=r\"n_components=100 must be between 1 and.*20 with svd_solver='arpack'\",\n ):\n sc.pp.pca(adata, n_comps=100)\n\n\n@pytest.mark.parametrize(\n (\"key_added\", \"keys_expected\"),\n [\n pytest.param(None, (\"X_pca\", \"PCs\", \"pca\"), id=\"None\"),\n pytest.param(\"custom_key\", (\"custom_key\",) * 3, id=\"custom_key\"),\n ],\n)\ndef test_pca_sparse(key_added: str | None, keys_expected: tuple[str, str, str]):\n \"\"\"\n Tests that implicitly centered pca on sparse arrays returns equivalent results to\n explicit centering on dense arrays.\n \"\"\"\n pbmc = pbmc3k_normalized()[:200].copy()\n\n pbmc_dense = pbmc.copy()\n pbmc_dense.X = pbmc_dense.X.toarray()\n\n implicit = sc.pp.pca(pbmc, dtype=np.float64, copy=True)\n explicit = sc.pp.pca(pbmc_dense, dtype=np.float64, key_added=key_added, copy=True)\n\n key_obsm, key_varm, key_uns = keys_expected\n\n np.testing.assert_allclose(\n implicit.uns[\"pca\"][\"variance\"], explicit.uns[key_uns][\"variance\"]\n )\n np.testing.assert_allclose(\n implicit.uns[\"pca\"][\"variance_ratio\"], explicit.uns[key_uns][\"variance_ratio\"]\n )\n np.testing.assert_allclose(implicit.obsm[\"X_pca\"], explicit.obsm[key_obsm])\n np.testing.assert_allclose(implicit.varm[\"PCs\"], explicit.varm[key_varm])\n\n\ndef test_pca_reproducible(array_type):\n pbmc = pbmc3k_normalized()\n pbmc.X = array_type(pbmc.X)\n\n a = sc.pp.pca(pbmc, copy=True, dtype=np.float64, random_state=42)\n b = sc.pp.pca(pbmc, copy=True, dtype=np.float64, random_state=42)\n c = sc.pp.pca(pbmc, copy=True, dtype=np.float64, random_state=0)\n\n assert_equal(a, b)\n # Test that changing random seed changes result\n # Does not show up reliably with 32 bit computation\n assert not np.array_equal(a.obsm[\"X_pca\"], c.obsm[\"X_pca\"])\n\n\ndef test_pca_chunked():\n \"\"\"\n See https://github.com/scverse/scanpy/issues/1590\n But this is also a more general test\n \"\"\"\n\n # Subsetting for speed of test\n pbmc_full = pbmc3k_normalized()\n pbmc = pbmc_full[::6].copy()\n pbmc.X = pbmc.X.astype(np.float64)\n chunked = sc.pp.pca(pbmc_full, chunked=True, copy=True)\n default = sc.pp.pca(pbmc_full, copy=True)\n\n # Taking absolute value since sometimes dimensions are flipped\n np.testing.assert_allclose(\n np.abs(chunked.obsm[\"X_pca\"]), np.abs(default.obsm[\"X_pca\"])\n )\n np.testing.assert_allclose(np.abs(chunked.varm[\"PCs\"]), np.abs(default.varm[\"PCs\"]))\n np.testing.assert_allclose(\n np.abs(chunked.uns[\"pca\"][\"variance\"]), np.abs(default.uns[\"pca\"][\"variance\"])\n )\n np.testing.assert_allclose(\n np.abs(chunked.uns[\"pca\"][\"variance_ratio\"]),\n np.abs(default.uns[\"pca\"][\"variance_ratio\"]),\n )\n\n\ndef test_pca_n_pcs():\n \"\"\"\n Tests that the n_pcs parameter also works for\n representations not called \"X_pca\"\n \"\"\"\n pbmc = pbmc3k_normalized()\n sc.pp.pca(pbmc, dtype=np.float64)\n pbmc.obsm[\"X_pca_test\"] = pbmc.obsm[\"X_pca\"]\n original = sc.pp.neighbors(pbmc, n_pcs=5, use_rep=\"X_pca\", copy=True)\n renamed = sc.pp.neighbors(pbmc, n_pcs=5, use_rep=\"X_pca_test\", copy=True)\n\n assert np.allclose(original.obsm[\"X_pca\"], renamed.obsm[\"X_pca_test\"])\n assert np.allclose(\n original.obsp[\"distances\"].toarray(), renamed.obsp[\"distances\"].toarray()\n )\n\n\n# We use all ARRAY_TYPES here since this error should be raised before\n# PCA can realize that it got a Dask array\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\ndef test_mask_highly_var_error(array_type):\n \"\"\"Check if use_highly_variable=True throws an error if the annotation is missing.\"\"\"\n adata = AnnData(array_type(A_list).astype(\"float32\"))\n with (\n pytest.warns(\n FutureWarning,\n match=r\"Argument `use_highly_variable` is deprecated, consider using the mask argument\\.\",\n ),\n pytest.raises(\n ValueError,\n match=r\"Did not find `adata\\.var\\['highly_variable'\\]`\\.\",\n ),\n ):\n sc.pp.pca(adata, use_highly_variable=True)\n\n\ndef test_mask_length_error():\n \"\"\"Check error for n_obs / mask length mismatch.\"\"\"\n adata = AnnData(A_list)\n mask_var = np.random.choice([True, False], adata.shape[1] + 1)\n with pytest.raises(\n ValueError, match=r\"The shape of the mask do not match the data\\.\"\n ):\n sc.pp.pca(adata, mask_var=mask_var, copy=True)\n\n\ndef test_mask_var_argument_equivalence(float_dtype, array_type):\n \"\"\"Test if pca result is equal when given mask as boolarray vs string\"\"\"\n\n adata_base = AnnData(array_type(np.random.random((100, 10))).astype(float_dtype))\n mask_var = np.random.choice([True, False], adata_base.shape[1])\n\n adata = adata_base.copy()\n sc.pp.pca(adata, mask_var=mask_var, dtype=float_dtype)\n\n adata_w_mask = adata_base.copy()\n adata_w_mask.var[\"mask\"] = mask_var\n sc.pp.pca(adata_w_mask, mask_var=\"mask\", dtype=float_dtype)\n\n assert np.allclose(\n adata.X.toarray() if issparse(adata.X) else adata.X,\n adata_w_mask.X.toarray() if issparse(adata_w_mask.X) else adata_w_mask.X,\n )\n\n\ndef test_mask(request: pytest.FixtureRequest, array_type):\n if array_type in DASK_CONVERTERS.values():\n reason = \"TODO: Dask arrays are not supported\"\n request.applymarker(pytest.mark.xfail(reason=reason))\n adata = sc.datasets.blobs(n_variables=10, n_centers=3, n_observations=100)\n adata.X = array_type(adata.X)\n\n if isinstance(adata.X, np.ndarray) and Version(ad.__version__) < Version(\"0.9\"):\n reason = (\n \"TODO: Previous version of anndata would return an F ordered array for one\"\n \" case here, which surprisingly considerably changes the results of PCA.\"\n )\n request.applymarker(pytest.mark.xfail(reason=reason))\n mask_var = np.random.choice([True, False], adata.shape[1])\n\n adata_masked = adata[:, mask_var].copy()\n sc.pp.pca(adata, mask_var=mask_var)\n sc.pp.pca(adata_masked)\n\n masked_var_loadings = adata.varm[\"PCs\"][~mask_var]\n np.testing.assert_equal(masked_var_loadings, np.zeros_like(masked_var_loadings))\n\n np.testing.assert_equal(adata.obsm[\"X_pca\"], adata_masked.obsm[\"X_pca\"])\n # There are slight difference based on whether the matrix was column or row major\n np.testing.assert_allclose(\n adata.varm[\"PCs\"][mask_var], adata_masked.varm[\"PCs\"], rtol=1e-11\n )\n\n\ndef test_mask_order_warning(request: pytest.FixtureRequest):\n if Version(ad.__version__) >= Version(\"0.9\"):\n reason = \"Not expected to warn in later versions of anndata\"\n request.applymarker(pytest.mark.xfail(reason=reason))\n\n adata = ad.AnnData(X=np.random.randn(50, 5))\n mask = np.array([True, False, True, False, True])\n\n with pytest.warns(\n UserWarning,\n match=\"When using a mask parameter with anndata<0.9 on a dense array\",\n ):\n sc.pp.pca(adata, mask_var=mask)\n\n\ndef test_mask_defaults(array_type, float_dtype):\n \"\"\"\n Test if pca result is equal without highly variable and with-but mask is None\n and if pca takes highly variable as mask as default\n \"\"\"\n A = array_type(A_list).astype(\"float64\")\n adata = AnnData(A)\n\n without_var = sc.pp.pca(adata, copy=True, dtype=float_dtype)\n\n rng = np.random.default_rng(8)\n mask = rng.choice([True, False], adata.shape[1])\n adata.var[\"highly_variable\"] = mask\n with_var = sc.pp.pca(adata, copy=True, dtype=float_dtype)\n assert without_var.uns[\"pca\"][\"params\"][\"mask_var\"] is None\n assert with_var.uns[\"pca\"][\"params\"][\"mask_var\"] == \"highly_variable\"\n assert not np.array_equal(without_var.obsm[\"X_pca\"], with_var.obsm[\"X_pca\"])\n with_no_mask = sc.pp.pca(adata, mask_var=None, copy=True, dtype=float_dtype)\n assert np.array_equal(without_var.obsm[\"X_pca\"], with_no_mask.obsm[\"X_pca\"])\n\n\ndef test_pca_layer():\n \"\"\"\n Tests that layers works the same way as .X\n \"\"\"\n X_adata = pbmc3k_normalized()\n\n layer_adata = X_adata.copy()\n layer_adata.layers[\"counts\"] = X_adata.X.copy()\n del layer_adata.X\n\n sc.pp.pca(X_adata)\n sc.pp.pca(layer_adata, layer=\"counts\")\n\n assert layer_adata.uns[\"pca\"][\"params\"][\"layer\"] == \"counts\"\n assert \"layer\" not in X_adata.uns[\"pca\"][\"params\"]\n\n np.testing.assert_equal(\n X_adata.uns[\"pca\"][\"variance\"], layer_adata.uns[\"pca\"][\"variance\"]\n )\n np.testing.assert_equal(\n X_adata.uns[\"pca\"][\"variance_ratio\"], layer_adata.uns[\"pca\"][\"variance_ratio\"]\n )\n np.testing.assert_equal(X_adata.obsm[\"X_pca\"], layer_adata.obsm[\"X_pca\"])\n np.testing.assert_equal(X_adata.varm[\"PCs\"], layer_adata.varm[\"PCs\"])\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pytest\n\nimport scanpy as sc\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\nfrom testing.scanpy._pytest.marks import needs\n\nn_neighbors = 5\nkey = \"test\"\n\n\n@pytest.fixture\ndef adata():\n return sc.AnnData(pbmc68k_reduced().X)\n\n\ndef test_neighbors_key_added(adata):\n sc.pp.neighbors(adata, n_neighbors=n_neighbors, random_state=0)\n sc.pp.neighbors(adata, n_neighbors=n_neighbors, random_state=0, key_added=key)\n\n conns_key = adata.uns[key][\"connectivities_key\"]\n dists_key = adata.uns[key][\"distances_key\"]\n\n assert adata.uns[\"neighbors\"][\"params\"] == adata.uns[key][\"params\"]\n assert np.allclose(\n adata.obsp[\"connectivities\"].toarray(), adata.obsp[conns_key].toarray()\n )\n assert np.allclose(\n adata.obsp[\"distances\"].toarray(), adata.obsp[dists_key].toarray()\n )\n\n\ndef test_neighbors_pca_keys_added_without_previous_pca_run(adata):\n assert \"pca\" not in adata.uns\n assert \"X_pca\" not in adata.obsm\n with pytest.warns(\n UserWarning,\n match=r\".*Falling back to preprocessing with `sc.pp.pca` and default params\",\n ):\n sc.pp.neighbors(adata, n_neighbors=n_neighbors, random_state=0)\n assert \"pca\" in adata.uns\n\n\n# test functions with neighbors_key and obsp\n@needs.igraph\n@needs.leidenalg\n@pytest.mark.parametrize(\"field\", [\"neighbors_key\", \"obsp\"])\ndef test_neighbors_key_obsp(adata, field):\n adata1 = adata.copy()\n\n sc.pp.neighbors(adata, n_neighbors=n_neighbors, random_state=0)\n sc.pp.neighbors(adata1, n_neighbors=n_neighbors, random_state=0, key_added=key)\n\n if field == \"neighbors_key\":\n arg = {field: key}\n else:\n arg = {field: adata1.uns[key][\"connectivities_key\"]}\n\n sc.tl.draw_graph(adata, layout=\"fr\", random_state=1)\n sc.tl.draw_graph(adata1, layout=\"fr\", random_state=1, **arg)\n\n assert adata.uns[\"draw_graph\"][\"params\"] == adata1.uns[\"draw_graph\"][\"params\"]\n assert np.allclose(adata.obsm[\"X_draw_graph_fr\"], adata1.obsm[\"X_draw_graph_fr\"])\n\n sc.tl.leiden(adata, random_state=0)\n sc.tl.leiden(adata1, random_state=0, **arg)\n\n assert adata.uns[\"leiden\"][\"params\"] == adata1.uns[\"leiden\"][\"params\"]\n assert np.all(adata.obs[\"leiden\"] == adata1.obs[\"leiden\"])\n\n # no obsp in umap, paga\n if field == \"neighbors_key\":\n sc.tl.umap(adata, random_state=0)\n sc.tl.umap(adata1, random_state=0, neighbors_key=key)\n\n assert adata.uns[\"umap\"][\"params\"] == adata1.uns[\"umap\"][\"params\"]\n assert np.allclose(adata.obsm[\"X_umap\"], adata1.obsm[\"X_umap\"])\n\n sc.tl.paga(adata, groups=\"leiden\")\n sc.tl.paga(adata1, groups=\"leiden\", neighbors_key=key)\n\n assert np.allclose(\n adata.uns[\"paga\"][\"connectivities\"].toarray(),\n adata1.uns[\"paga\"][\"connectivities\"].toarray(),\n )\n assert np.allclose(\n adata.uns[\"paga\"][\"connectivities_tree\"].toarray(),\n adata1.uns[\"paga\"][\"connectivities_tree\"].toarray(),\n )\n\n\n@needs.louvain\n@pytest.mark.parametrize(\"field\", [\"neighbors_key\", \"obsp\"])\ndef test_neighbors_key_obsp_louvain(adata, field):\n adata1 = adata.copy()\n\n sc.pp.neighbors(adata, n_neighbors=n_neighbors, random_state=0)\n sc.pp.neighbors(adata1, n_neighbors=n_neighbors, random_state=0, key_added=key)\n\n if field == \"neighbors_key\":\n arg = {field: key}\n else:\n arg = {field: adata1.uns[key][\"connectivities_key\"]}\n\n sc.tl.louvain(adata, random_state=0)\n sc.tl.louvain(adata1, random_state=0, **arg)\n\n assert adata.uns[\"louvain\"][\"params\"] == adata1.uns[\"louvain\"][\"params\"]\n assert np.all(adata.obs[\"louvain\"] == adata1.obs[\"louvain\"])\n\n\nfrom __future__ import annotations\n\nimport pickle\nfrom functools import partial\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nimport pytest\nimport scipy\nfrom anndata import AnnData\nfrom numpy.random import binomial, negative_binomial, seed\nfrom packaging.version import Version\nfrom scipy.stats import mannwhitneyu\n\nimport scanpy as sc\nfrom scanpy._utils import elem_mul, select_groups\nfrom scanpy.get import rank_genes_groups_df\nfrom scanpy.tools import rank_genes_groups\nfrom scanpy.tools._rank_genes_groups import _RankGenes\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\nfrom testing.scanpy._pytest.params import ARRAY_TYPES, ARRAY_TYPES_MEM\n\nif TYPE_CHECKING:\n from collections.abc import Callable\n from typing import Any\n\n from numpy.typing import NDArray\n\nHERE = Path(__file__).parent\nDATA_PATH = HERE / \"_data\"\n\n\n# We test results for a simple generic example\n# Tests are conducted for sparse and non-sparse AnnData objects.\n# Due to minor changes in multiplication implementation for sparse and non-sparse objects,\n# results differ (very) slightly\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES)\ndef get_example_data(array_type: Callable[[np.ndarray], Any]) -> AnnData:\n # create test object\n adata = AnnData(\n np.multiply(binomial(1, 0.15, (100, 20)), negative_binomial(2, 0.25, (100, 20)))\n )\n # adapt marker_genes for cluster (so as to have some form of reasonable input\n adata.X[0:10, 0:5] = np.multiply(\n binomial(1, 0.9, (10, 5)), negative_binomial(1, 0.5, (10, 5))\n )\n\n adata.X = array_type(adata.X)\n\n # Create cluster according to groups\n adata.obs[\"true_groups\"] = pd.Categorical(\n np.concatenate((np.zeros((10,), dtype=int), np.ones((90,), dtype=int)))\n )\n\n return adata\n\n\ndef get_true_scores() -> (\n tuple[\n NDArray[np.object_],\n NDArray[np.object_],\n NDArray[np.floating],\n NDArray[np.floating],\n ]\n):\n with (DATA_PATH / \"objs_t_test.pkl\").open(\"rb\") as f:\n true_scores_t_test, true_names_t_test = pickle.load(f)\n with (DATA_PATH / \"objs_wilcoxon.pkl\").open(\"rb\") as f:\n true_scores_wilcoxon, true_names_wilcoxon = pickle.load(f)\n\n return (\n true_names_t_test,\n true_names_wilcoxon,\n true_scores_t_test,\n true_scores_wilcoxon,\n )\n\n\n# TODO: Make dask compatible\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES_MEM)\ndef test_results(array_type):\n seed(1234)\n\n adata = get_example_data(array_type)\n assert adata.raw is None # Assumption for later checks\n\n (\n true_names_t_test,\n true_names_wilcoxon,\n true_scores_t_test,\n true_scores_wilcoxon,\n ) = get_true_scores()\n\n rank_genes_groups(adata, \"true_groups\", n_genes=20, method=\"t-test\")\n\n adata.uns[\"rank_genes_groups\"][\"names\"] = adata.uns[\"rank_genes_groups\"][\n \"names\"\n ].astype(true_names_t_test.dtype)\n\n for name in true_scores_t_test.dtype.names:\n assert np.allclose(\n true_scores_t_test[name], adata.uns[\"rank_genes_groups\"][\"scores\"][name]\n )\n assert np.array_equal(true_names_t_test, adata.uns[\"rank_genes_groups\"][\"names\"])\n assert adata.uns[\"rank_genes_groups\"][\"params\"][\"use_raw\"] is False\n\n rank_genes_groups(adata, \"true_groups\", n_genes=20, method=\"wilcoxon\")\n\n adata.uns[\"rank_genes_groups\"][\"names\"] = adata.uns[\"rank_genes_groups\"][\n \"names\"\n ].astype(true_names_wilcoxon.dtype)\n\n for name in true_scores_t_test.dtype.names:\n assert np.allclose(\n true_scores_wilcoxon[name][:7],\n adata.uns[\"rank_genes_groups\"][\"scores\"][name][:7],\n )\n assert np.array_equal(\n true_names_wilcoxon[:7], adata.uns[\"rank_genes_groups\"][\"names\"][:7]\n )\n assert adata.uns[\"rank_genes_groups\"][\"params\"][\"use_raw\"] is False\n\n\n@pytest.mark.parametrize(\"array_type\", ARRAY_TYPES_MEM)\ndef test_results_layers(array_type):\n seed(1234)\n\n adata = get_example_data(array_type)\n adata.layers[\"to_test\"] = adata.X.copy()\n adata.X = elem_mul(adata.X, np.random.randint(0, 2, adata.shape, dtype=bool))\n\n _, _, true_scores_t_test, true_scores_wilcoxon = get_true_scores()\n\n # Wilcoxon\n rank_genes_groups(\n adata,\n \"true_groups\",\n method=\"wilcoxon\",\n layer=\"to_test\",\n n_genes=20,\n )\n assert adata.uns[\"rank_genes_groups\"][\"params\"][\"use_raw\"] is False\n for name in true_scores_t_test.dtype.names:\n assert np.allclose(\n true_scores_wilcoxon[name][:7],\n adata.uns[\"rank_genes_groups\"][\"scores\"][name][:7],\n )\n\n rank_genes_groups(adata, \"true_groups\", method=\"wilcoxon\", n_genes=20)\n for name in true_scores_t_test.dtype.names:\n assert not np.allclose(\n true_scores_wilcoxon[name][:7],\n adata.uns[\"rank_genes_groups\"][\"scores\"][name][:7],\n )\n\n # t-test\n rank_genes_groups(\n adata,\n \"true_groups\",\n method=\"t-test\",\n layer=\"to_test\",\n use_raw=False,\n n_genes=20,\n )\n for name in true_scores_t_test.dtype.names:\n assert np.allclose(\n true_scores_t_test[name][:7],\n adata.uns[\"rank_genes_groups\"][\"scores\"][name][:7],\n )\n\n rank_genes_groups(adata, \"true_groups\", method=\"t-test\", n_genes=20)\n for name in true_scores_t_test.dtype.names:\n assert not np.allclose(\n true_scores_t_test[name][:7],\n adata.uns[\"rank_genes_groups\"][\"scores\"][name][:7],\n )\n\n\ndef test_rank_genes_groups_use_raw():\n # https://github.com/scverse/scanpy/issues/1929\n pbmc = pbmc68k_reduced()\n assert pbmc.raw is not None\n\n sc.tl.rank_genes_groups(pbmc, groupby=\"bulk_labels\", use_raw=True)\n\n pbmc = pbmc68k_reduced()\n del pbmc.raw\n assert pbmc.raw is None\n\n with pytest.raises(\n ValueError, match=\"Received `use_raw=True`, but `adata.raw` is empty\"\n ):\n sc.tl.rank_genes_groups(pbmc, groupby=\"bulk_labels\", use_raw=True)\n\n\ndef test_singlets():\n pbmc = pbmc68k_reduced()\n pbmc.obs[\"louvain\"] = pbmc.obs[\"louvain\"].cat.add_categories([\"11\"])\n pbmc.obs[\"louvain\"][0] = \"11\"\n\n with pytest.raises(ValueError, match=rf\"Could not calculate statistics.*{'11'}\"):\n rank_genes_groups(pbmc, groupby=\"louvain\")\n\n\ndef test_emptycat():\n pbmc = pbmc68k_reduced()\n pbmc.obs[\"louvain\"] = pbmc.obs[\"louvain\"].cat.add_categories([\"11\"])\n\n with pytest.raises(ValueError, match=rf\"Could not calculate statistics.*{'11'}\"):\n rank_genes_groups(pbmc, groupby=\"louvain\")\n\n\ndef test_log1p_save_restore(tmp_path):\n \"\"\"tests the sequence log1p→save→load→rank_genes_groups\"\"\"\n from anndata import read_h5ad\n\n pbmc = pbmc68k_reduced()\n sc.pp.log1p(pbmc)\n\n path = tmp_path / \"test.h5ad\"\n pbmc.write(path)\n\n pbmc = read_h5ad(path)\n\n sc.tl.rank_genes_groups(pbmc, groupby=\"bulk_labels\", use_raw=True)\n\n\ndef test_wilcoxon_symmetry():\n pbmc = pbmc68k_reduced()\n\n rank_genes_groups(\n pbmc,\n groupby=\"bulk_labels\",\n groups=[\"CD14+ Monocyte\", \"Dendritic\"],\n reference=\"Dendritic\",\n method=\"wilcoxon\",\n rankby_abs=True,\n )\n assert pbmc.uns[\"rank_genes_groups\"][\"params\"][\"use_raw\"] is True\n\n stats_mono = (\n rank_genes_groups_df(pbmc, group=\"CD14+ Monocyte\")\n .drop(columns=\"names\")\n .to_numpy()\n )\n\n rank_genes_groups(\n pbmc,\n groupby=\"bulk_labels\",\n groups=[\"CD14+ Monocyte\", \"Dendritic\"],\n reference=\"CD14+ Monocyte\",\n method=\"wilcoxon\",\n rankby_abs=True,\n )\n\n stats_dend = (\n rank_genes_groups_df(pbmc, group=\"Dendritic\").drop(columns=\"names\").to_numpy()\n )\n\n assert np.allclose(np.abs(stats_mono), np.abs(stats_dend))\n\n\n@pytest.mark.parametrize(\"reference\", [True, False])\ndef test_wilcoxon_tie_correction(reference):\n pbmc = pbmc68k_reduced()\n\n groups = [\"CD14+ Monocyte\", \"Dendritic\"]\n groupby = \"bulk_labels\"\n\n _, groups_masks = select_groups(pbmc, groups, groupby)\n\n X = pbmc.raw.X[groups_masks[0]].toarray()\n\n mask_rest = groups_masks[1] if reference else ~groups_masks[0]\n Y = pbmc.raw.X[mask_rest].toarray()\n\n # Handle scipy versions\n if Version(scipy.__version__) >= Version(\"1.7.0\"):\n pvals = mannwhitneyu(X, Y, use_continuity=False, alternative=\"two-sided\").pvalue\n pvals[np.isnan(pvals)] = 1.0\n else:\n # Backwards compat, to drop once we drop scipy < 1.7\n n_genes = X.shape[1]\n pvals = np.zeros(n_genes)\n\n for i in range(n_genes):\n try:\n _, pvals[i] = mannwhitneyu(\n X[:, i], Y[:, i], use_continuity=False, alternative=\"two-sided\"\n )\n except ValueError:\n pvals[i] = 1\n\n if reference:\n ref = groups[1]\n else:\n ref = \"rest\"\n groups = groups[:1]\n\n test_obj = _RankGenes(pbmc, groups, groupby, reference=ref)\n test_obj.compute_statistics(\"wilcoxon\", tie_correct=True)\n\n np.testing.assert_allclose(test_obj.stats[groups[0]][\"pvals\"], pvals)\n\n\n@pytest.mark.parametrize(\n (\"n_genes_add\", \"n_genes_out_add\"),\n [pytest.param(0, 0, id=\"equal\"), pytest.param(2, 1, id=\"more\")],\n)\ndef test_mask_n_genes(n_genes_add, n_genes_out_add):\n \"\"\"\\\n Check that no. genes in output is\n 1. =n_genes when n_genessum(mask)\n \"\"\"\n\n pbmc = pbmc68k_reduced()\n mask_var = np.zeros(pbmc.shape[1]).astype(bool)\n mask_var[:6].fill(True) # noqa: FBT003\n no_genes = sum(mask_var) - 1\n\n rank_genes_groups(\n pbmc,\n mask_var=mask_var,\n groupby=\"bulk_labels\",\n groups=[\"CD14+ Monocyte\", \"Dendritic\"],\n reference=\"CD14+ Monocyte\",\n n_genes=no_genes + n_genes_add,\n method=\"wilcoxon\",\n )\n\n assert len(pbmc.uns[\"rank_genes_groups\"][\"scores\"]) == no_genes + n_genes_out_add\n\n\ndef test_mask_not_equal():\n \"\"\"\\\n Check that mask is applied successfully to data set \\\n where test statistics are already available (test stats overwritten).\n \"\"\"\n\n pbmc = pbmc68k_reduced()\n mask_var = np.random.choice([True, False], pbmc.shape[1])\n n_genes = sum(mask_var)\n\n run = partial(\n rank_genes_groups,\n pbmc,\n groupby=\"bulk_labels\",\n groups=[\"CD14+ Monocyte\", \"Dendritic\"],\n reference=\"CD14+ Monocyte\",\n method=\"wilcoxon\",\n )\n\n run(n_genes=n_genes)\n no_mask = pbmc.uns[\"rank_genes_groups\"][\"names\"]\n\n run(mask_var=mask_var)\n with_mask = pbmc.uns[\"rank_genes_groups\"][\"names\"]\n\n assert not np.array_equal(no_mask, with_mask)\n\n\nfrom __future__ import annotations\n\nfrom pathlib import PurePosixPath, PureWindowsPath\n\nimport pytest\n\nfrom scanpy.readwrite import _slugify\n\n\n@pytest.mark.parametrize(\n \"path\",\n [\n PureWindowsPath(r\"C:\\foo\\bar\"),\n PureWindowsPath(r\".\\C\\foo\\bar\"),\n PureWindowsPath(r\"C\\foo\\bar\"),\n PurePosixPath(\"/C/foo/bar\"),\n PurePosixPath(\"./C/foo/bar\"),\n PurePosixPath(\"C/foo/bar\"),\n ],\n)\ndef test_slugify(path):\n assert _slugify(path) == \"C-foo-bar\"\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\nimport pytest\nimport scipy.sparse as sp\n\nimport scanpy as sc\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\n\nn_neighbors = 5\nkey = \"test\"\n\n\n@pytest.mark.parametrize(\"groupby\", [\"bulk_labels\", [\"bulk_labels\", \"phase\"]])\n@pytest.mark.parametrize(\"key_added\", [None, \"custom_key\"])\ndef test_dendrogram_key_added(groupby, key_added):\n adata = pbmc68k_reduced()\n sc.tl.dendrogram(adata, groupby=groupby, key_added=key_added, use_rep=\"X_pca\")\n if isinstance(groupby, list):\n dendrogram_key = f'dendrogram_{\"_\".join(groupby)}'\n else:\n dendrogram_key = f\"dendrogram_{groupby}\"\n\n if key_added is None:\n key_added = dendrogram_key\n assert key_added in adata.uns\n\n\nREP_PCA_0 = [\n *(1.50808525e00, -1.67258829e-01, -7.12063432e-01, -2.07935140e-01),\n *(-3.55730444e-01, -2.24421427e-01, -1.46907698e-02, -7.01090470e-02),\n *(-1.31467551e-01, -3.75757217e-02, -1.07698059e-02, -4.37555499e-02),\n *(1.06897885e-02, 1.10454357e-03, -5.37674241e-02, -4.94170748e-03),\n *(1.11988001e-02, -4.48330259e-03, -2.56892946e-02, -3.50749046e-02),\n *(-3.15931924e-02, 2.84416862e-02, -3.70664597e-02, -2.38820408e-02),\n *(-4.57040370e-02, 2.99325008e-02, 9.56365839e-03, -4.28026691e-02),\n *(5.36734704e-03, -3.08445804e-02, -1.16719725e-02, -2.35078149e-02),\n *(2.87542702e-04, -1.70532353e-02, -1.79676879e-02, -3.09410989e-02),\n *(-1.09178647e-02, -1.60753895e-02, -1.04769412e-02, -1.36501975e-02),\n *(-6.83976896e-03, -1.17562497e-02, -4.65345643e-02, 1.91588048e-02),\n *(-1.38043752e-02, 4.75460896e-03, -1.41307563e-02, -1.03387292e-02),\n *(-1.68043356e-02, 1.33516011e-03),\n]\nREP_PCA_1_6 = [\n *(-2.70745814e-01, -3.45929652e-01, 6.27844110e-02, -8.34012777e-02),\n *(-1.08290315e-01, -1.38125733e-01, -2.57148240e-02, -2.73127705e-02),\n *(-1.45030200e-01, -6.88858554e-02, -4.28490154e-02, -1.88931823e-02),\n *(-2.56232135e-02, -7.66322482e-03, -5.49384989e-02, -1.43514248e-02),\n *(2.42769364e-02, -3.01547404e-02, -3.37253511e-02, -3.81337740e-02),\n *(-3.42049589e-03, -4.34436463e-03, -4.15385924e-02, -2.66448390e-02),\n *(-2.74285320e-02, 1.47806173e-02, 1.19129466e-02, -6.70884028e-02),\n *(2.58150720e-03, -1.64280720e-02, -1.07431635e-02, -3.04328315e-02),\n *(-3.82748269e-03, -2.95090005e-02, -3.10521629e-02, -3.43420058e-02),\n *(-4.49432433e-03, -2.15906072e-02, -1.23507539e-02, -2.88041346e-02),\n *(-7.31994957e-03, -7.28111062e-03, -7.61008039e-02, 2.40524579e-02),\n *(-1.20806806e-02, 5.05997473e-03, -2.53410172e-02, -1.83318909e-02),\n *(-1.81263424e-02, -3.35110351e-03),\n]\nREP_PCA = np.array([REP_PCA_0, *([REP_PCA_1_6] * 6)], dtype=np.float32)\n\n\ndef test_dendrogram_cor():\n rep = sc.AnnData(\n sp.csr_matrix(\n (\n np.array([1.2762934659055623, 1.6916760106710726, 1.6916760106710726]),\n np.array([12, 5, 44]),\n np.array([0, 0, 0, 0, 0, 1, 2, 3]),\n ),\n shape=(7, 51),\n ),\n dict(leiden=pd.Categorical([\"372\", \"366\", \"357\", \"357\", \"357\", \"357\", \"357\"])),\n obsm=dict(X_pca=REP_PCA),\n )\n sc.tl.dendrogram(rep, groupby=\"leiden\")\n\n\n\"\",\"means\",\"dispersions\",\"dispersions_norm\",\"highly_variable\"\n\"HES4\",1.88077032191863,3.55366810363084,0.64023993665225,TRUE\n\"TNFRSF4\",0.862206964359899,3.53394568786647,0.312503920962044,FALSE\n\"SSU72\",2.78955473500241,3.05248251287271,-0.546511229064831,FALSE\n\"PARK7\",3.51562963004536,2.45678328702603,-0.734551702076537,FALSE\n\"RBP7\",0.779661894396393,3.59259048136138,0.848527753345566,TRUE\n\"SRM\",2.49550683986714,3.23588798534154,-0.249684360972197,FALSE\n\"MAD2L2\",2.05198283629147,3.2607335950542,-0.661689679526853,FALSE\n\"AGTRAP\",2.20008855043616,3.33014916346082,0.0255378327794182,FALSE\n\"TNFRSF1B\",2.12140619189998,3.37607011045442,0.387214229590131,FALSE\n\"EFHD2\",2.27299086606244,3.26768979892266,-0.466396227648297,FALSE\n\"NECAP2\",2.34002252354718,3.28416770240885,0.104030378053144,FALSE\n\"HP1BP3\",2.03090714091537,3.35886246940694,-0.225561862620218,FALSE\n\"C1QA\",2.26780119055147,3.93296434445563,4.77334982009271,TRUE\n\"C1QB\",1.56524398237555,3.97964719581811,3.33604014219507,TRUE\n\"HNRNPR\",2.46599869940325,3.22905980837902,-0.299710064912194,FALSE\n\"GALE\",0.676448698010205,3.42185061730017,-0.676457727295982,FALSE\n\"STMN1\",2.24648448379207,3.78470712108589,3.60566618419126,TRUE\n\"CD52\",3.9383240596554,2.65726626877817,0.254094755148372,FALSE\n\"FGR\",2.69591551909111,3.21700205644603,0.167425178928824,FALSE\n\"ATPIF1\",2.96814435737181,3.00283733638007,-0.18347721876371,FALSE\n\"SESN2\",0.205440249022099,3.48953831056964,0.441192697680527,FALSE\n\"EIF3I\",2.97101781554166,2.9000046561409,-0.6398134720155,FALSE\n\"LCK\",2.61248206065248,3.49304289851706,1.98594725205813,TRUE\n\"MARCKSL1\",2.40099871205148,3.32493702299919,0.402721225241548,FALSE\n\"SFPQ\",2.16157141818453,3.25187041411575,-0.590990742163032,FALSE\n\"PSMB2\",2.81001099723218,3.03368873008977,-0.617756413426228,FALSE\n\"MEAF6\",2.51071071704727,3.15328542182515,-0.854860736722014,FALSE\n\"NDUFS5\",3.11828489136254,2.82952861863563,-0.952562014836549,FALSE\n\"CAP1\",3.33127429016772,2.66489777846549,-0.898238151329864,FALSE\n\"SMAP2\",2.12149230757857,3.30882718006133,-0.142395509924957,FALSE\n\"C1orf228\",1.34773840883918,3.52729637972859,0.163933471425264,FALSE\n\"PRDX1\",3.07493711474154,3.00454271280564,-0.175909341639264,FALSE\n\"TMEM69\",0.958612646535407,3.32754139299571,-1.57406800748461,FALSE\n\"SCP2\",2.74799354433686,3.04717258564314,-0.566640585075036,FALSE\n\"MAGOH\",2.27062297589741,3.19748898661502,-1.01930243953784,FALSE\n\"JAK1\",2.73745653446834,3.11810685254423,-0.484083887167165,FALSE\n\"CCBL2\",1.13356732074892,3.41539203317771,-1.01249628906537,FALSE\n\"GBP2\",2.17540327076784,3.45016016075597,0.970752335551299,TRUE\n\"CD53\",3.51568067232204,2.55102239654541,-0.407812099116343,FALSE\n\"DENND2D\",1.80773774122602,3.42843268138354,-0.0804581789000627,FALSE\n\"C1orf162\",3.24942561210149,3.01582481520873,0.64252400956364,FALSE\n\"RHOC\",2.985741750694,3.32204178661091,1.23304292347939,TRUE\n\"CD2\",2.53557133944634,3.36110621201421,1.11676508734336,TRUE\n\"RP11-782C8.1\",0.370210663657041,3.37004316966741,-0.73064188045319,FALSE\n\"TXNIP\",3.2731625108326,2.98027688141431,0.486449083571333,FALSE\n\"CD160\",0.626919468581951,3.68140402798016,0.894211411042321,TRUE\n\"RP11-277L2.3\",0.664996704554698,3.44846098615156,-0.515426959080756,FALSE\n\"APH1A\",2.43813050354468,3.16948589068915,-0.73617021895599,FALSE\n\"MRPS21\",2.75295580601676,3.06523192160569,-0.498179608647018,FALSE\n\"CTSS\",3.54762757164911,3.14905736097152,1.66565514637711,FALSE\n\"MRPL9\",2.08300092575028,3.24120889728365,-0.748466011881596,FALSE\n\"S100A10\",3.81536096876538,2.44223867599515,-0.492237754855533,FALSE\n\"S100A9\",2.93029830903276,3.58932753136182,1.48860978715106,TRUE\n\"S100A8\",1.79100124352522,3.64342527255502,1.16835405436805,TRUE\n\"S100A6\",4.01980558791941,2.200924885923,-0.438355066417699,FALSE\n\"S100A4\",4.19267544737091,2.19511910191538,-0.134003732316128,FALSE\n\"RAB13\",0.690678797190794,3.36105158683678,-1.04437876728369,FALSE\n\"TPM3\",3.38289950746532,2.58912183328031,-1.2309361107997,FALSE\n\"HAX1\",2.44928220942326,3.18071193608576,-0.653924133872012,FALSE\n\"PMVK\",2.45046175895629,3.18505655637903,-0.622093867943636,FALSE\n\"PBXIP1\",1.8229903618879,3.44149071306331,-0.00460891537313223,FALSE\n\"DAP3\",2.25542389981013,3.25354017657867,-0.577839583255053,FALSE\n\"CCT3\",2.72444627797158,3.08593103645915,-0.696054084521543,FALSE\n\"SH2D2A\",1.05291375289055,3.54952016366369,0.331799604605673,FALSE\n\"MNDA\",2.50137095628744,3.33497476625665,0.476261377346501,FALSE\n\"FCER1A\",2.74486704433425,3.99108486884635,3.01162811231844,TRUE\n\"TAGLN2\",3.6933593313478,2.52503477022935,-0.342701464152289,FALSE\n\"LY9\",1.19787954406166,3.55566714133674,0.39340753736688,FALSE\n\"FCER1G\",3.61782460590848,3.42844644985511,1.28893143832626,FALSE\n\"SDHC\",2.64245219336588,3.08660335050013,-0.691624964850644,FALSE\n\"FCGR3A\",3.03212523741908,3.88389506038811,3.72635552834778,FALSE\n\"FCRLA\",1.20589759528181,3.64802336988037,1.31904556971929,TRUE\n\"CD247\",2.35082305573306,3.48621802034511,1.58432437768794,TRUE\n\"CREG1\",1.53902857465827,3.34760775908679,-0.756759826238331,FALSE\n\"RCSD1\",2.44375827328678,3.23729611150838,-0.239367917341764,FALSE\n\"XCL2\",0.540378449629801,3.80171120788367,1.35868462509435,TRUE\n\"XCL1\",0.673182330404454,3.57923209204653,0.275925154834737,FALSE\n\"PRDX6\",2.87323039456323,3.04722400641683,-0.566445654520422,FALSE\n\"C1orf21\",0.875691073768353,3.34781133709859,-1.3887971168783,FALSE\n\"TPR\",2.49690081334352,3.17458841749232,-0.698787254717891,FALSE\n\"PTPRC\",3.13800013048897,2.90841006663251,-0.602513135956354,FALSE\n\"PTPN7\",1.44125985833881,3.4445339272991,-0.129110038669163,FALSE\n\"NUCKS1\",2.48688778047874,3.27964792433752,0.0709168426377314,FALSE\n\"G0S2\",1.18912873845411,3.72012413383654,2.04167370570866,TRUE\n\"TRAF3IP3\",3.16899765019373,2.99825210787012,-0.203824894168744,FALSE\n\"NENF\",2.57797947202887,3.13285736625516,-0.386909372801589,FALSE\n\"CAPN2\",2.21017564094795,3.20940125154174,-0.92548065800644,FALSE\n\"COA6\",1.67581756008637,3.31473724112648,-0.740872828010337,FALSE\n\"ARID4B\",2.17720305391113,3.25324734391835,-0.580145952534994,FALSE\n\"RP11-156E8.1\",0.284484072581432,3.38119127267806,-1.04002606033147,FALSE\n\"SH3YL1\",1.694717034835,3.46219556579768,0.115657895877465,FALSE\n\"ID2\",3.20837502043912,3.14766745584641,1.22138551243336,FALSE\n\"HPCAL1\",1.95324651319074,3.32800286314145,-0.362715506743191,FALSE\n\"RAB10\",2.08675184028479,3.3288422298726,-0.358984992330866,FALSE\n\"OST4\",3.38848298306141,2.62297846214517,-1.082286941062,FALSE\n\"PPM1G\",2.17873901921501,3.2635955037283,-0.498643165708662,FALSE\n\"LBH\",2.06320580886345,3.4330046497431,0.103958550856389,FALSE\n\"SRSF7\",2.92332809912547,3.01009588806467,-0.707194308878235,FALSE\n\"ZFP36L2\",2.98697085195399,3.05750567641627,0.0591221709010196,FALSE\n\"ERLEC1\",1.47260668125104,3.46731442399555,0.0184060983653875,FALSE\n\"CAPG\",2.75040027041101,3.3135801149545,0.443281353486645,FALSE\n\"RNF181\",2.67271003354135,3.04683439915985,-0.953617774664238,FALSE\n\"GNLY\",2.7495375274892,4.06006663022984,3.27313045935005,TRUE\n\"CD8A\",1.4634273769048,3.59348828554246,0.835450647400119,TRUE\n\"CD8B\",1.77317700698711,3.68394018560444,1.40369015953436,TRUE\n\"MAL\",1.64318096077802,3.47144089598116,0.0451272544056004,FALSE\n\"DUSP2\",2.531132387685,3.380728830085,1.24603640758204,TRUE\n\"MGAT4A\",1.38759438748501,3.46413340182902,-0.293913731468961,FALSE\n\"LIMS1\",2.16137334569797,3.37009247185121,0.340133955053407,FALSE\n\"IL1B\",1.25491202982231,3.55771267704059,0.384410990215494,FALSE\n\"NIFK\",1.78674809979223,3.3331205081553,-0.634091247175682,FALSE\n\"GYPC\",3.26593350000607,3.06138394280262,0.842553567079129,FALSE\n\"MZT2B\",3.27153526537439,2.8848499252847,0.0674724082537133,FALSE\n\"CCDC115\",2.41118903796896,3.16293800176348,-0.784142430957143,FALSE\n\"CXCR4\",2.46374667112144,3.30203613064927,0.234940971896115,FALSE\n\"PPIG\",2.3491076588617,3.2642249511761,-0.0420774592796026,FALSE\n\"SSB\",2.61475888290781,3.14110121837938,-0.332599920102376,FALSE\n\"WIPF1\",2.00804851083515,3.33284278723216,-0.341204758336087,FALSE\n\"ITGA4\",2.18805519540149,3.31652567882021,-0.0817616276935953,FALSE\n\"HSPD1\",2.56038959284022,3.18612018177459,-0.0360206964289463,FALSE\n\"BZW1\",1.88473444093461,3.29736748706907,-0.49887257340366,FALSE\n\"NBEAL1\",3.04309527455068,2.9010704255307,-0.635083952090102,FALSE\n\"CD28\",1.17833159289949,3.55652783303207,0.40203379874517,FALSE\n\"AC079767.4\",0.75609360948958,3.68939377289349,0.94256078825022,TRUE\n\"SLC11A1\",1.37615070991924,3.47772767618663,-0.195373404572057,FALSE\n\"SP110\",2.40978638296584,3.24003539037167,-0.219298965676284,FALSE\n\"SP140\",1.5961055279512,3.3802462541091,-0.545407775031058,FALSE\n\"SP100\",2.47521852699296,3.19000451274842,-0.585843342832518,FALSE\n\"NCL\",2.64474405046805,3.14645992506896,-0.297297439768953,FALSE\n\"ARL4C\",2.42223987529934,3.30597023620197,0.263763657765005,FALSE\n\"UBE2F\",2.05581859860972,3.33834619910936,-0.316745178794656,FALSE\n\"OGG1\",0.842049416232588,3.46358161767188,-0.330636182704285,FALSE\n\"MRPS25\",1.83637521214718,3.33994598099191,-0.594444606657735,FALSE\n\"ACAA1\",2.42286250511087,3.15931449544319,-0.810689554061706,FALSE\n\"EXOG\",1.20605526350558,3.42932766021026,-0.872826809384204,FALSE\n\"EIF1B\",2.25854504434191,3.2539598165724,-0.574534471082372,FALSE\n\"UQCRC1\",2.60427554021496,3.0685228131831,-0.810737252577224,FALSE\n\"NDUFAF3\",2.83560702243022,3.06616628397382,-0.494637542637929,FALSE\n\"IMPDH2\",2.42585028766239,3.24211646139579,-0.204052283655239,FALSE\n\"GPX1\",3.6105586131361,2.84302749100406,0.604608636949215,FALSE\n\"GNAI2\",2.62659107028335,3.14492000647294,-0.307442228259969,FALSE\n\"MANF\",2.23349723834358,3.43751742582445,0.871177324249738,TRUE\n\"TEX264\",2.47180296893221,3.18819046951439,-0.599133682297661,FALSE\n\"TKT\",3.25564846508779,2.849134855879,-0.0893363347393464,FALSE\n\"ARL6IP5\",3.22468408835907,2.82309971146852,-0.203644897947614,FALSE\n\"PCNP\",2.2494031628663,3.28822351050334,-0.30467136560034,FALSE\n\"BBX\",1.92891438174297,3.35610292890043,-0.237826472651718,FALSE\n\"CD47\",2.49222840449586,3.19157160415785,-0.574362262109909,FALSE\n\"TIGIT\",0.96818939105192,3.60013416487041,0.917478360958406,TRUE\n\"COX17\",2.31104990672131,3.23389195422206,-0.264308009771308,FALSE\n\"EAF2\",2.01877998181135,3.53354084510981,0.550785559671043,TRUE\n\"GATA2\",0.189738871344609,3.61939208308945,2.21643129578065,TRUE\n\"RAB7A\",2.8987150914124,3.00905528851261,-0.711139108485027,FALSE\n\"H1FX\",2.62614265316577,3.26193175061855,0.463416308280052,FALSE\n\"SELT\",2.5748117702135,3.19186416871151,0.00181996133090909,FALSE\n\"SIAH2\",1.90641093943005,3.33636929145934,-0.325531424670991,FALSE\n\"GPR171\",0.773750066473151,3.49700202824371,-0.221684270216929,FALSE\n\"P2RY13\",1.01490152740574,3.52410523315076,0.0770790877187818,FALSE\n\"SSR3\",2.6346205193935,3.22265416230132,0.204660535770916,FALSE\n\"MFSD1\",1.62246493235478,3.38991666808116,-0.48278657182849,FALSE\n\"SEC62\",2.63945478975412,3.19986329318145,0.0545171792685254,FALSE\n\"EIF4A2\",3.30739005339261,2.84435879857266,-0.11030584579257,FALSE\n\"CCDC50\",1.72345530596264,3.4965446694018,0.315179100533326,FALSE\n\"HES1\",1.35371879587411,3.59134813206372,0.628223101558617,TRUE\n\"ATP5I\",2.67091313276188,3.06589134864023,-0.828073007387867,FALSE\n\"SPON2\",1.39974189083309,3.73302162636687,1.65516665691299,TRUE\n\"LYAR\",1.96041828509616,3.35888177318778,-0.2254760681397,FALSE\n\"CYTL1\",0.494106397368457,3.77214774939208,1.21559384247036,TRUE\n\"MRFAP1\",2.44679501225525,3.20491542523992,-0.476600583327349,FALSE\n\"BLOC1S4\",2.19704394854576,3.29186896345672,-0.275959538531454,FALSE\n\"BST1\",1.08042068875366,3.40819759541628,-1.08460236483721,FALSE\n\"FGFBP2\",1.5871240843349,3.82309830014627,2.32230068540171,TRUE\n\"MED28\",1.95596091462539,3.27795177635202,-0.585164519424375,FALSE\n\"KLF3\",1.96615774778099,3.3995331200336,-0.0448036282373467,FALSE\n\"SMIM14\",1.8733167730832,3.52480611379101,0.51196457730003,TRUE\n\"HOPX\",1.99199632234595,3.6895845704617,1.24431241125872,TRUE\n\"SPINK2\",0.64367493790273,3.75372270451607,1.33184277586563,TRUE\n\"IGFBP7\",2.12535811325291,3.46257699304813,1.06854812281712,TRUE\n\"IGJ\",2.02952034671665,4.31110894566778,4.00663971502981,TRUE\n\"CCNI\",3.27938208538782,2.81324330174249,-0.246919942495526,FALSE\n\"HNRNPDL\",3.2598845827918,2.80695519518584,-0.27452817905681,FALSE\n\"PLAC8\",3.00339922727606,3.36001913842592,1.40157341782591,FALSE\n\"HSD17B11\",2.46937497489751,3.23188418539812,-0.279017653255639,FALSE\n\"BANK1\",1.43566745072261,3.67783394119589,1.38163475670373,TRUE\n\"CCDC109B\",2.61533399141984,3.20338247650842,0.0777011128903178,FALSE\n\"SNHG8\",2.57262896000928,3.23038728345501,0.255605357583006,FALSE\n\"ANXA5\",3.30607435588407,2.7834626341202,-0.377673406874909,FALSE\n\"HMGB2\",2.85267319857206,3.35791655991469,0.611355989210591,TRUE\n\"SUB1\",3.08179304336015,3.14585089367294,0.451168019962388,FALSE\n\"IL7R\",2.22330155102622,3.59018870846112,2.0736263810646,TRUE\n\"FYB\",2.62833568552046,3.28191306290392,0.595050660531786,TRUE\n\"EMB\",1.60525338597389,3.34100009177806,-0.799548074852102,FALSE\n\"GZMK\",1.82348368517279,3.89375393761685,2.62242043667214,TRUE\n\"GZMA\",2.43305136877294,3.78614829441993,3.7817224790948,TRUE\n\"NSA2\",2.29678066911439,3.24559647737468,-0.640404680308627,FALSE\n\"COX7C\",3.59614114322243,2.22843708096564,-1.52625855378757,FALSE\n\"GLRX\",3.02324468069108,3.01487027279973,-0.130079163703886,FALSE\n\"CAMK4\",1.33153829226669,3.46161160457309,-0.312193392407886,FALSE\n\"EPB41L4A-AS1\",2.07925483366828,3.35132896289923,-0.259044074339529,FALSE\n\"REEP5\",2.93993627952589,3.00513323554622,-0.726007184268013,FALSE\n\"SNX2\",2.61398752309147,3.20516412861831,0.0894384111254829,FALSE\n\"IRF1\",2.08293528255313,3.36868185297734,-0.181920209251845,FALSE\n\"SKP1\",3.17378605057298,2.86758922239192,-0.783662076857227,FALSE\n\"TXNDC15\",1.58440901561121,3.39225463635209,-0.467646952511503,FALSE\n\"H2AFY\",3.13961463675365,2.94172091620841,-0.45469098201903,FALSE\n\"EGR1\",0.755854624853946,3.46616246971674,-0.408307681141864,FALSE\n\"MZB1\",1.89502940314269,4.19894567224949,3.50813686461313,TRUE\n\"NDUFA2\",2.85485509122861,3.03381640323039,-0.617272418450524,FALSE\n\"NDFIP1\",2.7535666656136,3.06568029273253,-0.496479882521461,FALSE\n\"CD74\",4.39880166182438,2.60402299400562,0.707106781186547,FALSE\n\"NUDCD2\",1.98067806054445,3.36181368053905,-0.212445384144632,FALSE\n\"NPM1\",3.90178321099788,2.20586010931427,-0.430866386484064,FALSE\n\"ATP6V0E1\",3.34379860542157,2.69986516791462,-0.744712133419028,FALSE\n\"HIGD2A\",3.62934687928222,2.3627226327381,-0.635850014226434,FALSE\n\"LMAN2\",2.91974587368354,3.02706685494151,-0.642859221158197,FALSE\n\"F12\",0.240142663431215,3.35755263308639,-1.3631912991643,FALSE\n\"PRR7\",1.25417390301911,3.44610729039542,-0.424578956152598,FALSE\n\"NHP2\",2.87524692361751,3.04847637234876,-0.561698071623503,FALSE\n\"SQSTM1\",2.36043469270379,3.19265611498893,-0.566416741943501,FALSE\n\"RNF130\",2.59071798702997,3.27293445728469,0.535900744954109,TRUE\n\"SERPINB1\",2.77391025775678,3.15138730332642,-0.171573936690829,FALSE\n\"SERPINB6\",2.18562732509154,3.2683987789217,-0.460812254576051,FALSE\n\"LY86\",2.81317057210154,3.25245517733428,0.211563369931743,FALSE\n\"ADTRP\",0.676051990032346,3.52107861259993,-0.0759865199703292,FALSE\n\"DEK\",2.76343860238718,3.10724137875725,-0.338926330355305,FALSE\n\"SOX4\",1.10798430666125,3.55882381561274,0.42504522777171,FALSE\n\"FAM65B\",2.43119761902019,3.28395555473758,0.102476107176322,FALSE\n\"HIST1H4C\",2.16308906306601,3.39128869694704,0.507076817092301,TRUE\n\"HIST1H1E\",2.46011240294088,3.23831352043661,-0.231914010082303,FALSE\n\"ZNRD1\",1.95954047684283,3.3140703241945,-0.424637829148126,FALSE\n\"LTB\",3.79717633154273,3.36188072620072,1.1687084798,FALSE\n\"LST1\",3.63853174752907,3.49409660575534,1.40750080911858,FALSE\n\"AIF1\",3.65665169135743,3.53135765627296,1.47479721160771,FALSE\n\"DDAH2\",2.11340429878934,3.32865209691498,0.0137468382652377,FALSE\n\"CLIC1\",3.92330153224379,1.73580916802688,-1.14411901412209,FALSE\n\"C6orf48\",3.26808655348311,2.89201107681185,0.0989137910660921,FALSE\n\"HLA-DRA\",4.14245186724542,3.33325849456824,1.12153477577529,FALSE\n\"HLA-DRB5\",3.91408203666622,3.00981880472529,0.789055970109514,FALSE\n\"HLA-DRB1\",4.12317921160857,3.05565265897838,0.815293889198523,FALSE\n\"HLA-DQA1\",3.68334996316207,3.40805018890775,1.25209417885776,FALSE\n\"HLA-DQB1\",3.54240798739379,3.27740161855204,2.11064186131899,FALSE\n\"HLA-DQA2\",3.49808501697905,3.28641585974323,2.14189544173609,FALSE\n\"HLA-DMB\",3.00743880994498,3.29519571695649,1.11390925106961,FALSE\n\"HLA-DMA\",3.31313113840681,3.23950601917576,1.62460717526643,FALSE\n\"HLA-DPA1\",4.04364135819571,3.18712882360912,1.05810518534791,FALSE\n\"HLA-DPB1\",4.03173419935203,3.2760181401218,1.19298532918502,FALSE\n\"HSD17B8\",1.41476751550311,3.43002959578333,-0.541120762000296,FALSE\n\"TAPBP\",2.35771334653687,3.26760910155185,-0.0172839447419166,FALSE\n\"DAXX\",1.78971204570193,3.29640775234363,-0.847342030957476,FALSE\n\"CUTA\",3.23620771231059,2.92721041573804,0.253458194396349,FALSE\n\"HMGA1\",2.68823776570973,3.19839966219859,0.0448749639018696,FALSE\n\"CCDC167\",2.2417364768391,3.34809811149758,0.166904927264235,FALSE\n\"CNPY3\",3.01119556212996,2.96333107129326,-0.358792507130755,FALSE\n\"MEA1\",2.33090504699858,3.19456717900589,-0.552415592991462,FALSE\n\"HSP90AB1\",3.2733330113404,2.804935751696,-0.283394643562035,FALSE\n\"ELOVL5\",1.94096442998644,3.33080521468527,-0.350260625658804,FALSE\n\"UBE2J1\",2.28703739021791,3.41116421484514,0.663617702434484,TRUE\n\"PNISR\",2.84910153982415,3.04456680162047,-0.576518828600864,FALSE\n\"RP3-467N11.1\",0.228668589920236,3.41011296999176,-0.644635827167253,FALSE\n\"CD164\",2.632249719051,3.13283307285002,-0.387069414674882,FALSE\n\"GTF3C6\",2.50336540233086,3.21077613822447,-0.433662871804668,FALSE\n\"RWDD1\",2.71267840038055,3.10855368493143,-0.547018944319712,FALSE\n\"GOPC\",1.29738004317114,3.37600577515729,-0.932721284740385,FALSE\n\"SAMD3\",1.45178351218028,3.45699836876203,-0.0483959840370171,FALSE\n\"DYNLT1\",1.97139648287987,3.32396100564693,-0.380679296677205,FALSE\n\"SOD2\",2.30358639457007,3.33924632341811,0.0971876625405603,FALSE\n\"TCP1\",2.27049072699976,3.25288183558515,-0.583024720191396,FALSE\n\"MRPL18\",2.41433158620565,3.16931059842845,-0.737454473711627,FALSE\n\"RNASET2\",3.55413776411645,2.52836778496456,-0.486358668909856,FALSE\n\"MAD1L1\",1.93519971071393,3.34319018071071,-0.295216397028156,FALSE\n\"CHST12\",1.87351132993537,3.31104759754241,-0.438072154004845,FALSE\n\"EIF2AK1\",1.56342739361332,3.35462386364832,-0.711326723714702,FALSE\n\"NDUFA4\",3.5313509543955,2.4196384731799,-0.863337742076391,FALSE\n\"TOMM7\",3.47234138295704,2.5953242492034,-0.254211647863172,FALSE\n\"CPVL\",2.90014579363954,3.5320455353688,1.27145997690549,TRUE\n\"LSM5\",1.94349075302528,3.30911772969625,-0.44664933433243,FALSE\n\"SEPT7\",2.96719887949065,3.057044987757,0.0570777922443779,FALSE\n\"AOAH\",1.67883149723539,3.36460215023991,-0.451226063103958,FALSE\n\"STK17A\",2.28547537054658,3.406941310819,0.630357832761845,TRUE\n\"COA1\",1.85962174058007,3.29801434064504,-0.838009955309245,FALSE\n\"PPIA\",3.84072050060515,1.97395756653606,-0.782753987303413,FALSE\n\"SNHG15\",1.98659188274985,3.30186164622693,-0.478898556221889,FALSE\n\"UPP1\",1.70251680017246,3.44110647375852,-0.00684081900160539,FALSE\n\"CCT6A\",2.46699995045641,3.17710426010359,-0.68035527817101,FALSE\n\"SBDS\",1.86319408097774,3.33261067921621,-0.637052654415536,FALSE\n\"WBSCR22\",2.3493036223267,3.19241126155589,-0.568210627107461,FALSE\n\"EIF4H\",2.4691986216715,3.15843222700747,-0.817153372999493,FALSE\n\"LAT2\",2.09148787357433,3.27395693790332,-0.417035829110541,FALSE\n\"NCF1\",2.05389185978319,3.33473810706238,-0.332781124565232,FALSE\n\"HSPB1\",2.49467999604256,3.25969132982628,-0.0752924155800499,FALSE\n\"RSBN1L-AS1\",0.303696615126449,3.45377698244236,-0.0477025325384536,FALSE\n\"CDK6\",0.74369361209756,3.55655353954441,0.138687495641845,FALSE\n\"CCDC132\",0.348488159454098,3.32205965300362,-0.962888014795907,FALSE\n\"BRI3\",2.44602987057203,3.32008159723281,0.367148612928619,FALSE\n\"PDAP1\",2.40859502009045,3.21403510533497,-0.409786495326051,FALSE\n\"ATP5J2\",3.21330592099154,2.71228684873763,-0.690174141805705,FALSE\n\"MCM7\",1.19119917917552,3.45599025313887,-0.60560162657811,FALSE\n\"LAMTOR4\",3.51216970285824,2.44221792511759,-0.785051760822904,FALSE\n\"PILRA\",2.56765585415311,3.49057588404772,1.96969487348407,TRUE\n\"ZNHIT1\",2.97766060424037,2.90776608029641,-0.605370926991997,FALSE\n\"FAM185A\",0.26161221766767,3.50499179216143,0.652458157777893,TRUE\n\"RINT1\",0.120314869557555,3.46943002356181,0.166291117654948,FALSE\n\"SYPL1\",1.85032922833223,3.40973410094924,-0.189071298374853,FALSE\n\"RP11-390E23.6\",0.386946146446636,3.37178116836425,-0.722229752562229,FALSE\n\"CPA5\",0.80794929426823,3.47986840813046,-0.181772024122253,FALSE\n\"MTPN\",2.22318096874595,3.26281401808078,-0.504798198033965,FALSE\n\"C7orf55\",1.5484530032343,3.34630014841228,-0.765227318322415,FALSE\n\"NDUFB2\",3.16518984246629,2.77285027462809,-1.20408111445315,FALSE\n\"MRPS33\",2.01485208304681,3.35999162428856,-0.220543402389014,FALSE\n\"GSTK1\",3.4814505781444,2.60137286692168,-0.233240280860595,FALSE\n\"ZYX\",2.36189161198549,3.22648582519528,-0.318568000405514,FALSE\n\"GIMAP7\",3.33751674898341,3.1838687105225,1.38032887541682,FALSE\n\"TMEM176B\",2.51113532566192,3.62661746163109,2.61294166772995,TRUE\n\"TMEM176A\",1.51193913471295,3.51183285909641,0.306687239045162,FALSE\n\"OFD1\",1.73282506417802,3.40124525700978,-0.238379844777042,FALSE\n\"NDUFB11\",3.33687528578276,2.71504562397523,-0.678061605433606,FALSE\n\"TIMP1\",3.31636498152831,3.13320094586693,1.15786959931528,FALSE\n\"CFP\",3.10993350047188,3.35593348697256,1.38344269448016,FALSE\n\"EBP\",2.00817689496213,3.32142831124919,-0.391935702968046,FALSE\n\"WDR13\",1.42412700270031,3.36050865719832,-1.04505468551383,FALSE\n\"PQBP1\",2.44420339226037,3.22064512426307,-0.361359096365817,FALSE\n\"PLP2\",2.87666000690641,3.17018239563911,-0.100323788044715,FALSE\n\"MSN\",2.57944941850045,3.16219705958519,-0.193623192614587,FALSE\n\"IGBP1\",2.82613831967294,3.07243996825482,-0.470854688574563,FALSE\n\"IL2RG\",3.15102075797078,3.02435185591424,-0.0880031417300249,FALSE\n\"COX7B\",2.23638412234519,3.27919865592683,-0.375751713430052,FALSE\n\"ITM2A\",2.03683717403862,3.55312675230888,0.637833933609996,TRUE\n\"TCEAL8\",1.76493938389587,3.31923539215111,-0.714744736823374,FALSE\n\"NGFRAP1\",1.42625360351265,3.53052896191991,0.187365373524195,FALSE\n\"RPL39\",2.4746451537889,3.28385381743747,0.101730742771905,FALSE\n\"NDUFA1\",3.08277499344226,2.82728056744339,-0.962538096912818,FALSE\n\"CD40LG\",1.13908642258007,3.62261294522185,1.06437021264756,TRUE\n\"SSR4\",3.43267775977527,2.82707067190924,0.549284209944994,FALSE\n\"MPP1\",1.43587248580675,3.35865952559766,-0.685193612173036,FALSE\n\"BLK\",1.40709315652298,3.66154429360133,1.13705148936854,TRUE\n\"BIN3\",1.47925428927643,3.41635553315693,-0.311580504835096,FALSE\n\"PNOC\",1.32209115379711,3.61891732479677,0.828062915934563,TRUE\n\"RP11-489E7.4\",0.839191116734345,3.39447833977714,-0.962252420736102,FALSE\n\"LEPROTL1\",2.190983710905,3.3619002313788,0.275611330633594,FALSE\n\"GTF2E2\",1.58934064713754,3.30980778860192,-1.00153524532824,FALSE\n\"PPAPDC1B\",1.52794987271751,3.54282262012451,0.507362838677888,TRUE\n\"GOLGA7\",2.12054388683268,3.27405241684299,-0.416283830706009,FALSE\n\"CHCHD7\",1.78939801782059,3.32479413278222,-0.682456073926386,FALSE\n\"TRAM1\",2.45675474953881,3.2390988589173,-0.226160335193182,FALSE\n\"TPD52\",1.72478331813181,3.4417388773682,-0.00316742095644467,FALSE\n\"MTDH\",3.05280739350945,2.90419025102596,-0.621239233913001,FALSE\n\"ZNF706\",2.97642929127324,3.00867737168276,-0.157561140113553,FALSE\n\"EIF3E\",2.80476202747249,3.23224722511165,0.134957223191815,FALSE\n\"LYPD2\",1.48400536912563,3.93487286137792,3.04610191376563,TRUE\n\"LY6E\",3.52876106620447,2.69712548928742,0.0987468736630405,FALSE\n\"COMMD5\",1.80082344851781,3.29410196863405,-0.860735473438945,FALSE\n\"PLGRKT\",1.89151195740484,3.35462448195361,-0.24439734023361,FALSE\n\"PTPLAD2\",1.97973182028553,3.3047987040352,-0.465844981333787,FALSE\n\"SIT1\",2.30867132505524,3.43445841645396,1.20511472414158,TRUE\n\"CCDC107\",2.38671189360534,3.25136998870585,-0.136257582352883,FALSE\n\"TLN1\",2.25112946854537,3.28757012178372,-0.309817498101959,FALSE\n\"DCAF10\",0.481516391549768,3.36824458252357,-0.739347263894807,FALSE\n\"ANXA1\",3.25038837946519,3.06685872744265,0.866590874351591,FALSE\n\"OSTF1\",2.91874199622537,3.00811793836013,-0.714692500859333,FALSE\n\"HNRNPK\",3.21429230631956,2.78646598228775,-0.364487061201765,FALSE\n\"HIATL1\",0.41347302038409,3.42662282947277,-0.456789353880501,FALSE\n\"ANP32B\",3.17699292897818,2.90427022125329,-0.620884353403957,FALSE\n\"TXN\",2.74419136757791,3.11334414322823,-0.315791414682894,FALSE\n\"ATP6V1G1\",3.3624065422185,2.67533139326953,-0.852428859915367,FALSE\n\"NDUFA8\",2.29102036642718,3.24193855002767,-0.669214756521754,FALSE\n\"ARPC5L\",2.47398015958645,3.25774525032343,-0.0895501006968455,FALSE\n\"C9orf78\",2.69694232632053,3.1099048951144,-0.538117342930835,FALSE\n\"NUP214\",1.52019913561372,3.39592830859287,-0.443857921705363,FALSE\n\"FCN1\",3.00566712020149,3.66796422708078,2.76812836550596,FALSE\n\"EGFL7\",0.399810426341325,3.69789824376537,0.856217084416377,TRUE\n\"SNHG7\",2.87596116491333,3.1531742059832,-0.164799983267586,FALSE\n\"PHPT1\",2.49749987471086,3.16465937202615,-0.771531047332516,FALSE\n\"C9orf142\",2.66017680966191,3.25386481848245,0.410272382033987,FALSE\n\"CLIC3\",1.8141475170406,3.75786503540309,1.83309219738145,TRUE\n\"KLF6\",2.96287747966498,3.05535843198089,0.0495934347232158,FALSE\n\"AKR1C3\",0.874161648288546,3.68549922435558,1.69773017207376,TRUE\n\"RBM17\",2.1937098366361,3.2222877835298,-0.82398548506327,FALSE\n\"PRKCQ-AS1\",1.71687725243083,3.47143905449621,0.169349893829095,FALSE\n\"GATA3\",1.23423492994735,3.49010705475675,-0.105639449011703,FALSE\n\"VIM\",4.06739551743169,1.9219411241981,-0.435360027336482,FALSE\n\"NSUN6\",0.20596103820399,3.47281238673302,0.212531604456449,FALSE\n\"DNAJC1\",1.96217309942806,3.32776983822773,-0.36375117180688,FALSE\n\"COMMD3\",2.22770289465426,3.28622758770585,-0.320391384741297,FALSE\n\"APBB1IP\",2.48652270934974,3.1514430844842,-0.868358369088895,FALSE\n\"ABI1\",1.9545422921601,3.35519898919245,-0.241843977734274,FALSE\n\"EPC1\",1.98110160133363,3.32679869892316,-0.368067341411495,FALSE\n\"HNRNPF\",2.9008348399592,3.00853895646476,-0.713096467062922,FALSE\n\"ZNF22\",1.77069596051533,3.30773405385662,-0.781551745766572,FALSE\n\"SRGN\",3.72197863086736,2.5051152677656,-0.378677660114587,FALSE\n\"PPA1\",3.07620328737387,3.00059713651996,-0.193418459314089,FALSE\n\"PRF1\",1.8335541021299,3.69343093664082,1.45881841271069,TRUE\n\"C10orf54\",3.10529102324294,3.11671626265077,0.321878495177014,FALSE\n\"PSAP\",3.46133598638492,3.14536293573533,1.65284607972742,FALSE\n\"ANAPC16\",3.10599776502874,2.90271739964013,-0.627775244463765,FALSE\n\"RPS24\",4.21077942971783,1.07699270335599,-1.36746490532121,FALSE\n\"ANXA11\",2.59306967007297,3.17767650974705,-0.0916465373194863,FALSE\n\"HHEX\",2.09034326754452,3.32248723087241,-0.0348080518899119,FALSE\n\"PDLIM1\",2.14187005036965,3.44068961198054,0.896161670916329,TRUE\n\"PGAM1\",2.48769771347841,3.10654000418404,-1.19733464100159,FALSE\n\"NPM3\",1.82637503444921,3.29752206894404,-0.840869379064064,FALSE\n\"ADD3\",1.81659996207802,3.36901610265621,-0.425587050408244,FALSE\n\"C10orf118\",1.55583445100185,3.44113631092627,-0.151111457212627,FALSE\n\"RGS10\",3.11344058999419,2.98653088838909,-0.255839656101156,FALSE\n\"TIAL1\",1.74825858844177,3.30627166797725,-0.790046203021887,FALSE\n\"FAM175B\",0.637380484828048,3.44589579613699,-0.530950025332543,FALSE\n\"ZNF511\",2.08386519741503,3.29430495572696,-0.512483807785518,FALSE\n\"FUOM\",1.88410913859839,3.30974848647591,-0.443845974168261,FALSE\n\"PSMD13\",2.35688939270209,3.19424541014928,-0.554772988478762,FALSE\n\"IFITM2\",3.68160552077843,2.79230472645207,0.140009202813686,FALSE\n\"IFITM1\",2.43556259778112,3.45732566196546,1.37264846853062,TRUE\n\"IFITM3\",2.77828655358999,3.61091668782621,1.57045192879319,TRUE\n\"RNH1\",3.26444325814092,2.91788175144473,0.212500241745586,FALSE\n\"IRF7\",2.02560475814499,3.30093399139887,-0.483021461714036,FALSE\n\"TALDO1\",3.2495587119335,2.8413084720404,-0.12369845225257,FALSE\n\"CTSD\",2.52001367158521,3.27471482545831,0.0347751688948523,FALSE\n\"CARS\",1.41363644116194,3.47795422679604,-0.193731215329537,FALSE\n\"ILK\",2.01411380987544,3.2684345129826,-0.627463417925022,FALSE\n\"NUCB2\",2.19571108474883,3.46016863191925,1.04957971226943,TRUE\n\"LDHA\",3.39712417337079,2.63270732937401,-1.03957187802874,FALSE\n\"CAT\",2.68053409953225,3.1405792354425,-0.33603867751737,FALSE\n\"CD82\",1.61837904247347,3.52631429390243,0.400462425327005,FALSE\n\"SPI1\",3.36083675348535,3.18607058054864,1.38999629247011,FALSE\n\"PSMC3\",2.37639006864026,3.17648331331814,-0.684904559805325,FALSE\n\"MTCH2\",2.22067237147074,3.20646806105152,-0.948582659099955,FALSE\n\"TIMM10\",2.22482618831079,3.19240079283312,-1.0593773882006,FALSE\n\"LPXN\",1.76265157982474,3.38799642004087,-0.3153374254694,FALSE\n\"MS4A1\",1.82431674142266,3.89874243045918,2.65139674163744,TRUE\n\"CYB561A3\",1.86270639868611,3.47322837237393,0.17974337782185,FALSE\n\"POLR2G\",2.95246554993266,2.97156414890618,-0.853263936893935,FALSE\n\"SLC3A2\",2.11634154429574,3.27708068132422,-0.392433020610266,FALSE\n\"OTUB1\",2.52290522943674,3.15701231479031,-0.82755616551445,FALSE\n\"BAD\",1.94136074781356,3.29094728778422,-0.527406758847808,FALSE\n\"CAPN1\",1.87628679434777,3.35167320661519,-0.257514104070181,FALSE\n\"NEAT1\",2.74255371475709,3.16384281381358,-0.182781172394637,FALSE\n\"KAT5\",1.2824138523676,3.36696528440057,-0.998252764467951,FALSE\n\"CTSW\",2.50184992092505,3.6280099264227,2.6231433704384,TRUE\n\"SF3B2\",2.72419388741045,3.09907065820968,-0.609491922174002,FALSE\n\"ADRBK1\",1.90484899588517,3.33468077078783,-0.333035952152058,FALSE\n\"POLD4\",3.28883662808768,2.78210585353592,-0.383630417780375,FALSE\n\"TBC1D10C\",2.93736587657673,3.1334831260796,-0.239446723320405,FALSE\n\"PTPRCAP\",3.54018037063507,3.33524683974809,2.31119898354919,FALSE\n\"CORO1B\",2.83313843837161,3.19031629023234,-0.023998385714872,FALSE\n\"GSTP1\",3.77041958250188,2.71995033635047,0.00933145512566443,FALSE\n\"UNC93B1\",2.28536280184206,3.27465212032249,-0.411560526684235,FALSE\n\"NDUFS8\",2.88862913685384,3.01809661674621,-0.676864417523986,FALSE\n\"MRPL21\",2.1553790638201,3.25544960469411,-0.562800801915119,FALSE\n\"LAMTOR1\",3.18383343358424,2.81331710139359,-0.246595921551035,FALSE\n\"MRPL48\",2.00051481315874,3.29806612811421,-0.495767505748207,FALSE\n\"SPCS2\",2.83155534622748,3.15482445635242,-0.158544063788521,FALSE\n\"TMEM126B\",1.76592133461715,3.32021335578926,-0.709064108721915,FALSE\n\"CTSC\",3.02764656960464,3.04078979991481,-0.0150571700901549,FALSE\n\"CWC15\",2.38279312796731,3.22725438234769,-0.31293727162123,FALSE\n\"FDX1\",2.21824013630881,3.28086232098308,-0.362648578097767,FALSE\n\"POU2AF1\",1.2522944791464,3.49211481385886,-0.0910858779238025,FALSE\n\"IL18\",1.03335103697836,3.42054386391006,-0.960862192076992,FALSE\n\"AMICA1\",2.90919616463609,3.26972963656569,0.277048964125279,FALSE\n\"CD3E\",2.92322571312313,3.45878855490342,0.993750739797926,TRUE\n\"CD3D\",3.09327082029195,3.59156091384064,2.42907660084387,FALSE\n\"CD3G\",2.2518104436951,3.5321563210233,1.61655948325521,TRUE\n\"FLI1\",1.80565531723311,3.3051818761287,-0.796376399726668,FALSE\n\"NINJ2\",1.5609515143884,3.46499337118086,0.00337601585060111,FALSE\n\"CD27\",2.90048060223493,3.52186727817359,1.23287531203383,TRUE\n\"CHD4\",1.3426981733221,3.36509561660317,-1.01180535808916,FALSE\n\"MLF2\",2.58958487797847,3.09999260264193,-0.603418269131707,FALSE\n\"LAG3\",0.892093375483209,3.53825618418845,0.351902623221687,FALSE\n\"CD4\",1.94017466695926,3.2872142166916,-0.543998166389395,FALSE\n\"KLRG1\",1.50777997640318,3.55248491474149,0.569931464629263,TRUE\n\"KLRB1\",1.1924799123468,3.60226689776107,0.860452451661331,TRUE\n\"KLRC2\",0.377902755908066,3.61136368610661,0.437379160465871,FALSE\n\"KLRC1\",0.599887622791987,3.6309163749101,0.588688943290777,TRUE\n\"RP11-291B21.2\",0.978938283551164,3.52872409157692,0.26477760311724,FALSE\n\"PRR4\",0.904315732416752,3.430788953806,-0.630366956945641,FALSE\n\"H2AFJ\",2.38184034537358,3.28237801066042,0.0909184464142142,FALSE\n\"WBP11\",1.3632085384555,3.40002055106161,-0.758646243607779,FALSE\n\"LDHB\",3.71062938452355,2.76529358903266,0.0912249535718192,FALSE\n\"FGFR1OP2\",1.76035231530845,3.40513812546283,-0.215767615582579,FALSE\n\"FKBP11\",2.135253356219,3.65349385725856,2.57222189391006,TRUE\n\"TMBIM6\",3.3237217096618,2.81448072395967,-0.241486980285002,FALSE\n\"COX14\",2.52408398744522,3.13571565361898,-0.983583238701814,FALSE\n\"HNRNPA1\",3.98648113617842,1.7958536858051,-1.05300780399819,FALSE\n\"NFE2\",0.8730624102893,3.48102103731114,-0.171236788614978,FALSE\n\"CD63\",3.11515145536368,3.07268154888682,0.12646750087714,FALSE\n\"WIBG\",2.0294320307228,3.33272357388905,-0.341734594792701,FALSE\n\"CNPY2\",2.57769129295769,3.14878839713026,-0.281957761015772,FALSE\n\"ATP5B\",3.38007438412281,2.66905319984354,-0.87999357239605,FALSE\n\"RP11-620J15.3\",1.01814228650138,3.51284571233393,-0.0357691831393403,FALSE\n\"TMBIM4-1\",2.68455938124627,3.13435538068185,-0.377040643665837,FALSE\n\"LYZ\",3.75075224876102,3.78102784301741,1.9257213037436,FALSE\n\"ATXN7L3B\",1.17805656257772,3.41012741810466,-1.0652607643385,FALSE\n\"NAP1L1\",3.55567504337354,2.61347062537124,-0.19129573349932,FALSE\n\"OSBPL8\",2.46106481956778,3.20877395865184,-0.448331566404239,FALSE\n\"BTG1\",3.77566666663771,2.63181890853916,-0.149840869742786,FALSE\n\"ISCU\",2.99603328783625,3.0106828226902,-0.148661634657808,FALSE\n\"HVCN1\",1.95519124092409,3.46154263802075,0.230793904961229,FALSE\n\"PPP1CC\",2.59884593528146,3.160535098199,-0.204571983535569,FALSE\n\"MAPKAPK5-AS1\",1.68065698646624,3.3484481281919,-0.545058786548605,FALSE\n\"ERP29\",3.60119967242132,2.3653440623085,-1.05158372977254,FALSE\n\"OAS1\",2.41730767894551,3.38488413031527,0.841915502003081,TRUE\n\"COX6A1\",3.50438682070113,2.38237205057007,-0.992545415932638,FALSE\n\"TRIAP1\",1.58815147511883,3.38383843261868,-0.522146460568415,FALSE\n\"POP5\",2.01722334743994,3.29899166797298,-0.491654000108527,FALSE\n\"ACADS\",0.963008418267332,3.40491964251251,-0.866817058553811,FALSE\n\"RNF34\",1.15273322429456,3.46085051376541,-0.556889783814366,FALSE\n\"AC084018.1\",0.963804155489779,3.42214197170866,-0.709401909686537,FALSE\n\"MPHOSPH9\",0.83013817581624,3.4636988298338,-0.329564842718418,FALSE\n\"SAP18\",3.29207895163967,2.73110560471937,-0.607549481703003,FALSE\n\"POMP\",2.55771840396182,3.10827532628545,-0.548852735791161,FALSE\n\"ALOX5AP\",2.25336594515534,3.47230897984495,1.1451978904923,TRUE\n\"WBP4\",1.18779191823964,3.40446825046509,-1.12197963301957,FALSE\n\"TSC22D1\",0.510620462965645,3.52372010161378,0.0131734099573617,FALSE\n\"ESD\",1.87835099052462,3.31057927051557,-0.440153605006518,FALSE\n\"EBPL\",1.83724170186105,3.34633700171372,-0.557321537360457,FALSE\n\"UCHL3\",1.70565827196434,3.27625488864867,-0.964402542580352,FALSE\n\"TNFSF13B\",1.82566382890225,3.35420686661042,-0.511608410672598,FALSE\n\"APEX1\",2.76202388943155,3.1294290812593,-0.254815165850972,FALSE\n\"DHRS4L2\",2.22120701005766,3.20754177788256,-0.940125994765445,FALSE\n\"PSME2\",3.65622261394295,2.32823873494105,-0.698130659385556,FALSE\n\"NEDD8\",3.40027342167025,2.6073602953745,-0.212481065229147,FALSE\n\"TINF2\",1.7595825909509,3.3571167498208,-0.494705978218076,FALSE\n\"GZMH\",1.87399710375762,3.89137561842566,2.14116045726274,TRUE\n\"GZMB\",2.0486734718415,3.98502854304607,2.55739518767246,TRUE\n\"NFKBIA\",3.09724085002722,3.05115538005322,0.030941728238131,FALSE\n\"PNN\",2.17417094634476,3.28000583093699,-0.369394349996051,FALSE\n\"RN7SL1\",0.310033909671626,3.46443195456467,0.0979622210236809,FALSE\n\"RPL36AL\",3.81477657430148,2.08048952933797,-1.14558530773851,FALSE\n\"ARF6\",2.89712400777323,3.04212544263297,-0.585773754720087,FALSE\n\"LGALS3\",2.95714605386322,3.18277617655643,-0.0525821357321126,FALSE\n\"DAAM1\",0.57574442907445,3.3389868067356,-1.1779022261292,FALSE\n\"DHRS7\",2.63315560935975,3.12874487999587,-0.414001910926923,FALSE\n\"ERH\",2.94360458991256,2.98295189076515,-0.810094247344307,FALSE\n\"COX16\",2.37305761054682,3.21971122305615,-0.368201195734782,FALSE\n\"FOS\",3.5425532392012,2.83233297420364,0.567529316155106,FALSE\n\"AHSA1\",2.11580830099645,3.27233952829737,-0.429774653475961,FALSE\n\"CALM1\",3.6435137992878,2.4942334037929,-0.398331166592955,FALSE\n\"EVL\",3.10536424801186,3.0527650495379,0.0380848907026398,FALSE\n\"PPP2R5C\",2.16753490785345,3.36382049675213,0.290735466929016,FALSE\n\"PLD4\",2.13199666321673,3.33876460147225,0.0933935888323726,FALSE\n\"CRIP1\",3.28862271314085,2.88640782945323,0.0743124620023625,FALSE\n\"AL928768.3\",1.7146392005646,4.15897724941942,4.16300429829345,TRUE\n\"KIAA0125\",1.01945656848426,3.66270174888895,1.46615926283861,TRUE\n\"NDNL2\",2.21232432750462,3.27493874593466,-0.40930304452347,FALSE\n\"EMC7\",2.3954251974654,3.19457782918572,-0.552337565906698,FALSE\n\"NOP10\",2.80317292512519,3.08273225448191,-0.431837752057912,FALSE\n\"SRP14\",3.78208186613075,2.05364490308828,-1.19406882464621,FALSE\n\"GCHFR\",1.85443389356539,3.403698874406,-0.224127691233205,FALSE\n\"ZNF106\",1.57011312783586,3.35824131646188,-0.687901744353156,FALSE\n\"CEP152\",0.384044007868824,3.339003760918,-0.880876444227342,FALSE\n\"EID1\",3.27396963601886,2.72379885596588,-0.639630116193344,FALSE\n\"MYO5A\",0.387572249547068,3.33901136594997,-0.880839634935318,FALSE\n\"SLTM\",2.3921926680529,3.22437706851683,-0.334017517583643,FALSE\n\"RPS27L\",2.49398843531957,3.20787309734392,-0.454931603475441,FALSE\n\"PPIB\",3.42580923411985,2.84692516143766,0.61812238189483,FALSE\n\"SPG21\",2.21463548061152,3.23828286509114,-0.698007171362482,FALSE\n\"PKM\",3.32942161679077,2.71947340695577,-0.658621209685313,FALSE\n\"CSK\",2.48614248640561,3.16790612038109,-0.747744189894056,FALSE\n\"SCAMP2\",2.34337481252477,3.22258586920363,-0.347140494189483,FALSE\n\"IMP3\",2.67094856594968,3.11208595458494,-0.523748799637702,FALSE\n\"ETFA\",2.382941320182,3.23501720275826,-0.256064030384178,FALSE\n\"CTSH\",2.83992491495104,3.19157178082816,-0.0192389575573612,FALSE\n\"IL16\",2.32376187865812,3.28175962625499,0.0863879377064578,FALSE\n\"ISG20\",3.05616653278971,3.33691307040955,1.29903659313704,FALSE\n\"ZNF710\",0.302510697367751,3.33310817634192,-1.69737291191455,FALSE\n\"IDH2\",2.72799970062989,3.13704308313857,-0.359334400608181,FALSE\n\"NGRN\",0.293903724983969,3.43376727742207,-0.321256392386435,FALSE\n\"SLCO3A1\",0.493085615604123,3.35090980631333,-0.823249717162938,FALSE\n\"VIMP\",2.19695313202799,3.39113008491449,0.505827578298676,TRUE\n\"POLR3K\",1.79200575384459,3.35693182884598,-0.495780115583827,FALSE\n\"NME4\",1.53699849852633,3.37119182644184,-0.604040131047366,FALSE\n\"C16orf13\",2.65421059498564,3.11214437158573,-0.523363955842171,FALSE\n\"STUB1\",2.6255955188528,3.12871673890011,-0.414187300898267,FALSE\n\"NDUFB10\",3.1714067608239,2.77984017918255,-1.17306230946285,FALSE\n\"SRRM2\",2.3237448190312,3.25887370066073,-0.0812826637521603,FALSE\n\"TCEB2\",3.35644682211781,2.64219840729023,-0.99790084049594,FALSE\n\"IL32\",3.18508097437666,3.81022388494304,4.13037162214629,FALSE\n\"CORO7\",1.49121264588086,3.4097277909081,-0.354498749631726,FALSE\n\"HMOX2\",2.43012488739796,3.20685758971942,-0.462371581093321,FALSE\n\"RSL1D1\",2.93058749992548,3.02165993383195,-0.663356270407244,FALSE\n\"TNFRSF17\",1.29279173449958,4.00988201127786,3.66203455705021,TRUE\n\"BFAR\",1.55489148281805,3.39206751374831,-0.468858673398396,FALSE\n\"NDUFAB1\",2.71570586891656,3.11380696089873,-0.512411028219006,FALSE\n\"LAT\",2.44442334440561,3.49183730868594,1.62549332460236,TRUE\n\"BRD7\",1.16657333980853,3.39613503906858,-1.20549903958126,FALSE\n\"TMEM208\",2.18834303466666,3.28070010433175,-0.363926207105802,FALSE\n\"DPEP2\",1.81715482167584,3.44782424269604,0.0321802095079433,FALSE\n\"PSMD7\",2.69331054932511,3.1592198759391,-0.213236501049554,FALSE\n\"GABARAPL2\",3.05887265215854,2.88321460607683,-0.71432196990088,FALSE\n\"C16orf74\",1.64545519715339,3.41688132963283,-0.308175685966357,FALSE\n\"IRF8\",1.85937966824741,3.63118625107702,1.09726211737773,TRUE\n\"GLOD4\",1.82939397728794,3.39627496597286,-0.267250422204789,FALSE\n\"SERPINF1\",1.41748893074723,3.36530116349807,-1.01031541771555,FALSE\n\"PSMB6\",3.18074359800394,2.73596222179072,-0.586226269097547,FALSE\n\"RNF167\",2.38891216415749,3.26093462172524,-0.0661836076523231,FALSE\n\"CLEC10A\",2.36594049161634,3.820537860374,4.03367292714074,TRUE\n\"TMEM256\",2.28584368835803,3.22156759273921,-0.82965775506666,FALSE\n\"EIF4A1\",3.87441317833487,1.90582900374495,-0.886131881399053,FALSE\n\"LSMD1\",2.61240381830552,3.13612548110962,-0.365379446448042,FALSE\n\"TRAPPC1\",3.45808144976913,2.49271102899477,-0.609985413133518,FALSE\n\"NCOR1\",2.52274379556912,3.18745292908864,-0.604537171279372,FALSE\n\"TNFRSF13B\",1.38031143188512,3.59876201313415,0.681963834240456,TRUE\n\"SNORD3B-2\",1.24675450771303,3.48593386192884,-0.13588952155318,FALSE\n\"IFT20\",2.13208648542484,3.2609666690596,-0.519348040347258,FALSE\n\"UNC119\",2.39855608213416,3.26340416172117,-0.048090860879279,FALSE\n\"NSRP1\",1.78792689808665,3.37028157877281,-0.418236368941893,FALSE\n\"CCL5\",2.77338811709309,3.87529260416926,2.57267224378342,TRUE\n\"CCL4\",1.0511249797538,3.64169233678438,1.25559294600193,TRUE\n\"GGNBP2\",2.27922701317041,3.22255558857385,-0.821876234934065,FALSE\n\"CWC25\",1.54356057470269,3.4153841296945,-0.317870871906012,FALSE\n\"MIEN1\",2.70482405569605,3.14408683121987,-0.312931081203359,FALSE\n\"CCR7\",1.90994749768619,3.51670403136384,0.475955364469793,FALSE\n\"NT5C3B\",0.771948123880509,3.343965121875,-1.14777630356906,FALSE\n\"DNAJC7\",2.25154446759174,3.20804863342093,-0.936133967230667,FALSE\n\"CCR10\",0.801337425291691,3.76664423842912,2.43941001832113,TRUE\n\"COA3\",2.19740659286139,3.27287174272182,-0.42558289768762,FALSE\n\"SLC25A39\",2.24100230144089,3.21426086975865,-0.887205985517354,FALSE\n\"ABI3\",2.89832295263727,3.20587699709897,0.0349905598481674,FALSE\n\"PHB\",2.77621828268808,3.06437536075699,-0.501426737566584,FALSE\n\"SUPT4H1\",2.93970298604634,2.94878583383306,-0.939614049352395,FALSE\n\"VMP1\",1.92883847386978,3.38186782705919,-0.123315949006977,FALSE\n\"PSMC5\",2.86999435160577,3.01449034114468,-0.690535415696962,FALSE\n\"CD79B\",2.84176860254764,3.6051815423987,1.54871061689013,TRUE\n\"DDX5\",3.77464779124756,2.16627034942249,-0.990658366225575,FALSE\n\"CD300A\",2.28388397984871,3.39974816577667,0.573704149617894,TRUE\n\"SUMO2\",3.42782335250183,2.43712610232806,-0.802705791777586,FALSE\n\"ACOX1\",0.492072973180953,3.46177239089959,-0.286661144595184,FALSE\n\"UBALD2\",1.53192280630342,3.40107933861356,-0.410502193754082,FALSE\n\"ST6GALNAC1\",0.274560898986033,3.48501115446387,0.379301679016756,FALSE\n\"SRSF2\",2.98666118989634,2.89705236613655,-0.652914725002025,FALSE\n\"SEPT9\",2.61413368580122,3.1943011067012,0.017874199869861,FALSE\n\"DCXR\",2.66768237951364,3.13918674981164,-0.345212196324342,FALSE\n\"CD7\",2.79237087781125,3.55157866193919,1.34550793306925,TRUE\n\"ANKRD12\",2.1434646211977,3.25284478100334,-0.583316564513242,FALSE\n\"PSMG2\",2.13606686709123,3.25441029869955,-0.570986444239716,FALSE\n\"TTC39C\",1.82848276874254,3.43921111202697,-0.0178502724154508,FALSE\n\"RNF138\",1.34054558502666,3.4430159746887,-0.446986864972786,FALSE\n\"ACAA2\",1.66861938161565,3.32980632480297,-0.653342109222304,FALSE\n\"NOP56\",2.02440052302235,3.27665807807809,-0.590914282760244,FALSE\n\"IDH3B\",2.36858948015518,3.18773341041145,-0.602482263257409,FALSE\n\"C20orf27\",2.81700460469908,3.07708427075547,-0.453248643441959,FALSE\n\"PCNA\",1.92681044999994,3.48948646538109,0.354988546987522,FALSE\n\"DTD1\",1.89180152659098,3.30012283589523,-0.486626593040638,FALSE\n\"NAA20\",2.02489479179352,3.2624009076075,-0.654279410237939,FALSE\n\"CST3\",3.87266101830714,3.70706007871455,1.84704591777024,FALSE\n\"CST7\",2.36352013310648,3.5912068490142,2.35351066106011,TRUE\n\"APMAP\",1.82385488816104,3.39210066558381,-0.291497385147886,FALSE\n\"EIF2S2\",2.81877706032528,3.05188746162684,-0.548767003551868,FALSE\n\"DYNLRB1\",2.92893109715304,3.01257319717096,-0.69780309977272,FALSE\n\"RBM39\",2.72760321845281,3.07463165385955,-0.770492984634845,FALSE\n\"TOP1\",2.03182368754573,3.2688526549337,-0.625605011441792,FALSE\n\"YWHAB\",3.37074808174454,2.59216218508851,-1.21758729884904,FALSE\n\"STK4\",1.91145116685317,3.32130289305684,-0.392493116499919,FALSE\n\"CD40\",1.34237652462335,3.40954750323109,-0.689588568559011,FALSE\n\"ZFAS1\",3.26221173651,2.84065226714444,-0.126579551648249,FALSE\n\"BCAS4\",1.31078818485541,3.51301727545523,0.0604290425842048,FALSE\n\"PSMA7\",3.70184262015965,2.15297593885639,-1.01466912254841,FALSE\n\"ADRM1\",2.74433349911799,3.03145981157427,-0.626206000845517,FALSE\n\"PPDPF\",3.70917800551077,2.29780596243752,-0.753094652048315,FALSE\n\"RGS19\",3.15313752271327,2.93950689261703,-0.464516061244376,FALSE\n\"GZMM\",2.47874240483735,3.47488674930115,1.50130737130099,TRUE\n\"PRSS57\",0.709220822554557,3.94826714249575,2.50911470602256,TRUE\n\"CFD\",2.84723814010623,3.71046642164492,1.94783413124585,TRUE\n\"CNN2\",3.00323234114854,3.08482057741565,0.180336356420705,FALSE\n\"HMHA1\",2.39283661108827,3.22489277945028,-0.330239232015325,FALSE\n\"GPX4\",3.44506841519488,2.59625049630521,-0.2510002252189,FALSE\n\"C19orf24\",2.46814911759545,3.18846497134477,-0.597122582208558,FALSE\n\"ABHD17A\",1.57635083390745,3.41584169039307,-0.314907916900643,FALSE\n\"TIMM13\",2.75910432364373,3.08053149182117,-0.440180603697268,FALSE\n\"GNG7\",1.76203899779364,3.49720326738288,0.319004651967895,FALSE\n\"SLC39A3\",1.6026774054119,3.30949755044099,-1.00354420657569,FALSE\n\"AES\",3.19645833825022,3.16698599022316,1.30620447379883,FALSE\n\"S1PR4\",2.92801073856382,3.15140301087611,-0.171514391080131,FALSE\n\"C19orf77\",0.511817382652722,3.73309107285334,1.0265547145141,TRUE\n\"MATK\",1.58652817616651,3.55807415089641,0.606124816228831,TRUE\n\"TMIGD2\",1.40700405541319,3.49012338119997,-0.105521104110771,FALSE\n\"SH3GL1\",1.02312430586355,3.39589871127882,-1.20786762905002,FALSE\n\"NDUFA11\",3.31978216914435,2.68460425184431,-0.811715924976139,FALSE\n\"CLPP\",2.31718877187444,3.24894571250279,-0.154018710142789,FALSE\n\"ALKBH7\",2.8621718237541,3.02838692923039,-0.637854963251816,FALSE\n\"STXBP2\",2.97132016287779,3.25948423046367,0.955433891813368,TRUE\n\"PRAM1\",1.73034077734327,3.43531150841942,-0.0405016236246931,FALSE\n\"EIF3G\",3.40489212234929,2.59543960636747,-0.253811689139252,FALSE\n\"ICAM4\",1.7772643639441,3.56261956954824,0.698983691962379,TRUE\n\"ICAM3\",3.29400678585182,2.72425881385486,-0.637610648792036,FALSE\n\"S1PR5\",0.887915380404925,3.51604071185922,0.148849258950584,FALSE\n\"ILF3-AS1\",2.05267070540039,3.3759332336717,-0.14969188855979,FALSE\n\"ILF3\",1.73355287731612,3.28416581667598,-0.918450895394821,FALSE\n\"ACP5\",1.79560618918677,3.40998194366918,-0.18763167192587,FALSE\n\"C19orf43\",3.66343251324588,2.19887261277014,-0.931776101613565,FALSE\n\"JUNB\",4.05380477204222,2.22221272665815,-0.406053017836545,FALSE\n\"PRDX2\",2.95057551234195,3.08153363722124,-0.436381579593953,FALSE\n\"CALR\",2.65133166821602,3.19118238315577,-0.00267155547852289,FALSE\n\"LYL1\",2.33536803427462,3.31724532018289,0.346369017325006,FALSE\n\"C19orf53\",3.16436473031736,2.79951637906891,-1.08574606609069,FALSE\n\"DDX39A\",2.30013644905545,3.27443938366181,-0.413236054606464,FALSE\n\"DNAJB1\",2.00278626978874,3.38532640069678,-0.107944528717884,FALSE\n\"NDUFB7\",2.93963730681195,2.92874368394805,-1.01559165748205,FALSE\n\"TPM4\",2.29263724839099,3.23979404425685,-0.686105024926598,FALSE\n\"BST2\",3.17506747117006,2.83325099983036,-0.936043360614697,FALSE\n\"IFI30\",2.85076823438554,3.44322765211275,0.934761051508159,TRUE\n\"LSM4\",2.9543320592816,2.97652355768902,-0.83446335815266,FALSE\n\"LRRC25\",2.54049014676706,3.38840801344387,1.2966258940397,TRUE\n\"COPE\",3.54598383391014,2.39720120837223,-0.941130741523855,FALSE\n\"UQCRFS1\",3.09869205893859,2.82343803607911,-0.979589936480713,FALSE\n\"U2AF1L4\",1.76255121620343,3.36702337520004,-0.437162065208865,FALSE\n\"HCST\",3.23021908139091,3.05521156186913,0.81545342947447,FALSE\n\"TYROBP\",3.63507593900109,3.45796190467109,1.34223868273705,FALSE\n\"POLR2I\",2.1695581156555,3.26862109163297,-0.459061305050001,FALSE\n\"SPINT2\",2.38844560562016,3.25656712708157,-0.0981814593780597,FALSE\n\"PPP1R14A\",1.69312644261049,3.63819086307192,1.13794931083336,TRUE\n\"SERTAD3\",1.41154117961891,3.33287725651987,-1.245345424383,FALSE\n\"CD79A\",2.55117267541198,4.08044186650481,5.85565718702568,TRUE\n\"RABAC1\",2.94951585872543,3.0419475800826,-0.586448012283793,FALSE\n\"PAFAH1B3\",1.97867292918424,3.31297635916622,-0.429499890215301,FALSE\n\"ZNF428\",2.23109792169309,3.27731524022097,-0.39058561932245,FALSE\n\"CALM3\",3.0194689152838,3.08156711100402,0.165898585578688,FALSE\n\"AP2S1\",3.44353021999209,2.64439744868238,-0.084068298721592,FALSE\n\"FTL\",4.49103232958047,1.5469629152355,-0.707106781186548,FALSE\n\"SNRNP70\",2.07997867397491,3.25168248470156,-0.701916789281399,FALSE\n\"CD37\",3.82224431575387,2.17497573497486,-0.974935751811395,FALSE\n\"FLT3LG\",1.970299806232,3.45944945648318,0.221490886859855,FALSE\n\"NOSIP\",3.23284966047349,3.24435342976625,1.64588996637151,FALSE\n\"FUZ\",0.659916465170035,3.38785814142607,-0.882160794927724,FALSE\n\"SPIB\",1.67547557426147,3.56558468612117,0.716206954536968,TRUE\n\"JOSD2\",2.05004428645473,3.29956793377812,-0.489092821768203,FALSE\n\"CD33\",2.21506666007505,3.32265189002931,-0.0335111855474503,FALSE\n\"NKG7\",2.86310987439543,3.91107961376435,2.70833690085605,TRUE\n\"FPR1\",1.36148532323964,3.46244095456051,-0.306181713025613,FALSE\n\"ZNF600\",0.317701654354738,3.52661104566874,0.948016250111536,TRUE\n\"ZNF524\",1.86511753362464,3.32878690279856,-0.659263553598737,FALSE\n\"CTD-3138B18.5\",0.344095934259196,3.38497144796139,-0.658387173686617,FALSE\n\"ATP6V1E1\",2.40396621188404,3.21436560753577,-0.407365116187893,FALSE\n\"BID\",2.97622732038644,3.10862320211162,0.285964261344062,FALSE\n\"MRPL40\",2.20172857816021,3.37711898332359,0.395475221247938,FALSE\n\"UFD1L\",2.37918109310491,3.18918750031914,-0.591829072570739,FALSE\n\"COMT\",2.57810709279703,3.10849926696943,-0.547377442954696,FALSE\n\"DGCR6L\",2.20262456816448,3.26603503595417,-0.479429249552333,FALSE\n\"SDF2L1\",2.53099539846667,3.26378978729801,0.475656818376051,FALSE\n\"IGLL5\",2.02317776150909,4.32095716633225,4.05040953310977,TRUE\n\"IGLL1\",0.396912866528015,4.03353506809634,2.48074067946924,TRUE\n\"CHCHD10\",2.84803778259013,3.09285195248981,-0.393475078689755,FALSE\n\"SMARCB1\",2.45535848295947,3.35156104743066,0.59777849619555,TRUE\n\"MIF\",2.9974728247457,3.00190085723968,-0.187632992795672,FALSE\n\"ASCC2\",1.46183606611481,3.37169593795579,-0.600775734130829,FALSE\n\"PIK3IP1\",1.97267972224139,3.41938476024801,0.0434257799346298,FALSE\n\"HMOX1\",2.26625251593555,3.54277002201964,1.70015368991908,TRUE\n\"EIF3D\",3.1456873345638,2.80838608730384,-1.04638533549461,FALSE\n\"IL2RB\",1.07703697479522,3.52569334603516,0.0929959100993111,FALSE\n\"LGALS2\",2.34537707362106,3.68218825129432,3.02007345219663,TRUE\n\"EIF3L\",3.55411882736079,2.47934890932978,-0.656313670014485,FALSE\n\"ADSL\",2.11303477285882,3.25989089946784,-0.52782087236007,FALSE\n\"RBX1\",2.86726791020208,3.05150916596198,-0.550201081230497,FALSE\n\"TTC38\",0.993488234346266,3.51766916363223,0.163733597494548,FALSE\n\"TYMP\",3.14942249530857,3.24050437387103,0.871207781742147,FALSE\n\"CCT8\",2.72267647287639,3.02332133497672,-1.10851886068876,FALSE\n\"SOD1\",3.37875548385482,2.66646555338911,-0.891354759552662,FALSE\n\"PAXBP1\",0.425662638175597,3.47008375124559,-0.246433136193615,FALSE\n\"ATP5O\",3.4277180993434,2.55583373605459,-0.391130541226468,FALSE\n\"MRPS6\",2.32597825960564,3.27872423956319,0.0641495925638163,FALSE\n\"TTC3\",1.59182076431075,3.38937472893264,-0.486295923304141,FALSE\n\"U2AF1\",3.15925424977627,2.79930398551865,-1.08668859600157,FALSE\n\"CSTB\",3.4772808013323,2.56546395439329,-0.357741285614294,FALSE\n\"SUMO3\",2.51254534173561,3.17021610371403,-0.730820413173816,FALSE\n\"ITGB2\",3.51163693858791,2.60117887552272,-0.233912874999023,FALSE\n\"S100B\",1.30307756616295,3.70544932468219,1.45530430679135,TRUE\n\"PRMT2\",2.53377804135866,3.233181935983,0.274016174151852,FALSE\n\"MT-ND3\",2.1881501449375,3.30109707049301,-0.203278360969233,FALSE\n\n\n\"\",\"means\",\"dispersions\",\"dispersions_norm\",\"highly_variable\"\n\"HES4\",0.5304635122844151,3.1288092668857144,2.6498873233795166,True\n\"TNFRSF4\",0.14171751499176025,3.4165066884559034,0.2908051908016205,False\n\"SSU72\",1.5785272424561636,1.971202021348294,-0.46825626492500305,False\n\"PARK7\",2.9118247635023935,0.8317761719103588,-1.2987037897109985,False\n\"RBP7\",0.117945088999612,3.4874560718097767,0.8794848918914795,False\n\"SRM\",1.1533739699636187,2.3919357802700856,-0.48200079798698425,False\n\"MAD2L2\",0.7493968660490854,2.7181935494443783,-1.1559326648712158,False\n\"AGTRAP\",0.8358520524842399,2.707010108440194,0.3076137900352478,False\n\"TNFRSF1B\",0.7568694932120187,2.7966985420761294,0.11162508279085159,False\n\"EFHD2\",0.9243933268955775,2.592517310372801,0.038651369512081146,False\n\"NECAP2\",0.979150595664978,2.5569515407911947,1.0841343402862549,False\n\"HP1BP3\",0.7030399485996791,2.8093159942644563,0.3153490424156189,False\n\"C1QA\",0.6261157955442156,3.385543681626872,10.756184577941895,True\n\"C1QB\",0.28881434781210763,3.6373533671918135,3.586956262588501,True\n\"HNRNPR\",1.115585243361337,2.4360244979119967,0.22135843336582184,False\n\"GALE\",0.11096609081540788,3.2957238845045014,-0.7113512754440308,False\n\"STMN1\",0.6996363959993634,3.111273483645198,5.190816879272461,True\n\"CD52\",3.4654853027207513,0.5954857524590023,0.01910829171538353,False\n\"FGR\",1.3675572446414404,2.2385974006937257,0.9593446850776672,False\n\"ATPIF1\",1.8348273948260716,1.7636630352024354,-0.6449624300003052,False\n\"SESN2\",0.024847239085606165,3.4679912301743254,0.7179816365242004,False\n\"EIF3I\",1.892265351840428,1.675388041191036,-1.9044612646102905,False\n\"LCK\",1.1270791363716126,2.6058680057188113,2.930917978286743,True\n\"MARCKSL1\",1.0122270372935704,2.569097370041412,1.3148876428604126,False\n\"SFPQ\",0.8402972320147923,2.6379961497199353,-0.880909264087677,False\n\"PSMB2\",1.604980263369424,1.9556445151936688,-0.65134596824646,False\n\"MEAF6\",1.2069518164225987,2.3029466998470602,0.018622349947690964,False\n\"NDUFS5\",2.14812363079616,1.451206658282785,-1.0763150453567505,False\n\"CAP1\",2.5468868037632535,1.1190614095007507,0.6637553572654724,False\n\"SMAP2\",0.7773683203969683,2.7489692881650103,-0.659021258354187,False\n\"C1orf228\",0.2898688234601702,3.2874463066453457,0.01257624477148056,False\n\"PRDX1\",1.968205041885376,1.686806370500477,0.695281982421875,False\n\"TMEM69\",0.1914095071383885,3.176824323316833,-1.6978821754455566,False\n\"SCP2\",1.5313029742240907,2.001290496389565,-0.11415766924619675,False\n\"MAGOH\",0.9533904068810599,2.521659061942851,-1.2912111282348633,False\n\"JAK1\",1.4743659768785748,2.092173777797029,-0.9138151407241821,False\n\"CCBL2\",0.23243971007210867,3.242827987120256,-0.44321003556251526,False\n\"GBP2\",0.7749368071556091,2.8204381467568003,0.49492961168289185,False\n\"CD53\",2.8928838889939446,0.8517409232738253,-1.1623235940933228,False\n\"DENND2D\",0.5294922345025199,3.0185494716856773,1.0842756032943726,False\n\"C1orf162\",2.184737410204751,1.573690905652336,-0.15529216825962067,False\n\"RHOC\",1.6602056353432793,2.082339812911737,0.8396775722503662,False\n\"CD2\",1.1210023679052081,2.517352537254768,1.5188066959381104,True\n\"RP11-782C8.1\",0.05293079308101109,3.330090082626875,-0.4262087941169739,False\n\"TXNIP\",2.281451116970607,1.441476391370049,0.6699948906898499,False\n\"CD160\",0.08323188781738282,3.5767392858542695,1.620283603668213,True\n\"RP11-277L2.3\",0.10405415092195783,3.3761168316854033,-0.04431665688753128,False\n\"APH1A\",1.1272542803628105,2.3677500297559635,-0.8678426146507263,False\n\"MRPS21\",1.5200760640416826,2.0316203966896293,0.24278214573860168,False\n\"CTSS\",2.600600915295737,1.30354158590256,1.9239482879638672,True\n\"MRPL9\",0.7814652310098921,2.676444012398946,-1.8300292491912842,False\n\"S100A10\",3.3607992339134216,0.5893342527101975,0.0,False\n\"S100A9\",1.409582656792232,2.465007269923572,3.855747938156128,True\n\"S100A8\",0.4613748264312744,3.2327125900516456,2.340056896209717,True\n\"S100A6\",3.7293049018723625,0.3582083481355116,-0.7179422378540039,False\n\"S100A4\",3.9585082391330175,0.2761895939069958,-0.9727155566215515,False\n\"RAB13\",0.1162391321999686,3.285544880542617,-0.7958083152770996,False\n\"TPM3\",2.6585403493472506,1.0232566416929298,0.00930843222886324,False\n\"HAX1\",1.1220479621206012,2.400309197613223,-0.3484174311161041,False\n\"PMVK\",1.1341235051836287,2.3685606953904,-0.8549098968505859,False\n\"PBXIP1\",0.5324374226161411,3.034081309820934,1.304816722869873,False\n\"DAP3\",0.9161491434914725,2.587464533062356,-0.05617878586053848,False\n\"CCT3\",1.4862880553518023,2.049297888778499,-1.462315320968628,False\n\"SH2D2A\",0.18897649220057897,3.400267518491947,0.15606589615345,False\n\"MNDA\",1.0991167637280055,2.517804503015002,1.5260169506072998,True\n\"FCER1A\",0.9420430513790675,3.21133792328958,11.652631759643555,True\n\"TAGLN2\",3.1507280370167323,0.7208878645285276,0.40864259004592896,False\n\"LY9\",0.2372690953527178,3.3198660981378607,0.34375178813934326,False\n\"FCER1G\",2.397678366388593,1.724497371427266,2.7781076431274414,True\n\"SDHC\",1.3813321280479431,2.1398495418109613,-0.30391132831573486,False\n\"FCGR3A\",1.2829865860939025,2.8652993179701114,9.280588150024414,True\n\"FCRLA\",0.22628040041242328,3.4028767751669573,1.1917246580123901,False\n\"CD247\",0.891135379246303,2.7810521250870672,3.577059030532837,True\n\"CREG1\",0.41137202501297,3.03348728249702,-1.3830684423446655,False\n\"RCSD1\",1.0958523975099836,2.443483884848663,0.34036004543304443,False\n\"XCL2\",0.06376952069146292,3.6799765942747578,2.4768619537353516,True\n\"XCL1\",0.09707019363130842,3.4941512028361577,0.9350355863571167,False\n\"PRDX6\",1.6806023379734585,1.9038159582034995,-1.2612943649291992,False\n\"C1orf21\",0.1641563814026969,3.229735484427098,-1.2588688135147095,False\n\"TPR\",1.1804531162125724,2.339397006805721,0.6189600825309753,False\n\"PTPRC\",2.12730660370418,1.5089899960552557,-0.6418120265007019,False\n\"PTPN7\",0.34402357612337386,3.1679505563704122,0.22867675125598907,False\n\"NUCKS1\",1.117167865548815,2.458023967826461,0.5723219513893127,False\n\"G0S2\",0.20984560932431903,3.4893325551481222,2.07489013671875,True\n\"TRAF3IP3\",2.1267591844286238,1.5469611141116486,-0.3562873899936676,False\n\"NENF\",1.2834286178861345,2.2435754253522218,-0.9592245221138,False\n\"CAPN2\",0.893832643372672,2.5788666413207455,-0.21754339337348938,False\n\"COA6\",0.48626756804330007,2.96749372243911,-1.0681225061416626,False\n\"ARID4B\",0.851666648047311,2.632559205303406,-0.9745415449142456,False\n\"RP11-156E8.1\",0.03860488619123186,3.3544368842736594,-0.22419904172420502,False\n\"SH3YL1\",0.4620246924672808,3.0782708629148376,0.3554126024246216,False\n\"ID2\",2.0568542160306658,1.7171223734566026,0.9232438206672668,False\n\"HPCAL1\",0.6492441235269819,2.8610645831228916,0.04494196176528931,False\n\"RAB10\",0.7523934701510838,2.756633099257716,-0.5352797508239746,False\n\"OST4\",2.656659406593868,1.0312111569255207,0.0636461079120636,False\n\"PPM1G\",0.8485996399606978,2.6429166729029396,-0.7961705327033997,False\n\"LBH\",0.6921050041062491,2.891877143501132,1.6483982801437378,True\n\"SRSF7\",1.7653309873172216,1.8248941913893455,0.22867758572101593,False\n\"ZFP36L2\",1.830921059335981,1.7927647316918736,-0.22974233329296112,False\n\"ERLEC1\",0.3578668246950422,3.147433133644511,-0.017255501821637154,False\n\"CAPG\",1.3814118017469135,2.2803317327738117,1.4932411909103394,False\n\"RNF181\",1.4380535347121102,2.0739707580413835,-1.1466816663742065,False\n\"GNLY\",1.0032860824040004,2.999422838365487,9.490450859069824,True\n\"CD8A\",0.3241312152998788,3.3003264356119013,1.815401315689087,True\n\"CD8B\",0.44246684312820433,3.273543690808798,2.8647544384002686,True\n\"MAL\",0.4297808020455497,3.128882818675537,1.005798578262329,False\n\"DUSP2\",1.1207810878753661,2.504273889398826,1.31015944480896,False\n\"MGAT4A\",0.31925102846963066,3.1955486141460714,0.5594810843467712,False\n\"LIMS1\",0.7920101846967424,2.7605458075660456,1.2295782566070557,False\n\"IL1B\",0.2570977234840393,3.2971713911053646,0.11192019283771515,False\n\"NIFK\",0.5474454225812639,2.923202247080712,-0.26958805322647095,False\n\"GYPC\",2.2203396940231324,1.5258614178678656,-0.5149469971656799,False\n\"MZT2B\",2.321262831347329,1.384265641952834,0.243854358792305,False\n\"CCDC115\",1.0950205942562647,2.404726569365529,-0.277945876121521,False\n\"CXCR4\",1.086656345980508,2.4897024867232056,1.0776978731155396,False\n\"PPIG\",0.9968291626657758,2.5297641244822695,0.5676126480102539,False\n\"SSB\",1.3223110280718122,2.215775082043864,-1.4170970916748047,False\n\"WIPF1\",0.6886635957445417,2.822894835405091,-0.734584629535675,False\n\"ITGA4\",0.8359883594512939,2.6870599618627664,-0.03595740348100662,False\n\"HSPD1\",1.2332227740968977,2.3219639147831264,0.3318365216255188,False\n\"BZW1\",0.6169549431119646,2.8532494815576617,-0.11466296017169952,False\n\"NBEAL1\",1.9933375239372253,1.6027220872149128,0.06300842016935349,False\n\"CD28\",0.22834476845605034,3.3463397443139953,0.6141861081123352,False\n\"AC079767.4\",0.1084918076651437,3.5331158892640326,1.2583324909210205,False\n\"SLC11A1\",0.30814815282821656,3.2428901013454476,-0.44257551431655884,False\n\"SP110\",1.0571888521739414,2.4843437491704345,-0.2953089475631714,False\n\"SP140\",0.43017298664365494,3.049996283569581,-0.007928198203444481,False\n\"SP100\",1.1510275983810425,2.3706471690624444,-0.8216238021850586,False\n\"NCL\",1.345952959060669,2.213487630522882,0.6381218433380127,False\n\"ARL4C\",1.0411439589091709,2.5342394118625093,0.6526366472244263,False\n\"UBE2F\",0.7239883623804365,2.7911613073139634,0.02221975289285183,False\n\"OGG1\",0.1437217548915318,3.3459452409171737,-0.2946557402610779,False\n\"MRPS25\",0.5746803440366473,2.9108047748360284,-0.4456234574317932,False\n\"ACAA1\",1.1095116427966527,2.3890495658081807,-0.5280453562736511,False\n\"EXOG\",0.2566750294821603,3.219165769692556,-0.6849249601364136,False\n\"EIF1B\",0.9172415174756732,2.5901936764847533,-0.00495842145755887,False\n\"UQCRC1\",1.3424256590434482,2.1660377150102073,0.03110724501311779,False\n\"NDUFAF3\",1.625256486279624,1.9499109903718916,-0.7188214063644409,False\n\"IMPDH2\",1.0820608033452714,2.445885899458562,-1.0259525775909424,False\n\"GPX1\",2.8411599666731697,1.052730919270263,0.21064862608909607,False\n\"GNAI2\",1.3338008305004665,2.208873101514469,0.579089343547821,False\n\"MANF\",0.8267381763458252,2.7701468356474312,1.394922137260437,False\n\"TEX264\",1.1493920115062168,2.3683195499177883,-0.8587568998336792,False\n\"TKT\",2.32977936404092,1.3481921466552433,-0.024843018501996994,False\n\"ARL6IP5\",2.2997865087645395,1.3544619890005456,0.021858587861061096,False\n\"PCNP\",0.9019928847040449,2.608156523163588,0.33216696977615356,False\n\"BBX\",0.6258379769325256,2.89382326193006,0.7139602899551392,False\n\"CD47\",1.167584800379617,2.359159624565145,-1.0048877000808716,False\n\"TIGIT\",0.16308624676295688,3.431156893100665,0.4123605489730835,False\n\"COX17\",0.9749383292879377,2.5266938649006994,0.5092821717262268,False\n\"EAF2\",0.6361635388646807,2.981489276051897,2.50433087348938,True\n\"GATA2\",0.02065466948917934,3.6053354664900152,1.8575512170791626,True\n\"RAB7A\",1.7316541000774928,1.851180282324229,0.6037248969078064,False\n\"H1FX\",1.2759774603162493,2.3191694233756097,0.2858111560344696,False\n\"SELT\",1.2521630345072066,2.3006853382342896,-0.018622349947690964,False\n\"SIAH2\",0.6234958239964077,2.8600963498517955,0.025168094784021378,False\n\"GPR171\",0.12484916210174561,3.3887073296941,0.060148950666189194,False\n\"P2RY13\",0.1822306227684021,3.3755764030955335,-0.04880068823695183,False\n\"SSR3\",1.3090353230067662,2.2646294170526837,-0.6124645471572876,False\n\"MFSD1\",0.4444763374328613,3.032452032490228,-0.2333795577287674,False\n\"SEC62\",1.3226326707431248,2.2461287756813846,-0.9171707630157471,False\n\"EIF4A2\",2.4276457606043134,1.260371513988436,-0.6789846420288086,False\n\"CCDC50\",0.46988354001726423,3.091734123464428,0.5284214615821838,False\n\"HES1\",0.2825032785960606,3.3338783969396846,0.486890584230423,False\n\"ATP5I\",1.422962064402444,2.1008111652004344,-0.8033192753791809,False\n\"SPON2\",0.2778847098350525,3.4251988961414366,1.4197502136230469,False\n\"LYAR\",0.6492481378146581,2.8655540393584085,0.13662846386432648,False\n\"CYTL1\",0.05800771815436227,3.6590551057329663,2.3032727241516113,True\n\"MRFAP1\",1.1185571449143545,2.397478309129657,-0.3935793340206146,False\n\"BLOC1S4\",0.8435735678672791,2.687142834166228,-0.034530218690633774,False\n\"BST1\",0.216778883934021,3.2453370329740827,-0.4175795614719391,False\n\"FGFBP2\",0.3279827870641436,3.4724997132473914,3.879157781600952,True\n\"MED28\",0.6768724032810756,2.7755042702316324,-1.7024245262145996,False\n\"KLF3\",0.6425517780440194,2.891692646708277,0.6704475283622742,False\n\"SMIM14\",0.549254503250122,3.0380474447677948,1.361133098602295,False\n\"HOPX\",0.5609079912730626,3.1768674230821143,3.3322792053222656,True\n\"SPINK2\",0.0866508081981114,3.5272536441528373,1.2096924781799316,False\n\"IGFBP7\",0.7338042909758432,2.858800205561481,1.1143312454223633,False\n\"IGJ\",0.41660849128450667,3.7034051283671636,8.388663291931152,True\n\"CCNI\",2.3846312495640345,1.296249511108144,-0.4117434322834015,False\n\"HNRNPDL\",2.3629807911600387,1.3019859173512682,-0.3690151870250702,False\n\"PLAC8\",1.665989602293287,2.0920173798969977,0.9535687565803528,False\n\"HSD17B11\",1.123522107260568,2.421642110920764,-0.008087675087153912,False\n\"BANK1\",0.3008819491522653,3.3619913162471877,0.7740704417228699,False\n\"CCDC109B\",1.2974642705917359,2.26003965325865,-0.6880581378936768,False\n\"SNHG8\",1.2345471130098615,2.329453261673833,0.45518630743026733,False\n\"ANXA5\",2.4337044647761754,1.2579877071783607,-0.6967406868934631,False\n\"HMGB2\",1.5129692786080498,2.158400935845091,1.7348089218139648,True\n\"SUB1\",1.9149133661815099,1.7741164588515903,1.3518120050430298,False\n\"IL7R\",0.7544083384105137,2.9421889881944194,2.4607436656951904,True\n\"FYB\",1.2590651869773866,2.359428561746921,0.9488804936408997,False\n\"EMB\",0.44575908592769076,3.0021261924147176,-0.6230799555778503,False\n\"GZMK\",0.411912488256182,3.467607131862128,3.8205127716064453,True\n\"GZMA\",0.8198124470029559,3.066636345233417,6.500912666320801,True\n\"NSA2\",0.9535107489994594,2.5590329629649484,-0.5897804498672485,False\n\"COX7C\",3.1460759837286814,0.6037324147664256,0.04472475126385689,False\n\"GLRX\",1.8970978804997036,1.7331939701500767,-1.079692006111145,False\n\"CAMK4\",0.2955056534494673,3.228014595417946,-0.5945321917533875,False\n\"EPB41L4A-AS1\",0.7334964432035174,2.8017803827777286,0.19367752969264984,False\n\"REEP5\",1.7951795479229518,1.7943711789313865,-0.20682169497013092,False\n\"SNX2\",1.2902411535808018,2.274426155022765,-0.45111197233200073,False\n\"IRF1\",0.7273444042887006,2.8259228063128767,0.583486020565033,False\n\"SKP1\",2.205871346337455,1.4317556549676123,-1.2225773334503174,False\n\"TXNDC15\",0.42446238313402446,3.0506132428010764,0.0,False\n\"H2AFY\",2.102547113895416,1.5527049731877816,-0.3130963444709778,False\n\"EGR1\",0.12374017681394305,3.3571881826144856,-0.20137102901935577,False\n\"MZB1\",0.38472775902066914,3.643412870804385,5.927809715270996,True\n\"NDUFA2\",1.6624157694407873,1.913403205598164,-1.1484661102294922,False\n\"NDFIP1\",1.5259128734043665,2.018526590035048,0.08868664503097534,False\n\"CD74\",4.12582847901753,0.30904436481048864,-0.8706594109535217,False\n\"NUDCD2\",0.6666762467793056,2.838733392491702,-0.411119669675827,False\n\"NPM1\",3.6014694012914386,0.3721969248978591,-0.6744897365570068,False\n\"ATP6V0E1\",2.5409100696018765,1.1457857980379478,0.8463109731674194,False\n\"HIGD2A\",3.1212137351717266,0.6834339935249308,0.2923003137111664,False\n\"LMAN2\",1.7619913738114492,1.8226149026577876,0.19615693390369415,False\n\"F12\",0.0330258093561445,3.2966116392244755,-0.7039853930473328,False\n\"PRR7\",0.2693428771836417,3.237854683304181,-0.4940134584903717,False\n\"NHP2\",1.6793759053094046,1.9110689086917452,-1.175937533378601,False\n\"SQSTM1\",1.0375721618107387,2.4559081954580066,-0.8355434536933899,False\n\"RNF130\",1.2203602889605931,2.3884078517044336,1.4261703491210938,False\n\"SERPINB1\",1.5039421936443873,2.085371752579504,0.8753591775894165,False\n\"SERPINB6\",0.8504585446630205,2.648841929413822,-0.6941288113594055,False\n\"LY86\",1.47219874892916,2.2025843541884615,0.4986390173435211,False\n\"ADTRP\",0.10101056609834944,3.4481685000820357,0.5535088777542114,False\n\"DEK\",1.5087253100531441,2.0655981285248735,0.6426517367362976,False\n\"SOX4\",0.2103937956265041,3.3251341296374988,0.39756593108177185,False\n\"FAM65B\",1.0588049936294555,2.5079820063963676,0.15378384292125702,False\n\"HIST1H4C\",0.7881362237249102,2.7708035089645917,-0.3064814507961273,False\n\"HIST1H1E\",1.1152915106500898,2.422656031574563,0.008087675087153912,False\n\"ZNRD1\",0.6638743696893965,2.8217776261284206,-0.7574009895324707,False\n\"LTB\",2.8712917688914708,1.215964406563696,1.3257043361663818,False\n\"LST1\",2.3984740846497674,1.741513492260081,2.9048540592193604,True\n\"AIF1\",2.3917279178755626,1.7763478010736589,3.164321184158325,True\n\"DDAH2\",0.770357038293566,2.750185246221578,-0.6393881440162659,False\n\"CLIC1\",3.739155138560704,0.2242716484012899,-1.133987307548523,False\n\"C6orf48\",2.339820236819131,1.3425456372492157,-0.06690166145563126,False\n\"HLA-DRA\",3.2884157282965525,1.1054795721498065,1.6032928228378296,True\n\"HLA-DRB5\",3.154324907915933,1.0063751810099069,1.2954466342926025,False\n\"HLA-DRB1\",3.446997114590236,0.8677241145786997,0.8647573590278625,False\n\"HLA-DQA1\",2.5476609880583627,1.570376504835609,3.7467103004455566,True\n\"HLA-DQB1\",2.452441107204982,1.5292111965907413,1.3234971761703491,False\n\"HLA-DQA2\",2.3444376816068377,1.6477707912737234,2.20660138130188,True\n\"HLA-DMB\",1.6760586064202445,2.0951059164789236,0.9899164438247681,False\n\"HLA-DMA\",2.1292821376664297,1.7393766169589273,1.0905851125717163,False\n\"HLA-DPA1\",3.2330788959775654,1.0596003132738443,1.4607789516448975,False\n\"HLA-DPB1\",3.16840386731284,1.1301109135152296,1.679804801940918,True\n\"HSD17B8\",0.34022767066955567,3.1348648285625913,-0.16790559887886047,False\n\"TAPBP\",1.0028094267845153,2.5280142554997447,0.5343676805496216,False\n\"DAXX\",0.5590433457919529,2.8906131104106447,-0.7323309183120728,False\n\"CUTA\",2.2513053277560644,1.4442655688953754,0.6907703876495361,False\n\"HMGA1\",1.3753544402122497,2.211347885867605,0.6107485890388489,False\n\"CCDC167\",0.8610743641853332,2.7035271330848105,0.24763178825378418,False\n\"CNPY3\",1.9137607056753976,1.6892247123927515,0.7134667634963989,False\n\"MEA1\",1.0072646692820957,2.483654244744258,-0.3084085285663605,False\n\"HSP90AB1\",2.3867687017577035,1.2828696964441078,-0.5114044547080994,False\n\"ELOVL5\",0.645506854738508,2.8481827513011635,-0.2181389182806015,False\n\"UBE2J1\",0.8947798136302403,2.670492969036405,1.5020928382873535,False\n\"PNISR\",1.6540176173618861,1.9174103609219555,-1.1013076305389404,False\n\"RP3-467N11.1\",0.029356536184038436,3.405342138675657,0.19817093014717102,False\n\"CD164\",1.3470230272838049,2.1907374991342996,0.34708523750305176,False\n\"GTF3C6\",1.1803169584274291,2.342655696233183,0.6726308465003967,False\n\"RWDD1\",1.44691650390625,2.1123171500166498,-0.6561261415481567,False\n\"GOPC\",0.29898412942886354,3.147857445738742,-1.4133557081222534,False\n\"SAMD3\",0.34568540096282957,3.180138629167676,0.3747692108154297,False\n\"DYNLT1\",0.6663589119911194,2.83300111917329,-0.5281877517700195,False\n\"SOD2\",0.9205976619039263,2.642633113035837,0.9792211055755615,False\n\"TCP1\",0.9338056281634739,2.5636135733186625,-0.5038118958473206,False\n\"MRPL18\",1.0986694458552768,2.398655604769184,-0.37479764223098755,False\n\"RNASET2\",2.9407299634388515,0.8370573705911849,-1.2626277208328247,False\n\"MAD1L1\",0.6341510653495789,2.8770988733880554,0.3724043369293213,False\n\"CHST12\",0.6126794375692095,2.8404418394810413,-0.3762286901473999,False\n\"EIF2AK1\",0.4188102793693542,3.046149048194831,-0.05736686661839485,False\n\"NDUFA4\",2.9592636115210396,0.7837307422954507,-1.6269043684005737,False\n\"TOMM7\",2.8213674572535923,0.8942258964180442,-0.8721067309379578,False\n\"CPVL\",1.4130316621916634,2.421491929714827,3.299067258834839,True\n\"LSM5\",0.6533228519984654,2.829257689018655,-0.6046384572982788,False\n\"SEPT7\",1.8130666937146869,1.7952363822936783,-0.19447706639766693,False\n\"AOAH\",0.47285788978849136,3.0247918433301044,-0.3318163752555847,False\n\"STK17A\",0.8681095252718244,2.7487604185375454,1.0266162157058716,False\n\"COA1\",0.6039574517522539,2.851430495384938,-1.288696527481079,False\n\"PPIA\",3.5606521875517707,0.3562480931691587,-0.7240313291549683,False\n\"SNHG15\",0.6873580013002668,2.7928120337349354,-1.3489545583724976,False\n\"UPP1\",0.46531397104263306,3.089963383721524,0.5056666731834412,False\n\"CCT6A\",1.145482864379883,2.3730372145209375,-0.783494770526886,False\n\"SBDS\",0.5968672043936594,2.8765601879455054,-0.931872546672821,False\n\"WBSCR22\",1.026856778689793,2.464154093390568,-0.6788833141326904,False\n\"EIF4H\",1.163014680658068,2.3355280385784853,-1.381888747215271,False\n\"LAT2\",0.7729718971252442,2.7128792661073886,-1.241738200187683,False\n\"NCF1\",0.7237173124722072,2.7894038273395605,-0.006156879477202892,False\n\"HSPB1\",1.1349364835875375,2.4314607175300473,0.14855121076107025,False\n\"RSBN1L-AS1\",0.03947643859045846,3.4271648188678583,0.3792375922203064,False\n\"CDK6\",0.115386267048972,3.4192639664480127,0.31368282437324524,False\n\"CCDC132\",0.05164613587515695,3.2576960768298404,-1.0268747806549072,False\n\"BRI3\",1.0589042724881854,2.5264102345798345,0.5038936138153076,False\n\"PDAP1\",1.0740721126965114,2.442179759204409,-1.09636390209198,False\n\"ATP5J2\",2.341696172441755,1.2777849206137528,-0.549278974533081,False\n\"MCM7\",0.2507415097100394,3.2184273582969998,-0.692467987537384,False\n\"LAMTOR4\",2.934967553274972,0.7882563134366036,-1.5959900617599487,False\n\"PILRA\",1.0828603843280247,2.6414314504728,3.4982712268829346,True\n\"ZNHIT1\",1.9015019583702086,1.6663747693152677,-2.0330617427825928,False\n\"FAM185A\",0.033765993458884105,3.3792286628347243,-0.01849723979830742,False\n\"RINT1\",0.014512692519596644,3.403195955268523,0.18036365509033203,False\n\"SYPL1\",0.5652138178689139,2.9579592640077195,0.22393718361854553,False\n\"RP11-390E23.6\",0.05689254828861782,3.286793717687195,-0.7854464650154114,False\n\"CPA5\",0.13437814678464616,3.3654090125494265,-0.13316133618354797,False\n\"MTPN\",0.8822556029047285,2.6248019123251116,0.6445664167404175,False\n\"C7orf55\",0.4160502713067191,3.0288028236282476,-0.2802734971046448,False\n\"NDUFB2\",2.2544069262913293,1.344193389075016,-0.05462820082902908,False\n\"MRPS33\",0.6887507067407881,2.827236389391286,0.6046954393386841,False\n\"GSTK1\",2.827765314238412,0.8969790653233523,-0.8532997369766235,False\n\"ZYX\",1.0235753835950578,2.489530041579394,-0.196776881814003,False\n\"GIMAP7\",2.2397186613082884,1.5876096278661298,1.7584842443466187,True\n\"TMEM176B\",0.9490635667528425,2.871800016929958,5.280209064483643,True\n\"TMEM176A\",0.3573712778091431,3.231418092533679,0.9894309043884277,False\n\"OFD1\",0.49818977628435407,3.0086811906603628,-0.5388453602790833,False\n\"NDUFB11\",2.517864238875253,1.1725489993281453,1.0291316509246826,False\n\"TIMP1\",2.2173807883262633,1.6034321140820327,0.06834747642278671,False\n\"CFP\",1.7583173383985247,2.089187811462487,3.9995925426483154,True\n\"EBP\",0.69867180619921,2.7901664683699527,0.006156879477202892,False\n\"WDR13\",0.35334653207233974,3.0984666428353043,-0.6041927337646484,False\n\"PQBP1\",1.1066113696779523,2.4196659177273814,-0.039614420384168625,False\n\"PLP2\",1.6222328853607177,2.0064834073669857,-0.05304449051618576,False\n\"MSN\",1.2662034085818699,2.281929009749784,-0.32753971219062805,False\n\"IGBP1\",1.618926956312997,1.9447770691682476,-0.7792403697967529,False\n\"IL2RG\",2.0749206624712264,1.6082441419039284,0.10453162342309952,False\n\"COX7B\",0.8857673301015582,2.635271083018891,0.8410510420799255,False\n\"ITM2A\",0.6361515644618443,3.0110888999558854,3.108832836151123,True\n\"TCEAL8\",0.539815628528595,2.914090055192999,-0.39897477626800537,False\n\"NGFRAP1\",0.3234098836353847,3.2381122715098525,1.0696707963943481,False\n\"RPL39\",1.1081793393407549,2.45699714739544,0.5559408068656921,False\n\"NDUFA1\",2.0945171880722047,1.4909836892363224,-0.7772108316421509,False\n\"CD40LG\",0.20659316710063388,3.4277449064888623,1.44575834274292,False\n\"SSR4\",2.6547780190195356,1.0810629118808102,0.40418583154678345,False\n\"MPP1\",0.3585300963265555,3.0951642098340564,-0.6437773704528809,False\n\"BLK\",0.29053961072649276,3.3730008507488067,0.8865353465080261,False\n\"BIN3\",0.3703190929549081,3.1070248154834927,-0.5016101598739624,False\n\"PNOC\",0.27039243289402554,3.334161462678466,0.4897821843624115,False\n\"RP11-489E7.4\",0.15074032442910332,3.26897496726013,-0.9332917928695679,False\n\"LEPROTL1\",0.8173508821214949,2.7371271477689523,0.826274037361145,False\n\"GTF2E2\",0.4438890027999878,2.9866488284389066,-0.8219709396362305,False\n\"PPAPDC1B\",0.3715251350402832,3.171007459799798,0.265318363904953,False\n\"GOLGA7\",0.7948394434792655,2.6987533604340714,0.1654203236103058,False\n\"CHCHD7\",0.5513905014310564,2.9137123338134714,-0.40433815121650696,False\n\"TRAM1\",1.1111518195697239,2.428138798946678,0.09555574506521225,False\n\"TPD52\",0.4878309239659991,3.024190582171708,-0.33954283595085144,False\n\"MTDH\",2.002582537787301,1.6009606285753135,0.049763090908527374,False\n\"ZNF706\",1.8430475289481028,1.7611412038924252,-0.6809436678886414,False\n\"EIF3E\",1.4990359786578586,2.1318132727444024,1.4219098091125488,False\n\"LYPD2\",0.2695499757358006,3.615220481829752,3.3608639240264893,True\n\"LY6E\",2.8343881038257055,0.9498866870112674,-0.491885244846344,False\n\"COMMD5\",0.5678990956715175,2.877028204677167,-0.9252270460128784,False\n\"PLGRKT\",0.6059730056353978,2.8920885018839155,0.6785319447517395,False\n\"PTPLAD2\",0.6819068455696106,2.797172391899942,-1.2599046230316162,False\n\"SIT1\",0.8835776056562151,2.7373692410772663,2.7572219371795654,True\n\"CCDC107\",1.0341274997166225,2.499956034529484,0.001301950658671558,False\n\"TLN1\",0.8933609758104597,2.638256754198833,0.8970859050750732,False\n\"DCAF10\",0.07251328502382551,3.32401222875022,-0.4766378402709961,False\n\"ANXA1\",2.172457705906459,1.5877249211937419,-0.049763090908527374,False\n\"OSTF1\",1.7721871655327932,1.8030162945417845,-0.0834740549325943,False\n\"HNRNPK\",2.2995847078732083,1.343789417262253,-0.057637229561805725,False\n\"HIATL1\",0.058406964370182585,3.3672887779322815,-0.1175645962357521,False\n\"ANP32B\",2.181985102721623,1.4753827626529357,-0.8945223093032837,False\n\"TXN\",1.488051608289991,2.0750060630092437,-1.1334372758865356,False\n\"ATP6V1G1\",2.5903410400663103,1.0940277692270626,0.4927493929862976,False\n\"NDUFA8\",0.9543200090953282,2.5454485738925956,-0.8447312712669373,False\n\"ARPC5L\",1.1225670995031085,2.4242290318166617,0.03318217396736145,False\n\"C9orf78\",1.43552109003067,2.109446208137029,-0.6928533315658569,False\n\"NUP214\",0.3878690736634391,3.107276439999808,-0.4985940456390381,False\n\"FCN1\",1.419729381629399,2.555718118144937,5.016188621520996,True\n\"EGFL7\",0.046753500189099996,3.6133860799049646,1.9243485927581787,True\n\"SNHG7\",1.642782987526485,1.9652695056213405,-0.5380735397338867,False\n\"PHPT1\",1.1817486844744,2.3400272195953016,0.6293397545814514,False\n\"C9orf142\",1.3177240756579809,2.281642536725902,-0.3322579264640808,False\n\"CLIC3\",0.4501452023642404,3.2944315768386736,3.1331729888916016,True\n\"KLF6\",1.791573168209621,1.8320175433271169,0.3303128778934479,False\n\"AKR1C3\",0.13095156465257918,3.554385532172659,1.4348105192184448,False\n\"RBM17\",0.8748487618991306,2.6013348468905093,0.20413824915885925,False\n\"PRKCQ-AS1\",0.46681215490613664,3.103100917574293,0.6744897365570068,False\n\"GATA3\",0.2590657837050302,3.251678202418415,-0.3528030514717102,False\n\"VIM\",3.885628358295986,0.21233870271118252,-1.171054482460022,False\n\"NSUN6\",0.024934874602726528,3.474595904983479,0.7727817893028259,False\n\"DNAJC1\",0.6634162780216762,2.8232553997600425,-0.7272209525108337,False\n\"COMMD3\",0.8854918708120073,2.6176850215840988,0.5109971761703491,False\n\"APBB1IP\",1.1748897525242397,2.340432011980694,0.6360066533088684,False\n\"ABI1\",0.6449177377564567,2.871763896273733,0.2634500563144684,False\n\"EPC1\",0.669435395513262,2.842406744945392,-0.33610016107559204,False\n\"HNRNPF\",1.7424403936522348,1.8326575481032996,0.3394443988800049,False\n\"ZNF22\",0.5449531759534564,2.9080615878019396,-0.4845747947692871,False\n\"SRGN\",3.1820203178269524,0.7187670703930441,0.4020548164844513,False\n\"PPA1\",1.9723249432018826,1.6812761727173977,0.6536975502967834,False\n\"PRF1\",0.4672342995234898,3.275639186480046,2.8916826248168945,True\n\"C10orf54\",1.9480348338399616,1.7539579326550647,1.200229525566101,False\n\"PSAP\",2.4776945972442626,1.3736345890505057,0.16466780006885529,False\n\"ANAPC16\",2.096301405089242,1.5138270419198916,-0.6054397821426392,False\n\"RPS24\",4.16888646875109,0.03951180933621988,-1.707903504371643,False\n\"ANXA11\",1.2801857791628157,2.268022200146689,-0.5565853118896484,False\n\"HHEX\",0.7483355024882725,2.7820411128241203,-0.12503677606582642,False\n\"PDLIM1\",0.7521954165186201,2.832517061902885,0.6899582743644714,False\n\"PGAM1\",1.2006995878900801,2.2877982710090805,-0.23087278008460999,False\n\"NPM3\",0.5835609034129552,2.865885252861492,-1.083449125289917,False\n\"ADD3\",0.5499678979601179,2.9657473341933214,0.3345223069190979,False\n\"C10orf118\",0.3986073684692383,3.111001656390669,-0.4539417028427124,False\n\"RGS10\",2.047165349211012,1.6040314410144636,0.07285413146018982,False\n\"TIAL1\",0.5335880361284528,2.910171923069395,-0.4546095132827759,False\n\"FAM175B\",0.09877219642911639,3.3717517386351727,-0.08053461462259293,False\n\"ZNF511\",0.7602233764103481,2.7354446135042756,-0.8773934245109558,False\n\"FUOM\",0.6160800736291068,2.8513776886293734,-0.1528898924589157,False\n\"PSMD13\",1.0358003786631993,2.453071104581682,-0.88944411277771,False\n\"IFITM2\",3.025044196333204,0.8708068091830532,0.8743330836296082,False\n\"IFITM1\",0.9825507824761527,2.6816280869914118,3.4528088569641113,True\n\"IFITM3\",1.233267947946276,2.6049925324807095,4.9933271408081055,True\n\"RNH1\",2.2854771607262747,1.430862690039207,0.5909375548362732,False\n\"IRF7\",0.7144035414287022,2.7756465632086846,-0.22828449308872223,False\n\"TALDO1\",2.315921594074794,1.363848686102709,0.09177643060684204,False\n\"CTSD\",1.150955012185233,2.433645389775409,0.183403879404068,False\n\"CARS\",0.3293207386561802,3.1871696070730824,0.45904606580734253,False\n\"ILK\",0.7130265188217163,2.767055800512124,-0.366992712020874,False\n\"NUCB2\",0.7804963626180377,2.8388301473961355,0.7918906211853027,False\n\"LDHA\",2.646963887895857,1.0612974821123924,0.2691672146320343,False\n\"CAT\",1.3958466352735246,2.1633590126821045,-0.003160706255584955,False\n\"CD82\",0.4070742334638323,3.177156608603619,0.3390251696109772,False\n\"SPI1\",2.231482674053737,1.6421315097406062,0.3593484163284302,False\n\"PSMC3\",1.057702433041164,2.435412908399148,-1.2249243259429932,False\n\"MTCH2\",0.9038760454314095,2.5695294343274675,-0.39278340339660645,False\n\"TIMM10\",0.9125858875683376,2.555437698134146,-0.6572561264038086,False\n\"LPXN\",0.5208022686413356,2.9724967887828257,-1.003830909729004,False\n\"MS4A1\",0.3979865292140416,3.5480273204811525,4.784470081329346,True\n\"CYB561A3\",0.55853326184409,2.991263798988882,0.6968382000923157,False\n\"POLR2G\",1.8253445369856698,1.7620458795909237,-0.6680358648300171,False\n\"SLC3A2\",0.7898644290651594,2.7050488519728555,0.2738380432128906,False\n\"OTUB1\",1.2153184212957109,2.3037832457407155,0.03240028768777847,False\n\"BAD\",0.6657410158429827,2.7825584997611923,-1.5583586692810059,False\n\"CAPN1\",0.5994882069315229,2.888645002994283,-0.7602766752243042,False\n\"NEAT1\",1.4566774565832956,2.130449105919701,-0.4241686761379242,False\n\"KAT5\",0.2968501506532942,3.127999098077351,-1.6162132024765015,False\n\"CTSW\",0.9621502413068499,2.813046329991027,4.177524089813232,True\n\"SF3B2\",1.4724382727486747,2.077045175298248,-1.1073514223098755,False\n\"ADRBK1\",0.6168780016899109,2.883235462426564,0.4977295994758606,False\n\"POLD4\",2.4242102306229727,1.2463729817337565,-0.7832542061805725,False\n\"TBC1D10C\",1.7153395128250122,1.9253617431514982,1.66213858127594,True\n\"PTPRCAP\",2.3851946640014647,1.6333942228268428,2.099515914916992,True\n\"CORO1B\",1.5599733216421945,2.057563574525523,0.5480964779853821,False\n\"GSTP1\",3.119873720577785,0.8714259378963397,0.8762562870979309,False\n\"UNC93B1\",0.9315736372130258,2.5934637616120333,0.056414298713207245,False\n\"NDUFS8\",1.7151016981261118,1.8656834018318127,0.8106539845466614,False\n\"MRPL21\",0.8315154310635158,2.6519930911928933,-0.6398611068725586,False\n\"LAMTOR1\",2.2409815730367386,1.3932246889024167,0.3105868101119995,False\n\"MRPL48\",0.6947724696568081,2.795760143583927,0.09647350758314133,False\n\"SPCS2\",1.583583630834307,2.014408752528413,0.0402255542576313,False\n\"TMEM126B\",0.5399540260859899,2.9149846414807135,-0.38627228140830994,False\n\"CTSC\",1.8882055456297737,1.7540570450853805,-0.7820197343826294,False\n\"CWC15\",1.0441237218039376,2.4719411812930745,-0.5309398770332336,False\n\"FDX1\",0.8730441645213536,2.638431645274597,-0.8734093904495239,False\n\"POU2AF1\",0.26468634639467514,3.2528270328841224,-0.34106749296188354,False\n\"IL18\",0.20164505788258144,3.2570738661670697,-1.032037377357483,False\n\"AMICA1\",1.5803277056557792,2.130784304219579,1.409800410270691,False\n\"CD3E\",1.4863548026766096,2.314327366060956,1.9281386137008667,True\n\"CD3D\",1.597190865448543,2.335945691488548,3.824258327484131,True\n\"CD3G\",0.7958931735583714,2.874686919074812,3.1952579021453857,True\n\"FLI1\",0.5684784987994603,2.8810853876799003,-0.8676179051399231,False\n\"NINJ2\",0.39574057000023977,3.131716985087926,-0.20563724637031555,False\n\"CD27\",1.4169702465193612,2.4144899758867426,3.2094931602478027,True\n\"CHD4\",0.3184319577898298,3.127275857394397,-0.2588708698749542,False\n\"MLF2\",1.3171423414775303,2.1922417776912413,-1.8046914339065552,False\n\"LAG3\",0.14760625158037458,3.4282075205781326,0.3878890872001648,False\n\"CD4\",0.6600866266659328,2.804439425764517,-1.11149263381958,False\n\"KLRG1\",0.3516923270906721,3.239973909166978,1.0919852256774902,False\n\"KLRB1\",0.2260218334197998,3.3903462304760947,1.0637223720550537,False\n\"KLRC2\",0.04603870017187936,3.5524425370682007,1.4186891317367554,False\n\"KLRC1\",0.08168333121708461,3.5181544168477346,1.1341944932937622,False\n\"RP11-291B21.2\",0.1719919524874006,3.3853792623738665,0.032535381615161896,False\n\"PRR4\",0.16564905541283745,3.268730880594785,-0.9353170394897461,False\n\"H2AFJ\",1.0108332354681833,2.5471112430404825,0.8971828818321228,False\n\"WBP11\",0.32068588869912285,3.1503122876713943,0.017255501821637154,False\n\"LDHB\",3.090368707180023,0.8168236151401055,0.7066460847854614,False\n\"FGFR1OP2\",0.5136887632097517,2.996302626757417,-0.6979153752326965,False\n\"FKBP11\",0.6760352529798235,3.026186457422854,3.4171648025512695,True\n\"TMBIM6\",2.4643773542131697,1.2290757748601833,-0.9120943546295166,False\n\"COX14\",1.2297626870019096,2.2744939273574203,-0.449995756149292,False\n\"HNRNPA1\",3.810504821709224,0.21062666187887483,-1.1763725280761719,False\n\"NFE2\",0.14978008372443063,3.3596810038081437,-0.18068765103816986,False\n\"CD63\",1.9836902158600944,1.7124564837311567,0.8881585597991943,False\n\"WIBG\",0.7042777190889631,2.810107214841377,0.3281242549419403,False\n\"CNPY2\",1.2761631846427917,2.2577829311966244,-0.7252264022827148,False\n\"ATP5B\",2.5996967690331596,1.1091054208588431,0.5957455635070801,False\n\"RP11-620J15.3\",0.18905291829790388,3.314683790084808,-0.5540375709533691,False\n\"TMBIM4-1\",1.4108897934641156,2.1371014371892754,-0.33906713128089905,False\n\"LYZ\",2.32092558997018,2.0034402274302705,4.855844020843506,True\n\"ATXN7L3B\",0.25039225373949325,3.205269601427195,-0.8268774747848511,False\n\"NAP1L1\",2.914099838052477,0.8732971017743051,-1.0150723457336426,False\n\"OSBPL8\",1.1245168273789543,2.4081749062298576,-0.22293363511562347,False\n\"BTG1\",3.263639827115195,0.6591327372688829,0.21681377291679382,False\n\"ISCU\",1.8713946012088232,1.7385922299795884,-1.002670168876648,False\n\"HVCN1\",0.6137770799228123,2.960789849442122,2.0815939903259277,True\n\"PPP1CC\",1.297231457574027,2.2420686157896124,-0.984041690826416,False\n\"MAPKAPK5-AS1\",0.4799438408442906,2.9987331096006606,-0.6666826009750366,False\n\"ERP29\",3.0997669761521474,0.6733285132770047,0.2609098553657532,False\n\"OAS1\",1.0008396107809885,2.6152677244653435,2.1920578479766846,True\n\"COX6A1\",2.9413888655390057,0.7728634491770028,-1.7011394500732422,False\n\"TRIAP1\",0.42828919036047797,3.040210829921559,-0.13367559015750885,False\n\"POP5\",0.7117045303753444,2.7676167058689862,-0.35793620347976685,False\n\"ACADS\",0.18171605110168457,3.2706955540822853,-0.9190158247947693,False\n\"RNF34\",0.23365106412342618,3.265410289859481,-0.21252667903900146,False\n\"AC084018.1\",0.1785874148777553,3.304151777634601,-0.6414235234260559,False\n\"MPHOSPH9\",0.14263780798230852,3.3228616833605376,-0.48618412017822266,False\n\"SAP18\",2.4612904606546673,1.194226452443494,-1.17167329788208,False\n\"POMP\",1.2791384216717312,2.2226097845101864,-1.3045293092727661,False\n\"ALOX5AP\",0.8205571668488639,2.817019737696731,2.2021431922912598,True\n\"WBP4\",0.25492370912006923,3.195421653593422,-0.9274764657020569,False\n\"TSC22D1\",0.07034264905112131,3.4627494652358646,0.6744897365570068,False\n\"ESD\",0.6112283757754735,2.857631623666684,-0.025168094784021378,False\n\"EBPL\",0.5743373210089547,2.911140628405641,-0.44085457921028137,False\n\"UCHL3\",0.5158664427484785,2.906478392976629,-1.8521963357925415,False\n\"TNFSF13B\",0.5592069693974087,2.950253576124085,0.114521823823452,False\n\"APEX1\",1.4997389898981366,2.0783920997492284,0.7932186126708984,False\n\"DHRS4L2\",0.9008679515974862,2.5793652945943517,-0.20818470418453217,False\n\"PSME2\",3.1926214323725017,0.6151405421380615,0.08016160875558853,False\n\"NEDD8\",2.677662648473467,1.0168477402452565,-0.03447107970714569,False\n\"TINF2\",0.5248728227615357,2.9586227677804393,0.2333584725856781,False\n\"GZMH\",0.4276356611933027,3.49082709004423,5.656941890716553,True\n\"GZMB\",0.5011476033074516,3.447795450085074,5.103966236114502,True\n\"NFKBIA\",1.970262508051736,1.710929472693763,0.8766761422157288,False\n\"PNN\",0.8359825416973659,2.667499932998655,-0.3728102147579193,False\n\"RN7SL1\",0.03981003795351301,3.452677876978633,0.5909239649772644,False\n\"RPL36AL\",3.5186046256337846,0.37371933376110616,-0.6697607040405273,False\n\"ARF6\",1.7209269029753549,1.8634674529040895,0.7790370583534241,False\n\"LGALS3\",1.7085593152046203,1.9631794610996076,2.2017180919647217,True\n\"DAAM1\",0.0946424126625061,3.2396136126711124,-1.176908254623413,False\n\"DHRS7\",1.3602573806898934,2.1619897444801097,-0.02067740261554718,False\n\"ERH\",1.809483583654676,1.7766632177612538,-0.45947712659835815,False\n\"COX16\",1.0406308320590427,2.464616606656494,-0.670096218585968,False\n\"FOS\",2.7579654117992947,1.0824986615322134,0.4139935076236725,False\n\"AHSA1\",0.7930625823565892,2.694530409887485,0.09269482642412186,False\n\"CALM1\",3.1145014020374844,0.7020010676480116,0.3499749004840851,False\n\"EVL\",1.994544268335615,1.6767580625500653,0.6197234988212585,False\n\"PPP2R5C\",0.8032375778470721,2.7357859984771857,0.8031774759292603,False\n\"PLD4\",0.781629514013018,2.745792729404775,-0.7103106379508972,False\n\"CRIP1\",2.3570457751410347,1.3485928162050307,-0.021858587861061096,False\n\"AL928768.3\",0.30944340535572595,3.750983402099297,4.747713088989258,True\n\"KIAA0125\",0.170427519934518,3.4702203119494848,0.736476719379425,False\n\"NDNL2\",0.8685464893068586,2.643079578218124,-0.7933650612831116,False\n\"EMC7\",1.070540543283735,2.432631661243977,-1.277764081954956,False\n\"NOP10\",1.5708321292059761,2.006079303975137,-0.05780021473765373,False\n\"SRP14\",3.4579191841397967,0.4189183782295364,-0.529359757900238,False\n\"GCHFR\",0.5725268135751996,2.936848478391105,-0.07582114636898041,False\n\"ZNF106\",0.42008651086262294,3.053906966662721,0.04232580214738846,False\n\"CEP152\",0.056845097882407054,3.2824405567379205,-0.8215653896331787,False\n\"EID1\",2.422059579236167,1.23438981786237,-0.8725121021270752,False\n\"MYO5A\",0.057138428347451344,3.2945381304245833,-0.7211896777153015,False\n\"SLTM\",1.0512443804740905,2.471111930697448,-0.5466943979263306,False\n\"RPS27L\",1.166659072126661,2.3599716750059816,-0.9919328093528748,False\n\"PPIB\",2.6301378543036327,1.1083946847644455,0.5908904671669006,False\n\"SPG21\",0.8901063888413565,2.5904578730209464,0.0,False\n\"PKM\",2.5076510569027493,1.176702845651336,-1.3021998405456543,False\n\"CSK\",1.1698884044374738,2.347671320564952,-1.1881637573242188,False\n\"SCAMP2\",1.00727518762861,2.5009361443943363,0.01992262527346611,False\n\"IMP3\",1.4027804013660976,2.1361745176242195,-0.3509249687194824,False\n\"ETFA\",1.0373080359186444,2.488858599784022,-0.20953330397605896,False\n\"CTSH\",1.5475437102999006,2.0970698580920826,1.0130292177200317,False\n\"IL16\",0.9671092472757612,2.56156350103441,-0.5422874689102173,False\n\"ISG20\",1.7505244146074568,2.016135050125821,2.9572830200195312,True\n\"ZNF710\",0.04286399330411639,3.305014996963431,-0.6342612504959106,False\n\"IDH2\",1.4521725981576101,2.121438067271259,-0.5394445657730103,False\n\"NGRN\",0.03887676886149815,3.3872271487355894,0.04786762222647667,False\n\"SLCO3A1\",0.07637244156428746,3.2834020878003316,-0.8135874271392822,False\n\"VIMP\",0.8211317297390529,2.7300311384015576,0.7040702104568481,False\n\"POLR3K\",0.5473924248559134,2.926626550452363,-0.22096535563468933,False\n\"NME4\",0.4067884768758501,3.044676912518125,-1.24894380569458,False\n\"C16orf13\",1.3815849273545402,2.1544656183162068,-0.11693161725997925,False\n\"STUB1\",1.3442519756725857,2.186247330566634,0.28964367508888245,False\n\"NDUFB10\",2.246188083716801,1.3694475150795709,0.13347992300987244,False\n\"SRRM2\",0.9750618079730443,2.5447020911968687,0.8514124751091003,False\n\"TCEB2\",2.586166467666626,1.0965636060937665,0.5100718140602112,False\n\"IL32\",1.5635816890852792,2.5077126695832486,5.8457112312316895,True\n\"CORO7\",0.37258016960961476,3.123058584602567,-0.30942124128341675,False\n\"HMOX2\",1.0961791280337743,2.4258323814159017,0.05876084417104721,False\n\"RSL1D1\",1.7715168908664158,1.8223612309477548,0.1925375610589981,False\n\"TNFRSF17\",0.20039225918906076,3.734794972418854,2.931699752807617,True\n\"BFAR\",0.40954332079206196,3.064362635162185,-1.0129806995391846,False\n\"NDUFAB1\",1.4510624824251448,2.106938780347512,-0.7249302268028259,False\n\"LAT\",0.9726561835833958,2.7155758463641546,2.348205089569092,True\n\"BRD7\",0.24906224352972847,3.19008838473117,-0.9819570183753967,False\n\"TMEM208\",0.8501164514677865,2.6506047962283876,-0.6637696027755737,False\n\"DPEP2\",0.5275195918764387,3.0412005951364733,1.4059056043624878,False\n\"PSMD7\",1.4038766472680229,2.1638531531899914,0.003160706255584955,False\n\"GABARAPL2\",2.030773697921208,1.5607095132755997,-0.25290602445602417,False\n\"C16orf74\",0.4480143519810268,3.056765224247439,0.07905567437410355,False\n\"IRF8\",0.5049569334302629,3.1696558103814794,1.5297493934631348,True\n\"GLOD4\",0.5516447394234794,2.976833097668488,0.49193236231803894,False\n\"SERPINF1\",0.35220787933894565,3.0890103539471983,-0.7175406217575073,False\n\"PSMB6\",2.2829623106547765,1.3251462531754836,-0.19650287926197052,False\n\"RNF167\",1.0312209701538086,2.5099021337051965,0.19026349484920502,False\n\"CLEC10A\",0.7441702171734401,3.1773799703520935,6.258185386657715,True\n\"TMEM256\",0.9545732556070601,2.54198221782856,-0.9097875952720642,False\n\"EIF4A1\",3.622343404974256,0.31811460098691197,-0.842484712600708,False\n\"LSMD1\",1.3186237233025686,2.2217600062490663,-1.3185251951217651,False\n\"TRAPPC1\",2.822690201146262,0.8819645309100325,-0.9558647274971008,False\n\"NCOR1\",1.2070997834205628,2.3179943715682136,0.26645800471305847,False\n\"TNFRSF13B\",0.28890597139086044,3.345732288271596,0.6079807877540588,False\n\"SNORD3B-2\",0.26696110793522426,3.221208837092627,-0.6640545725822449,False\n\"IFT20\",0.8111377028056553,2.670073632416282,-0.3284872770309448,False\n\"UNC119\",1.0415094583375113,2.499818976682488,-0.001301950658671558,False\n\"NSRP1\",0.5396455366270883,2.9475280417771756,0.07582114636898041,False\n\"CCL5\",1.1013995705332076,2.8478928737961957,6.792006015777588,True\n\"CCL4\",0.1806995838029044,3.4532010856590265,0.5952651500701904,False\n\"GGNBP2\",0.9487677652495248,2.5455469032268185,-0.8428857922554016,False\n\"CWC25\",0.40028065511158534,3.081570445083094,-0.806719183921814,False\n\"MIEN1\",1.426257335458483,2.1357348730881753,-0.3565492331981659,False\n\"CCR7\",0.5681145552226475,3.0379638708079306,1.359946370124817,False\n\"NT5C3B\",0.13863806247711183,3.22747431124715,-1.2776302099227905,False\n\"DNAJC7\",0.9331264993122645,2.5410453384673524,-0.9273708462715149,False\n\"CCR10\",0.1108936790057591,3.612490398034411,1.9169169664382935,True\n\"COA3\",0.8571394637652806,2.6493598220411614,-0.6852099299430847,False\n\"SLC25A39\",0.9169763238089426,2.5680784815214026,-0.42001479864120483,False\n\"ABI3\",1.6107087775639126,2.061000545116925,0.5885447263717651,False\n\"PHB\",1.5484174258368355,2.0109907055325333,0.0,False\n\"SUPT4H1\",1.835239235333034,1.7239034438957301,-1.2122483253479004,False\n\"VMP1\",0.6176909235545567,2.9160707618254262,1.1683127880096436,False\n\"PSMC5\",1.6979237767628261,1.8689210557218192,0.8568484783172607,False\n\"CD79B\",1.335678538254329,2.474470654803971,3.9768106937408447,True\n\"DDX5\",3.427597258431571,0.44473973976606224,-0.44915130734443665,False\n\"CD300A\",0.8749203951018197,2.7276036211613053,2.573941469192505,True\n\"SUMO2\",2.8082289954594204,0.86314143367416,-1.0844461917877197,False\n\"ACOX1\",0.07262007066181728,3.3391575254827672,-0.3509746193885803,False\n\"UBALD2\",0.39571391378130233,3.086186104539514,-0.7513935565948486,False\n\"ST6GALNAC1\",0.03464632511138916,3.4462553316689273,0.5376349687576294,False\n\"SRSF2\",1.9151688402039664,1.6577228768188745,0.47658801078796387,False\n\"SEPT9\",1.2919991755485534,2.2744533539365155,-0.4506640136241913,False\n\"DCXR\",1.3798010148320878,2.177184438311973,0.17370441555976868,False\n\"CD7\",1.29038943869727,2.507973829010733,3.395426034927368,True\n\"ANKRD12\",0.8216382837295533,2.66114185179545,-0.4823058247566223,False\n\"PSMG2\",0.8190717816352844,2.6542091840076822,-0.6016966700553894,False\n\"TTC39C\",0.5427869493620736,2.99837517020116,0.797814667224884,False\n\"RNF138\",0.306676995073046,3.1767744772718745,-1.1179616451263428,False\n\"ACAA2\",0.48080614839281355,2.9721650355712104,-1.008094072341919,False\n\"NOP56\",0.7246573805809021,2.7437968114646294,-0.7425371408462524,False\n\"IDH3B\",1.0446737861633302,2.453920613779574,-0.8733046650886536,False\n\"C20orf27\",1.6007519773074559,1.9665105232146776,-0.5234684944152832,False\n\"PCNA\",0.5862160124097552,3.0054421182283693,0.8981603384017944,False\n\"DTD1\",0.6217433295931135,2.84821110259958,-0.21755990386009216,False\n\"NAA20\",0.725691396508898,2.745415635296214,-0.7163992524147034,False\n\"CST3\",2.4901576014927453,1.9259647344594246,4.278759479522705,True\n\"CST7\",0.8640804076194764,2.855755926249558,2.8692381381988525,True\n\"APMAP\",0.5536924280439104,2.957822274723249,0.22199203073978424,False\n\"EIF2S2\",1.6079588062422616,1.9606558379870191,-0.5923698544502258,False\n\"DYNLRB1\",1.7748636896269663,1.814717253134203,0.0834740549325943,False\n\"RBM39\",1.4892547130584717,2.0513054995678712,-1.436632513999939,False\n\"TOP1\",0.7303502157756261,2.7408506171971974,-0.7901069521903992,False\n\"YWHAB\",2.647990154879434,1.0205313110758139,-0.00930843222886324,False\n\"STK4\",0.6256038151468549,2.866859343111854,0.16328619420528412,False\n\"CD40\",0.31097863640104023,3.162470080393206,0.16298498213291168,False\n\"ZFAS1\",2.3486714036124092,1.326688016061616,-0.1850188821554184,False\n\"BCAS4\",0.2786700984409877,3.2849840520721623,-0.01257624477148056,False\n\"PSMA7\",3.319489393915449,0.5015205471537102,-0.2727741301059723,False\n\"ADRM1\",1.5278737575667245,2.0060961798587895,-0.05760160833597183,False\n\"PPDPF\",3.281096433912005,0.5611161456751806,-0.08765339851379395,False\n\"RGS19\",2.1279496557371957,1.527910199934606,-0.49954116344451904,False\n\"GZMM\",1.0099424144199916,2.6794868526182127,3.412128448486328,True\n\"PRSS57\",0.08252408197947911,3.7968947524402057,3.4469528198242188,True\n\"CFD\",1.2352295902797155,2.696244388393237,6.496248245239258,True\n\"CNN2\",1.8224455891336713,1.8333902777508315,0.34989890456199646,False\n\"HMHA1\",1.0577020171710423,2.454463492947017,-0.8629907369613647,False\n\"GPX4\",2.7614921736717224,0.9494776394675492,-0.49467945098876953,False\n\"C19orf24\",1.1449876771654401,2.373465790163825,-0.7766575813293457,False\n\"ABHD17A\",0.41261717898505074,3.0879841268355372,-0.7298415303230286,False\n\"TIMM13\",1.5215113874844142,2.0354400299559323,0.287733793258667,False\n\"GNG7\",0.4967505850110735,3.045439968675094,-0.06647885590791702,False\n\"SLC39A3\",0.4524819002832685,2.9719183851831987,-1.0112636089324951,False\n\"AES\",2.055302165235792,1.6936659738532043,0.7468628883361816,False\n\"S1PR4\",1.692176855632237,1.953677942401159,-0.6744897365570068,False\n\"C19orf77\",0.06244105236870902,3.6118648571256156,1.911726713180542,True\n\"MATK\",0.3906022116116115,3.187216783243197,0.45961153507232666,False\n\"TMIGD2\",0.32168435164860315,3.218144053138174,0.8303215503692627,False\n\"SH3GL1\",0.20087484087262836,3.2439662957305617,-1.1407932043075562,False\n\"NDUFA11\",2.5167468558038983,1.149988825777601,0.8750220537185669,False\n\"CLPP\",0.9730332446098328,2.539993741562218,-0.9471071362495422,False\n\"ALKBH7\",1.6917789271899633,1.8646706221168927,-1.7219793796539307,False\n\"STXBP2\",1.694889534200941,1.9980211338177163,2.698835611343384,True\n\"PRAM1\",0.4840992430278233,3.061402011177607,0.13864043354988098,False\n\"EIF3G\",2.695952774797167,0.9951579794744126,-0.1826348751783371,False\n\"ICAM4\",0.4749988055229187,3.1621921166520788,1.433837652206421,False\n\"ICAM3\",2.453131310939789,1.2123284357757655,-1.0368387699127197,False\n\"S1PR5\",0.15060103859220234,3.3814580038221638,0.0,False\n\"ILF3-AS1\",0.7054658828462873,2.8405309613536533,0.8193523287773132,False\n\"ILF3\",0.5308586181913103,2.8962605441557594,-0.6521413326263428,False\n\"ACP5\",0.5303129165513175,2.9930761110519413,0.7225717306137085,False\n\"C19orf43\",3.25658225774765,0.5364403892661752,-0.1643032431602478,False\n\"JUNB\",3.7901645251682825,0.316238569419281,-0.8483121991157532,False\n\"PRDX2\",1.7645213208879744,1.858968715222697,0.7148494720458984,False\n\"CALR\",1.3290880775451661,2.254487291795686,1.162619948387146,False\n\"LYL1\",0.9563667801448277,2.6056230709740698,0.2846193313598633,False\n\"C19orf53\",2.228918822152274,1.3852338816065533,-1.5723987817764282,False\n\"DDX39A\",0.9464807830538069,2.578392564158921,-0.22644084692001343,False\n\"DNAJB1\",0.6687950457845415,2.870630568093778,0.24030452966690063,False\n\"NDUFB7\",1.8374175548553466,1.7236877196558829,-1.215326189994812,False\n\"TPM4\",0.9554985867227827,2.546753693118915,-0.82023686170578,False\n\"BST2\",2.2237748670578004,1.4074733152959267,-1.405168890953064,False\n\"IFI30\",1.4106743250574385,2.3598219176986577,2.510138750076294,True\n\"LSM4\",1.8264865061214992,1.762657844769331,-0.6593043804168701,False\n\"LRRC25\",1.1167027596064976,2.528556031671244,1.6975390911102295,True\n\"COPE\",2.990876786708832,0.7581176602903894,0.5242888331413269,False\n\"UQCRFS1\",2.1159608466284614,1.4791192411398744,-0.8664258122444153,False\n\"U2AF1L4\",0.5218728055272783,2.974725334467154,0.462003618478775,False\n\"HCST\",2.1652586228506907,1.5675971058267806,-0.20111462473869324,False\n\"TYROBP\",2.3660371429579596,1.806212533315188,3.3867719173431396,True\n\"POLR2I\",0.835576035295214,2.6638993562133986,-0.43481749296188354,False\n\"SPINT2\",1.0317874605315072,2.509554928580431,0.18366709351539612,False\n\"PPP1R14A\",0.4159930263246809,3.2532410194071963,1.251011610031128,False\n\"SERTAD3\",0.35232430900846207,3.085454079799817,-0.7601679563522339,False\n\"CD79A\",0.7608414111818587,3.382065228465363,9.56307601928711,True\n\"RABAC1\",1.8045469270433698,1.7858469616716552,-0.32844439148902893,False\n\"PAFAH1B3\",0.6875877237319946,2.772837167790801,-1.7568938732147217,False\n\"ZNF428\",0.8841501631055559,2.6306985117349675,0.7552334070205688,False\n\"CALM3\",1.856072565146855,1.7954805380654666,-0.19099347293376923,False\n\"AP2S1\",2.706616437775748,1.034152338549398,0.08373745530843735,False\n\"FTL\",4.454551051684788,0.025404013925400155,-1.7517262697219849,False\n\"SNRNP70\",0.7736265073503766,2.6933479864368017,-1.5570942163467407,False\n\"CD37\",3.511182520389557,0.3889651977709266,-0.6224027872085571,False\n\"FLT3LG\",0.6243492075375148,2.9516206954676707,1.8943358659744263,True\n\"NOSIP\",2.0801830649375916,1.6914672586162376,0.7303296327590942,False\n\"FUZ\",0.10724892105375017,3.318005736170058,-0.5264747738838196,False\n\"SPIB\",0.42876665217535836,3.172395062036034,1.5649499893188477,True\n\"JOSD2\",0.7369095052991594,2.7451994056458484,-0.7198905348777771,False\n\"CD33\",0.8506120961053031,2.6911529660162996,0.034530218690633774,False\n\"NKG7\",1.1686289865630013,2.8233462110279635,6.400406360626221,True\n\"FPR1\",0.3064187911578587,3.2248570169781057,-0.626787543296814,False\n\"ZNF600\",0.03999556234904698,3.4740517967015125,0.7682672142982483,False\n\"ZNF524\",0.5976441264152527,2.8791170006224784,-0.8955675959587097,False\n\"CTD-3138B18.5\",0.048145558834075924,3.3418048508666027,-0.3290092945098877,False\n\"ATP6V1E1\",1.0629272035190038,2.464006032760394,-0.6816962361335754,False\n\"BID\",1.7931452764783586,1.8393706826537046,0.435226708650589,False\n\"MRPL40\",0.8259185668400356,2.7255363568028974,0.6266633868217468,False\n\"UFD1L\",1.0518067049980164,2.453920650372201,-0.8733039498329163,False\n\"COMT\",1.3008981384549823,2.206929746524636,-1.5627800226211548,False\n\"DGCR6L\",0.8596877070835659,2.654527350536305,-0.5962173938751221,False\n\"SDF2L1\",1.1725469865117755,2.40114649171226,1.6359761953353882,True\n\"IGLL5\",0.4182624476296561,3.6736587063238733,8.00640869140625,True\n\"IGLL1\",0.03814483574458531,3.8329614999459465,3.7462050914764404,True\n\"CHCHD10\",1.6264535624640328,1.9636597449862905,-0.5570181012153625,False\n\"SMARCB1\",1.0624028178623743,2.5237692032176917,0.45371779799461365,False\n\"MIF\",1.8698141769000463,1.745761250672554,-0.9003832936286926,False\n\"ASCC2\",0.36864139182226996,3.090039723107883,-0.7052021026611328,False\n\"PIK3IP1\",0.6333167733464922,2.9331575274605077,1.5172693729400635,True\n\"HMOX1\",0.8037361277852739,2.876400650217808,3.224771022796631,True\n\"EIF3D\",2.197960192135402,1.4083684151285283,-1.3984380960464478,False\n\"IL2RB\",0.2014488993372236,3.343736181879373,-0.3129846751689911,False\n\"LGALS2\",0.7938459042140416,3.016556624575689,5.638465404510498,True\n\"EIF3L\",2.969286025592259,0.7963243898596808,0.6429697275161743,False\n\"ADSL\",0.7988648244312831,2.672640308144182,-0.284285306930542,False\n\"RBX1\",1.6827683806419373,1.887271751688752,-1.4559962749481201,False\n\"TTC38\",0.17977709089006697,3.339751939734414,-0.346042662858963,False\n\"TYMP\",1.9089728954860141,1.8813097525520956,1.03360915184021,False\n\"CCT8\",1.504336954184941,2.0209129161724997,0.11677031219005585,False\n\"SOD1\",2.6116693735122682,1.0862257675778242,0.4394535422325134,False\n\"PAXBP1\",0.05895839384623936,3.399797897897179,0.1521693766117096,False\n\"ATP5O\",2.7453843239375524,0.9545133806869737,-0.46028006076812744,False\n\"MRPS6\",0.9709907872336251,2.5545194480196622,-0.6744897365570068,False\n\"TTC3\",0.42875579629625593,3.043738477490395,-0.08834376186132431,False\n\"U2AF1\",2.212861317907061,1.4066743372050075,-1.4111768007278442,False\n\"CSTB\",2.812350261892591,0.9215837236658532,-0.6852241158485413,False\n\"SUMO3\",1.1913197013310024,2.342881429343698,0.6763486862182617,False\n\"ITGB2\",2.850207292011806,0.9088416474784733,-0.7722658514976501,False\n\"S100B\",0.24995170593261717,3.4242507353331937,1.4100645780563354,False\n\"PRMT2\",1.1855203751155308,2.3803963652038784,1.2942209243774414,False\n\"MT-ND3\",0.8427824320111956,2.6685150092724306,-0.35532906651496887,False\n\n\n\"\",\"x\"\n\"1\",\"LYZ\"\n\"2\",\"GNLY\"\n\"3\",\"S100A9\"\n\"4\",\"FTL\"\n\"5\",\"FTH1\"\n\"6\",\"S100A8\"\n\"7\",\"HLA-DRA\"\n\"8\",\"CST3\"\n\"9\",\"CD74\"\n\"10\",\"NKG7\"\n\"11\",\"GZMB\"\n\"12\",\"IGLL5\"\n\"13\",\"HLA-DPB1\"\n\"14\",\"FCER1A\"\n\"15\",\"CCL4\"\n\"16\",\"HLA-DRB1\"\n\"17\",\"PPBP\"\n\"18\",\"FCGR3A\"\n\"19\",\"PF4\"\n\"20\",\"GNG11\"\n\"21\",\"CCL5\"\n\"22\",\"LST1\"\n\"23\",\"HLA-DPA1\"\n\"24\",\"FCN1\"\n\"25\",\"CD79A\"\n\"26\",\"CCL3\"\n\"27\",\"FCER1G\"\n\"28\",\"FGFBP2\"\n\"29\",\"TYROBP\"\n\"30\",\"GZMH\"\n\"31\",\"HLA-DQA1\"\n\"32\",\"IFITM3\"\n\"33\",\"GZMK\"\n\"34\",\"AIF1\"\n\"35\",\"APOBEC3B\"\n\"36\",\"CLEC10A\"\n\"37\",\"IFI27\"\n\"38\",\"AL928768.3\"\n\"39\",\"PRDX1\"\n\"40\",\"S100B\"\n\"41\",\"STMN1\"\n\"42\",\"GIMAP5\"\n\"43\",\"ACTB\"\n\"44\",\"C1QA\"\n\"45\",\"CLU\"\n\"46\",\"WARS\"\n\"47\",\"LGALS2\"\n\"48\",\"HLA-DQB1\"\n\"49\",\"STK17A\"\n\"50\",\"SAT1\"\n\"51\",\"CST7\"\n\"52\",\"G0S2\"\n\"53\",\"RALY\"\n\"54\",\"TUBA1B\"\n\"55\",\"GIMAP4\"\n\"56\",\"PRF1\"\n\"57\",\"IL8\"\n\"58\",\"YWHAB\"\n\"59\",\"C1QB\"\n\"60\",\"TUBB1\"\n\"61\",\"ATP5H\"\n\"62\",\"MYL9\"\n\"63\",\"CTSS\"\n\"64\",\"CD9\"\n\"65\",\"GSTO1\"\n\"66\",\"COTL1\"\n\"67\",\"LGALS1\"\n\"68\",\"CD1C\"\n\"69\",\"TREML1\"\n\"70\",\"HLA-DRB5\"\n\"71\",\"GP9\"\n\"72\",\"GZMA\"\n\"73\",\"NPC2\"\n\"74\",\"LTB\"\n\"75\",\"SPARC\"\n\"76\",\"HLA-DMA\"\n\"77\",\"MZB1\"\n\"78\",\"S100A4\"\n\"79\",\"HMGB2\"\n\"80\",\"KIAA0101\"\n\"81\",\"APOBEC3A\"\n\"82\",\"IGSF6\"\n\"83\",\"SPON2\"\n\"84\",\"CLIC3\"\n\"85\",\"ANXA1\"\n\"86\",\"MS4A6A\"\n\"87\",\"VMO1\"\n\"88\",\"GAPDH\"\n\"89\",\"IL1B\"\n\"90\",\"TMEM40\"\n\"91\",\"GSTP1\"\n\"92\",\"TCL1A\"\n\"93\",\"C10orf32\"\n\"94\",\"TNFRSF13B\"\n\"95\",\"SDPR\"\n\"96\",\"CDKN1C\"\n\"97\",\"RETN\"\n\"98\",\"IGJ\"\n\"99\",\"H2AFY\"\n\"100\",\"TMEM219\"\n\"101\",\"IL7R\"\n\"102\",\"TALDO1\"\n\"103\",\"TYMS\"\n\"104\",\"MS4A7\"\n\"105\",\"TMSB4X\"\n\"106\",\"S100A11\"\n\"107\",\"PYCARD\"\n\"108\",\"CXCL2\"\n\"109\",\"ABI3\"\n\"110\",\"HOPX\"\n\"111\",\"SWAP70\"\n\"112\",\"SPTSSB\"\n\"113\",\"CFD\"\n\"114\",\"S100A12\"\n\"115\",\"PSAP\"\n\"116\",\"ISG15\"\n\"117\",\"ABT1\"\n\"118\",\"AKR1C3\"\n\"119\",\"BIRC5\"\n\"120\",\"NCF2\"\n\"121\",\"VAMP8\"\n\"122\",\"CDA\"\n\"123\",\"TK1\"\n\"124\",\"PHACTR4\"\n\"125\",\"CEBPB\"\n\"126\",\"HES1\"\n\"127\",\"CD37\"\n\"128\",\"PRKCD\"\n\"129\",\"SRSF3\"\n\"130\",\"ANXA5\"\n\"131\",\"PLA2G12A\"\n\"132\",\"TMEM176A\"\n\"133\",\"GPX1\"\n\"134\",\"FCGR2B\"\n\"135\",\"ARPC1B\"\n\"136\",\"TYMP\"\n\"137\",\"CTD-2267D19.2\"\n\"138\",\"CAT\"\n\"139\",\"HBA1\"\n\"140\",\"SNX3\"\n\"141\",\"NRGN\"\n\"142\",\"TIMP1\"\n\"143\",\"IRF7\"\n\"144\",\"RAB32\"\n\"145\",\"RHOG\"\n\"146\",\"PRSS57\"\n\"147\",\"LYPD2\"\n\"148\",\"TNFSF13B\"\n\"149\",\"CKB\"\n\"150\",\"LILRB2\"\n\"151\",\"ZWINT\"\n\"152\",\"RP11-290F20.3\"\n\"153\",\"CA2\"\n\"154\",\"SCPEP1\"\n\"155\",\"NT5C3A\"\n\"156\",\"TPM4\"\n\"157\",\"ITM2C\"\n\"158\",\"CRIP2\"\n\"159\",\"XCL1\"\n\"160\",\"XCL2\"\n\"161\",\"IFIT2\"\n\"162\",\"MCM5\"\n\"163\",\"SLC39A3\"\n\"164\",\"GCA\"\n\"165\",\"RP1-313I6.12\"\n\"166\",\"TUBB\"\n\"167\",\"HN1\"\n\"168\",\"H2AFZ\"\n\"169\",\"CCDC50\"\n\"170\",\"CCL4L1\"\n\"171\",\"ATP5O\"\n\"172\",\"RBP7\"\n\"173\",\"GMNN\"\n\"174\",\"KRT1\"\n\"175\",\"SLC25A11\"\n\"176\",\"CCT7\"\n\"177\",\"ARHGDIA\"\n\"178\",\"RP5-887A10.1\"\n\"179\",\"ID1\"\n\"180\",\"UBXN1\"\n\"181\",\"GMPR\"\n\"182\",\"UBE2D3\"\n\"183\",\"HIST1H4C\"\n\"184\",\"FKBP2\"\n\"185\",\"FPR1\"\n\"186\",\"PGRMC1\"\n\"187\",\"UBB\"\n\"188\",\"IFNGR2\"\n\"189\",\"COPS6\"\n\"190\",\"COQ7\"\n\"191\",\"PSMA7\"\n\"192\",\"RP11-291B21.2\"\n\"193\",\"RBM3\"\n\"194\",\"MED30\"\n\"195\",\"PRELID1\"\n\"196\",\"PPP1R14A\"\n\"197\",\"NIT1\"\n\"198\",\"TNFRSF17\"\n\"199\",\"ATP6V0E1\"\n\"200\",\"ITGA2B\"\n\"201\",\"ZNF263\"\n\"202\",\"CHTF8\"\n\"203\",\"C1orf162\"\n\"204\",\"PTAFR\"\n\"205\",\"VPREB3\"\n\"206\",\"MANBA\"\n\"207\",\"HLA-DQA2\"\n\"208\",\"RARRES3\"\n\"209\",\"PPIB\"\n\"210\",\"SNRNP25\"\n\"211\",\"ARL6IP5\"\n\"212\",\"ASF1B\"\n\"213\",\"PRPF19\"\n\"214\",\"KLRC1\"\n\"215\",\"SH3BP1\"\n\"216\",\"RGS1\"\n\"217\",\"VPS29\"\n\"218\",\"ABRACL\"\n\"219\",\"HNRNPA2B1\"\n\"220\",\"SERPINA1\"\n\"221\",\"S100A6\"\n\"222\",\"MT-CO2\"\n\"223\",\"TRAPPC3\"\n\"224\",\"FERMT3\"\n\"225\",\"APOBEC3H\"\n\"226\",\"GPR183\"\n\"227\",\"LGALS3BP\"\n\"228\",\"PGM1\"\n\"229\",\"FAM96B\"\n\"230\",\"PSMB8\"\n\"231\",\"HSP90AA1\"\n\"232\",\"TMSB10\"\n\"233\",\"LILRA3\"\n\"234\",\"IL32\"\n\"235\",\"SRSF6\"\n\"236\",\"GNS\"\n\"237\",\"SF3B5\"\n\"238\",\"IDH2\"\n\"239\",\"ANAPC11\"\n\"240\",\"CENPN\"\n\"241\",\"IFI35\"\n\"242\",\"NCR3\"\n\"243\",\"RRM2\"\n\"244\",\"BLOC1S1\"\n\"245\",\"ERH\"\n\"246\",\"C19orf59\"\n\"247\",\"SRGN\"\n\"248\",\"MRPL23\"\n\"249\",\"SLA\"\n\"250\",\"MRP63\"\n\"251\",\"PITPNA-AS1\"\n\"252\",\"EPN1\"\n\"253\",\"DCAF5\"\n\"254\",\"IDH3G\"\n\"255\",\"PTCRA\"\n\"256\",\"C14orf166\"\n\"257\",\"RAMP1\"\n\"258\",\"RABL6\"\n\"259\",\"TIGIT\"\n\"260\",\"PCNA\"\n\"261\",\"CD160\"\n\"262\",\"MS4A4A\"\n\"263\",\"SIVA1\"\n\"264\",\"LSM6\"\n\"265\",\"RBM17\"\n\"266\",\"RGS16\"\n\"267\",\"ZNF185\"\n\"268\",\"SNX9\"\n\"269\",\"EGFL7\"\n\"270\",\"SELL\"\n\"271\",\"RBCK1\"\n\"272\",\"ACP1\"\n\"273\",\"PMVK\"\n\"274\",\"CAPZA2\"\n\"275\",\"PRKCB\"\n\"276\",\"SH3BGRL3\"\n\"277\",\"COX5A\"\n\"278\",\"CWC15\"\n\"279\",\"ISOC2\"\n\"280\",\"GINS2\"\n\"281\",\"IFIT1\"\n\"282\",\"GSTA4\"\n\"283\",\"CD14\"\n\"284\",\"KARS\"\n\"285\",\"MANF\"\n\"286\",\"STAMBP\"\n\"287\",\"NAA20\"\n\"288\",\"FEN1\"\n\"289\",\"FCRL2\"\n\"290\",\"TNFSF10\"\n\"291\",\"ACRBP\"\n\"292\",\"CLEC2B\"\n\"293\",\"CHI3L2\"\n\"294\",\"DDT\"\n\"295\",\"ACAP1\"\n\"296\",\"IL18RAP\"\n\"297\",\"PCNP\"\n\"298\",\"HAGH\"\n\"299\",\"BIK\"\n\"300\",\"GUSB\"\n\"301\",\"SRM\"\n\"302\",\"CHMP4A\"\n\"303\",\"SOX4\"\n\"304\",\"IRF8\"\n\"305\",\"IFITM2\"\n\"306\",\"C16orf13\"\n\"307\",\"VAPA\"\n\"308\",\"SLC25A5\"\n\"309\",\"ENHO\"\n\"310\",\"EIF3H\"\n\"311\",\"HNRNPH3\"\n\"312\",\"NDUFA12\"\n\"313\",\"CTD-2006K23.1\"\n\"314\",\"PPM1N\"\n\"315\",\"IFI6\"\n\"316\",\"IFNG\"\n\"317\",\"NDFIP1\"\n\"318\",\"AHNAK\"\n\"319\",\"SERPINF1\"\n\"320\",\"BLK\"\n\"321\",\"SRSF7\"\n\"322\",\"DSCR3\"\n\"323\",\"HNRNPF\"\n\"324\",\"SEC61B\"\n\"325\",\"GPBAR1\"\n\"326\",\"CSNK2B\"\n\"327\",\"HMOX1\"\n\"328\",\"SEPT5\"\n\"329\",\"NIT2\"\n\"330\",\"HNRNPM\"\n\"331\",\"IER3\"\n\"332\",\"SP140\"\n\"333\",\"RP11-295P9.3\"\n\"334\",\"ZBP1\"\n\"335\",\"SMIM7\"\n\"336\",\"EREG\"\n\"337\",\"SURF1\"\n\"338\",\"BANK1\"\n\"339\",\"MALAT1\"\n\"340\",\"SLC40A1\"\n\"341\",\"ALDH2\"\n\"342\",\"POLR2G\"\n\"343\",\"TMEM141\"\n\"344\",\"PTGDS\"\n\"345\",\"RGS18\"\n\"346\",\"C5orf15\"\n\"347\",\"NSA2\"\n\"348\",\"ACTG1\"\n\"349\",\"SDHB\"\n\"350\",\"IFI30\"\n\"351\",\"UBE2D2\"\n\"352\",\"IL6\"\n\"353\",\"CD79B\"\n\"354\",\"PHLDA2\"\n\"355\",\"NCOA4\"\n\"356\",\"C17orf62\"\n\"357\",\"TMEM208\"\n\"358\",\"HIST1H2AC\"\n\"359\",\"S1PR4\"\n\"360\",\"SULF2\"\n\"361\",\"SARM1\"\n\"362\",\"PSMG2\"\n\"363\",\"VIM\"\n\"364\",\"ATP5D\"\n\"365\",\"XBP1\"\n\"366\",\"MS4A1\"\n\"367\",\"NDUFB9\"\n\"368\",\"BIN2\"\n\"369\",\"HES4\"\n\"370\",\"OAZ1\"\n\"371\",\"RUFY1\"\n\"372\",\"NOP58\"\n\"373\",\"TNNT1\"\n\"374\",\"CXCR3\"\n\"375\",\"CLIC2\"\n\"376\",\"FCRLA\"\n\"377\",\"FABP5\"\n\"378\",\"CFP\"\n\"379\",\"TPM1\"\n\"380\",\"ACOT7\"\n\"381\",\"CCND2\"\n\"382\",\"SH3BGRL\"\n\"383\",\"RBM39\"\n\"384\",\"SPRY1\"\n\"385\",\"LY6G6F\"\n\"386\",\"ANXA2\"\n\"387\",\"IGFBP7\"\n\"388\",\"GFI1B\"\n\"389\",\"PFN1\"\n\"390\",\"P2RY13\"\n\"391\",\"TRAF3IP3\"\n\"392\",\"PDLIM1\"\n\"393\",\"RP11-164H13.1\"\n\"394\",\"GFER\"\n\"395\",\"CMTM5\"\n\"396\",\"EZH2\"\n\"397\",\"PTTG1\"\n\"398\",\"SUMO3\"\n\"399\",\"FFAR2\"\n\"400\",\"LILRA4\"\n\"401\",\"CTSW\"\n\"402\",\"RAD51\"\n\"403\",\"NDUFA11\"\n\"404\",\"CCNA2\"\n\"405\",\"KCTD10\"\n\"406\",\"KIR3DL2\"\n\"407\",\"UBE2J1\"\n\"408\",\"CD8B\"\n\"409\",\"C6orf25\"\n\"410\",\"KIFC1\"\n\"411\",\"RP11-367G6.3\"\n\"412\",\"FCER2\"\n\"413\",\"DHRS9\"\n\"414\",\"MYBL2\"\n\"415\",\"TSPAN15\"\n\"416\",\"TRPM4\"\n\"417\",\"ARHGDIB\"\n\"418\",\"LINC00926\"\n\"419\",\"MAX\"\n\"420\",\"BATF3\"\n\"421\",\"EWSR1\"\n\"422\",\"MCM7\"\n\"423\",\"CENPM\"\n\"424\",\"IFFO1\"\n\"425\",\"KLRD1\"\n\"426\",\"ZNF703\"\n\"427\",\"SNX29P2\"\n\"428\",\"F13A1\"\n\"429\",\"PID1\"\n\"430\",\"IL13RA1\"\n\"431\",\"FAM212A\"\n\"432\",\"HAVCR2\"\n\"433\",\"AC022182.3\"\n\"434\",\"PPP6C\"\n\"435\",\"OSM\"\n\"436\",\"RHOC\"\n\"437\",\"KCNG1\"\n\"438\",\"CD72\"\n\"439\",\"NAGA\"\n\"440\",\"UBE2Q1\"\n\"441\",\"LPGAT1\"\n\"442\",\"MYCL\"\n\"443\",\"HBP1\"\n\"444\",\"KIR2DL3\"\n\"445\",\"FAM13A\"\n\"446\",\"STAP1\"\n\"447\",\"GGNBP2\"\n\"448\",\"FCRL5\"\n\"449\",\"RXRA\"\n\"450\",\"SH2D1B\"\n\"451\",\"UBA5\"\n\"452\",\"NFE2\"\n\"453\",\"ACSM3\"\n\"454\",\"CCR10\"\n\"455\",\"MAP3K7CL\"\n\"456\",\"FH\"\n\"457\",\"RCE1\"\n\"458\",\"NCOR2\"\n\"459\",\"TPPP3\"\n\"460\",\"FKBP3\"\n\"461\",\"GPR56\"\n\"462\",\"PPIL2\"\n\"463\",\"PILRA\"\n\"464\",\"AP001189.4\"\n\"465\",\"BASP1\"\n\"466\",\"DHRS4\"\n\"467\",\"NEIL1\"\n\"468\",\"TSC22D1\"\n\"469\",\"FCGRT\"\n\"470\",\"FBXO41\"\n\"471\",\"SPIB\"\n\"472\",\"DDX17\"\n\"473\",\"FCRL6\"\n\"474\",\"LDHB\"\n\"475\",\"LMNA\"\n\"476\",\"PLBD1\"\n\"477\",\"SLC4A10\"\n\"478\",\"AC079767.4\"\n\"479\",\"TSPAN4\"\n\"480\",\"CLEC7A\"\n\"481\",\"PLD4\"\n\"482\",\"HIST1H1B\"\n\"483\",\"TNNI2\"\n\"484\",\"CPNE2\"\n\"485\",\"RLN2\"\n\"486\",\"ALDH1A1\"\n\"487\",\"FRAT1\"\n\"488\",\"CLYBL\"\n\"489\",\"HLA-DMB\"\n\"490\",\"COCH\"\n\"491\",\"HCK\"\n\"492\",\"SYCE1L\"\n\"493\",\"TNFAIP8L1\"\n\"494\",\"MMD\"\n\"495\",\"TMEM176B\"\n\"496\",\"PRSS23\"\n\"497\",\"ZNF212\"\n\"498\",\"HCAR3\"\n\"499\",\"H1F0\"\n\"500\",\"VPS37C\"\n\"501\",\"TMEM140\"\n\"502\",\"SPI1\"\n\"503\",\"MAPK7\"\n\"504\",\"CD38\"\n\"505\",\"CTBP2\"\n\"506\",\"FAM43A\"\n\"507\",\"CACNA2D3\"\n\"508\",\"CHST2\"\n\"509\",\"RP13-188A5.1\"\n\"510\",\"RAB11B-AS1\"\n\"511\",\"SERPINE2\"\n\"512\",\"ADAP2\"\n\"513\",\"LILRB4\"\n\"514\",\"NDRG2\"\n\"515\",\"TNFRSF4\"\n\"516\",\"C12orf75\"\n\"517\",\"RP13-270P17.3\"\n\"518\",\"ZDHHC1\"\n\"519\",\"LMNB1\"\n\"520\",\"SPTLC1\"\n\"521\",\"HELQ\"\n\"522\",\"CLEC1B\"\n\"523\",\"POU2AF1\"\n\"524\",\"MKI67\"\n\"525\",\"HAPLN3\"\n\"526\",\"SERPING1\"\n\"527\",\"WARS2\"\n\"528\",\"S100A10\"\n\"529\",\"ADAM28\"\n\"530\",\"RPS2\"\n\"531\",\"FAH\"\n\"532\",\"HLA-DQB2\"\n\"533\",\"UBE2C\"\n\"534\",\"KIAA0930\"\n\"535\",\"MCM3\"\n\"536\",\"STXBP2\"\n\"537\",\"PHACTR1\"\n\"538\",\"CORO1C\"\n\"539\",\"HERC5\"\n\"540\",\"BLOC1S5\"\n\"541\",\"RP11-18H21.1\"\n\"542\",\"ZNF467\"\n\"543\",\"LILRA2\"\n\"544\",\"TOPBP1\"\n\"545\",\"PON2\"\n\"546\",\"RP11-407N17.5\"\n\"547\",\"KYNU\"\n\"548\",\"MCM4\"\n\"549\",\"KIF16B\"\n\"550\",\"RELT\"\n\"551\",\"SBNO2\"\n\"552\",\"ACSL1\"\n\"553\",\"RP11-428G5.5\"\n\"554\",\"PPP1R14B\"\n\"555\",\"EIF4A1\"\n\"556\",\"CD8A\"\n\"557\",\"ACBD3\"\n\"558\",\"SENCR\"\n\"559\",\"GSN\"\n\"560\",\"TCF7L2\"\n\"561\",\"PKIG\"\n\"562\",\"RP5-1028K7.2\"\n\"563\",\"NPDC1\"\n\"564\",\"QPCT\"\n\"565\",\"CCL3L3\"\n\"566\",\"LRRC25\"\n\"567\",\"SOD2\"\n\"568\",\"KIAA0125\"\n\"569\",\"EMR1\"\n\"570\",\"FCGR2A\"\n\"571\",\"RP11-403A21.2\"\n\"572\",\"RP11-1399P15.1\"\n\"573\",\"GRN\"\n\"574\",\"CD68\"\n\"575\",\"FOLR3\"\n\"576\",\"SYNGR1\"\n\"577\",\"EEPD1\"\n\"578\",\"CSTA\"\n\"579\",\"CAPN12\"\n\"580\",\"C9orf37\"\n\"581\",\"PPM1F\"\n\"582\",\"YBX1\"\n\"583\",\"C16orf74\"\n\"584\",\"STK32C\"\n\"585\",\"CXCR6\"\n\"586\",\"TTC38\"\n\"587\",\"NDUFAF7\"\n\"588\",\"AC011899.9\"\n\"589\",\"KLRB1\"\n\"590\",\"FCGR1A\"\n\"591\",\"ARHGAP24\"\n\"592\",\"CSF2RA\"\n\"593\",\"KCNQ1OT1\"\n\"594\",\"KHK\"\n\"595\",\"LYRM4\"\n\"596\",\"SLC7A7\"\n\"597\",\"CPVL\"\n\"598\",\"EPB41L3\"\n\"599\",\"LAMTOR4\"\n\"600\",\"SIGLEC9\"\n\"601\",\"YAE1D1\"\n\"602\",\"DIS3\"\n\"603\",\"STK3\"\n\"604\",\"CXCL16\"\n\"605\",\"DENND5A\"\n\"606\",\"MARCO\"\n\"607\",\"FBXO33\"\n\"608\",\"WDYHV1\"\n\"609\",\"WDR76\"\n\"610\",\"KLRF1\"\n\"611\",\"TUBG1\"\n\"612\",\"TMEM131\"\n\"613\",\"MOCS2\"\n\"614\",\"CEP170\"\n\"615\",\"EIF2AK4\"\n\"616\",\"CRCP\"\n\"617\",\"F2R\"\n\"618\",\"RP11-222K16.2\"\n\"619\",\"SMARCD3\"\n\"620\",\"DCTN4\"\n\"621\",\"LXN\"\n\"622\",\"TRAF4\"\n\"623\",\"BCDIN3D\"\n\"624\",\"CARD9\"\n\"625\",\"IL2RA\"\n\"626\",\"C19orf48\"\n\"627\",\"NEURL1\"\n\"628\",\"CAMK1\"\n\"629\",\"EGR1\"\n\"630\",\"TUBA1C\"\n\"631\",\"HLA-DOB\"\n\"632\",\"ADPRM\"\n\"633\",\"RP11-110A12.2\"\n\"634\",\"RCL1\"\n\"635\",\"WDR60\"\n\"636\",\"AIM2\"\n\"637\",\"XXbac-B135H6.15\"\n\"638\",\"RP11-792A8.4\"\n\"639\",\"ZSWIM8\"\n\"640\",\"SHMT1\"\n\"641\",\"DLEU1\"\n\"642\",\"RP11-554J4.1\"\n\"643\",\"PERP\"\n\"644\",\"MGLL\"\n\"645\",\"HOXB-AS1\"\n\"646\",\"ABHD5\"\n\"647\",\"ICAM4\"\n\"648\",\"CYBA\"\n\"649\",\"LGALS3\"\n\"650\",\"TNFRSF18\"\n\"651\",\"CD7\"\n\"652\",\"SLC48A1\"\n\"653\",\"ARIH2OS\"\n\"654\",\"MTMR11\"\n\"655\",\"EXOC3\"\n\"656\",\"REC8\"\n\"657\",\"LILRB1\"\n\"658\",\"SECTM1\"\n\"659\",\"MT1E\"\n\"660\",\"FAM110A\"\n\"661\",\"EMR2\"\n\"662\",\"CSF1R\"\n\"663\",\"RAB13\"\n\"664\",\"TFDP2\"\n\"665\",\"AC016629.8\"\n\"666\",\"WDR4\"\n\"667\",\"TNFRSF8\"\n\"668\",\"CDK12\"\n\"669\",\"SGK1\"\n\"670\",\"NBPF1\"\n\"671\",\"UBXN7\"\n\"672\",\"LILRA5\"\n\"673\",\"YIPF1\"\n\"674\",\"PAICS\"\n\"675\",\"STAG3\"\n\"676\",\"IFI27L1\"\n\"677\",\"SIGLEC10\"\n\"678\",\"LOH12CR2\"\n\"679\",\"CEBPD\"\n\"680\",\"PHLDA1\"\n\"681\",\"CTSL\"\n\"682\",\"ADAM17\"\n\"683\",\"TSPAN13\"\n\"684\",\"PNOC\"\n\"685\",\"GBE1\"\n\"686\",\"RPS17\"\n\"687\",\"PTMS\"\n\"688\",\"MARCH8\"\n\"689\",\"HIST1H2BC\"\n\"690\",\"ARID1B\"\n\"691\",\"HIST2H2BE\"\n\"692\",\"AP003733.1\"\n\"693\",\"RP11-25K19.1\"\n\"694\",\"LPAR5\"\n\"695\",\"DUS2\"\n\"696\",\"ACP2\"\n\"697\",\"C10orf11\"\n\"698\",\"ASRGL1\"\n\"699\",\"GMPPA\"\n\"700\",\"ZNF576\"\n\"701\",\"C12orf66\"\n\"702\",\"FCF1\"\n\"703\",\"GOLGA2\"\n\"704\",\"LIG1\"\n\"705\",\"C14orf142\"\n\"706\",\"TSPAN5\"\n\"707\",\"HSPA5\"\n\"708\",\"STX17\"\n\"709\",\"ITGAX\"\n\"710\",\"MT2A\"\n\"711\",\"LY6E\"\n\"712\",\"NR2C1\"\n\"713\",\"C17orf59\"\n\"714\",\"MFSD1\"\n\"715\",\"CDCA7L\"\n\"716\",\"AP001258.4\"\n\"717\",\"CCDC88A\"\n\"718\",\"B2M\"\n\"719\",\"MAPK8\"\n\"720\",\"UNG\"\n\"721\",\"RP11-727F15.9\"\n\"722\",\"CCDC18\"\n\"723\",\"KLF11\"\n\"724\",\"CD3D\"\n\"725\",\"LAG3\"\n\"726\",\"CLEC4E\"\n\"727\",\"RP11-138A9.2\"\n\"728\",\"SULT1A1\"\n\"729\",\"RP11-22N19.2\"\n\"730\",\"SH2B3\"\n\"731\",\"SHC1\"\n\"732\",\"USP7\"\n\"733\",\"DTYMK\"\n\"734\",\"ATF7IP2\"\n\"735\",\"SLAMF7\"\n\"736\",\"E2F3\"\n\"737\",\"FUT7\"\n\"738\",\"CDK16\"\n\"739\",\"SMPD2\"\n\"740\",\"CLEC4A\"\n\"741\",\"HSPA6\"\n\"742\",\"C2orf76\"\n\"743\",\"AC013264.2\"\n\"744\",\"CHST7\"\n\"745\",\"TCP11L2\"\n\"746\",\"APP\"\n\"747\",\"TRIB1\"\n\"748\",\"ZBTB43\"\n\"749\",\"PLAGL2\"\n\"750\",\"LPAR2\"\n\"751\",\"CTD-3138B18.5\"\n\"752\",\"TSHZ2\"\n\"753\",\"LTB4R\"\n\"754\",\"EMP3\"\n\"755\",\"DYNLL2\"\n\"756\",\"TCEA3\"\n\"757\",\"GNB1L\"\n\"758\",\"VSTM1\"\n\"759\",\"PGM2L1\"\n\"760\",\"KCNK6\"\n\"761\",\"MT-CO1\"\n\"762\",\"AGL\"\n\"763\",\"KCTD20\"\n\"764\",\"TAB2\"\n\"765\",\"TBC1D19\"\n\"766\",\"TPT1\"\n\"767\",\"ORC2\"\n\"768\",\"ANKRD32\"\n\"769\",\"CD27-AS1\"\n\"770\",\"RPL10A\"\n\"771\",\"CDK5\"\n\"772\",\"MLYCD\"\n\"773\",\"DHX8\"\n\"774\",\"LCN8\"\n\"775\",\"METTL2A\"\n\"776\",\"CHAF1A\"\n\"777\",\"SH2B2\"\n\"778\",\"RIPK3\"\n\"779\",\"CD180\"\n\"780\",\"RNF144B\"\n\"781\",\"C1orf21\"\n\"782\",\"MTIF2\"\n\"783\",\"DSE\"\n\"784\",\"THAP7-AS1\"\n\"785\",\"PRKAG2-AS1\"\n\"786\",\"GPR171\"\n\"787\",\"RAB34\"\n\"788\",\"HDGF\"\n\"789\",\"NACA\"\n\"790\",\"ROGDI\"\n\"791\",\"MAP7D3\"\n\"792\",\"EPOR\"\n\"793\",\"RNF157\"\n\"794\",\"CKS2\"\n\"795\",\"AGPAT1\"\n\"796\",\"WIPI1\"\n\"797\",\"PLEKHO2\"\n\"798\",\"SFXN3\"\n\"799\",\"RAB31\"\n\"800\",\"TPRG1L\"\n\"801\",\"TREM1\"\n\"802\",\"MYL6B\"\n\"803\",\"IFIT3\"\n\"804\",\"COG3\"\n\"805\",\"KLHDC10\"\n\"806\",\"XPNPEP1\"\n\"807\",\"ALDH3B1\"\n\"808\",\"AP1S2\"\n\"809\",\"PIBF1\"\n\"810\",\"RALGAPA2\"\n\"811\",\"FBP1\"\n\"812\",\"RGS2\"\n\"813\",\"FBXO42\"\n\"814\",\"MEGF9\"\n\"815\",\"ABI2\"\n\"816\",\"GSTM3\"\n\"817\",\"LRR1\"\n\"818\",\"GTF3C1\"\n\"819\",\"TMEM161A\"\n\"820\",\"RPL36AL\"\n\"821\",\"C11orf57\"\n\"822\",\"MAP4K5\"\n\"823\",\"B3GALT6\"\n\"824\",\"BRF1\"\n\"825\",\"AEN\"\n\"826\",\"SLFN12L\"\n\"827\",\"RPS5\"\n\"828\",\"THAP2\"\n\"829\",\"AGPAT3\"\n\"830\",\"ID3\"\n\"831\",\"EEA1\"\n\"832\",\"SIGLEC1\"\n\"833\",\"RP5-1073O3.7\"\n\"834\",\"PDCD4-AS1\"\n\"835\",\"LINC00877\"\n\"836\",\"RPS6KA4\"\n\"837\",\"FUS\"\n\"838\",\"ODC1\"\n\"839\",\"ATP5A1\"\n\"840\",\"SH3GLB1\"\n\"841\",\"BBC3\"\n\"842\",\"AQP3\"\n\"843\",\"GTF3A\"\n\"844\",\"JTB\"\n\"845\",\"CYTIP\"\n\"846\",\"CORO1B\"\n\"847\",\"ID2\"\n\"848\",\"NFKBIA\"\n\"849\",\"NDUFB10\"\n\"850\",\"REEP3\"\n\"851\",\"TNFRSF1A\"\n\"852\",\"SH3KBP1\"\n\"853\",\"TMBIM6\"\n\"854\",\"PNRC1\"\n\"855\",\"LDHA\"\n\"856\",\"STOML2\"\n\"857\",\"COMMD10\"\n\"858\",\"CLDN5\"\n\"859\",\"MYADM\"\n\"860\",\"C14orf1\"\n\"861\",\"CD2\"\n\"862\",\"KLF6\"\n\"863\",\"CISD3\"\n\"864\",\"CIR1\"\n\"865\",\"MRPS6\"\n\"866\",\"MRPL52\"\n\"867\",\"GIMAP7\"\n\"868\",\"CCND3\"\n\"869\",\"WDR1\"\n\"870\",\"UXS1\"\n\"871\",\"ATP1B3\"\n\"872\",\"PTRHD1\"\n\"873\",\"H2AFX\"\n\"874\",\"MGST2\"\n\"875\",\"LYAR\"\n\"876\",\"UBLCP1\"\n\"877\",\"UPK3A\"\n\"878\",\"BMPR2\"\n\"879\",\"LRRFIP1\"\n\"880\",\"LINC00936\"\n\"881\",\"PGK1\"\n\"882\",\"NAP1L1\"\n\"883\",\"PLEKHB2\"\n\"884\",\"CCDC12\"\n\"885\",\"LMAN2\"\n\"886\",\"MPP1\"\n\"887\",\"PNMA1\"\n\"888\",\"RAD21\"\n\"889\",\"ARRB2\"\n\"890\",\"KLRG1\"\n\"891\",\"THYN1\"\n\"892\",\"NCKAP1L\"\n\"893\",\"ASB8\"\n\"894\",\"HSP90B1\"\n\"895\",\"THAP11\"\n\"896\",\"SCGB3A1\"\n\"897\",\"FYB\"\n\"898\",\"ZUFSP\"\n\"899\",\"IFIT5\"\n\"900\",\"CSTB\"\n\"901\",\"FLNA\"\n\"902\",\"RBM4\"\n\"903\",\"TNFAIP8\"\n\"904\",\"CUTA\"\n\"905\",\"FN3KRP\"\n\"906\",\"CEPT1\"\n\"907\",\"VBP1\"\n\"908\",\"ADAM10\"\n\"909\",\"AURKB\"\n\"910\",\"PSMA2.1\"\n\"911\",\"IRF9\"\n\"912\",\"SMC4\"\n\"913\",\"NUP54\"\n\"914\",\"XRCC5\"\n\"915\",\"ANKRD22\"\n\"916\",\"BOLA1\"\n\"917\",\"GLRX5\"\n\"918\",\"ZC3H15\"\n\"919\",\"LYL1\"\n\"920\",\"RPUSD3\"\n\"921\",\"YWHAE\"\n\"922\",\"UBALD2\"\n\"923\",\"ARRDC3\"\n\"924\",\"ERICH1\"\n\"925\",\"JAKMIP1\"\n\"926\",\"SPCS2\"\n\"927\",\"YPEL5\"\n\"928\",\"ALKBH7\"\n\"929\",\"COMMD5\"\n\"930\",\"NDUFB5\"\n\"931\",\"SDF2L1\"\n\"932\",\"TUBA4A\"\n\"933\",\"NDUFB11\"\n\"934\",\"OARD1\"\n\"935\",\"LINC-PINT\"\n\"936\",\"CTNNBL1\"\n\"937\",\"JAK1\"\n\"938\",\"VAMP5\"\n\"939\",\"HTATIP2\"\n\"940\",\"CCT5\"\n\"941\",\"EI24\"\n\"942\",\"FBXO3\"\n\"943\",\"ACD\"\n\"944\",\"SMARCC2\"\n\"945\",\"ATG4C\"\n\"946\",\"ZFP36L1\"\n\"947\",\"STX11\"\n\"948\",\"FEM1B\"\n\"949\",\"SAFB2\"\n\"950\",\"SLC16A3\"\n\"951\",\"VPS28\"\n\"952\",\"RPL7L1\"\n\"953\",\"UBA2\"\n\"954\",\"XRCC6\"\n\"955\",\"CD247\"\n\"956\",\"PDZD4\"\n\"957\",\"TNFSF4\"\n\"958\",\"CLIC1\"\n\"959\",\"STUB1\"\n\"960\",\"ORAI3\"\n\"961\",\"CARD16\"\n\"962\",\"RAC2\"\n\"963\",\"ARFGAP2\"\n\"964\",\"TMEM66\"\n\"965\",\"CHCHD1\"\n\"966\",\"DNAJB14\"\n\"967\",\"MAL\"\n\"968\",\"PPP2CA\"\n\"969\",\"CCT2\"\n\"970\",\"MYO9B\"\n\"971\",\"NDUFC2\"\n\"972\",\"MVD\"\n\"973\",\"PLEKHA3\"\n\"974\",\"HMGN1\"\n\"975\",\"USP3\"\n\"976\",\"LYPLA1\"\n\"977\",\"OAZ2\"\n\"978\",\"TRADD\"\n\"979\",\"GOLGB1\"\n\"980\",\"CD19\"\n\"981\",\"ALOX5AP\"\n\"982\",\"MLX\"\n\"983\",\"PQBP1\"\n\"984\",\"PPP2R1B\"\n\"985\",\"RPN2\"\n\"986\",\"HSPD1\"\n\"987\",\"DUSP23\"\n\"988\",\"RIC3\"\n\"989\",\"TMCO1\"\n\"990\",\"C14orf119\"\n\"991\",\"KDM3B\"\n\"992\",\"RTN4\"\n\"993\",\"C6orf48\"\n\"994\",\"PRPF8\"\n\"995\",\"TBCC\"\n\"996\",\"ATP6V0B\"\n\"997\",\"NIFK\"\n\"998\",\"UQCRC1\"\n\"999\",\"ARHGEF40\"\n\"1000\",\"GDI2\"\n\"1001\",\"FAM49B\"\n\"1002\",\"ATP5SL\"\n\"1003\",\"EXOSC8\"\n\"1004\",\"FAM32A\"\n\"1005\",\"RNF126\"\n\"1006\",\"MYO1G\"\n\"1007\",\"EAF2\"\n\"1008\",\"SSBP1\"\n\"1009\",\"ALG13\"\n\"1010\",\"BCL2A1\"\n\"1011\",\"ARPC5\"\n\"1012\",\"ATP5C1\"\n\"1013\",\"PCBP1\"\n\"1014\",\"ADH5\"\n\"1015\",\"PTPN18\"\n\"1016\",\"CISD1\"\n\"1017\",\"SNRPE\"\n\"1018\",\"MCM2\"\n\"1019\",\"PPP1R18\"\n\"1020\",\"SAT2\"\n\"1021\",\"C9orf16\"\n\"1022\",\"DRAP1\"\n\"1023\",\"QRICH1\"\n\"1024\",\"AATF\"\n\"1025\",\"UBAC2\"\n\"1026\",\"PHF3\"\n\"1027\",\"AHSA1\"\n\"1028\",\"ITSN2\"\n\"1029\",\"NEAT1\"\n\"1030\",\"DHFR\"\n\"1031\",\"PSMD14\"\n\"1032\",\"PPA1\"\n\"1033\",\"TCL1B\"\n\"1034\",\"MAFB\"\n\"1035\",\"CAP1\"\n\"1036\",\"SPG7\"\n\"1037\",\"MRPL12\"\n\"1038\",\"PTGES2\"\n\"1039\",\"DHRS4L2\"\n\"1040\",\"NXT2\"\n\"1041\",\"KIF5B\"\n\"1042\",\"PMEPA1\"\n\"1043\",\"NME3\"\n\"1044\",\"TCP1\"\n\"1045\",\"PICALM\"\n\"1046\",\"GNB2\"\n\"1047\",\"FAM96A\"\n\"1048\",\"AAMP\"\n\"1049\",\"WDR45\"\n\"1050\",\"FMNL1\"\n\"1051\",\"GBP1\"\n\"1052\",\"ZNF593\"\n\"1053\",\"LMAN1\"\n\"1054\",\"SLC39A1\"\n\"1055\",\"FGR\"\n\"1056\",\"PEX16\"\n\"1057\",\"CTA-217C2.1\"\n\"1058\",\"EIF2B1\"\n\"1059\",\"NME1\"\n\"1060\",\"TGFB1\"\n\"1061\",\"CMTM7\"\n\"1062\",\"HAUS5\"\n\"1063\",\"AP3S1\"\n\"1064\",\"GANAB\"\n\"1065\",\"NUDC\"\n\"1066\",\"GIMAP2\"\n\"1067\",\"GPR42\"\n\"1068\",\"AP001053.11\"\n\"1069\",\"RP11-349A22.5\"\n\"1070\",\"AHCY\"\n\"1071\",\"DPH5\"\n\"1072\",\"MAEA\"\n\"1073\",\"SCAND1\"\n\"1074\",\"GINM1\"\n\"1075\",\"METTL23\"\n\"1076\",\"MGST1\"\n\"1077\",\"PSMC4\"\n\"1078\",\"ERV3-1\"\n\"1079\",\"PINK1\"\n\"1080\",\"RANBP1\"\n\"1081\",\"ZCCHC9\"\n\"1082\",\"PTX3\"\n\"1083\",\"COPS8\"\n\"1084\",\"GLRX3\"\n\"1085\",\"PTPN7\"\n\"1086\",\"TMEM9B\"\n\"1087\",\"TTC3\"\n\"1088\",\"MX1\"\n\"1089\",\"ADI1\"\n\"1090\",\"STK38\"\n\"1091\",\"RPL22L1\"\n\"1092\",\"IL23A\"\n\"1093\",\"NUDT1\"\n\"1094\",\"PACS1\"\n\"1095\",\"NCOR1\"\n\"1096\",\"MT-ND6\"\n\"1097\",\"DUT\"\n\"1098\",\"PRNP\"\n\"1099\",\"THOC7\"\n\"1100\",\"ERP44\"\n\"1101\",\"DIAPH1\"\n\"1102\",\"ICAM2\"\n\"1103\",\"ARL2\"\n\"1104\",\"PRPF31\"\n\"1105\",\"TUBB2A\"\n\"1106\",\"GID8\"\n\"1107\",\"MFF\"\n\"1108\",\"COQ2\"\n\"1109\",\"REEP5\"\n\"1110\",\"CARHSP1\"\n\"1111\",\"CPQ\"\n\"1112\",\"LTC4S\"\n\"1113\",\"PPAPDC2\"\n\"1114\",\"MRPL9\"\n\"1115\",\"CD47\"\n\"1116\",\"APOBEC3G\"\n\"1117\",\"GADD45G\"\n\"1118\",\"SSR2\"\n\"1119\",\"TAF5\"\n\"1120\",\"BST2\"\n\"1121\",\"METTL9\"\n\"1122\",\"OAF\"\n\"1123\",\"TMEM50A\"\n\"1124\",\"HNRNPA0\"\n\"1125\",\"PPP1R2\"\n\"1126\",\"CTSC\"\n\"1127\",\"PHGDH\"\n\"1128\",\"WDR83\"\n\"1129\",\"GADD45B\"\n\"1130\",\"CENPW\"\n\"1131\",\"C5orf30\"\n\"1132\",\"BTN3A1\"\n\"1133\",\"CDC123\"\n\"1134\",\"ZNF493\"\n\"1135\",\"FAM107B\"\n\"1136\",\"NDUFA2\"\n\"1137\",\"RAC1\"\n\"1138\",\"CYB5B\"\n\"1139\",\"WTAP\"\n\"1140\",\"ARF6\"\n\"1141\",\"TMX2\"\n\"1142\",\"ANXA6\"\n\"1143\",\"EMG1\"\n\"1144\",\"P2RX5\"\n\"1145\",\"JUND\"\n\"1146\",\"POLR3GL\"\n\"1147\",\"MRPS12\"\n\"1148\",\"PITHD1\"\n\"1149\",\"MMADHC\"\n\"1150\",\"FBXO21\"\n\"1151\",\"ESYT1\"\n\"1152\",\"TRIM16L\"\n\"1153\",\"GABARAPL2\"\n\"1154\",\"IFRD1\"\n\"1155\",\"GMFG\"\n\"1156\",\"MAD2L1\"\n\"1157\",\"MRPS18B\"\n\"1158\",\"RNASE4\"\n\"1159\",\"BCL11A\"\n\"1160\",\"WDR83OS\"\n\"1161\",\"TAF12\"\n\"1162\",\"LAT2\"\n\"1163\",\"NAT9\"\n\"1164\",\"REXO2\"\n\"1165\",\"CTSB\"\n\"1166\",\"EMC7\"\n\"1167\",\"BBS2\"\n\"1168\",\"OSCAR\"\n\"1169\",\"RP11-412D9.4\"\n\"1170\",\"CTA-250D10.23\"\n\"1171\",\"INTS12\"\n\"1172\",\"PRR5\"\n\"1173\",\"TMEM242\"\n\"1174\",\"VDAC3\"\n\"1175\",\"WDR5\"\n\"1176\",\"A2M-AS1\"\n\"1177\",\"SNX17\"\n\"1178\",\"PHF12\"\n\"1179\",\"MOB2\"\n\"1180\",\"ACTN4\"\n\"1181\",\"COX7A2L\"\n\"1182\",\"EIF3M\"\n\"1183\",\"MRPS33\"\n\"1184\",\"TUBA1A\"\n\"1185\",\"DAGLB\"\n\"1186\",\"PPP1CA\"\n\"1187\",\"CYFIP1\"\n\"1188\",\"HMGB1\"\n\"1189\",\"TMEM205\"\n\"1190\",\"PSMB6\"\n\"1191\",\"MLLT11\"\n\"1192\",\"BSDC1\"\n\"1193\",\"LAMTOR1\"\n\"1194\",\"BABAM1\"\n\"1195\",\"HDAC2\"\n\"1196\",\"ELOF1\"\n\"1197\",\"ORAI1\"\n\"1198\",\"PRDX3\"\n\"1199\",\"LARP1\"\n\"1200\",\"ERGIC3\"\n\"1201\",\"PSMD4\"\n\"1202\",\"EIF5\"\n\"1203\",\"FHL1\"\n\"1204\",\"ANAPC13\"\n\"1205\",\"UQCRH\"\n\"1206\",\"ZNF567\"\n\"1207\",\"PROCA1\"\n\"1208\",\"SMARCA4\"\n\"1209\",\"NKAP\"\n\"1210\",\"POU2F2\"\n\"1211\",\"METTL8\"\n\"1212\",\"HRASLS2\"\n\"1213\",\"GHITM\"\n\"1214\",\"RFNG\"\n\"1215\",\"ANKRD44\"\n\"1216\",\"LILRB3\"\n\"1217\",\"CARS\"\n\"1218\",\"PBXIP1\"\n\"1219\",\"TKT\"\n\"1220\",\"THEM6\"\n\"1221\",\"CTNNAL1\"\n\"1222\",\"TXNL4B\"\n\"1223\",\"TMEM91\"\n\"1224\",\"SEPT11\"\n\"1225\",\"PFKFB3\"\n\"1226\",\"CCDC91\"\n\"1227\",\"RP11-430B1.2\"\n\"1228\",\"HP1BP3\"\n\"1229\",\"SCP2\"\n\"1230\",\"OCIAD1\"\n\"1231\",\"TNFRSF9\"\n\"1232\",\"EXOC3L1\"\n\"1233\",\"ZNF559\"\n\"1234\",\"ARSD\"\n\"1235\",\"CCDC115\"\n\"1236\",\"LIMD2\"\n\"1237\",\"ZBTB32\"\n\"1238\",\"MTERFD2\"\n\"1239\",\"CDC42EP3\"\n\"1240\",\"FGFR1OP2\"\n\"1241\",\"KIF3A\"\n\"1242\",\"ATRAID\"\n\"1243\",\"DNAJA3\"\n\"1244\",\"MX2\"\n\"1245\",\"TBXAS1\"\n\"1246\",\"APMAP\"\n\"1247\",\"RAD51D\"\n\"1248\",\"PACSIN2\"\n\"1249\",\"GAS6\"\n\"1250\",\"METTL3\"\n\"1251\",\"PITPNM1\"\n\"1252\",\"TMEM14B\"\n\"1253\",\"RP11-1070N10.3\"\n\"1254\",\"MED7\"\n\"1255\",\"POLR3K\"\n\"1256\",\"PLEKHG5\"\n\"1257\",\"CRELD2\"\n\"1258\",\"NOSIP\"\n\"1259\",\"NUSAP1\"\n\"1260\",\"RNF213\"\n\"1261\",\"DEXI\"\n\"1262\",\"MLEC\"\n\"1263\",\"GNAI2\"\n\"1264\",\"NOP10\"\n\"1265\",\"ADD1\"\n\"1266\",\"LAMTOR2\"\n\"1267\",\"RAN\"\n\"1268\",\"TRIP12\"\n\"1269\",\"GP1BA\"\n\"1270\",\"SPATS2L\"\n\"1271\",\"CEP78\"\n\"1272\",\"PGRMC2\"\n\"1273\",\"PDIA3\"\n\"1274\",\"PHF14\"\n\"1275\",\"CISD2\"\n\"1276\",\"FAM221A\"\n\"1277\",\"FXYD5\"\n\"1278\",\"AC079305.10\"\n\"1279\",\"LTV1\"\n\"1280\",\"DEPTOR\"\n\"1281\",\"CTSZ\"\n\"1282\",\"PPP2R5C\"\n\"1283\",\"ASB7\"\n\"1284\",\"TBCB\"\n\"1285\",\"TMEM97\"\n\"1286\",\"CMPK1\"\n\"1287\",\"CENPT\"\n\"1288\",\"MYCBP2\"\n\"1289\",\"CLEC4C\"\n\"1290\",\"BAZ2A\"\n\"1291\",\"NOL7\"\n\"1292\",\"RRAGC\"\n\"1293\",\"HDAC1\"\n\"1294\",\"GMEB1\"\n\"1295\",\"RFC3\"\n\"1296\",\"MNDA\"\n\"1297\",\"BRK1\"\n\"1298\",\"NCLN\"\n\"1299\",\"KCNC3\"\n\"1300\",\"NLRC4\"\n\"1301\",\"RNF181\"\n\"1302\",\"TMEM165\"\n\"1303\",\"CYTH4\"\n\"1304\",\"UXT\"\n\"1305\",\"RNF113A\"\n\"1306\",\"MBNL1-AS1\"\n\"1307\",\"RFC2\"\n\"1308\",\"DNAJC2\"\n\"1309\",\"MRPL20\"\n\"1310\",\"JMJD6\"\n\"1311\",\"MRPL41\"\n\"1312\",\"RFC1\"\n\"1313\",\"LIPT2\"\n\"1314\",\"HIST1H2BD\"\n\"1315\",\"RHOB\"\n\"1316\",\"GPSM3\"\n\"1317\",\"ZFAT\"\n\"1318\",\"SCFD2\"\n\"1319\",\"TSPAN33\"\n\"1320\",\"RP6-91H8.3\"\n\"1321\",\"ZNF526\"\n\"1322\",\"IRAK1\"\n\"1323\",\"SPATA5L1\"\n\"1324\",\"PARVB\"\n\"1325\",\"ZNF688\"\n\"1326\",\"MIR4435-1HG\"\n\"1327\",\"ACY3\"\n\"1328\",\"UBXN4\"\n\"1329\",\"BYSL\"\n\"1330\",\"FCGR3B\"\n\"1331\",\"RER1\"\n\"1332\",\"DERL1\"\n\"1333\",\"SHOC2\"\n\"1334\",\"GTPBP2\"\n\"1335\",\"RP11-258F1.1\"\n\"1336\",\"SPCS1\"\n\"1337\",\"MZT2B\"\n\"1338\",\"SUPT4H1\"\n\"1339\",\"NEMF\"\n\"1340\",\"ZNF92\"\n\"1341\",\"RMI2\"\n\"1342\",\"N6AMT1\"\n\"1343\",\"CD300C\"\n\"1344\",\"CCBL1\"\n\"1345\",\"ATG16L1\"\n\"1346\",\"TFAM\"\n\"1347\",\"LARS\"\n\"1348\",\"IRF4\"\n\"1349\",\"CXCL10\"\n\"1350\",\"ZNF394\"\n\"1351\",\"LRRK1\"\n\"1352\",\"PPIE\"\n\"1353\",\"CBX5\"\n\"1354\",\"EIF4A3\"\n\"1355\",\"TESC\"\n\"1356\",\"MARCH2\"\n\"1357\",\"UNC45A\"\n\"1358\",\"RP11-362F19.1\"\n\"1359\",\"PTGIR\"\n\"1360\",\"MRPL28\"\n\"1361\",\"TERF2IP\"\n\"1362\",\"SSX2IP\"\n\"1363\",\"RASD1\"\n\"1364\",\"GALM\"\n\"1365\",\"LNPEP\"\n\"1366\",\"NDUFS2\"\n\"1367\",\"TAOK2\"\n\"1368\",\"POMT1\"\n\"1369\",\"DONSON\"\n\"1370\",\"TUBB6\"\n\"1371\",\"MUTYH\"\n\"1372\",\"SLC25A3\"\n\"1373\",\"UTRN\"\n\"1374\",\"FBXL12\"\n\"1375\",\"MAF1\"\n\"1376\",\"BLNK\"\n\"1377\",\"RP11-706O15.1\"\n\"1378\",\"THAP5\"\n\"1379\",\"C12orf45\"\n\"1380\",\"CDC40\"\n\"1381\",\"C19orf52\"\n\"1382\",\"RBM7\"\n\"1383\",\"NDUFA4\"\n\"1384\",\"RP11-293M10.5\"\n\"1385\",\"CTC-338M12.5\"\n\"1386\",\"TIMM17A\"\n\"1387\",\"LINC00528\"\n\"1388\",\"NUPL2\"\n\"1389\",\"EGLN2\"\n\"1390\",\"TCEAL1\"\n\"1391\",\"CEP120\"\n\"1392\",\"ACTR3\"\n\"1393\",\"RP11-452F19.3\"\n\"1394\",\"LARP4\"\n\"1395\",\"DEAF1\"\n\"1396\",\"CENPQ\"\n\"1397\",\"HDAC5\"\n\"1398\",\"ELP5\"\n\"1399\",\"PCBP4\"\n\"1400\",\"MEOX1\"\n\"1401\",\"PTPRC\"\n\"1402\",\"TCF4\"\n\"1403\",\"DCTPP1\"\n\"1404\",\"QRSL1\"\n\"1405\",\"BBX\"\n\"1406\",\"TNFAIP1\"\n\"1407\",\"PDE12\"\n\"1408\",\"PGLYRP2\"\n\"1409\",\"COMMD3\"\n\"1410\",\"MT-ND5\"\n\"1411\",\"NPRL2\"\n\"1412\",\"RBBP8\"\n\"1413\",\"RDH14\"\n\"1414\",\"FADS1\"\n\"1415\",\"WDR55\"\n\"1416\",\"MLTK\"\n\"1417\",\"CTB-61M7.2\"\n\"1418\",\"SLBP\"\n\"1419\",\"CD48\"\n\"1420\",\"HRH2\"\n\"1421\",\"C19orf33\"\n\"1422\",\"FAM45A\"\n\"1423\",\"JUP\"\n\"1424\",\"ISCA2\"\n\"1425\",\"IER2\"\n\"1426\",\"GRK6\"\n\"1427\",\"RP11-383C5.4\"\n\"1428\",\"CSNK1A1\"\n\"1429\",\"EIF2S3\"\n\"1430\",\"CCDC152\"\n\"1431\",\"C3AR1\"\n\"1432\",\"SRSF2\"\n\"1433\",\"TTF1\"\n\"1434\",\"EHD4\"\n\"1435\",\"OTUB1\"\n\"1436\",\"PRKCI\"\n\"1437\",\"CDK19\"\n\"1438\",\"ARPC5L\"\n\"1439\",\"EIF3K\"\n\"1440\",\"CCR6\"\n\"1441\",\"TP53BP2\"\n\"1442\",\"HGD\"\n\"1443\",\"MARCH7\"\n\"1444\",\"IL4I1\"\n\"1445\",\"C9orf142\"\n\"1446\",\"MFN2\"\n\"1447\",\"SNHG7\"\n\"1448\",\"ZFP36\"\n\"1449\",\"ZFP69\"\n\"1450\",\"ING5\"\n\"1451\",\"GAS2L1\"\n\"1452\",\"RIOK2\"\n\"1453\",\"C1orf228\"\n\"1454\",\"VPS25\"\n\"1455\",\"ENO1\"\n\"1456\",\"MCM6\"\n\"1457\",\"TPI1\"\n\"1458\",\"PRKCE\"\n\"1459\",\"PORCN\"\n\"1460\",\"ABCD4\"\n\"1461\",\"DSCC1\"\n\"1462\",\"TM7SF3\"\n\"1463\",\"L3MBTL2\"\n\"1464\",\"EFNB1\"\n\"1465\",\"FANCG\"\n\"1466\",\"FXYD2\"\n\"1467\",\"MPV17\"\n\"1468\",\"TMEM138\"\n\"1469\",\"LCP1\"\n\"1470\",\"HMG20A\"\n\"1471\",\"SNAPIN\"\n\"1472\",\"MRPS15\"\n\"1473\",\"ASPHD2\"\n\"1474\",\"KLF10\"\n\"1475\",\"NOP2\"\n\"1476\",\"TRAK1\"\n\"1477\",\"CDC34\"\n\"1478\",\"NSDHL\"\n\"1479\",\"SETD8\"\n\"1480\",\"GPATCH4\"\n\"1481\",\"C19orf24\"\n\"1482\",\"FXYD6\"\n\"1483\",\"ARPC3\"\n\"1484\",\"GOT2\"\n\"1485\",\"DMTN\"\n\"1486\",\"STT3A\"\n\"1487\",\"SLC31A2\"\n\"1488\",\"TXNRD1\"\n\"1489\",\"NLN\"\n\"1490\",\"SEC24D\"\n\"1491\",\"TNFAIP2\"\n\"1492\",\"PLIN3\"\n\"1493\",\"CAPZB\"\n\"1494\",\"PPARGC1B\"\n\"1495\",\"C5AR1\"\n\"1496\",\"MBNL2\"\n\"1497\",\"CELA1\"\n\"1498\",\"SNRPB\"\n\"1499\",\"C11orf68\"\n\"1500\",\"OTUD1\"\n\"1501\",\"C19orf10\"\n\"1502\",\"EIF3G\"\n\"1503\",\"NDUFS8\"\n\"1504\",\"CENPL\"\n\"1505\",\"LIMS2\"\n\"1506\",\"SUSD3\"\n\"1507\",\"HMGA1\"\n\"1508\",\"RP11-70P17.1\"\n\"1509\",\"DCTN3\"\n\"1510\",\"HVCN1\"\n\"1511\",\"ADSL\"\n\"1512\",\"RMND5A\"\n\"1513\",\"C11orf58\"\n\"1514\",\"PRR7\"\n\"1515\",\"ZNF683\"\n\"1516\",\"ARRDC4\"\n\"1517\",\"RNASEL\"\n\"1518\",\"GPS1\"\n\"1519\",\"PRR13\"\n\"1520\",\"SSPN\"\n\"1521\",\"SURF6\"\n\"1522\",\"RP11-598F7.3\"\n\"1523\",\"ODF3B\"\n\"1524\",\"NFATC1\"\n\"1525\",\"PLAC8\"\n\"1526\",\"RP11-218M22.1\"\n\"1527\",\"C16orf93\"\n\"1528\",\"NLRP12\"\n\"1529\",\"PLEKHF1\"\n\"1530\",\"ASGR1\"\n\"1531\",\"CDKN3\"\n\"1532\",\"ARHGAP4\"\n\"1533\",\"FLT3LG\"\n\"1534\",\"CD33\"\n\"1535\",\"CCDC66\"\n\"1536\",\"AC092580.4\"\n\"1537\",\"CTNS\"\n\"1538\",\"JARID2\"\n\"1539\",\"ZNF626\"\n\"1540\",\"CLPX\"\n\"1541\",\"PAIP2B\"\n\"1542\",\"RBBP6\"\n\"1543\",\"ELOVL4\"\n\"1544\",\"COL6A2\"\n\"1545\",\"CCDC28B\"\n\"1546\",\"TULP3\"\n\"1547\",\"OLA1\"\n\"1548\",\"APC\"\n\"1549\",\"UTP6\"\n\"1550\",\"CHAC2\"\n\"1551\",\"FASLG\"\n\"1552\",\"LSM1\"\n\"1553\",\"MARC1\"\n\"1554\",\"ST20\"\n\"1555\",\"TRIOBP\"\n\"1556\",\"EIF1B\"\n\"1557\",\"CITED4\"\n\"1558\",\"ZNF844\"\n\"1559\",\"DPM1\"\n\"1560\",\"LINC00152\"\n\"1561\",\"ENKUR\"\n\"1562\",\"NOL9\"\n\"1563\",\"GPR82\"\n\"1564\",\"MON1B\"\n\"1565\",\"ZCWPW1\"\n\"1566\",\"NDUFS7\"\n\"1567\",\"NRG1\"\n\"1568\",\"CD82\"\n\"1569\",\"SNHG12\"\n\"1570\",\"TROAP\"\n\"1571\",\"ZNF398\"\n\"1572\",\"CLSTN3\"\n\"1573\",\"LGMN\"\n\"1574\",\"KIAA0226L\"\n\"1575\",\"GALNS\"\n\"1576\",\"HINT1\"\n\"1577\",\"PPM1B\"\n\"1578\",\"SAAL1\"\n\"1579\",\"BAZ1B\"\n\"1580\",\"GZMM\"\n\"1581\",\"MBOAT7\"\n\"1582\",\"RP11-23P13.6\"\n\"1583\",\"SEC11C\"\n\"1584\",\"MRPL19\"\n\"1585\",\"JAZF1\"\n\"1586\",\"AKAP7\"\n\"1587\",\"MYNN\"\n\"1588\",\"C10orf54\"\n\"1589\",\"MCFD2\"\n\"1590\",\"ANTXR2\"\n\"1591\",\"JUN\"\n\"1592\",\"STAT2\"\n\"1593\",\"USP25\"\n\"1594\",\"MYL12B\"\n\"1595\",\"MTDH\"\n\"1596\",\"SDAD1\"\n\"1597\",\"ALOX12\"\n\"1598\",\"ASXL2\"\n\"1599\",\"RP11-1094M14.11\"\n\"1600\",\"C1orf54\"\n\"1601\",\"ADAL\"\n\"1602\",\"SLC35A2\"\n\"1603\",\"CLK3\"\n\"1604\",\"CYTL1\"\n\"1605\",\"DAB2\"\n\"1606\",\"LMO2\"\n\"1607\",\"CD300E\"\n\"1608\",\"SLC25A14\"\n\"1609\",\"HELZ\"\n\"1610\",\"FXYD1\"\n\"1611\",\"SETD1B\"\n\"1612\",\"EIF1AY\"\n\"1613\",\"SAV1\"\n\"1614\",\"HECTD1\"\n\"1615\",\"MRPL2\"\n\"1616\",\"FLJ00104\"\n\"1617\",\"KLHL6\"\n\"1618\",\"FTSJ2\"\n\"1619\",\"UPF3B\"\n\"1620\",\"C9orf91\"\n\"1621\",\"HIST1H2BJ\"\n\"1622\",\"TCF7\"\n\"1623\",\"WDR34\"\n\"1624\",\"DDX51\"\n\"1625\",\"RASGRP4\"\n\"1626\",\"POP7\"\n\"1627\",\"CTC-378H22.1\"\n\"1628\",\"TMED9\"\n\"1629\",\"SPATA6\"\n\"1630\",\"CXXC1\"\n\"1631\",\"ABCC3\"\n\"1632\",\"UBE2R2\"\n\"1633\",\"PLA2G7\"\n\"1634\",\"LIN52\"\n\"1635\",\"SLC43A3\"\n\"1636\",\"DICER1\"\n\"1637\",\"MPP6\"\n\"1638\",\"ZNF148\"\n\"1639\",\"BANP\"\n\"1640\",\"CCDC167\"\n\"1641\",\"CEBPA\"\n\"1642\",\"OPRL1\"\n\"1643\",\"AC109826.1\"\n\"1644\",\"MRPS18A\"\n\"1645\",\"C19orf53\"\n\"1646\",\"CXCL3\"\n\"1647\",\"DOK2\"\n\"1648\",\"FAM76B\"\n\"1649\",\"ZBED6\"\n\"1650\",\"CD59\"\n\"1651\",\"FAM207A\"\n\"1652\",\"CDC37\"\n\"1653\",\"MT1F\"\n\"1654\",\"ATF5\"\n\"1655\",\"ALKBH4\"\n\"1656\",\"KIAA1147\"\n\"1657\",\"MAFF\"\n\"1658\",\"MGRN1\"\n\"1659\",\"C4orf3\"\n\"1660\",\"SKP1\"\n\"1661\",\"SLC24A4\"\n\"1662\",\"RBM12\"\n\"1663\",\"CTC-444N24.11\"\n\"1664\",\"MXD3\"\n\"1665\",\"ATP6V1A\"\n\"1666\",\"ICOSLG\"\n\"1667\",\"FBXL17\"\n\"1668\",\"SSB\"\n\"1669\",\"RP11-169K16.9\"\n\"1670\",\"ADIPOR2\"\n\"1671\",\"LIMK1\"\n\"1672\",\"DOHH\"\n\"1673\",\"PRAM1\"\n\"1674\",\"GPKOW\"\n\"1675\",\"ACTR6\"\n\"1676\",\"STRN\"\n\"1677\",\"FPR2\"\n\"1678\",\"MED9\"\n\"1679\",\"ARF1\"\n\"1680\",\"CLEC12A\"\n\"1681\",\"FXR2\"\n\"1682\",\"AFTPH\"\n\"1683\",\"TMEM62\"\n\"1684\",\"MED27\"\n\"1685\",\"IL17RA\"\n\"1686\",\"ASCC2\"\n\"1687\",\"PTPRN2\"\n\"1688\",\"SETD6\"\n\"1689\",\"PIGV\"\n\"1690\",\"SRPK1\"\n\"1691\",\"RP11-274B18.2\"\n\"1692\",\"DTNBP1\"\n\"1693\",\"FAM120A\"\n\"1694\",\"PIGF\"\n\"1695\",\"FOSL2\"\n\"1696\",\"WNT10A\"\n\"1697\",\"GSTZ1\"\n\"1698\",\"TMEM214\"\n\"1699\",\"CES4A\"\n\"1700\",\"MTCH2\"\n\"1701\",\"PCYT1A\"\n\"1702\",\"AGA\"\n\"1703\",\"IRS2\"\n\"1704\",\"PSMD7\"\n\"1705\",\"DIDO1\"\n\"1706\",\"LMNB2\"\n\"1707\",\"GAA\"\n\"1708\",\"ZNF528\"\n\"1709\",\"POLR3E\"\n\"1710\",\"MYBBP1A\"\n\"1711\",\"KCNE1\"\n\"1712\",\"GRAP\"\n\"1713\",\"PTGDR\"\n\"1714\",\"MIR4458HG\"\n\"1715\",\"BNIP2\"\n\"1716\",\"ARAP1\"\n\"1717\",\"CHN2\"\n\"1718\",\"SETDB1\"\n\"1719\",\"SPEF2\"\n\"1720\",\"SNAP47\"\n\"1721\",\"CD302\"\n\"1722\",\"TCF12\"\n\"1723\",\"NAF1\"\n\"1724\",\"DNAJB1\"\n\"1725\",\"LARP1B\"\n\"1726\",\"SLC27A1\"\n\"1727\",\"AAAS\"\n\"1728\",\"CD86\"\n\"1729\",\"C1orf63\"\n\"1730\",\"DLST\"\n\"1731\",\"RBM48\"\n\"1732\",\"PTK2\"\n\"1733\",\"TGIF2\"\n\"1734\",\"MYO5A\"\n\"1735\",\"S100Z\"\n\"1736\",\"VPS72\"\n\"1737\",\"THUMPD3\"\n\"1738\",\"TFEC\"\n\"1739\",\"BROX\"\n\"1740\",\"RP4-575N6.4\"\n\"1741\",\"NME6\"\n\"1742\",\"PGM2\"\n\"1743\",\"UBE3A\"\n\"1744\",\"TMEM248\"\n\"1745\",\"ARHGAP26\"\n\"1746\",\"RPUSD2\"\n\"1747\",\"RBM38\"\n\"1748\",\"AP4S1\"\n\"1749\",\"USF1\"\n\"1750\",\"MORN3\"\n\"1751\",\"NCBP1\"\n\"1752\",\"MRPS34\"\n\"1753\",\"SMCO4\"\n\"1754\",\"FBXW5\"\n\"1755\",\"MORC2-AS1\"\n\"1756\",\"ZNF174\"\n\"1757\",\"EPHB6\"\n\"1758\",\"GNPTAB\"\n\"1759\",\"PPIP5K2\"\n\"1760\",\"CLEC4G\"\n\"1761\",\"FBXO4\"\n\"1762\",\"ADO\"\n\"1763\",\"RP5-821D11.7\"\n\"1764\",\"UGDH\"\n\"1765\",\"MAP3K8\"\n\"1766\",\"DENND1A\"\n\"1767\",\"ATP10D\"\n\"1768\",\"SCIMP\"\n\"1769\",\"HLA-DOA\"\n\"1770\",\"SMC2\"\n\"1771\",\"COG4\"\n\"1772\",\"RHEBL1\"\n\"1773\",\"GEMIN4\"\n\"1774\",\"HCCS\"\n\"1775\",\"PRRT3\"\n\"1776\",\"SLC25A12\"\n\"1777\",\"L3MBTL3\"\n\"1778\",\"PREPL\"\n\"1779\",\"MRPL42\"\n\"1780\",\"C21orf119\"\n\"1781\",\"SNX14\"\n\"1782\",\"GTPBP6\"\n\"1783\",\"HMGCS1\"\n\"1784\",\"MYO15B\"\n\"1785\",\"POLI\"\n\"1786\",\"NUDT7\"\n\"1787\",\"TBK1\"\n\"1788\",\"MTA2\"\n\"1789\",\"NPL\"\n\"1790\",\"SRGAP2\"\n\"1791\",\"CCP110\"\n\"1792\",\"CDK6\"\n\"1793\",\"CHD7\"\n\"1794\",\"PDCD1\"\n\"1795\",\"PUM1\"\n\"1796\",\"SLC10A7\"\n\"1797\",\"STK38L\"\n\"1798\",\"ARHGAP18\"\n\"1799\",\"YEATS2\"\n\"1800\",\"METTL25\"\n\"1801\",\"TBCE\"\n\"1802\",\"LRRK2\"\n\"1803\",\"LRFN1\"\n\"1804\",\"IL1RN\"\n\"1805\",\"ACOT13\"\n\"1806\",\"B4GALT4\"\n\"1807\",\"LINC00847\"\n\"1808\",\"CD3E\"\n\"1809\",\"COX10\"\n\"1810\",\"SERP1\"\n\"1811\",\"C2orf88\"\n\"1812\",\"GLS\"\n\"1813\",\"RP11-103G8.2\"\n\"1814\",\"PEX26\"\n\"1815\",\"MACROD2\"\n\"1816\",\"TOP1MT\"\n\"1817\",\"TOR4A\"\n\"1818\",\"C3orf62\"\n\"1819\",\"ZNF669\"\n\"1820\",\"TIPARP\"\n\"1821\",\"ECE2\"\n\"1822\",\"NIPSNAP1\"\n\"1823\",\"MT-ATP6\"\n\"1824\",\"MAGEH1\"\n\"1825\",\"ARRB1\"\n\"1826\",\"SMAGP\"\n\"1827\",\"SLC41A1\"\n\"1828\",\"PANK2\"\n\"1829\",\"DOK3\"\n\"1830\",\"RCC1\"\n\"1831\",\"SLC6A6\"\n\"1832\",\"RP11-421L21.3\"\n\"1833\",\"RP11-356I2.4\"\n\"1834\",\"DHRS1\"\n\"1835\",\"PEMT\"\n\"1836\",\"LAMP3\"\n\"1837\",\"RP11-65J3.1\"\n\"1838\",\"HOMER1\"\n\"1839\",\"RPUSD4\"\n\"1840\",\"ANAPC7\"\n\"1841\",\"NCL\"\n\"1842\",\"WDR91\"\n\"1843\",\"CXorf65\"\n\"1844\",\"FAM76A\"\n\"1845\",\"NPRL3\"\n\"1846\",\"TUBB4B\"\n\"1847\",\"CANT1\"\n\"1848\",\"HAUS8\"\n\"1849\",\"RNH1\"\n\"1850\",\"C11orf24\"\n\"1851\",\"FAM50B\"\n\"1852\",\"HLA-E\"\n\"1853\",\"SYCE1\"\n\"1854\",\"MED10\"\n\"1855\",\"ADTRP\"\n\"1856\",\"LRIG2\"\n\"1857\",\"KDM5C\"\n\"1858\",\"OSGIN2\"\n\"1859\",\"NME7\"\n\"1860\",\"VAPB\"\n\"1861\",\"EFR3A\"\n\"1862\",\"NEK8\"\n\"1863\",\"ATP5G2\"\n\"1864\",\"RNASEH1-AS1\"\n\"1865\",\"RP3-395M20.12\"\n\"1866\",\"CCDC22\"\n\"1867\",\"CXCR4\"\n\"1868\",\"RFC5\"\n\"1869\",\"BEX5\"\n\"1870\",\"CD1D\"\n\"1871\",\"ARL4D\"\n\"1872\",\"ZWILCH\"\n\"1873\",\"DDI2\"\n\"1874\",\"SH2D3A\"\n\"1875\",\"ZBED5-AS1\"\n\"1876\",\"TRMT61A\"\n\"1877\",\"FUZ\"\n\"1878\",\"TMEM87B\"\n\"1879\",\"BCS1L\"\n\"1880\",\"MRI1\"\n\"1881\",\"NAPA-AS1\"\n\"1882\",\"TLE1\"\n\"1883\",\"DAAM1\"\n\"1884\",\"PDCD2L\"\n\"1885\",\"RP13-580F15.2\"\n\"1886\",\"GSTT1\"\n\"1887\",\"AP3M2\"\n\"1888\",\"NELFB\"\n\"1889\",\"NAA50\"\n\"1890\",\"CBFA2T3\"\n\"1891\",\"GPX4\"\n\"1892\",\"PHF10\"\n\"1893\",\"SEL1L\"\n\"1894\",\"C16orf58\"\n\"1895\",\"CLCN7\"\n\"1896\",\"LRRC45\"\n\"1897\",\"RNF214\"\n\"1898\",\"MAP2K7\"\n\"1899\",\"UIMC1\"\n\"1900\",\"GFOD2\"\n\"1901\",\"RNF168\"\n\"1902\",\"EID2B\"\n\"1903\",\"ZBTB17\"\n\"1904\",\"CLCF1\"\n\"1905\",\"OPA3\"\n\"1906\",\"VAMP2\"\n\"1907\",\"SFMBT1\"\n\"1908\",\"SGMS1\"\n\"1909\",\"TNFRSF13C\"\n\"1910\",\"CAMK2G\"\n\"1911\",\"GCSAM\"\n\"1912\",\"SLC35B4\"\n\"1913\",\"ETFA\"\n\"1914\",\"NUDT15\"\n\"1915\",\"DHRS13\"\n\"1916\",\"STK11IP\"\n\"1917\",\"CROT\"\n\"1918\",\"SUSD1\"\n\"1919\",\"NOM1\"\n\"1920\",\"ZBTB11\"\n\"1921\",\"LCORL\"\n\"1922\",\"ZNF333\"\n\"1923\",\"TAF9B\"\n\"1924\",\"MTRR\"\n\"1925\",\"SAMD9\"\n\"1926\",\"SNX22\"\n\"1927\",\"HIGD2A\"\n\"1928\",\"ABCB1\"\n\"1929\",\"BCL2L12\"\n\"1930\",\"POLD1\"\n\"1931\",\"ANO9\"\n\"1932\",\"OLIG1\"\n\"1933\",\"C2CD2L\"\n\"1934\",\"TRMT112\"\n\"1935\",\"HERC2\"\n\"1936\",\"GNG7\"\n\"1937\",\"SCARB2\"\n\"1938\",\"GLCCI1\"\n\"1939\",\"PIGU\"\n\"1940\",\"RBM5\"\n\"1941\",\"MYO1C\"\n\"1942\",\"KIAA0368\"\n\"1943\",\"AC009403.2\"\n\"1944\",\"TSPYL2\"\n\"1945\",\"ZNF609\"\n\"1946\",\"CDR2\"\n\"1947\",\"WHSC1\"\n\"1948\",\"FEM1A\"\n\"1949\",\"MTMR9\"\n\"1950\",\"PIK3C2A\"\n\"1951\",\"RWDD2B\"\n\"1952\",\"CTB-55O6.12\"\n\"1953\",\"FAAH\"\n\"1954\",\"ZNF211\"\n\"1955\",\"NDRG1\"\n\"1956\",\"FAM214A\"\n\"1957\",\"TBXA2R\"\n\"1958\",\"SLC2A13\"\n\"1959\",\"APBA3\"\n\"1960\",\"MICB\"\n\"1961\",\"MICU1\"\n\"1962\",\"PTRH2\"\n\"1963\",\"PAN2\"\n\"1964\",\"IFNG-AS1\"\n\"1965\",\"CSNK2A2\"\n\"1966\",\"PPP1R12C\"\n\"1967\",\"TESK1\"\n\"1968\",\"ZNF429\"\n\"1969\",\"BPNT1\"\n\"1970\",\"CR1\"\n\"1971\",\"SLC20A2\"\n\"1972\",\"RP11-104L21.3\"\n\"1973\",\"HBEGF\"\n\"1974\",\"TMEM181\"\n\"1975\",\"PRKAG1\"\n\"1976\",\"CXorf24\"\n\"1977\",\"FAM26F\"\n\"1978\",\"DPH6\"\n\"1979\",\"ERI3\"\n\"1980\",\"RASGRP3\"\n\"1981\",\"C1orf56\"\n\"1982\",\"ENOX2\"\n\"1983\",\"IL15\"\n\"1984\",\"DNAJC9\"\n\"1985\",\"ENDOV\"\n\"1986\",\"CAMK1D\"\n\"1987\",\"LRRC59\"\n\"1988\",\"ATXN3\"\n\"1989\",\"PPRC1\"\n\"1990\",\"CLEC4D\"\n\"1991\",\"SBNO1\"\n\"1992\",\"TMEM156\"\n\"1993\",\"ALG12\"\n\"1994\",\"DPP4\"\n\"1995\",\"NEK1\"\n\"1996\",\"ICAM1\"\n\"1997\",\"CLUAP1\"\n\"1998\",\"NUP98\"\n\"1999\",\"DUSP5\"\n\"2000\",\"HCFC2\"\n\n\nACGCCTGACACGCGCT-1,0,0,0,2546,2779\nTACCGATCCAACACTT-1,0,1,1,2666,2848\nATTAAAGCGGACGAGC-1,0,0,2,2546,2917\nGATAAGGGACGATTAG-1,0,1,3,2665,2986\nGTGCAAATCACCAATA-1,0,0,4,2545,3055\nTGTTGGCTGGCGGAAG-1,0,1,5,2665,3124\nGCATCCTCTCCTATTA-1,0,0,6,2545,3192\nGCGAGGGACTGCTAGA-1,0,1,7,2665,3261\nTGGTACCGGCACAGCC-1,0,0,8,2545,3330\nGCGCGTTTAAATCGTA-1,0,1,9,2664,3399\nTGCCTTGCCCTTACGG-1,0,0,10,2544,3467\nGACGACTTTCCAAGAA-1,0,1,11,2664,3536\nCCAGTGAGCTCCTTGT-1,0,0,12,2544,3605\nATACCCTGGCTCAAAT-1,0,1,13,2664,3674\nGGGTTTCCGGCTTCCA-1,0,0,14,2544,3742\nTAACCGTCCAGTTCAT-1,0,1,15,2663,3812\nAAACAACGAATAGTTC-1,0,0,16,2543,3880\nCAAGGGAGTGTATTTG-1,0,1,17,2663,3949\nCCAAGCTTGATCTCCT-1,0,0,18,2543,4018\nTTATTTCATCCCAAAC-1,0,1,19,2663,4087\nGAGCGCTATGTCAGGC-1,0,0,20,2543,4155\nTATGGCAGACTTTCGA-1,0,1,21,2662,4224\nCTTCGTGCCCGCATCG-1,0,0,22,2542,4293\nAAACGGGTTGGTATCC-1,0,1,23,2662,4362\nTGCAAACCCACATCAA-1,0,0,24,2542,4430\nGACGGGATGTCTTATG-1,0,1,25,2662,4499\nGGCGAGCATCGAGGAC-1,0,0,26,2542,4568\nCGCGTGCTATCAACGA-1,0,1,27,2661,4637\nTGAAACCTCAACTCAC-1,0,0,28,2541,4706\nCACATAAGGCGACCGT-1,0,1,29,2661,4775\nTGACCCAACTCACATT-1,0,0,30,2541,4843\nATACGCCGATCTACCG-1,0,1,31,2660,4912\nACTTATCTGATCTATA-1,0,0,32,2541,4981\nGTGTGAGCCGAGGTGC-1,0,1,33,2660,5050\nGATGATTTGAAACTGG-1,0,0,34,2540,5118\nGGGAACCACCTGTTTC-1,0,1,35,2660,5187\nGTTCGTTGCGGACCAG-1,0,0,36,2540,5256\nTGAGGTTGATCCCAAG-1,0,1,37,2659,5325\nGATGCCACACTACAGC-1,0,0,38,2540,5393\nAGGCAAAGAGGAATCA-1,0,1,39,2659,5463\nAAGTAAGCTTCCAAAC-1,0,0,40,2539,5531\nAACGTAGTCTACCCAT-1,0,1,41,2659,5600\nGTTTGAGCGGTTATGT-1,0,0,42,2539,5669\nGAAGCAAGGCAATGTT-1,0,1,43,2658,5738\nTCACTCAGCGCATTAG-1,0,0,44,2538,5806\nTACAATGAAACCAGCA-1,0,1,45,2658,5875\nGTGCGCTTACAAATGA-1,0,0,46,2538,5944\nGCACTCCCACAGTCCC-1,0,1,47,2658,6013\nCGAAGACTGCCCGGGA-1,0,0,48,2538,6081\nCAGGATCCGCCCGACC-1,0,1,49,2657,6150\nCACGATTGGTCGTTAA-1,0,0,50,2537,6219\nGGTTGTATCGTGAAAT-1,0,1,51,2657,6288\nTCTTATGGGTAGTACC-1,0,0,52,2537,6356\nTACAAGCTGTTCACTG-1,0,1,53,2657,6426\nGTATCTTGTTGCTCAC-1,0,0,54,2537,6494\nATACCAGGTGAGCGAT-1,0,1,55,2656,6563\nCCTAAACAGGGTCCGT-1,0,0,56,2536,6632\nATGGTGCTCAAAGCCA-1,0,1,57,2656,6701\nCAAATGCGGAGTGTTC-1,0,0,58,2536,6769\nCGTGCCCGACATTTGT-1,0,1,59,2656,6838\nGTATCTCCCTAACTGT-1,0,0,60,2536,6907\nATTTGCCTAGTTACGA-1,0,1,61,2655,6976\nACGTCCTAAACGAGAT-1,0,0,62,2535,7044\nCTGGGATCGCCCAGAT-1,0,1,63,2655,7113\nCTGCAAATGGGCTCCA-1,0,0,64,2535,7182\nCATTATAACAGGGTCC-1,0,1,65,2655,7251\nACCTTTCCTTTAGAAG-1,0,0,66,2535,7320\nATAGATTTGCAGTCGG-1,0,1,67,2654,7389\nCTCGGGCATCGTCGGG-1,0,0,68,2534,7457\nGTGGCGGGCCGTAGCT-1,0,1,69,2654,7526\nCAACAGTGCCAAACGG-1,0,0,70,2534,7595\nTGCGGGTATTGGGATC-1,0,1,71,2653,7664\nGTCTCGCCAACACGCC-1,0,0,72,2534,7732\nCTGGGCGGCCAAATGT-1,0,1,73,2653,7801\nTAAAGGAGAAACTAGT-1,0,0,74,2533,7870\nTCCCACGGAGGGAGCT-1,0,1,75,2653,7939\nAGCTTCAATACTTTGA-1,0,0,76,2533,8007\nTTCCACATTTCTCGTC-1,0,1,77,2652,8077\nACAAACCGACAAGGCG-1,0,0,78,2533,8145\nAGACGGGATTGGTATA-1,0,1,79,2652,8214\nAACCTAAAGCCGTCCG-1,0,0,80,2532,8283\nTACAAATTGCGGAGGT-1,0,1,81,2652,8352\nCCCGCTAGAGGGTTAA-1,0,0,82,2532,8420\nCATTGCAAAGCATAAT-1,0,1,83,2651,8489\nTGTACGCTATCAGCTT-1,0,0,84,2531,8558\nTTCTTCGCAATAGAGC-1,0,1,85,2651,8627\nTGTGATTCCAGCGCTT-1,0,0,86,2531,8695\nATTCAGGATCGCCTCT-1,0,1,87,2651,8764\nGCCCATGGGTGCAATG-1,0,0,88,2531,8833\nTTCCCGACGCTTCACT-1,0,1,89,2650,8902\nAGCGGTTGAGATGTAC-1,0,0,90,2530,8970\nGCTGTCTGTGATCGAC-1,0,1,91,2650,9040\nAAAGACATGAAGTTTA-1,0,0,92,2530,9108\nCAACAGAATAACGCTA-1,0,1,93,2650,9177\nTGCGGTCTACGAGTAA-1,0,0,94,2530,9246\nAAGACTCACGCCCACT-1,0,1,95,2649,9315\nCTTTGAAACATATTCC-1,0,0,96,2529,9383\nCTGGGCACTAGTCGGA-1,0,1,97,2649,9452\nCGCCCTTACATCCACC-1,0,0,98,2529,9521\nCACGACCACAGACTTT-1,0,1,99,2649,9590\nCAATCCATTATCCGTT-1,0,0,100,2529,9658\nGTGGCGTGCACCAGAG-1,0,1,101,2648,9727\nCGGAGTCCTAACCTGG-1,0,0,102,2528,9796\nGGTCCCATAACATAGA-1,0,1,103,2648,9865\nATCTCATAAACCTACC-1,0,0,104,2528,9934\nTGCATGGCAGTCTTGC-1,0,1,105,2648,10003\nTTGCAGGTCATGAAGT-1,0,0,106,2528,10071\nAGCTGCATTTGAGGTG-1,0,1,107,2647,10140\nTAATCAGGAATGCTGC-1,0,0,108,2527,10209\nCCATCATAAGAACAGG-1,0,1,109,2647,10278\nTCGTATCACCAAGCTA-1,0,0,110,2527,10346\nATTCAGATGAATCCCT-1,0,1,111,2646,10415\nAAAGGTCAACGACATG-1,0,0,112,2527,10484\nAGCTGCTGTGCCGAAT-1,0,1,113,2646,10553\nCTAGCGCCAATCCTAC-1,0,0,114,2526,10621\nGCTCGACCGAACTGAA-1,0,1,115,2646,10691\nACAGTGCAGCGCATTT-1,0,0,116,2526,10759\nCGGCTGAAGGTTACGC-1,0,1,117,2645,10828\nCACCTCTACGAGTGTG-1,0,0,118,2526,10897\nATACGACAGATGGGTA-1,0,1,119,2645,10966\nACTTCCTGTCGTGCGA-1,0,0,120,2525,11034\nCGTAACGGAACGATCA-1,0,1,121,2645,11103\nAAATCACTCCTAAACG-1,0,0,122,2525,11172\nCTCCGAGTAAATCCGC-1,0,1,123,2644,11241\nACGCTAGTATCAGTGC-1,0,0,124,2525,11309\nAGAGTGAACAGACACC-1,0,1,125,2644,11378\nACACCCGTAAATCTGT-1,0,0,126,2524,11447\nGCTTTGCTGCCGGGTA-1,0,1,127,2644,11516\nACAGGAGGCGCAGCCG-1,0,2,0,2786,2780\nAGGCAATACGGAGGAC-1,0,3,1,2905,2849\nTGGTGTGACAGACGAT-1,0,2,2,2785,2918\nATCTATCGATGATCAA-1,0,3,3,2905,2987\nCGGTAACAAGATACAT-1,0,2,4,2785,3055\nTCGCCGGAGAGTCTTA-1,0,3,5,2904,3124\nGGAGGAGTGTGTTTAT-1,0,2,6,2785,3193\nTTAGGTGTGACTGGTC-1,0,3,7,2904,3262\nCAGGGCTAACGAAACC-1,0,2,8,2784,3330\nCCCGTGGGTTAATTGA-1,0,3,9,2904,3399\nGACCGACCGCTAATAT-1,0,2,10,2784,3468\nGGTATCAAGCATAGAA-1,0,3,11,2903,3537\nTGCATGAGTAGATTCG-1,0,2,12,2783,3605\nAATTCCAACTTGGTGA-1,0,3,13,2903,3675\nTGCCGATGTCATCAAT-1,0,2,14,2783,3743\nGCTGGGTCCGCTGTTA-1,0,3,15,2903,3812\nTGAACACCCGAAGCAG-1,0,2,16,2783,3881\nAACATTGGTCAGCCGT-1,0,3,17,2902,3950\nGTGGGTCTTCTTTGCG-1,0,2,18,2782,4018\nCATCGAATGGATCTCT-1,0,3,19,2902,4087\nGCTACACTGTCCGAAC-1,0,2,20,2782,4156\nCGGGTTGTAGCTTTGG-1,0,3,21,2902,4225\nCCTAAGTGTCTAACCG-1,0,2,22,2782,4293\nTCTGTGACTGACCGTT-1,0,3,23,2901,4362\nTTATCATACTCGCAAA-1,0,2,24,2781,4431\nAGCGTAGCGCTAGACC-1,0,3,25,2901,4500\nTCCCTCCGAAATCGTT-1,0,2,26,2781,4569\nAGGTCGCCACTTCGGT-1,0,3,27,2901,4638\nCTAGCAACTAATTTAC-1,0,2,28,2781,4706\nTTGCTAGCTACCAATC-1,0,3,29,2900,4775\nGCCGGTTTGGGCGGAT-1,0,2,30,2780,4844\nTGTAACTTGTCAACCT-1,0,3,31,2900,4913\nCGAGATGTTGCCTATA-1,0,2,32,2780,4981\nGTTACGAAATCCACGC-1,0,3,33,2900,5050\nCTTGTCGTACGTGTCA-1,0,2,34,2780,5119\nGCGTCCAGCTCGTGGC-1,0,3,35,2899,5188\nCCCTTCTCGTACGCGA-1,0,2,36,2779,5256\nCCAAAGTCCCGCTAAC-1,0,3,37,2899,5326\nCCGCTTCGCGGTTAAC-1,0,2,38,2779,5394\nGTTACGGCCCGACTGC-1,0,3,39,2898,5463\nCCCGCTTGCCCTCGTC-1,0,2,40,2779,5532\nTAGTGAGAAGTGGTTG-1,0,3,41,2898,5601\nCGCTACCGCCCTATGA-1,0,2,42,2778,5669\nAAACAATCTACTAGCA-1,0,3,43,2898,5738\nGCGCGATGGGTCAAGT-1,0,2,44,2778,5807\nATAAACCATTGGACGG-1,0,3,45,2897,5876\nTCGGGCACTTCTGGAT-1,0,2,46,2778,5944\nTCTGTGGCTACATTTC-1,0,3,47,2897,6013\nCTCTGTGCCTGCTATG-1,0,2,48,2777,6082\nCACGACTAAAGTTCTG-1,0,3,49,2897,6151\nGAGGAGTAATTCCTAC-1,0,2,50,2777,6219\nAGAGGTATCTCGGTCC-1,0,3,51,2896,6289\nGGCGTACCCTATATAA-1,0,2,52,2776,6357\nGCCGGAAACACATCTT-1,0,3,53,2896,6426\nAAATGTGGGTGCTCCT-1,0,2,54,2776,6495\nACCAGGAGTGTGATCT-1,1,3,55,2896,6564\nTGTGGAGGAAGCTTAA-1,0,2,56,2776,6632\nAAGGAGAACTTATAAG-1,0,3,57,2895,6701\nCCCTCGGGAGCCTTGT-1,0,2,58,2775,6770\nACTGTTTAGTGTAGGC-1,0,3,59,2895,6839\nCGTCAGTTTATCGTCT-1,0,2,60,2775,6907\nGCGTGTATGTCGTATT-1,0,3,61,2895,6976\nACAATCGATCTTTATA-1,0,2,62,2775,7045\nCAGCCCTCACAGGCAG-1,1,3,63,2894,7114\nCGCGTCATATTAAACC-1,0,2,64,2774,7183\nGAAGACTTCAATGCCG-1,1,3,65,2894,7252\nTTGCGGCGACTCATGC-1,0,2,66,2774,7320\nACCAAACTAGAAATCC-1,1,3,67,2894,7389\nTTACTGTTTCTCTACG-1,0,2,68,2774,7458\nGACCAGGTCATTCATA-1,1,3,69,2893,7527\nTTCTTCCCTTTGATAT-1,0,2,70,2773,7595\nACGCCCAGCTGTCGAT-1,1,3,71,2893,7664\nAGTAGCGTGAACGAAC-1,0,2,72,2773,7733\nCCTCGACCCACTGCCT-1,1,3,73,2893,7802\nAGTTATTGAAAGGTAA-1,0,2,74,2773,7870\nTCAGTTACGGAATGAT-1,1,3,75,2892,7940\nGAATCTATACTCGGAC-1,0,2,76,2772,8008\nTCGGCTAACTTCCCTT-1,1,3,77,2892,8077\nACGTGGTCGAATGTGC-1,0,2,78,2772,8146\nATATCGTGCCAGACCC-1,1,3,79,2891,8215\nGTAGCTAGTAAGCGCG-1,0,2,80,2772,8283\nACGCTTAGTGTCTCTC-1,1,3,81,2891,8352\nTCCGGCCTGCATCGAT-1,0,2,82,2771,8421\nTAGTGGAACTCATACA-1,1,3,83,2891,8490\nATCATCTGCCCAGTGT-1,0,2,84,2771,8558\nGTTATTAAATACGACC-1,1,3,85,2890,8627\nGCGCTAAGTATGCATG-1,0,2,86,2771,8696\nCCTGACGCAACCTTTA-1,1,3,87,2890,8765\nCCCAAGAATGCACGGT-1,0,2,88,2770,8834\nAACTGGGTTCGAGCCG-1,1,3,89,2890,8903\nGGTTCCACCCGCTTCT-1,0,2,90,2770,8971\nCATGCACGTGTTACTG-1,0,3,91,2889,9040\nAGCGTTCCGATTTAAA-1,0,2,92,2769,9109\nCCTACGCGACCTTACA-1,1,3,93,2889,9178\nCGAATTACATGGTGTT-1,0,2,94,2769,9246\nGAGGTCTTAGTGGGTC-1,1,3,95,2889,9315\nGCCGCTAGATACGCAG-1,0,2,96,2769,9384\nGTCACCTGTCTATGTC-1,0,3,97,2888,9453\nCCGATTGGTCAATGAA-1,0,2,98,2768,9521\nCCTGTGCGGATTGTAA-1,1,3,99,2888,9591\nTTACGTAGCGCGTGCT-1,0,2,100,2768,9659\nGGAGGCGAAGAACCGC-1,1,3,101,2888,9728\nGGGTCACGTGCTTATG-1,0,2,102,2768,9797\nGCTCCGGACGTTGATA-1,1,3,103,2887,9866\nATGTTTGTAAGATCAT-1,0,2,104,2767,9934\nTGACCCAGCATTCCCG-1,1,3,105,2887,10003\nTGGTCGTTTGATAGAT-1,0,2,106,2767,10072\nTGTAATGCCTTCGGAC-1,1,3,107,2887,10141\nTGCTCACACAACAACC-1,0,2,108,2767,10209\nTACGATCCAAGCCACT-1,0,3,109,2886,10278\nTTGTAACTTCATAGCG-1,0,2,110,2766,10347\nAGATTCAAGCGGGTCG-1,1,3,111,2886,10416\nCTCAGCAGACTGCCGA-1,0,2,112,2766,10484\nGTAACATCAGCTCATC-1,1,3,113,2886,10554\nATGGAACAGAATAAAC-1,0,2,114,2766,10622\nGGGCCTATACAACCGG-1,1,3,115,2885,10691\nTCAAACAATTAGGACA-1,0,2,116,2765,10760\nAAACCACTACACAGAT-1,1,3,117,2885,10829\nAAACGACAGTCTTGCC-1,0,2,118,2765,10897\nTTGAGGGTCGAACGCG-1,0,3,119,2884,10966\nTGTTGATCACTGTTTA-1,0,2,120,2765,11035\nAGGGTGTGCTACACGC-1,0,3,121,2884,11104\nGTAGTTAGACAATATA-1,0,2,122,2764,11172\nAATGGCCGCCAATGCG-1,0,3,123,2884,11241\nTCGGCGGTATTAGATT-1,0,2,124,2764,11310\nGGGTCACTGAGTAGTG-1,0,3,125,2883,11379\nGAATTATGCAACCTAC-1,0,2,126,2764,11448\nGATCTTAGTGAACGTG-1,0,3,127,2883,11517\nCTAATGCGCCCAACAA-1,0,4,0,3025,2781\nGCCACCCATTCCACTT-1,1,5,1,3144,2850\nTACTCACAACGTAGTA-1,0,4,2,3025,2918\nGTTCGGTGTGGATTTA-1,0,5,3,3144,2987\nTCTTTCGGCGGGACAC-1,0,4,4,3024,3056\nGGAGACATTCACGGGC-1,0,5,5,3144,3125\nGGGATTATCTCACAAC-1,0,4,6,3024,3193\nTAGAACGCCAGTAACG-1,1,5,7,3143,3262\nACGAGTCGCCGGCGTT-1,0,4,8,3024,3331\nTGATGGGACTAAGTCA-1,0,5,9,3143,3400\nTGCGAGAAACGTTACG-1,0,4,10,3023,3468\nTCGCCTCGACCTGTTG-1,1,5,11,3143,3538\nAACTCGATAAACACGT-1,1,4,12,3023,3606\nAGGAAAGCCTCTGATG-1,1,5,13,3142,3675\nGAAGGACTAAATTGAA-1,1,4,14,3023,3744\nGTATCGGGACGAGCTG-1,1,5,15,3142,3813\nCCTGTGCATAGGAGAC-1,1,4,16,3022,3881\nCATACGGGTGCATGAT-1,1,5,17,3142,3950\nCCACTAAACTGAATCG-1,1,4,18,3022,4019\nAAATTGCGGCGGTTCT-1,1,5,19,3141,4088\nAGTCCAGCGGGTACGT-1,1,4,20,3021,4156\nCATTCAGGTCAGTGCG-1,1,5,21,3141,4225\nCTAAAGTCCGAAGCTA-1,1,4,22,3021,4294\nAATCAGACTGCAGGAC-1,1,5,23,3141,4363\nAGTATCCATAATAACG-1,1,4,24,3021,4432\nCTGGCTGCTAACGTAA-1,1,5,25,3140,4501\nGTTCCAAGACAGCGAC-1,1,4,26,3020,4569\nAAGACTAACCCGTTGT-1,1,5,27,3140,4638\nGATTAATCCTGGCTCA-1,1,4,28,3020,4707\nCGCGCAAGGAACTACA-1,1,5,29,3140,4776\nCAGTAGCGAGGTAGTA-1,0,4,30,3020,4844\nACGGCGGGTTGCCCTG-1,1,5,31,3139,4913\nCTAGGCGGCAGAGAAT-1,1,4,32,3019,4982\nGTGCGCAGCTTGCTCC-1,1,5,33,3139,5051\nTCACTATCGTGCAATC-1,1,4,34,3019,5119\nTATGATTCTGCTTGGT-1,1,5,35,3139,5189\nTAAGATTTAGCGGGAG-1,1,4,36,3019,5257\nTTACGGTGTCACCGAG-1,1,5,37,3138,5326\nCTACACTAGCTTGTTC-1,1,4,38,3018,5395\nTGAGCAGTCGTGAAGT-1,1,5,39,3138,5464\nCGCTGAGGACGTCCAA-1,1,4,40,3018,5532\nGTGTATGACTTTAAAG-1,1,5,41,3137,5601\nCTAAACGGGTGTAATC-1,1,4,42,3018,5670\nTGTACTGTGCCAAAGT-1,1,5,43,3137,5739\nGGCCACAAGCGATGGC-1,1,4,44,3017,5807\nGTCAATTGTACTGAAG-1,1,5,45,3137,5876\nAGGGACAGCACGGCGG-1,1,4,46,3017,5945\nAGCTTATAGAGACCTG-1,1,5,47,3136,6014\nAACTAGCGTATCGCAC-1,1,4,48,3017,6083\nAACTTTAGCTGCTGAG-1,1,5,49,3136,6152\nCCCAAGACAGAGTATG-1,1,4,50,3016,6220\nGGCATCAACGAGCACG-1,1,5,51,3136,6289\nATGCATTCCGTGATGG-1,1,4,52,3016,6358\nTTATAGATGCACATTA-1,1,5,53,3135,6427\nGAACCATCTGGGAGAC-1,1,4,54,3016,6495\nTGCTATACAAACGGAC-1,1,5,55,3135,6564\nACTTGCCATATTGTAC-1,1,4,56,3015,6633\nTATTCCGGCAGTCCTA-1,1,5,57,3135,6702\nGACGGACCGCGTTCCT-1,1,4,58,3015,6770\nATGTGTAGTTTAGTCA-1,1,5,59,3134,6840\nATACCAGCAAATTGCT-1,1,4,60,3014,6908\nAAGTTTACTAATGGCA-1,1,5,61,3134,6977\nCTCTCGATGTGCGCCT-1,1,4,62,3014,7046\nGATTGACACTCTGCTC-1,1,5,63,3134,7115\nTATCACAGCACGGGCA-1,1,4,64,3014,7183\nACCGTTCCCGCTCTGA-1,1,5,65,3133,7252\nCCGCCACCACAATCCA-1,1,4,66,3013,7321\nCATTCACTGACAGCTA-1,1,5,67,3133,7390\nCGGCTGCAAGATTAAG-1,1,4,68,3013,7458\nCATGAACCTCTTATCA-1,1,5,69,3133,7527\nTTAATGCGAGGTAACT-1,1,4,70,3013,7596\nAATAAGTCCTCGAGAC-1,1,5,71,3132,7665\nACCAGCCCGGTCTTTG-1,1,4,72,3012,7733\nCTACGAACTAGGTCGA-1,1,5,73,3132,7803\nACATCTCAACGCGTAA-1,1,4,74,3012,7871\nCACTACTCAGTTCTGT-1,1,5,75,3132,7940\nCCGACTCGCATAGTCT-1,1,4,76,3012,8009\nCATTTATCGTTCAAGA-1,1,5,77,3131,8078\nCAAACGTGGTCTTGCG-1,1,4,78,3011,8146\nTAGAAACCACTAAGTA-1,1,5,79,3131,8215\nACTGATTTAGTGATTC-1,1,4,80,3011,8284\nTCGTATTTCGTCCGGA-1,1,5,81,3130,8353\nCGGAAATTTCACATCC-1,1,4,82,3011,8421\nATCCACGCTAAATGTT-1,1,5,83,3130,8490\nGTTCAATCTATGTCAA-1,1,4,84,3010,8559\nATAAAGGTCAAGTACG-1,1,5,85,3130,8628\nCAACTCCAACGTTTAG-1,1,4,86,3010,8697\nTAGGAACAGCCTCCAG-1,1,5,87,3129,8766\nATGGGAACGGAAGCGG-1,1,4,88,3010,8834\nCACACGTTTCAATGGG-1,1,5,89,3129,8903\nGGTGTTCTGTTTCTAC-1,1,4,90,3009,8972\nAGTAACGTTCATCCTG-1,1,5,91,3129,9041\nGTATAGTGGCCCATGT-1,1,4,92,3009,9109\nTCTACACGTTCATGCA-1,1,5,93,3128,9178\nAATCTGGGTAGACCCT-1,1,4,94,3009,9247\nTCGGTTAGCCATGTAG-1,1,5,95,3128,9316\nTGCCATGGCTTATAAG-1,1,4,96,3008,9384\nTAAGTAAATGTGCCGC-1,1,5,97,3128,9454\nGTGTCCGATAAGGCAT-1,1,4,98,3008,9522\nTGGCACGAGCTCGAGT-1,1,5,99,3127,9591\nACCGGTCTGAGTACGG-1,1,4,100,3007,9660\nGAACTTAGCGCCCGGT-1,1,5,101,3127,9729\nAGTAGCTAGACGCCGA-1,1,4,102,3007,9797\nATAGGAATCTAAGCTT-1,1,5,103,3127,9866\nCTTCCTGCATATTTAC-1,1,4,104,3007,9935\nCAATATGTAGATTTAC-1,1,5,105,3126,10004\nACAAGGCCTACCAGCC-1,1,4,106,3006,10072\nTTATAGTCCAAGGTGC-1,1,5,107,3126,10141\nAAACGCCCGAGATCGG-1,1,4,108,3006,10210\nCCTCGTTACGCCTGTT-1,1,5,109,3126,10279\nGAACGGTGTAAAGCAG-1,1,4,110,3006,10348\nACGCATAAATGACATG-1,1,5,111,3125,10417\nGGTTCGATGCTGAGTT-1,0,4,112,3005,10485\nCTTTGGCAGACAGAGT-1,0,5,113,3125,10554\nTTCGTGGGCTGGAAGC-1,0,4,114,3005,10623\nCAAAGGTTAAATTCAG-1,0,5,115,3125,10692\nGTTTGGCGTCAGGCAC-1,0,4,116,3005,10760\nGCTTTCTATCTCAACT-1,1,5,117,3124,10829\nTGCATCTCCGGATCTT-1,1,4,118,3004,10898\nCTGAAACGGCCCTCAG-1,1,5,119,3124,10967\nTAGCAGTAAATACGCG-1,1,4,120,3004,11035\nCGGGCTACTTAAATTG-1,1,5,121,3124,11105\nATTATGCTCAGTATTG-1,1,4,122,3004,11173\nTGATGCTCACGTAGTC-1,1,5,123,3123,11242\nGTCTAAGATGCCCAGC-1,1,4,124,3003,11311\nAACCCGATAGGGCTTC-1,1,5,125,3123,11380\nCGCTATCGTGGCTTTA-1,0,4,126,3003,11448\nCGTCTCTCGCCGAGGC-1,0,5,127,3122,11517\nAGTGGGAGTATACACG-1,1,6,0,3264,2781\nGGTCTTGGTGTTAACT-1,1,7,1,3384,2850\nGGCTGGCAGCTTTATG-1,1,6,2,3264,2919\nCGCCAATTATTGCGTT-1,1,7,3,3384,2988\nGGTAACCGGCAAAGGT-1,1,6,4,3264,3056\nTGGGACCATTGGGAGT-1,1,7,5,3383,3125\nCTGCAGGTGCTCGGCC-1,1,6,6,3263,3194\nCCGGTGCGAGTGATAG-1,1,7,7,3383,3263\nGGGTACACTCTGGAGG-1,1,6,8,3263,3332\nTAGCCAGAGGGTCCGG-1,1,7,9,3382,3401\nCTTGTGAGGACAGCGG-1,1,6,10,3263,3469\nGAAGGGCATAACCATG-1,1,7,11,3382,3538\nCAACATGGCCTGATAA-1,1,6,12,3262,3607\nCAATTTGACCGGGAAG-1,1,7,13,3382,3676\nTCTGACTGTAATGGTT-1,1,6,14,3262,3744\nTTCATAGCCTTGTAAC-1,1,7,15,3381,3813\nTGGAAACGGAGTGAAC-1,1,6,16,3262,3882\nATCGCACGATTGTTCA-1,1,7,17,3381,3951\nCGCCACCCGCATTAAC-1,1,6,18,3261,4019\nTGGACCACGGCGTTGA-1,1,7,19,3381,4089\nGTATATGTTACGGCGG-1,1,6,20,3261,4157\nGTATTCTTACCGTGCT-1,1,7,21,3380,4226\nTTCAGAGTAACCTGAC-1,1,6,22,3261,4295\nGCGGTAACCCAAATGA-1,1,7,23,3380,4364\nCTACGTGTTGCCACCA-1,1,6,24,3260,4432\nCTAGATAAACTCCTCG-1,1,7,25,3380,4501\nTCCATTAGTTGGATAG-1,1,6,26,3260,4570\nCTGGCTCCTGCGGGAT-1,1,7,27,3379,4639\nCAGTCTCTCGGCTAAT-1,1,6,28,3259,4707\nGTATGACGTGGGAAAC-1,1,7,29,3379,4776\nAGTCACTCCGCCTCAT-1,1,6,30,3259,4845\nGCAGCGGTGGGCATTA-1,1,7,31,3379,4914\nTATGGAGTTTCTCGTT-1,1,6,32,3259,4982\nACTCAACGAATGTATT-1,1,7,33,3378,5052\nAACACGCGGCCGCGAA-1,1,6,34,3258,5120\nCGATATTAGCCGCAGG-1,1,7,35,3378,5189\nAGCGTCTGAACCCGCA-1,1,6,36,3258,5258\nGATGTCCGGATCACAT-1,1,7,37,3378,5327\nGGTCACGTTAGATTCA-1,1,6,38,3258,5395\nTTAAGGATACGGAGGT-1,1,7,39,3377,5464\nGTGCGGGACCATCGGC-1,1,6,40,3257,5533\nCCATCTTGTTCACAAT-1,1,7,41,3377,5602\nTCCGAGAAGGCTAAGC-1,1,6,42,3257,5670\nTGGCGGTGTGCGATTG-1,1,7,43,3377,5739\nATCCTGCTGCAGATAG-1,1,6,44,3257,5808\nTTATGCGTCCCGGTCC-1,1,7,45,3376,5877\nCATAATGAGCGGGCGA-1,1,6,46,3256,5946\nAGACATAGATCCTTCC-1,1,7,47,3376,6015\nGGTGAAACCGGGAATG-1,1,6,48,3256,6083\nAACTGGTGTGGGCCTT-1,1,7,49,3375,6152\nGTAGCGCTGTTGTAGT-1,1,6,50,3256,6221\nTTGTTTGTGTAAATTC-1,1,7,51,3375,6290\nGGATCAAAGGACGAGG-1,1,6,52,3255,6358\nCGTAGCGCCGACGTTG-1,1,7,53,3375,6427\nCAAGTGAACTTTGGTT-1,1,6,54,3255,6496\nGTAGACAACCGATGAA-1,1,7,55,3374,6565\nCAATGGTCGGCCTGGG-1,1,6,56,3255,6633\nACAGATTAGGTTAGTG-1,1,7,57,3374,6703\nGTTATCACCTTCTGAA-1,1,6,58,3254,6771\nTGGTATCGGTCTGTAT-1,1,7,59,3374,6840\nGGAATAACCTCAAGAA-1,1,6,60,3254,6909\nATTATCTCGACAGATC-1,1,7,61,3373,6978\nCCGAGGGATGTTAGGC-1,1,6,62,3254,7046\nTGAGATCAAATACTCA-1,1,7,63,3373,7115\nAAACGAAGAACATACC-1,1,6,64,3253,7184\nCTGGTCCTAACTTGGC-1,1,7,65,3373,7253\nTGCACGAGTCGGCAGC-1,1,6,66,3253,7321\nATAGTCTTTGACGTGC-1,1,7,67,3372,7390\nTGGAGCTAAAGTTCCC-1,1,6,68,3252,7459\nGGGTGGTCCAGCCTGT-1,1,7,69,3372,7528\nCATGCATGGAGACCCT-1,1,6,70,3252,7597\nACACGGCACTATGCAT-1,1,7,71,3372,7666\nCCCTGGTATGGGCGGC-1,1,6,72,3252,7734\nGGAGGATTGAAAGGAG-1,1,7,73,3371,7803\nCCGCTGGTGCCATTCA-1,1,6,74,3251,7872\nGTTAGAGTGTGCCGCT-1,1,7,75,3371,7941\nTCGGAATGACCATCAA-1,1,6,76,3251,8009\nTTCAATTAGCCATAAT-1,1,7,77,3371,8078\nGATGTGTTGTCACAAG-1,1,6,78,3251,8147\nTCTTTCTCTTAAGGAG-1,1,7,79,3370,8216\nACCCTTTAGTTCTCCA-1,1,6,80,3250,8284\nACCACAACTCAGAACA-1,1,7,81,3370,8354\nTATGATAAATCTAACG-1,1,6,82,3250,8422\nGATCCTCTTGCGCTTA-1,1,7,83,3370,8491\nTTCTACCTTTATGTTG-1,1,6,84,3250,8560\nGAAATACCTGCTGGCT-1,1,7,85,3369,8629\nATTCTGAGTATGAACT-1,1,6,86,3249,8697\nGGATTAAGCTAAGGTC-1,1,7,87,3369,8766\nAGTACGTGGCCTGTCT-1,1,6,88,3249,8835\nTCAGGGTGCACGAAAC-1,1,7,89,3368,8904\nAAATTTACCGAAATCC-1,1,6,90,3249,8972\nTTGAGGCATTTAACTC-1,1,7,91,3368,9041\nAACCAGTATCACTCTT-1,1,6,92,3248,9110\nCACCGGAGATATCTCC-1,1,7,93,3368,9179\nGACTGGGCGCCGCAAC-1,1,6,94,3248,9247\nCACGTCTATGATGTGG-1,1,7,95,3367,9317\nTTAAGACGAACGAACC-1,1,6,96,3248,9385\nTGACCAGCTTCAAAGT-1,1,7,97,3367,9454\nAGAGTTAGAGACCGAT-1,1,6,98,3247,9523\nTTCGGACTGATGCCTT-1,1,7,99,3367,9592\nCTCGAATGGAACGTAT-1,1,6,100,3247,9660\nGGACGGCTTGCGCAAC-1,1,7,101,3366,9729\nCTAAGTACAGGGCTAC-1,1,6,102,3247,9798\nACAAATTCAGATCTGA-1,1,7,103,3366,9867\nCATGGAAATGGGACCA-1,1,6,104,3246,9935\nGGTGGACCACGTGTTA-1,1,7,105,3366,10004\nCACGACGTAATAGTAA-1,1,6,106,3246,10073\nCGGGTTCGGCACGTAT-1,1,7,107,3365,10142\nCTGGGCTATCCTTTGG-1,1,6,108,3245,10211\nGTATTAGGGTTCGCGT-1,1,7,109,3365,10280\nTCATTCGTATAATTTG-1,1,6,110,3245,10348\nAATAGCAAGCCTCCTG-1,1,7,111,3365,10417\nCATCTACCCGAGAACG-1,1,6,112,3245,10486\nGCTTCAGTGGGATTAC-1,1,7,113,3364,10555\nTCTGTGATGGAGGTTG-1,1,6,114,3244,10623\nATCCACTTTCAGACTA-1,1,7,115,3364,10692\nATGGTTACGAAACATG-1,1,6,116,3244,10761\nGGCCCAATCTAGAGGG-1,1,7,117,3364,10830\nGATGGTGAAATAACCC-1,1,6,118,3244,10898\nAGAGGGACAATTGTCC-1,1,7,119,3363,10968\nCGCGTACATTCTGGAA-1,1,6,120,3243,11036\nCAAGAAACCCTAAACT-1,1,7,121,3363,11105\nTTGGTGCGGTGTTGAA-1,1,6,122,3243,11174\nGGTTCCCTAGTGTCTC-1,1,7,123,3363,11243\nCGATAACCAATTTGAG-1,1,6,124,3243,11311\nGCCCACTGGTCCACAA-1,0,7,125,3362,11380\nGAGGGCCGGCAGAGTC-1,0,6,126,3242,11449\nCGACACGGATGCCCAC-1,0,7,127,3362,11518\nCTGTCTGTGGCTGGCT-1,1,8,0,3504,2782\nATATTATCCCGTATTT-1,1,9,1,3623,2851\nGCGCTGGCGGAAAGTC-1,1,8,2,3503,2919\nATCTAACGTCCCTATG-1,1,9,3,3623,2988\nGTCAGACAGCGTTGGA-1,1,8,4,3503,3057\nGCCAGGCTTAGTGGTA-1,1,9,5,3623,3126\nATTCAAAGTACCTGTT-1,1,8,6,3503,3195\nTGGACGTAGGCGAATC-1,1,9,7,3622,3264\nACACATTGACGCAACA-1,1,8,8,3502,3332\nGATATCAGTATGTATC-1,1,9,9,3622,3401\nTGGGCCTTGCCTGCAT-1,1,8,10,3502,3470\nCAAAGTCAGGTTAGCT-1,1,9,11,3622,3539\nGGATCCCTACCAGCTA-1,1,8,12,3502,3607\nATCGTCCAATCGAGTC-1,1,9,13,3621,3676\nACATGGCTCAATTTAG-1,1,8,14,3501,3745\nAGGCCCAGTGACTGGT-1,1,9,15,3621,3814\nGCTTCCAGCTTAGATT-1,1,8,16,3501,3882\nTGCTTGAAACCATGCA-1,1,9,17,3620,3952\nCAATATTGGACTAGTG-1,1,8,18,3501,4020\nCGTGCTGGCCTAGTCG-1,1,9,19,3620,4089\nCCTGCGATAGAACTGT-1,1,8,20,3500,4158\nGGGTAATGCTGTGTTT-1,1,9,21,3620,4227\nAACGCGAACGGCAACA-1,1,8,22,3500,4295\nTGTCGGCATGGTGGAA-1,1,9,23,3619,4364\nAGCGTACGAGAGCTAG-1,1,8,24,3500,4433\nATACTCTCGCCACTCT-1,1,9,25,3619,4502\nAATCCATGCAAGGGTG-1,1,8,26,3499,4570\nTTAAACAGAGTCCCGC-1,1,9,27,3619,4639\nCCACAGCTGAAATCAT-1,1,8,28,3499,4708\nCGGTTCCGGCTTCTTG-1,1,9,29,3618,4777\nGACGTGAGACTCCATG-1,1,8,30,3499,4846\nTCGTTGGCTCGTCAAT-1,1,9,31,3618,4915\nGGTGAACGGGCTAGCC-1,1,8,32,3498,4983\nGCACTGTGCAAATGTA-1,1,9,33,3618,5052\nACGAGAACCCATCACG-1,1,8,34,3498,5121\nCCAGCTACGCCTCATA-1,1,9,35,3617,5190\nTCCCGGTCAGGAATTT-1,1,8,36,3497,5258\nTCGCATTCAATGACTT-1,1,9,37,3617,5327\nCTGGTTCAACGCATCA-1,1,8,38,3497,5396\nGGTGATTTCATCTTGT-1,1,9,39,3617,5465\nCACCCTTTCCTCGCTC-1,1,8,40,3497,5533\nCAACTTGTAGTGGGCA-1,1,9,41,3616,5603\nAATATCAAGGTCGGAT-1,1,8,42,3496,5671\nACTCAGACCTGCTTCT-1,1,9,43,3616,5740\nTTGGAGTCTCCCTTCT-1,1,8,44,3496,5809\nGGATACTCATGAATTG-1,1,9,45,3616,5878\nTGGGCACAAACAGAAC-1,1,8,46,3496,5946\nGAGCCACGGTAGTAGG-1,1,9,47,3615,6015\nTCGATAGGCTAGTCGC-1,1,8,48,3495,6084\nTAACCGCCCGCAGTGC-1,1,9,49,3615,6153\nGCCTATTTGCTACACA-1,1,8,50,3495,6221\nTTGACGATTCAGCACG-1,1,9,51,3615,6290\nTTAAACCGGTAGCGAC-1,1,8,52,3495,6359\nACCGAAAGGGCCCTGC-1,1,9,53,3614,6428\nACGTTCCGCGCTCCGT-1,1,8,54,3494,6496\nATACCAGGCTAATAGA-1,1,9,55,3614,6566\nCGGCTTTGTATGATAA-1,1,8,56,3494,6634\nCTTGACCCGAAAGATA-1,1,9,57,3613,6703\nCGCAGAAACATTTGCG-1,1,8,58,3494,6772\nGACCCGTCGCCGGCTA-1,1,9,59,3613,6841\nAATCGGGACACTACGA-1,1,8,60,3493,6909\nGTCACAAAGTTTCCAA-1,1,9,61,3613,6978\nTATATTCGCGTCGATA-1,1,8,62,3493,7047\nCCTCCCGACAATCCCT-1,1,9,63,3612,7116\nCGACATGCGATCTTCT-1,1,8,64,3493,7184\nAACACGACTGTACTGA-1,1,9,65,3612,7253\nCCCAACCACACTAACA-1,1,8,66,3492,7322\nCACCGCCGACCAGCGA-1,1,9,67,3612,7391\nTGGTATCGCATCCCAA-1,1,8,68,3492,7460\nCAGAGTGATTTAACGT-1,1,9,69,3611,7529\nAACCCTGGTGGAACCA-1,1,8,70,3492,7597\nGTCAGTTGTGCTCGTT-1,1,9,71,3611,7666\nATTGACGTAACTCGGT-1,1,8,72,3491,7735\nGATGTCGGTCAACTGC-1,1,9,73,3611,7804\nAGGGCAGCGGCGTGGT-1,1,8,74,3491,7872\nACATCGTTAACCTAGT-1,1,9,75,3610,7941\nTCCATTGTGACCTCGT-1,1,8,76,3490,8010\nTGTTTAATACTTCATC-1,1,9,77,3610,8079\nTTGCTGGCCGGGCTTC-1,1,8,78,3490,8147\nCATATTATTTGCCCTA-1,1,9,79,3610,8217\nCTGCCTAGCCACCAAG-1,1,8,80,3490,8285\nACGAGATATTTGCTTA-1,1,9,81,3609,8354\nGACTACAATTGCTCGT-1,1,8,82,3489,8423\nAACGTGATGAAGGACA-1,1,9,83,3609,8492\nACTCTCTTATACACGA-1,1,8,84,3489,8560\nCGCATCATGGCTTCAG-1,1,9,85,3609,8629\nCGGCTCTTCGTCGAAC-1,1,8,86,3489,8698\nATTCTTCGTACTTATG-1,1,9,87,3608,8767\nAGTGAGGGTTTCTGAC-1,1,8,88,3488,8835\nGCCAGGCGTTCGCATG-1,1,9,89,3608,8904\nGACTAACACAGCACCT-1,1,8,90,3488,8973\nCAATGGAATCTACATA-1,1,9,91,3608,9042\nGTGGTCAGCGAAGTAT-1,1,8,92,3488,9111\nATGGCTGGAAATGGCC-1,1,9,93,3607,9180\nATCAGGTCGCCATTGC-1,1,8,94,3487,9248\nTATCACCATGTAAAGT-1,1,9,95,3607,9317\nAGCGCTTATGGGCAAG-1,1,8,96,3487,9386\nAAGCGGCGTCATGGGT-1,1,9,97,3606,9455\nACTAATACGTCAGGCG-1,1,8,98,3487,9523\nGGCTGAGCATCGTAAG-1,1,9,99,3606,9592\nCGGTTGGGTTCAAGTT-1,1,8,100,3486,9661\nGACTGATTGGTCACAA-1,1,9,101,3606,9730\nAGACGGGCCGATTTAA-1,1,8,102,3486,9798\nACCAGTGCCCGGTCAA-1,1,9,103,3605,9868\nGTCCTTTAATGACTTC-1,1,8,104,3486,9936\nCCTACAAGTCCGGAAT-1,1,9,105,3605,10005\nGCCTGCTACACTGAGA-1,1,8,106,3485,10074\nGACTCGGTCGGCGGAT-1,1,9,107,3605,10143\nCTAGACATATATGTAG-1,1,8,108,3485,10211\nTCGCCCAACTGACTCC-1,1,9,109,3604,10280\nAAACTAACGTGGCGAC-1,1,8,110,3485,10349\nAACTGAGGTCAGCGTC-1,1,9,111,3604,10418\nACAATGATTCTTCTAC-1,1,8,112,3484,10486\nATAAGTACCCGATTGT-1,1,9,113,3604,10555\nATTGGGAGTTCTGTAA-1,1,8,114,3484,10624\nCGAACATAGTCAGAAA-1,1,9,115,3603,10693\nTAGCTCAGATCCTAGT-1,1,8,116,3483,10761\nGTGTCGTATTCACCTT-1,1,9,117,3603,10831\nCTCACCGATCCAAACT-1,1,8,118,3483,10899\nATATGTGCACAAACCA-1,1,9,119,3603,10968\nCAGTCCAACGCCTTCT-1,1,8,120,3483,11037\nTCGTCCGGGTACACTC-1,1,9,121,3602,11106\nGCAGAAACGTAATCCA-1,1,8,122,3482,11174\nTTCGAGCCGGCGCTAC-1,1,9,123,3602,11243\nGGAAGATAAGACTGTA-1,1,8,124,3482,11312\nATAAGCAAACACCGAG-1,0,9,125,3602,11381\nGCATAAATTGAACGCC-1,0,8,126,3482,11449\nCGCCGGTGTCGCAGTA-1,0,9,127,3601,11518\nGACCTGGTCTGGGCGT-1,1,10,0,3743,2782\nAGCCGCTTGATTAGCG-1,1,11,1,3863,2852\nCCCGGCTAGGTGAGAA-1,1,10,2,3743,2920\nCGAGCCGAGCACTCGA-1,1,11,3,3862,2989\nTAGTGCTTGAATCCTT-1,1,10,4,3742,3058\nCAACCGCACCTAGACA-1,1,11,5,3862,3127\nACCACTGTTCAAGAAG-1,1,10,6,3742,3195\nAGATGCTATAACGAGC-1,1,11,7,3862,3264\nAATTACTCGTACGCTC-1,1,10,8,3742,3333\nCGTCAATCTTTAACAT-1,1,11,9,3861,3402\nCCAAAGCAGTTGGTTG-1,1,10,10,3741,3470\nCCATATTGGATCATGA-1,1,11,11,3861,3539\nCGTACCGAAAGTCTAG-1,1,10,12,3741,3608\nCTCGAGATCCAAAGCA-1,1,11,13,3861,3677\nTGGATAGAGTAACAGA-1,1,10,14,3741,3745\nTCACAGATCCTCAAAC-1,1,11,15,3860,3815\nAGAGCTACGAAAGCAT-1,1,10,16,3740,3883\nTGCGTGATTGGGTGTC-1,1,11,17,3860,3952\nCACATGTTTGGACATG-1,1,10,18,3740,4021\nTTCGCATCCGGAAGCA-1,1,11,19,3860,4090\nCCCTAGTGTCAGGTGT-1,1,10,20,3740,4158\nTTACCGCCTTAGGGAA-1,1,11,21,3859,4227\nCCAGTCCATTATTCGA-1,1,10,22,3739,4296\nCGTAAACGCTTGAGTG-1,1,11,23,3859,4365\nATTCCTTCCAGGCGGT-1,1,10,24,3739,4433\nTTCCTTTCTGTGTTGC-1,1,11,25,3858,4502\nAGTTGACATCGGCTGG-1,1,10,26,3739,4571\nAACTCGATGGCGCAGT-1,1,11,27,3858,4640\nGATAAGGCAGATGCAA-1,1,10,28,3738,4709\nGGCTGGCTAGCTTAAA-1,1,11,29,3858,4778\nCCTCATGCAGCTACGA-1,1,10,30,3738,4846\nGACGCCTGTTGCAGGG-1,1,11,31,3857,4915\nTAATTAGATGGATATG-1,1,10,32,3738,4984\nGAGGGCATCGCGTATC-1,1,11,33,3857,5053\nCTTGTGAGTCTTTGAC-1,1,10,34,3737,5121\nTCAACACATTGGGTAA-1,1,11,35,3857,5190\nACTGTATACGCGAGCA-1,1,10,36,3737,5259\nGTGAAACGTGCTCCAC-1,1,11,37,3856,5328\nCGAGTGCTATAGTTCG-1,1,10,38,3736,5396\nGTACTGCATGAAGCGT-1,1,11,39,3856,5466\nGTAACTTGCGGCAGTC-1,1,10,40,3736,5534\nGAATCGCCGGACACGG-1,1,11,41,3856,5603\nGGGAGTAATGGCTGGC-1,1,10,42,3736,5672\nCATGAACCGACATTTG-1,1,11,43,3855,5741\nTCTGTCATACAAGAGC-1,1,10,44,3735,5809\nGTCGTCAATTATAAGG-1,1,11,45,3855,5878\nTAAAGAGCCCGAAACC-1,1,10,46,3735,5947\nGTACTGAGGTCGTAAC-1,1,11,47,3855,6016\nAAAGACCCAAGTCGCG-1,1,10,48,3735,6084\nCGTCAGTGCGCACAAG-1,1,11,49,3854,6153\nTGTATCCTTATTCCAT-1,1,10,50,3734,6222\nATTCTCGTCTCTTTAG-1,1,11,51,3854,6291\nAAAGTCACTGATGTAA-1,1,10,52,3734,6360\nTGTCTACAGTTTCTGT-1,1,11,53,3854,6429\nTTAACGTCGCAAGACC-1,1,10,54,3734,6497\nCTATGTCTATTGAAAC-1,1,11,55,3853,6566\nTCGGGTGAAACTGCTA-1,1,10,56,3733,6635\nTGTCCCGACATAGCAC-1,1,11,57,3853,6704\nACAGCATAGAGCCAGT-1,1,10,58,3733,6772\nATATTCCCACAGGTCA-1,1,11,59,3853,6841\nTTGGATCGACTTCTGG-1,1,10,60,3733,6910\nCACCATCGGAGGAGAC-1,1,11,61,3852,6979\nTCGTTCGTTATTATGT-1,1,10,62,3732,7047\nCTTAACTTCGAAGTAC-1,1,11,63,3852,7117\nGCACAAGTGTCGGAAG-1,1,10,64,3732,7185\nTACCAGCTAGGTTTAA-1,1,11,65,3851,7254\nACGTACAGATTTCTCT-1,1,10,66,3732,7323\nAATTTGGTTCCAAAGA-1,1,11,67,3851,7392\nGTAAGGATTTGTCGGA-1,1,10,68,3731,7460\nCATCATCTACCCGGAC-1,1,11,69,3851,7529\nACGATGGATCCGATGC-1,1,10,70,3731,7598\nCACTCAGCTCTTGAGG-1,1,11,71,3850,7667\nTAGATCCGAAGTCGCA-1,1,10,72,3731,7735\nTGAAACTTATGCAAGC-1,1,11,73,3850,7804\nGCGATTCTGGAAGCAG-1,1,10,74,3730,7873\nCAAACTATTGAGCTTC-1,1,11,75,3850,7942\nTAGAATTAAGGGCAAC-1,1,10,76,3730,8010\nCGAAACATAGATGGCA-1,1,11,77,3849,8080\nGATGGTGCCCTAGGCA-1,1,10,78,3729,8148\nCCCGCAGGGCCCAAAG-1,1,11,79,3849,8217\nACAGCGCACCCGCAGC-1,1,10,80,3729,8286\nGGTAAATGTGCGTTAC-1,1,11,81,3849,8355\nGTCCTTCTAGTGGGTT-1,1,10,82,3729,8423\nGGAAGCTCGCTTACAG-1,1,11,83,3848,8492\nCACCGATACACCGAGC-1,1,10,84,3728,8561\nCAGCCGGGCCCTCTAT-1,1,11,85,3848,8630\nCGGAGCTTATAACACC-1,1,10,86,3728,8698\nATTACAACTACCGGCC-1,1,11,87,3848,8767\nTCCTCTGGCCCATTAG-1,1,10,88,3728,8836\nCGGCACCGTTAGCGCC-1,1,11,89,3847,8905\nTCGGTCCCTGACTCCA-1,1,10,90,3727,8974\nTGGTTGGAGGATCCTG-1,1,11,91,3847,9043\nCTGCGGTAGTCACGTG-1,1,10,92,3727,9111\nGTGCCTCAGTGTACGG-1,1,11,93,3847,9180\nATCGTTCACTTTCGCC-1,1,10,94,3727,9249\nACTTACCGGGCGCGCA-1,1,11,95,3846,9318\nCTAGACTGCATTTCGT-1,1,10,96,3726,9386\nTTGCCGGTGATCCCTC-1,1,11,97,3846,9455\nCTGTCACGCCAGGCGC-1,1,10,98,3726,9524\nCGGTATAGGTATTAGC-1,1,11,99,3846,9593\nCCAACGCTTGCCAGGG-1,1,10,100,3726,9661\nCGTTGAGTAATTGCGT-1,1,11,101,3845,9731\nTGTATTTACCTAATGC-1,1,10,102,3725,9799\nTAAATGCCGTCTCATG-1,1,11,103,3845,9868\nCGGTTTATGAAGGAAC-1,1,10,104,3725,9937\nGCAAGATGTGTTCGCG-1,1,11,105,3844,10006\nAAAGGTAAGCTGTACC-1,1,10,106,3725,10074\nGTACGTCACGTATTAA-1,1,11,107,3844,10143\nAGTACCTTCGAGTGCT-1,1,10,108,3724,10212\nATTGTGACTTCGCTGC-1,1,11,109,3844,10281\nTGTATCAGACTGAAGC-1,1,10,110,3724,10349\nGAGACCCTGCAACGCC-1,1,11,111,3843,10418\nTGGGTGGGATGTCATT-1,1,10,112,3724,10487\nGGCTAATGATTGAAAT-1,1,11,113,3843,10556\nATAACGTTACCTCCAC-1,1,10,114,3723,10625\nTGCGAGATGGCGGCCA-1,1,11,115,3843,10694\nCACACTTGTATTGCGA-1,1,10,116,3723,10762\nGCTGGTGACTCGTAGT-1,1,11,117,3842,10831\nCGACACCGCTTAAGGA-1,1,10,118,3723,10900\nGTAACAACTGACCTTG-1,1,11,119,3842,10969\nCAACTGAGGGTATGAC-1,1,10,120,3722,11037\nCTAATTATGAAGCGTA-1,1,11,121,3842,11106\nCCGATCTTAAGAGGCT-1,1,10,122,3722,11175\nCGACTCGGTACACGGT-1,1,11,123,3841,11244\nTGCTGCGTCAGAGTTA-1,1,10,124,3721,11312\nAGAGTTGCAGGCCTCC-1,0,11,125,3841,11381\nACTGGCGAACCTGCGT-1,0,10,126,3721,11450\nACTAAGGACGCACACC-1,0,11,127,3841,11519\nCGTCCAGATGGCTCCA-1,1,12,0,3983,2783\nACTATCGCCGGCTAAA-1,1,13,1,4102,2852\nGATAGCGTACCACGCG-1,1,12,2,3982,2921\nAGGACTTATAGGAGAA-1,1,13,3,4102,2990\nTAGTCGGGATTCTTCG-1,1,12,4,3982,3058\nACCATTAAGGGTGTCA-1,1,13,5,4101,3127\nTTAATGTGTTTGCAGG-1,1,12,6,3981,3196\nTCCGCAGCCACCTAGC-1,1,13,7,4101,3265\nGAGGAGATCCTCATGC-1,1,12,8,3981,3333\nGGTCCTTCATACGACT-1,1,13,9,4101,3402\nCCCTGTTGGCAAAGAC-1,1,12,10,3981,3471\nGTGCCTAGCTATGCTT-1,1,13,11,4100,3540\nGTCATCTCCTACAGCT-1,1,12,12,3980,3609\nGCGAGAAACGGGAGTT-1,1,13,13,4100,3678\nCGTCGCGGCGGGATTT-1,1,12,14,3980,3746\nCATTGTGTGCTAGATC-1,1,13,15,4100,3815\nCGCGGCAGTATTACGG-1,1,12,16,3980,3884\nGAAATGGGATGTAAAC-1,1,13,17,4099,3953\nCATTCCCTAAGTACAA-1,1,12,18,3979,4021\nAGTTCTGCGTTGTATC-1,1,13,19,4099,4090\nACCCTATAGGACTGAG-1,1,12,20,3979,4159\nTTAGATAGGTCGATAC-1,1,13,21,4099,4228\nATTGATAGCAACGAGA-1,1,12,22,3979,4296\nTCTCGGCTCCAGGACT-1,1,13,23,4098,4366\nCTTGAGGTTATCCCGA-1,1,12,24,3978,4434\nAAGAAGGATCAGTTAG-1,1,13,25,4098,4503\nGTACGACGGCGCTGCG-1,1,12,26,3978,4572\nCTTATGTTGACTACCA-1,1,13,27,4098,4641\nCGGCACTCAAGAAAGT-1,1,12,28,3978,4709\nGTCAAAGTTTACATAG-1,1,13,29,4097,4778\nCTCCTAAGTTATGTCT-1,1,12,30,3977,4847\nACTGTGCTAGTAGATC-1,1,13,31,4097,4916\nGTTTGGCCGCTCAGCG-1,1,12,32,3977,4984\nTTATCCAATCGAACTC-1,1,13,33,4096,5053\nCCGTACCCAAGCGCCA-1,1,12,34,3977,5122\nCATACAAAGCCGAACC-1,1,13,35,4096,5191\nGGTCGGTAATTAGACA-1,1,12,36,3976,5259\nAGCGGGAAGGGTCCAT-1,1,13,37,4096,5329\nGCCCACCAAGGCTGTC-1,1,12,38,3976,5397\nGTAGACGTCGTTACAT-1,1,13,39,4095,5466\nGAGCATCATCCCTGGG-1,1,12,40,3976,5535\nAGGTAACCTCCTATTC-1,1,13,41,4095,5604\nGGTTTGAGTGCTGGAA-1,1,12,42,3975,5672\nGCACTAGTCGCGCTAT-1,1,13,43,4095,5741\nGGTAACTATGTATCTG-1,1,12,44,3975,5810\nGCGGTCCCTAGACGCA-1,1,13,45,4094,5879\nCGAGCGTTGATCAGCC-1,1,12,46,3974,5947\nAATCCAAGGGCCTGAG-1,1,13,47,4094,6016\nCCGTGCCCATGACGGC-1,1,12,48,3974,6085\nGAAATTCACATCGCTG-1,1,13,49,4094,6154\nCTCTGCGAAGCAAGCA-1,1,12,50,3974,6223\nAGTAGGTAACATACAT-1,1,13,51,4093,6292\nATTGGGAATATCTTGG-1,1,12,52,3973,6360\nTAGAGCTACGAAGAAC-1,1,13,53,4093,6429\nTGCGGCATAGTTCAAC-1,1,12,54,3973,6498\nCCGCCGGTCAACACAC-1,1,13,55,4093,6567\nTCGTATAGTGCAATTA-1,1,12,56,3973,6635\nTAGTTTATTCTTGCTT-1,1,13,57,4092,6704\nGATATCTCATGCAATA-1,1,12,58,3972,6773\nCGTTTAAGCGGAGCAC-1,1,13,59,4092,6842\nCATGCTGGCTCCAATT-1,1,12,60,3972,6910\nGAAACAGCCATGCAGT-1,1,13,61,4092,6980\nAGTTTCGCAGGTCGGA-1,1,12,62,3972,7048\nCTCATGGCTCACAATC-1,1,13,63,4091,7117\nAACCGTTGTGTTTGCT-1,1,12,64,3971,7186\nACCCTTCATCTGCGAA-1,1,13,65,4091,7255\nTGCGGTGAAATTTCAT-1,1,12,66,3971,7323\nCAAATTGTCAGCAAGC-1,1,13,67,4091,7392\nGAGGTACATCCATCTT-1,1,12,68,3971,7461\nAAATGGCATGTCTTGT-1,1,13,69,4090,7530\nTCATCCCAGAGGGTGG-1,1,12,70,3970,7598\nCGTAGCGAATTGTCAG-1,1,13,71,4090,7667\nCCTAGTTAGTCGCATG-1,1,12,72,3970,7736\nGAAACTCTAATGAAGG-1,1,13,73,4089,7805\nTTGTATCACACAGAAT-1,1,12,74,3970,7874\nTTCAAGCCGAGCTGAG-1,1,13,75,4089,7943\nAGGTACGATATTGCCA-1,1,12,76,3969,8011\nTTAAGCCGACAACTTC-1,1,13,77,4089,8080\nGTCTTAGTACAGCCGG-1,1,12,78,3969,8149\nTGGGTAAGGTTCCCGC-1,1,13,79,4088,8218\nCTACGCCATTTCCGAT-1,1,12,80,3969,8286\nGACCGTCAGGTCGTGA-1,1,13,81,4088,8355\nTAGTTAAGATAGGATA-1,1,12,82,3968,8424\nGAATATTCGGAGTCCC-1,1,13,83,4088,8493\nCAAACTACGATAGAGA-1,1,12,84,3968,8561\nCAGGAAGACTTTATAT-1,1,13,85,4087,8631\nTTCGTAATCCCAGCGG-1,1,12,86,3967,8699\nGTGAGGAGCGGTTGAG-1,1,13,87,4087,8768\nCAATCCCTATACCAGC-1,1,12,88,3967,8837\nAGGGAAACGAGGTACT-1,1,13,89,4087,8906\nTCCTGCCAACTGGAGA-1,1,12,90,3967,8974\nAATTTGGGACATAGTA-1,1,13,91,4086,9043\nAACTCCTAATCCCATG-1,1,12,92,3966,9112\nGCTCATTACTGCATGT-1,1,13,93,4086,9181\nTCTCGAACGAGGTCAC-1,1,12,94,3966,9249\nTCACTACGACCAATGC-1,1,13,95,4086,9318\nGTGATGCACAACATCT-1,1,12,96,3966,9387\nCCGACGTAAACACAAC-1,1,13,97,4085,9456\nTGGGATGCACTCATTC-1,1,12,98,3965,9524\nTTCCATCATGCGGTGA-1,1,13,99,4085,9594\nTGCACAGTGAAGTTAT-1,1,12,100,3965,9662\nCTATTGTGTTTGGTCA-1,1,13,101,4085,9731\nTGCGAGCCCTTCCGCG-1,1,12,102,3965,9800\nTTGAAAGGTGTAAAGG-1,1,13,103,4084,9869\nGGTGCAGAGCCTATCG-1,1,12,104,3964,9937\nACTATATGCTGTGTTC-1,1,13,105,4084,10006\nTCCTGGCGCTGCCTGG-1,1,12,106,3964,10075\nGAGCCGAGCGTTTATT-1,1,13,107,4084,10144\nAGAATGCGGGTTCGGA-1,1,12,108,3964,10212\nATGCGACAATTGGTCC-1,1,13,109,4083,10281\nTTCCGGCTCGACTTCT-1,1,12,110,3963,10350\nTGATTATGGCACGCAG-1,1,13,111,4083,10419\nGGTTTGACAAGAAGCT-1,1,12,112,3963,10488\nGCAGCTATGGACAGGT-1,1,13,113,4082,10557\nCACCATGATCGCAAAG-1,1,12,114,3963,10625\nGTCGGAAGGATACCAG-1,1,13,115,4082,10694\nGGCCCAGTTATCAGCA-1,1,12,116,3962,10763\nGGGCCTATTTAAGTAT-1,1,13,117,4082,10832\nGTTGTTACATTGCGCT-1,1,12,118,3962,10900\nCAACTCCGTAACTTGC-1,1,13,119,4081,10969\nGATCTTTGCTCAAAGA-1,1,12,120,3962,11038\nTCGCTTAATTACGAAG-1,1,13,121,4081,11107\nCGATCATTAGAGGCAC-1,1,12,122,3961,11175\nTGTTCTCTACTCCCTA-1,1,13,123,4081,11245\nGCTTAGGGAAGCGGTA-1,1,12,124,3961,11313\nCAGGTTTAGTACTACA-1,0,13,125,4080,11382\nAAGCGGAGTGCGCGCA-1,0,12,126,3960,11451\nTCAGATGGAGACGTAG-1,0,13,127,4080,11520\nTCGCACTAACGTTTGT-1,1,14,0,4222,2784\nCACGTCGGGTTCTAGA-1,1,15,1,4341,2853\nGGAGTACACATGAGCT-1,1,14,2,4222,2921\nGTGTGTCGACGTCGCT-1,1,15,3,4341,2990\nGAAGCTCTTTGCTTAG-1,1,14,4,4221,3059\nACACCGAGCGCTCTTT-1,1,15,5,4341,3128\nCGTAATAATTACGAGT-1,1,14,6,4221,3196\nCATCAACACCTACTAA-1,1,15,7,4340,3265\nCCAAGTTTCTACAGAT-1,1,14,8,4221,3334\nACGGGTCATGTGACTT-1,1,15,9,4340,3403\nAGTGTGCTAAGATCGC-1,1,14,10,4220,3472\nGGCGGTTTGCCGGTGC-1,1,15,11,4340,3541\nGTATAATCTCCCGGAT-1,1,14,12,4220,3609\nTAGTCCGTATGCATAA-1,1,15,13,4339,3678\nCACTTCGTCTTATCTC-1,1,14,14,4219,3747\nCATCCGCAGGCCCGAA-1,1,15,15,4339,3816\nCCCTGATGTAACTCGT-1,1,14,16,4219,3884\nCCATAGTCAGTAACCC-1,1,15,17,4339,3953\nCGGGCCATAGCCGCAC-1,1,14,18,4219,4022\nCTCCGGCTTGTAGACA-1,1,15,19,4338,4091\nAACTTGCGTTCTCGCG-1,1,14,20,4218,4159\nAATGAGTTCGCATATG-1,1,15,21,4338,4229\nCGAGGCCAGGCATTGG-1,1,14,22,4218,4297\nTCTGCGTCCGGTTTCT-1,1,15,23,4338,4366\nCAATCCTGCCGTGGAG-1,1,14,24,4218,4435\nCTGAGCAAGTAACAAG-1,1,15,25,4337,4504\nGGGTACCCACGGTCCT-1,1,14,26,4217,4572\nACGGAATTTAGCAAAT-1,1,15,27,4337,4641\nGGGCGGTCCTATTGTC-1,1,14,28,4217,4710\nATGTTACGAGCAATAC-1,1,15,29,4337,4779\nAACCATGGGATCGCTA-1,1,14,30,4217,4847\nTCGCATCCCTAAGTGT-1,1,15,31,4336,4916\nACTTAGTACGACAAGA-1,1,14,32,4216,4985\nGAGCTCTCGGACCTAA-1,1,15,33,4336,5054\nTCTATTACGCTGGCGA-1,1,14,34,4216,5123\nAGATACGACTTCATAT-1,1,15,35,4335,5192\nCGCTATACCGCCCACT-1,1,14,36,4216,5260\nCAGTGTCCGCAGAATG-1,1,15,37,4335,5329\nCCATCCATACCAAGTC-1,1,14,38,4215,5398\nAACCCAGAGACGGAGA-1,1,15,39,4335,5467\nGAAGAACGGTGCAGGT-1,1,14,40,4215,5535\nGATAAATCGGTGGATG-1,1,15,41,4334,5604\nCAGCTCGTGCTTGTGT-1,1,14,42,4215,5673\nGAGTACGGGTATACAA-1,1,15,43,4334,5742\nCATCGCCCGCGGCCAA-1,1,14,44,4214,5810\nTCTTACAGAGGTACCG-1,1,15,45,4334,5880\nTGGAAGACGAACACCA-1,1,14,46,4214,5948\nGTTGTCGTGTTAGTTG-1,1,15,47,4333,6017\nCCAAGGAACAGAGAGG-1,1,14,48,4214,6086\nCTGCACCTGGAACCGC-1,1,15,49,4333,6155\nCGCTTTCATACCGGTG-1,1,14,50,4213,6223\nGTTCTTCCCTCGATGT-1,1,15,51,4333,6292\nATTTAACTCGTATTAC-1,1,14,52,4213,6361\nAACGATAGAAGGGCCG-1,1,15,53,4332,6430\nTATCCTGCATGGGAAT-1,1,14,54,4212,6498\nAGGCCCATTGTACAGG-1,1,15,55,4332,6567\nCCGGCGCATATTGGAT-1,1,14,56,4212,6636\nATCTGTAATTGTACCC-1,1,15,57,4332,6705\nGAGCGAGGGAGTACCG-1,1,14,58,4212,6773\nTTATTAGGGAAGCATC-1,1,15,59,4331,6843\nCTTCTTACGTCGTATA-1,1,14,60,4211,6911\nGAAGTGCTGGATAGCT-1,1,15,61,4331,6980\nGTGCAACAAATGTGGC-1,1,14,62,4211,7049\nCATGCGTTAGACAGAA-1,1,15,63,4331,7118\nACACACTTTCTACACG-1,1,14,64,4211,7186\nAGCCCTAAGCGAAGTT-1,1,15,65,4330,7255\nATTAATTCGGTCACTC-1,1,14,66,4210,7324\nAACAGGAAATCGAATA-1,1,15,67,4330,7393\nACGTTTAGTTGTGATC-1,1,14,68,4210,7461\nTCCTTCAGTGGTCGAA-1,1,15,69,4330,7530\nCGAACGCCCAGTGCCG-1,1,14,70,4210,7599\nCCTCGAAGTGGACGGG-1,1,15,71,4329,7668\nCTCTGTTTGAGGATTC-1,1,14,72,4209,7737\nTGGGCACGTTCTATGG-1,1,15,73,4329,7806\nACTATTCGTCCGTGGT-1,1,14,74,4209,7874\nCCTCTGGCCTAGACGG-1,1,15,75,4328,7943\nCCATAAACAACCCGAC-1,1,14,76,4209,8012\nCATAGTACATTGAGAG-1,1,15,77,4328,8081\nATTTCATTATTTCGCG-1,1,14,78,4208,8149\nCAACTATATCGAATGC-1,1,15,79,4328,8218\nCTAGTATTCGGAATTA-1,1,14,80,4208,8287\nGTGGAACCTACATGCG-1,1,15,81,4327,8356\nCCTAAAGGCTGACGCT-1,1,14,82,4208,8424\nCGTGACATTGGGTCGT-1,1,15,83,4327,8494\nCCAATCGGTAGATCGA-1,1,14,84,4207,8562\nATTGTCGCAATACCTT-1,1,15,85,4327,8631\nAAATTACACGACTCTG-1,1,14,86,4207,8700\nCACTCCTCTCGGTCGG-1,1,15,87,4326,8769\nAAATAACCATACGGGA-1,1,14,88,4207,8837\nAGTTACTCTATCGTGG-1,1,15,89,4326,8906\nCGTTAGCTCACAACTG-1,1,14,90,4206,8975\nGAATGTATGGCAGGTC-1,1,15,91,4326,9044\nGCAACCACCAGACCGG-1,1,14,92,4206,9112\nTCACTCGTGCAACGGC-1,1,15,93,4325,9181\nAAACAGAGCGACTCCT-1,1,14,94,4205,9250\nCAGCCTCTCCTCAAGA-1,1,15,95,4325,9319\nTTGCGTGAACGCTTAG-1,1,14,96,4205,9387\nCCGCCTGCGAATTGGT-1,1,15,97,4325,9457\nAGATGAGGGTTGCGAT-1,1,14,98,4205,9525\nCGGTGGGCTCCAGCCT-1,1,15,99,4324,9594\nGGCAGCGGTAATCCTA-1,1,14,100,4204,9663\nGCTAGCAGGGAGTGGG-1,1,15,101,4324,9732\nCTCAAGACATTAGCGC-1,1,14,102,4204,9800\nCACGGCGCGCCAAAGG-1,1,15,103,4324,9869\nTGCAATTTGGGCACGG-1,1,14,104,4204,9938\nATGCCAATCGCTCTGC-1,1,15,105,4323,10007\nGCTGGACCCAAAGTGG-1,1,14,106,4203,10075\nATTCCTAAGACGTGGA-1,1,15,107,4323,10144\nTCCGGAGGAAGGGCTG-1,1,14,108,4203,10213\nTCGGTGACCGCTCCGG-1,1,15,109,4323,10282\nTCCGAAGTAGTCACCA-1,1,14,110,4203,10351\nCATGTAGGAGCGCCAA-1,1,15,111,4322,10420\nCACAAGAAAGATATTA-1,1,14,112,4202,10488\nAGGGTCAGTAACCCTA-1,1,15,113,4322,10557\nTAAGCCCTTACGACCA-1,1,14,114,4202,10626\nATACCGTCATCCATAA-1,1,15,115,4322,10695\nGGACGTCCATAGTTGG-1,1,14,116,4202,10763\nCATCAAACTGGCGCCC-1,1,15,117,4321,10832\nAAACGTGTTCGCCCTA-1,1,14,118,4201,10901\nAAATTGGTGAGAAGCA-1,1,15,119,4321,10970\nGGTCATTGTAGTCATA-1,1,14,120,4201,11038\nTGCAGTGAGGCTCGGG-1,1,15,121,4320,11108\nGAACATTAGTATGTTA-1,1,14,122,4201,11176\nGGTTTGCGAACACGTA-1,1,15,123,4320,11245\nACACAAATATTCCTAG-1,1,14,124,4200,11314\nTTGGGTTTATTCAGCG-1,0,15,125,4320,11383\nATTCGCAGAGGACACT-1,0,14,126,4200,11451\nGATTTAGTGCGTACTG-1,0,15,127,4319,11520\nTAGAAACACAATAGTG-1,1,16,0,4461,2784\nCAGTAGATGATGTCCG-1,1,17,1,4581,2853\nTCTTAACTCGGATGTA-1,1,16,2,4461,2922\nTACATCTTGTTTCTTG-1,1,17,3,4580,2991\nTTCATAGGGTGTCCAT-1,1,16,4,4461,3059\nTGAAGTAGCTTACGGA-1,1,17,5,4580,3129\nGCACAAGTGGATCATA-1,1,16,6,4460,3197\nGGGCGAATTTCTCCAC-1,1,17,7,4580,3266\nATGTTCCTGCCCACCT-1,1,16,8,4460,3335\nGCTCAACCTCTTAGAG-1,1,17,9,4579,3404\nATAGCTGCTCTTGTTA-1,1,16,10,4460,3472\nCGTCAGCTATTTACTC-1,1,17,11,4579,3541\nATCTGATAGTGTCTTA-1,1,16,12,4459,3610\nTGCACTATGTGAGTGC-1,1,17,13,4579,3679\nCCGACAAACACATGAG-1,1,16,14,4459,3747\nGCCTTGTATATGCAGT-1,1,17,15,4578,3816\nATAATACCGTTAGCCG-1,1,16,16,4459,3885\nACACTCCAATGTCACT-1,1,17,17,4578,3954\nAGTTGCTGACTGATAT-1,1,16,18,4458,4022\nGGCGCTCCTCATCAAT-1,1,17,19,4578,4092\nTGCCTGACATCGGTCA-1,1,16,20,4458,4160\nTTGGCCATCTTGCGCT-1,1,17,21,4577,4229\nCAGTGGTTGCACATGA-1,1,16,22,4457,4298\nAGAATTGTTTGACATA-1,1,17,23,4577,4367\nAAATGCTCGTTACGTT-1,1,16,24,4457,4435\nCACCTAATAGAGTCGT-1,1,17,25,4577,4504\nCATTTCTAGCAGACTA-1,1,16,26,4457,4573\nCCGAAAGTGGTGAGCA-1,1,17,27,4576,4642\nAGTCAGCCACCGCCTG-1,1,16,28,4456,4710\nTCATCACTCGAGCTCG-1,1,17,29,4576,4779\nCTCAGGACTCACCTGT-1,1,16,30,4456,4848\nCGGTGTACTTGATCCC-1,1,17,31,4576,4917\nCCTACGGCTCAGTCGA-1,1,16,32,4456,4986\nGTACTTGGGCACTTCT-1,1,17,33,4575,5055\nTGATTTCCTCCTGACG-1,1,16,34,4455,5123\nCCTCACCAATCTTGAC-1,1,17,35,4575,5192\nGGTGAGATGCAGATAA-1,1,16,36,4455,5261\nGCTAGTTTCATTGAGG-1,1,17,37,4575,5330\nAGGACATCGCACGTCG-1,1,16,38,4455,5398\nGTGGACGTGCTGAGAC-1,1,17,39,4574,5467\nTAAGGAACTTGTGGGA-1,1,16,40,4454,5536\nTCGCTGTGCGTAAATC-1,1,17,41,4574,5605\nGCATCCCTAACTTTGA-1,1,16,42,4454,5673\nCACCCACACGTCACCC-1,1,17,43,4573,5743\nCCCTCATTCTGGAATT-1,1,16,44,4454,5811\nAGGGCGTGATCGGCTA-1,1,17,45,4573,5880\nGGTGCGGATAAGTGGC-1,1,16,46,4453,5949\nTAATATTGAAATTCGC-1,1,17,47,4573,6018\nCTTACACTGGGAAATA-1,1,16,48,4453,6086\nACCAAGAACGCGTGTC-1,1,17,49,4572,6155\nGCCTTCAGCCCTACCG-1,1,16,50,4453,6224\nGATGCTACAAGCGCCT-1,1,17,51,4572,6293\nCCGGGACCCGCAGAGA-1,1,16,52,4452,6361\nGTTCCAGTCTGACCAT-1,1,17,53,4572,6430\nATGATCGGGAATAGAC-1,1,16,54,4452,6499\nTTGGATTGGGTACCAC-1,1,17,55,4571,6568\nTACCTCACGCTTGTAC-1,1,16,56,4452,6636\nCATGGCAGGAAGATCG-1,1,17,57,4571,6706\nATGACGCCGGCTCTAA-1,1,16,58,4451,6774\nAGCGACATCCCATTCA-1,1,17,59,4571,6843\nAGTAATGTCTTGCCGC-1,1,16,60,4451,6912\nTTCTTAGTGGCTCAGA-1,1,17,61,4570,6981\nCGTCTGGAAGGGCCCG-1,1,16,62,4450,7049\nACGTGCGCCTCGTGCA-1,1,17,63,4570,7118\nAGAGCGGGCTAATCAT-1,1,16,64,4450,7187\nGCGTCGAAATGTCGGT-1,1,17,65,4570,7256\nAACTGATATTAGGCCT-1,1,16,66,4450,7324\nCGAGCTGGGCTTTAGG-1,1,17,67,4569,7393\nGGGTGTTTCAGCTATG-1,1,16,68,4449,7462\nTTAATTTCAGACGCGG-1,1,17,69,4569,7531\nACTGCCGTCGTAACTC-1,1,16,70,4449,7600\nGTGCACGAAAGTGACT-1,1,17,71,4569,7669\nATCTCCCTGCAATCTA-1,1,16,72,4449,7737\nACGCCAGATGATTTCT-1,1,17,73,4568,7806\nAGCTATTTAATCCAAC-1,1,16,74,4448,7875\nCCACGAGAAGAGAATC-1,1,17,75,4568,7944\nGATTCCGCGTTTCCGT-1,1,16,76,4448,8012\nGTCGGATGTAGCGCGC-1,1,17,77,4568,8081\nTATTTATACCGAGTAG-1,1,16,78,4448,8150\nGTAGGTGATCCGTGTA-1,1,17,79,4567,8219\nAGTTAAGCGGTCCCGG-1,1,16,80,4447,8287\nCTGGCGACATAAGTCC-1,1,17,81,4567,8357\nTTGGCCTAGAATTTCG-1,1,16,82,4447,8425\nGGCATATCGGTTCTGC-1,1,17,83,4566,8494\nGGGCGTCCACTGGCTC-1,1,16,84,4447,8563\nTTACCCATTGCCGGGT-1,1,17,85,4566,8632\nTTAGACACGATCGTTG-1,1,16,86,4446,8700\nGCGCTGATCCAGACTC-1,1,17,87,4566,8769\nTTCGGCAACCCGCTGA-1,1,16,88,4446,8838\nGATATTTCCTACATGG-1,1,17,89,4565,8907\nCTGCGTTACGATATAA-1,1,16,90,4446,8975\nTAATAAACAAGGAGAT-1,1,17,91,4565,9044\nAACCTTTACGACGTCT-1,1,16,92,4445,9113\nAGTCCCGCCTTTAATT-1,1,17,93,4565,9182\nTGAGATTAGGCCCTAA-1,1,16,94,4445,9251\nAGTGTATTGCGCATTG-1,1,17,95,4564,9320\nGTTGGATTCAGTGGCT-1,1,16,96,4445,9388\nTAAAGCTGCAATAGGG-1,1,17,97,4564,9457\nAGTAGGAAGGAAGTTG-1,1,16,98,4444,9526\nTATCACTTCGAGTAAC-1,1,17,99,4564,9595\nTGATCTACGCTGATCT-1,1,16,100,4444,9663\nGGATCATCCCGTACGC-1,1,17,101,4563,9732\nTGACACTTCTCTTTGC-1,1,16,102,4443,9801\nAGCCCTTCTAATCCGA-1,1,17,103,4563,9870\nCACCGCGTCCACTCTA-1,1,16,104,4443,9938\nTAATTGGAATCGGGAA-1,1,17,105,4563,10008\nTCGTAAGCTCCGAGGA-1,1,16,106,4443,10076\nTATATTACAAATGTCG-1,1,17,107,4562,10145\nCGCGAGAGGGACTTGT-1,1,16,108,4442,10214\nGGACCTACGGTAACGT-1,1,17,109,4562,10283\nGAAATATGCTTGAATG-1,1,16,110,4442,10351\nCCGTATTAGCGCAGTT-1,1,17,111,4562,10420\nAGGCGTCTATGGACGG-1,1,16,112,4442,10489\nAACATCGATACGTCTA-1,1,17,113,4561,10558\nTGAATATGCTATAAAC-1,1,16,114,4441,10626\nACCAAACACCCAGCGA-1,1,17,115,4561,10695\nTGGCTTGTACAAGCTT-1,1,16,116,4441,10764\nGAATGAAGGTCTTCAG-1,1,17,117,4561,10833\nAGATACCAGCACTTCA-1,1,16,118,4441,10901\nGCGGTCCCGGTGAAGG-1,1,17,119,4560,10971\nGAGGCATTTGCAGCAG-1,1,16,120,4440,11039\nGGCAAGCCAGGGATAG-1,1,17,121,4560,11108\nTCTACGGGCTCAGTTG-1,1,16,122,4440,11177\nTCTGCGAATCGTTCGC-1,1,17,123,4559,11246\nAGCTCGTTGATGGAAA-1,1,16,124,4440,11314\nTGAATGAGATACAGCA-1,0,17,125,4559,11383\nACCCTTGCCTGGGTCG-1,0,16,126,4439,11452\nGGCGAACCGTTCTGAT-1,0,17,127,4559,11521\nGCGATGTCTGTGCTTG-1,1,18,0,4701,2785\nATTAACACCTGAGATA-1,1,19,1,4820,2854\nGAAATCTGACCAAGTT-1,1,18,2,4700,2922\nCCTGACAAACTCGCGC-1,1,19,3,4820,2992\nATGTCATTTCCCATTG-1,1,18,4,4700,3060\nGCGGTGCGGAGCATCG-1,1,19,5,4820,3129\nCGGAAAGCAAATGTGC-1,1,18,6,4700,3198\nGCTGAGCAACGGTTCT-1,1,19,7,4819,3267\nTCACTCTTCGTCTGTC-1,1,18,8,4699,3335\nGATCTTCATTGTCCTC-1,1,19,9,4819,3404\nACTCGATGTATTTCAT-1,1,18,10,4699,3473\nTGAAGAGCGGTCCTAG-1,1,19,11,4818,3542\nTAAACGTCGTCAATGA-1,1,18,12,4699,3610\nACGTTATTGGTCACTC-1,1,19,13,4818,3679\nATAGCCTCAGTACCCA-1,1,18,14,4698,3748\nCGGGAGTATACCGCCG-1,1,19,15,4818,3817\nGTCCTACAGGCGGCTC-1,1,18,16,4698,3886\nCGGATAAGCGGACATG-1,1,19,17,4817,3955\nAACTTCTGCGTCTATC-1,1,18,18,4698,4023\nGGGTTCAGACGAACAA-1,1,19,19,4817,4092\nAGTCTGGACATCCTTG-1,1,18,20,4697,4161\nTTGAACGAATCCTTTG-1,1,19,21,4817,4230\nGAAATACTAAACGTTT-1,1,18,22,4697,4298\nCCCGCGCAATGCACCC-1,1,19,23,4816,4367\nTTCGGCTAGAGATGGT-1,1,18,24,4697,4436\nGACACGAGTTAGAGGA-1,1,19,25,4816,4505\nGAGGTCCCAAAGATCT-1,1,18,26,4696,4573\nTAACTCCATGGAGGCT-1,1,19,27,4816,4642\nCTTGTTTATGTAGCCA-1,1,18,28,4696,4711\nGATGGCGCACACATTA-1,1,19,29,4815,4780\nATAATAGTGTAGGGAC-1,1,18,30,4695,4849\nCGCTATTCAATGTATG-1,1,19,31,4815,4918\nATATTGCTGTCAAAGT-1,1,18,32,4695,4986\nGGATTCAGTACGGTGG-1,1,19,33,4815,5055\nTTCTTAGTGAACGGTG-1,1,18,34,4695,5124\nAATGGTTCTCACAAGC-1,1,19,35,4814,5193\nTATACACGCAAAGTAT-1,1,18,36,4694,5261\nCTTCATAGCTCAAGAA-1,1,19,37,4814,5330\nCAACGGTTCTTGATAC-1,1,18,38,4694,5399\nACACCCGAGAAATCCG-1,1,19,39,4814,5468\nTCTATCATGCAGTTAC-1,1,18,40,4694,5536\nCCCGCCATGCTCCCGT-1,1,19,41,4813,5606\nCGCTTCCACTGAAATC-1,1,18,42,4693,5674\nCACTGTCCAAGTGAGA-1,1,19,43,4813,5743\nATTACTAGCCTCTTGC-1,1,18,44,4693,5812\nCATAGTAGCATAGTAG-1,1,19,45,4813,5881\nCAACTCCTTGATCCCG-1,1,18,46,4693,5949\nAAGTAGAAGACCGGGT-1,1,19,47,4812,6018\nGCGGGAACCAGGCCCT-1,1,18,48,4692,6087\nATTAGATTGATAGCGG-1,1,19,49,4812,6156\nCTCGGTCCGTAGCCTG-1,1,18,50,4692,6224\nTGGCTTTGGGTAGACA-1,1,19,51,4811,6293\nTATCCATATCATGCGA-1,1,18,52,4692,6362\nGGAGTGCCGCCCTGGA-1,1,19,53,4811,6431\nTGAGAATGCTTTACCG-1,1,18,54,4691,6500\nTTAACCAACCCTCCCT-1,1,19,55,4811,6569\nTGTTTCGGTACTTCTC-1,1,18,56,4691,6637\nTTGCTGAAGGAACCAC-1,1,19,57,4810,6706\nTATTTAGTCTAGATCG-1,1,18,58,4691,6775\nCTCCGGCCTAATATGC-1,1,19,59,4810,6844\nTTGTGGCCCTGACAGT-1,1,18,60,4690,6912\nTCGCCGGTCGATCCGT-1,1,19,61,4810,6981\nCCATAGGTTGGCGTGG-1,1,18,62,4690,7050\nGAACGACCGAATGATA-1,1,19,63,4809,7119\nTCCGATAATTGCCATA-1,1,18,64,4690,7187\nCATTACGTCGGCCCGT-1,1,19,65,4809,7257\nCAAGCACCAAATGCCT-1,1,18,66,4689,7325\nTGCATGGATCGGATCT-1,1,19,67,4809,7394\nGAAATCGCGCGCAACT-1,1,18,68,4689,7463\nCTGAAAGAGATCCGAC-1,1,19,69,4808,7532\nCACCTCGATGGTGGAC-1,1,18,70,4688,7600\nATTTGTTCCAGGGCTC-1,1,19,71,4808,7669\nTGGGCCACAAGAGCGC-1,1,18,72,4688,7738\nCCTTCTTGATCCAGTG-1,1,19,73,4808,7807\nCCTCGCCAGCAAATTA-1,1,18,74,4688,7875\nTTCATGGCGCAACAGG-1,1,19,75,4807,7944\nTTAATCAGTACGTCAG-1,1,18,76,4687,8013\nCCTATCTATATCGGAA-1,1,19,77,4807,8082\nATTATACTTTGCTCGT-1,1,18,78,4687,8150\nATGGATCCGGCGTCCG-1,1,19,79,4807,8220\nCGCCCGCTTCCGTACA-1,1,18,80,4687,8288\nGGATTCCGCTATACCC-1,1,19,81,4806,8357\nCGGTCTATCAACCCGT-1,1,18,82,4686,8426\nATGCCGGTTGATGGGA-1,1,19,83,4806,8495\nTCATGCAGGTTCTCAT-1,1,18,84,4686,8563\nTGAGCTTTAATGACGC-1,1,19,85,4806,8632\nTCCCTTAGATTACTCG-1,1,18,86,4686,8701\nATATCTCCCTCGTTAA-1,1,19,87,4805,8770\nAGCTCTTCCCAGTGCA-1,1,18,88,4685,8838\nTCGCTAAACCGCTATC-1,1,19,89,4805,8907\nCACATTCTTTCGATGG-1,1,18,90,4685,8976\nGATATGCGGTAGCCAA-1,1,19,91,4804,9045\nCGTTTCACTTCGGGCG-1,1,18,92,4685,9114\nCCAATTACGGGTCGAG-1,1,19,93,4804,9183\nGCAGGTAGAGTATGGT-1,1,18,94,4684,9251\nGTCGTATTGGCGTACA-1,1,19,95,4804,9320\nGAAATTAGCACGGATA-1,1,18,96,4684,9389\nAATGCACCAAGCAATG-1,1,19,97,4803,9458\nAGGACGCTCGATGTTG-1,1,18,98,4684,9526\nGGCTAAAGGGCGGGTC-1,1,19,99,4803,9595\nCATCTATCCCGTGTCT-1,1,18,100,4683,9664\nCAGTAACTATTTATTG-1,1,19,101,4803,9733\nCATATACTACTGATAA-1,1,18,102,4683,9801\nGCGTTCGGAGACCGGG-1,1,19,103,4802,9871\nAAGTTCAGTCTGCGTA-1,1,18,104,4683,9939\nCGAAGCTATAAATTCA-1,1,19,105,4802,10008\nCGCGGTCACAAACCAA-1,1,18,106,4682,10077\nGGGAATGAGCCCTCAC-1,1,19,107,4802,10146\nACGGAGCGCAAATTAC-1,1,18,108,4682,10214\nCGTTCTTCGCACACCT-1,1,19,109,4801,10283\nGAATAGCCCTGCGGTC-1,1,18,110,4681,10352\nAATAGCTACCGCGTGC-1,1,19,111,4801,10421\nCCGAGCTGTGCTTGTC-1,1,18,112,4681,10489\nGATGACGATGATCGCG-1,1,19,113,4801,10558\nGCCTATGCTGGGCCTT-1,1,18,114,4681,10627\nTTACTGTCTAGAGCTC-1,1,19,115,4800,10696\nAGCGGTTGCCGCTCTG-1,1,18,116,4680,10765\nGCTTGCAGCACAATTG-1,1,19,117,4800,10834\nCCGGAGGTCTTATGGT-1,1,18,118,4680,10902\nACAGTATACCGTGGGA-1,1,19,119,4800,10971\nGGGATCCCAATACAAA-1,1,18,120,4680,11040\nATTACGACTCCACAGT-1,1,19,121,4799,11109\nCTCACACGCAAGCCTA-1,1,18,122,4679,11177\nCCAGATGTAAATGGGT-1,1,19,123,4799,11246\nGAACTTGTGCACGGGA-1,0,18,124,4679,11315\nAAGCCGCTTTACCTTG-1,0,19,125,4799,11384\nTCCATTCCCACTAGAG-1,0,18,126,4679,11452\nAGAGCGCTTGTAACGG-1,0,19,127,4798,11522\nTGGGTTCCCGGACGGA-1,1,20,0,4940,2785\nGCTGCGCCTCCCACGT-1,1,21,1,5060,2855\nCTGTTGGCTCTTCTGA-1,1,20,2,4940,2923\nTTGTTCTAGATACGCT-1,1,21,3,5059,2992\nCCCTCAAGCTCTTAGT-1,1,20,4,4939,3061\nTGGTCTAGCTTACATG-1,1,21,5,5059,3130\nATGCACCTTCCTTAAT-1,1,20,6,4939,3198\nGGGATACGGTAATAAT-1,1,21,7,5059,3267\nAGTTCACCGGTTGGAC-1,1,20,8,4939,3336\nGACATACTGTCGCAGA-1,1,21,9,5058,3405\nTGGACACCGTTGCTTG-1,1,20,10,4938,3473\nTGCGATGCTAATGGCT-1,1,21,11,5058,3542\nTTCTGTTTCCTGTCGC-1,1,20,12,4938,3611\nCGTTGTAAACGTCAGG-1,1,21,13,5058,3680\nGATCGGCGATAAGTCG-1,1,20,14,4938,3749\nAGCCTTAAAGCGGAAG-1,1,21,15,5057,3818\nTCCGTAACCACAATCC-1,1,20,16,4937,3886\nGAATGCCGAAATGACC-1,1,21,17,5057,3955\nTATACTCATGCGGCAA-1,1,20,18,4937,4024\nTAGTGTCAGAAACGGC-1,1,21,19,5056,4093\nCGTCATACCATATCCA-1,1,20,20,4937,4161\nTAGTACCTTAGTGGTC-1,1,21,21,5056,4230\nCTGGCGGGAATAAGTA-1,1,20,22,4936,4299\nAGTGTGGTCTATTGTG-1,1,21,23,5056,4368\nGCTATCGCGGCGCAAC-1,1,20,24,4936,4436\nCAGTAATCCCTCCCAG-1,1,21,25,5055,4506\nGTATTAAGGCGTCTAA-1,1,20,26,4936,4574\nCTAATTTCAACAACAC-1,1,21,27,5055,4643\nTTAGCAACATGGATGT-1,1,20,28,4935,4712\nATGCTCAGTGTTGCAT-1,1,21,29,5055,4781\nGATGTTTGTGCGAGAT-1,1,20,30,4935,4849\nCACTTCGCCACAGGCT-1,1,21,31,5054,4918\nCTGTATGGTGTAGAAA-1,1,20,32,4934,4987\nTACGTGCACTATGCTG-1,1,21,33,5054,5056\nGTATCAGCTTGGGTTC-1,1,20,34,4934,5124\nTAACAAAGGGAGAAGC-1,1,21,35,5054,5193\nTTACATCGTGGCCTGG-1,1,20,36,4934,5262\nTCTGAACCGGTCGGCT-1,1,21,37,5053,5331\nGGCTCGTGCCACCAGC-1,1,20,38,4933,5399\nTATAAGTGAGGATAGC-1,1,21,39,5053,5469\nGCCCGCGCGTTTGACA-1,1,20,40,4933,5537\nTGCTGGTTGGACAATT-1,1,21,41,5053,5606\nGCCTAGCGATCTGACC-1,1,20,42,4933,5675\nCCATGCTCTGCAGGAA-1,1,21,43,5052,5744\nTATAAATCCACAAGCT-1,1,20,44,4932,5812\nCGCTCGACATAATGAT-1,1,21,45,5052,5881\nCCAATTGAATGTTAAT-1,1,20,46,4932,5950\nGCTAATACCGAATGCC-1,1,21,47,5052,6019\nTTCAACGACCCGACCG-1,1,20,48,4932,6087\nTTCCTCGAGGGTGTCT-1,1,21,49,5051,6156\nCAACGACCCGTTTACA-1,1,20,50,4931,6225\nCTTGTACTTGTTGACT-1,1,21,51,5051,6294\nTCTCTAATAGCTGGTA-1,1,20,52,4931,6363\nATTATGCCATAGGGAG-1,1,21,53,5051,6432\nGACAACGCAGCTTACG-1,1,20,54,4931,6500\nAGATGACTCGCCCACG-1,1,21,55,5050,6569\nGTGCGGGTCTCCAAAT-1,1,20,56,4930,6638\nGTACGAGATTGCGACA-1,1,21,57,5050,6707\nGTATAGGACTCAGTAG-1,1,20,58,4930,6775\nTTGCACGGAGCAGCAC-1,1,21,59,5049,6844\nCACAGCTAGGGAGTGA-1,1,20,60,4930,6913\nATACTAGCATGACCCT-1,1,21,61,5049,6982\nCCAAGACTTCTGCGAA-1,1,20,62,4929,7050\nACATAATAAGGCGGTG-1,1,21,63,5049,7120\nTAATACACAGTAGTAT-1,1,20,64,4929,7188\nTCTTGGTAACACCAAA-1,1,21,65,5048,7257\nAACTGGGTCCCGACGT-1,1,20,66,4929,7326\nATCACTTCATCCTCGC-1,1,21,67,5048,7395\nTGGAAGGATAAAGATG-1,1,20,68,4928,7463\nCATGATGCACAATTCT-1,1,21,69,5048,7532\nTGCCTGATCAAACGAT-1,1,20,70,4928,7601\nATAGGGATATCCTTGA-1,1,21,71,5047,7670\nCACCTAATCAGTTTAC-1,1,20,72,4927,7738\nTGTGACTACGCCAGTC-1,1,21,73,5047,7807\nCCCGACCATAGTCCGC-1,1,20,74,4927,7876\nCGCGCCCGACTTAATA-1,1,21,75,5047,7945\nTGCCACCTGGCGAAAC-1,1,20,76,4927,8014\nCTGCCAAGGTTGGAAG-1,1,21,77,5046,8083\nTCTCCAACGTAGGTTA-1,1,20,78,4926,8151\nTTCTTGGAGTAATGAG-1,1,21,79,5046,8220\nGTCTCGATCTGCTTTC-1,1,20,80,4926,8289\nTACTCTCCGAACAAAT-1,1,21,81,5046,8358\nATCACATTAGAATATC-1,1,20,82,4926,8426\nTACGGGATGCTAGCAG-1,1,21,83,5045,8495\nAGCTGAAGTAAACCAA-1,1,20,84,4925,8564\nCATGGGTCGGGTGTGG-1,1,21,85,5045,8633\nCACCCACGAGGCAATT-1,1,20,86,4925,8701\nTGCCATTACTAAAGAA-1,1,21,87,5045,8771\nCCGTTACGTTAGAACA-1,1,20,88,4925,8839\nGCCAGGAGTAACCGAT-1,1,21,89,5044,8908\nGGAAAGTGCCCATGCC-1,1,20,90,4924,8977\nTCTTACCGGAACTCGT-1,1,21,91,5044,9046\nTATGTCAAGACCGACT-1,1,20,92,4924,9114\nCCTAACTAAGGCTCTA-1,1,21,93,5044,9183\nGCGGTGAACTGCGCTC-1,1,20,94,4924,9252\nCCTCGCGCGATATAGG-1,1,21,95,5043,9321\nATCGGCAAGCAGTCCA-1,1,20,96,4923,9389\nAGATCGTGCATAAGAT-1,1,21,97,5043,9458\nATTCAGGACCTATTTC-1,1,20,98,4923,9527\nAGCTAGAAGCAGAAGT-1,1,21,99,5042,9596\nTTCGCTATCTGACGTG-1,1,20,100,4923,9664\nTTCCGCAGAGAAATAT-1,1,21,101,5042,9734\nCAGTACATTCTCTAAA-1,1,20,102,4922,9802\nGTGAAGATTTCAAGTG-1,1,21,103,5042,9871\nAACGCATGATCTGGGT-1,1,20,104,4922,9940\nCCCAGAGGAGGGCGTA-1,1,21,105,5041,10009\nGGAAACCTTGTTGAAT-1,1,20,106,4922,10077\nGTGAAGCCGTATAGTC-1,1,21,107,5041,10146\nGAGCTGTCGTCTCGGA-1,1,20,108,4921,10215\nTGTCATTTGTTGGGAA-1,1,21,109,5041,10284\nACCAACACCACACACT-1,1,20,110,4921,10352\nAAATGATTCGATCAGC-1,1,21,111,5040,10421\nCTGCTTTATGTCCGCG-1,1,20,112,4921,10490\nGCGAGAGGCCATGTAA-1,1,21,113,5040,10559\nATTGACCGGCGATGAC-1,1,20,114,4920,10628\nACCCTGGTAACGCCCT-1,1,21,115,5040,10697\nGTTGGACCGCATCAGG-1,1,20,116,4920,10765\nCGTTTACAAGGCAGCT-1,1,21,117,5039,10834\nCGGTCCATGAGACTCC-1,1,20,118,4919,10903\nGTTCGTCTGGGTCCCT-1,1,21,119,5039,10972\nTGTACTACTCTCACGG-1,1,20,120,4919,11040\nAGTGAAGATGGTGTCC-1,1,21,121,5039,11109\nGAGGGCCATAATATTA-1,1,20,122,4919,11178\nACTCCCGTAGACTAGG-1,1,21,123,5038,11247\nCGTCAAATGGTCGCAG-1,0,20,124,4918,11315\nAAGTCTAGTAGCTGCC-1,0,21,125,5038,11385\nAAGAAATCACCAGATT-1,0,20,126,4918,11453\nGAGTGCACGGACAACA-1,0,21,127,5038,11522\nGTTCAAATCAGATGTC-1,1,22,0,5179,2786\nCATGGCTCCCTATGTC-1,1,23,1,5299,2855\nGGCCACACGAAAGCCT-1,1,22,2,5179,2924\nGAAGCCGGGTAAGCTC-1,1,23,3,5299,2993\nTTGCGCTCTCTCGCTT-1,1,22,4,5179,3061\nTACATGACCTTATCCG-1,1,23,5,5298,3130\nGTCCTTCTACAACCCA-1,1,22,6,5178,3199\nTATATGCTGGGTTGCC-1,1,23,7,5298,3268\nGTTTATGGGATTTAGA-1,1,22,8,5178,3336\nGGAACCCGAACAAGAA-1,1,23,9,5298,3405\nAACGTTATCAGCACCT-1,1,22,10,5178,3474\nCATCGTCCGGTTACTA-1,1,23,11,5297,3543\nAGACAGCTCAGAATCC-1,1,22,12,5177,3612\nGCAGATCCTCGCAAAT-1,1,23,13,5297,3681\nGGGTCATGCGTACCAT-1,1,22,14,5177,3749\nCTGGTCATTCCAATCC-1,1,23,15,5297,3818\nTCAGGGCGCAAACTCG-1,1,22,16,5177,3887\nGATGCCAGCAGAAGGC-1,1,23,17,5296,3956\nGTTATTAACGTGGGAG-1,1,22,18,5176,4024\nAATACAATGTTTCAGG-1,1,23,19,5296,4093\nTTGCTGCACCTATCCA-1,1,22,20,5176,4162\nCCAGAGACAAAGCCGG-1,1,23,21,5296,4231\nCCGAAGTATATTGTTC-1,1,22,22,5176,4299\nGCTAAACCTGAGGTGA-1,1,23,23,5295,4369\nTCATACTTACAGATCC-1,1,22,24,5175,4437\nCGAGCACTTCAAGTTT-1,1,23,25,5295,4506\nTAGCAACCTGTCACAA-1,1,22,26,5175,4575\nTGGGAAATGCCTTTCC-1,1,23,27,5294,4644\nAGACCATGGGATACAA-1,1,22,28,5175,4712\nTAAATGAATCCGTTTC-1,1,23,29,5294,4781\nACAACGGTCCCTGCGA-1,1,22,30,5174,4850\nGTCACTCTCCAAATCT-1,1,23,31,5294,4919\nTTCTACTTGCGAGGGC-1,1,22,32,5174,4987\nCGCAATTACTTTCGGT-1,1,23,33,5293,5056\nCTGTTCATCTCACGGG-1,1,22,34,5174,5125\nTTCTTGTAACCTAATG-1,1,23,35,5293,5194\nGCTTGATGATAATCAG-1,1,22,36,5173,5263\nTTGGCTCGCATGAGAC-1,1,23,37,5293,5332\nGCCCAGTTGGTATGCC-1,1,22,38,5173,5400\nATTCCTCCGCCAGTGC-1,1,23,39,5292,5469\nTCGTCCGCTGGCGTCT-1,1,22,40,5172,5538\nGGAGAAGTCATTGGCA-1,1,23,41,5292,5607\nTTGTTAGCAAATTCGA-1,1,22,42,5172,5675\nTCTAGCATCTTCGATG-1,1,23,43,5292,5744\nTTCTAGGCCAATTGTG-1,1,22,44,5172,5813\nTCACGGTCATCGCACA-1,1,23,45,5291,5882\nATGAAGCCAAGGAGCC-1,1,22,46,5171,5950\nAATGACTGTCAGCCGG-1,1,23,47,5291,6020\nCCAAACAGAACCCTCG-1,1,22,48,5171,6088\nTATCGATGATTAAACG-1,1,23,49,5291,6157\nGAACACACATCAACCA-1,1,22,50,5171,6226\nCCCGTCAGCGTCTGAC-1,1,23,51,5290,6295\nAGCATCGTCGATAATT-1,1,22,52,5170,6363\nGACTAAGATCATGCAC-1,1,23,53,5290,6432\nTAGGGTGTTTCAAGAG-1,1,22,54,5170,6501\nTGGTTCGTAGCAAAGG-1,1,23,55,5290,6570\nCTGTTCACTGCCTGTG-1,1,22,56,5170,6638\nATGTGCATCCGACGCA-1,1,23,57,5289,6707\nTTGTCGTTCAGTTACC-1,1,22,58,5169,6776\nCGGGATCAATGTAAGA-1,1,23,59,5289,6845\nTTATCTGTATCATAAC-1,1,22,60,5169,6913\nATCGACTCTTTCCGTT-1,1,23,61,5289,6983\nCTCATTTGATGGGCGG-1,1,22,62,5169,7051\nGTAAGCGGGCAGTCAG-1,1,23,63,5288,7120\nTCTATCGGTCGCAACA-1,1,22,64,5168,7189\nAACGCGGTCTCCAGCC-1,1,23,65,5288,7258\nATTAATACTACGCGGG-1,1,22,66,5168,7326\nCTTTAACTTTCAAAGG-1,1,23,67,5287,7395\nCGTACCTGATAGGCCT-1,1,22,68,5168,7464\nGAATGTTGGGTAATCT-1,1,23,69,5287,7533\nTGCGGAGTAAAGGTGC-1,1,22,70,5167,7601\nCCTGAATATTTACATA-1,1,23,71,5287,7670\nTTGCTCCCATACCGGA-1,1,22,72,5167,7739\nCCTCTAATCTGCCAAG-1,1,23,73,5286,7808\nAGGTTGAGGCACGCTT-1,1,22,74,5167,7877\nTCCCGTCAGTCCCGCA-1,1,23,75,5286,7946\nTCCGATGACTGAGCTC-1,1,22,76,5166,8014\nCAGCCTCCTGCAGAGG-1,1,23,77,5286,8083\nCTTAGCCTTCCACATG-1,1,22,78,5166,8152\nATTAATGAACCAGTCG-1,1,23,79,5285,8221\nACGATACATAGAACTA-1,1,22,80,5165,8289\nAGCCACTCCCGTGCTT-1,1,23,81,5285,8358\nATACGGGTTTCGATTG-1,1,22,82,5165,8427\nCTGTCAAATGGCTCGG-1,1,23,83,5285,8496\nGCTCGGAATTTAAAGC-1,1,22,84,5165,8564\nTAGGCATGTTACGCCA-1,1,23,85,5284,8634\nTGGCAACTCGCGCGCC-1,1,22,86,5164,8702\nATCAGTAGGCAGGGAT-1,1,23,87,5284,8771\nTATCGATCTATGCATA-1,1,22,88,5164,8840\nCGACTCAGGATGTTAT-1,1,23,89,5284,8909\nGCCATATTGCACACAG-1,1,22,90,5164,8977\nAATTCATAAGGGATCT-1,1,23,91,5283,9046\nCGGTAGAGGTGCAGGT-1,1,22,92,5163,9115\nAATGATGATACGCTAT-1,1,23,93,5283,9184\nCTTGTGCTCACCGATT-1,1,22,94,5163,9252\nTTCCAATCAGAGCTAG-1,1,23,95,5283,9321\nCGATGGACCCTACGCC-1,1,22,96,5163,9390\nGGTCGGATAAACGGCG-1,1,23,97,5282,9459\nTTAGCTAATACGATCT-1,1,22,98,5162,9528\nCTCGATATTTGCGAGC-1,1,23,99,5282,9597\nATTACTTACTGGGCAT-1,1,22,100,5162,9665\nCTAGCCGATGTTATGA-1,1,23,101,5282,9734\nTACTGCAATCAATTAC-1,1,22,102,5162,9803\nTAGTCTGTGACGTTGC-1,1,23,103,5281,9872\nCTCGTTTCTAATGTTT-1,1,22,104,5161,9940\nTTCGTTCAACGAAGTT-1,1,23,105,5281,10009\nCTGAATTTATTGCCAG-1,1,22,106,5161,10078\nTGGAATATCCTTGACC-1,1,23,107,5280,10147\nCAGATCATTTAAAGTC-1,1,22,108,5161,10215\nCTCCTTTACGCAAGTC-1,1,23,109,5280,10285\nTCCCAAACAGACAACG-1,1,22,110,5160,10353\nATCGCTGCGTGCAGCA-1,1,23,111,5280,10422\nTTAGTTCAAGTGTTCG-1,1,22,112,5160,10491\nAAACTCGTGATATAAG-1,1,23,113,5279,10560\nTTAACGAACAAGCAGT-1,1,22,114,5160,10628\nGTTATATCAGGAGCCA-1,1,23,115,5279,10697\nCAAATTGGATTATGCC-1,1,22,116,5159,10766\nCGAGGAGCTTCCATAT-1,1,23,117,5279,10835\nGGAGACCAATGTGCTT-1,1,22,118,5159,10903\nCATTGATGAACACGCC-1,1,23,119,5278,10972\nGTCAATGCTATAATTT-1,1,22,120,5158,11041\nACCACCCTCTCTTCTA-1,1,23,121,5278,11110\nTGGAGGGAAACACCTC-1,1,22,122,5158,11178\nCACGGACGTGGATGGC-1,1,23,123,5278,11248\nAACTTTCTCGATCATG-1,0,22,124,5158,11316\nCGTATTGTTTCCTAAT-1,0,23,125,5277,11385\nCCTACTGCGGCGGCCA-1,0,22,126,5157,11454\nCTTAGGTCCCAATCGT-1,0,23,127,5277,11523\nCGCAATCGATCATTAG-1,1,24,0,5419,2787\nTGGTTATGCTTGCGGT-1,1,25,1,5538,2856\nGGCTTGGCTCTCACCT-1,1,24,2,5419,2924\nATTGGTAGGATCCGCT-1,0,25,3,5538,2993\nTCAGGGCGACTTCCTT-1,0,24,4,5418,3062\nTCTGCAGATTCGAGTC-1,0,25,5,5538,3131\nCTCTCGCTGTACTATG-1,1,24,6,5418,3199\nAATAGTCGCGAGTCGG-1,1,25,7,5537,3269\nAGTTACCCTTAAGACT-1,1,24,8,5417,3337\nCTTAAATAAGACCCAT-1,1,25,9,5537,3406\nGGTTGTGCTCTTGTCC-1,1,24,10,5417,3475\nGTGAGTCTAAGACGGA-1,1,25,11,5537,3544\nCGCGACACTGCGCAGC-1,1,24,12,5417,3612\nGCTCGCGGTTCCGCTC-1,1,25,13,5536,3681\nTTAACTCACGCGTGGA-1,1,24,14,5416,3750\nGGAACGGCCTGCAGCC-1,1,25,15,5536,3819\nGTAGAAACGGGTGGAG-1,1,24,16,5416,3887\nTAATGAAAGACCCTTG-1,1,25,17,5536,3956\nAGGCTTGCTAGACACC-1,1,24,18,5416,4025\nTTGCGTAGTTTGAGGA-1,1,25,19,5535,4094\nCGCCCTTGAAGGCTGA-1,1,24,20,5415,4162\nCCCGGTGGAAGAACCT-1,1,25,21,5535,4232\nTTAACACCTCGAACAT-1,1,24,22,5415,4300\nGATTCCTATACGGCGC-1,1,25,23,5535,4369\nTTACCCTAACAGTCCT-1,1,24,24,5415,4438\nACCCACCTACATGCTC-1,1,25,25,5534,4507\nAAAGGGCAGCTTGAAT-1,1,24,26,5414,4575\nCACACAGGGATAGATT-1,1,25,27,5534,4644\nAGAGCGTACAAGCTCG-1,1,24,28,5414,4713\nTCTTACGGCATCCGAC-1,1,25,29,5533,4782\nGCCTATTCCGATATAG-1,1,24,30,5414,4850\nGAAAGTGACTAACTGC-1,1,25,31,5533,4919\nCCGGAATGGTTTCAGT-1,1,24,32,5413,4988\nAGTATAATACTAGGCA-1,1,25,33,5533,5057\nTAACTATCGAAGGTCC-1,1,24,34,5413,5126\nATGAGGAGTGTTAATC-1,1,25,35,5532,5195\nTGTGTCGCGAGTTGCA-1,1,24,36,5413,5263\nATCCAACGCAGTCATA-1,1,25,37,5532,5332\nAAGGCGCGTAAAGCTT-1,1,24,38,5412,5401\nAGTCGGCCCAAACGAC-1,1,25,39,5532,5470\nAACGTCAGACTAGTGG-1,1,24,40,5412,5538\nACTACCAGCTCTCTGG-1,1,25,41,5531,5607\nGCAAGTGCACAGAGAA-1,1,24,42,5412,5676\nACACCTTAAGTAGGGC-1,1,25,43,5531,5745\nTTCGACGGGAAGGGCG-1,1,24,44,5411,5813\nTTCGCACTCGCGTGCT-1,1,25,45,5531,5883\nTATTTGTTACCCTTTA-1,1,24,46,5411,5951\nCGCTGTGACGCCGCAC-1,1,25,47,5530,6020\nGTTGCACGGAGTTTCG-1,1,24,48,5410,6089\nGTTTCCTGGAGGGTGA-1,1,25,49,5530,6158\nACACCCAGCATGCAGC-1,1,24,50,5410,6226\nTCAACCATGTTCGGGC-1,1,25,51,5530,6295\nTTACAACTACGCATCC-1,1,24,52,5410,6364\nTCCGATGGTGCGACAT-1,1,25,53,5529,6433\nGGGCGTACATTTATAT-1,1,24,54,5409,6501\nAGCGACCAACGATATT-1,1,25,55,5529,6570\nACACAAAGACGGGTGG-1,1,24,56,5409,6639\nATCGCACGCCGGGAGA-1,1,25,57,5529,6708\nGCTCTAAACCCTGACG-1,1,24,58,5409,6777\nAATGCAACCGGGTACC-1,1,25,59,5528,6846\nTCAAACAACCGCGTCG-1,1,24,60,5408,6914\nTATGCTCCCTACTTAC-1,1,25,61,5528,6983\nAAAGGGATGTAGCAAG-1,1,24,62,5408,7052\nACGATCATACATAGAG-1,1,25,63,5528,7121\nTTGTTCAGTGTGCTAC-1,1,24,64,5408,7189\nATGCATGATCCAGGAT-1,1,25,65,5527,7258\nAGTCTTCTCCTCAAAT-1,1,24,66,5407,7327\nGATTCCCTTGTCGCAG-1,1,25,67,5527,7396\nCTCGCACCTATATAGT-1,1,24,68,5407,7464\nACTCAATAAAGGCACG-1,1,25,69,5526,7534\nAACCGAGCTTGGTCAT-1,1,24,70,5407,7602\nTAAGGCAACATAAGAT-1,1,25,71,5526,7671\nCACGCACAGCGCAGCT-1,1,24,72,5406,7740\nGGTTTACAATCTCAAT-1,1,25,73,5526,7809\nTGCAGGATCGGCAAAG-1,1,24,74,5406,7877\nATAACGGAGTCCAACG-1,1,25,75,5525,7946\nAACGATATGTCAACTG-1,1,24,76,5406,8015\nGACAACGACCATTGAA-1,1,25,77,5525,8084\nTTGACCATGTTCTCCG-1,1,24,78,5405,8152\nAGTACGGGCACCTGGC-1,1,25,79,5525,8221\nCGCCATCCGATTATGA-1,1,24,80,5405,8290\nAAGGTATCCTAATATA-1,1,25,81,5524,8359\nTGTTGTCAAGAAGTCT-1,1,24,82,5405,8427\nCAGTGAATAAATGACT-1,1,25,83,5524,8497\nCACCTTGCGAAACTCG-1,1,24,84,5404,8565\nCATTTAGCGGACCATG-1,1,25,85,5524,8634\nCCAGTCTAGACGGCGC-1,1,24,86,5404,8703\nTCGCTTTAAACGTTTG-1,1,25,87,5523,8772\nGTGAAACGGCGCCACC-1,1,24,88,5403,8840\nGGGCTCATCGAACCCA-1,1,25,89,5523,8909\nTTGATGTGTAGTCCCG-1,1,24,90,5403,8978\nCAGTAGCCCACGCGGT-1,1,25,91,5523,9047\nAGCGCGGGTGCCAATG-1,1,24,92,5403,9115\nTAATCGATCCGTACGT-1,1,25,93,5522,9184\nAGTGGCGGCAATTTGA-1,1,24,94,5402,9253\nCCTTTCAATGAAGAAA-1,1,25,95,5522,9322\nCTCAGTCACGACAAAT-1,1,24,96,5402,9391\nATAGGCTAGCTTCGCA-1,1,25,97,5522,9460\nCGGTTCAAGTAGGTGT-1,1,24,98,5402,9528\nCAGTCGAGGATGCAAT-1,1,25,99,5521,9597\nTATCACCCAACCGACC-1,1,24,100,5401,9666\nAATGATGCGACTCCTG-1,1,25,101,5521,9735\nTGGAACCACTGACACA-1,1,24,102,5401,9803\nGCCAATAGGGCATCTC-1,1,25,103,5521,9872\nTTCTTTGGTCGCGACG-1,1,24,104,5401,9941\nATTAGATTCCTCAGCA-1,1,25,105,5520,10010\nCCGTGGAACGATCCAA-1,1,24,106,5400,10078\nGGGTCGTGGCAAGTGT-1,1,25,107,5520,10148\nTCGCTCGGCACCAGCG-1,1,24,108,5400,10216\nACGCAATCACTACAGC-1,1,25,109,5520,10285\nCTCTAATGCATTGATC-1,1,24,110,5400,10354\nGTCTCGACTAAGTTTG-1,1,25,111,5519,10423\nTGGTTTAAACGTGGGT-1,1,24,112,5399,10491\nCGCAGATCTTCACCCG-1,1,25,113,5519,10560\nTCCAGATGTACGCCAA-1,1,24,114,5399,10629\nCATTGCGGGTCAATTC-1,1,25,115,5518,10698\nGACGTTCGTAAATACA-1,1,24,116,5399,10766\nTACACCGTCGTTAGTC-1,1,25,117,5518,10835\nACGGGCGTATGCGACA-1,1,24,118,5398,10904\nGAAGGCTACCATTGTT-1,1,25,119,5518,10973\nTAAATCTTTACACCTC-1,1,24,120,5398,11042\nAGTTTATGTAAAGACA-1,1,25,121,5517,11111\nAGGAGACATCCACAGT-1,1,24,122,5398,11179\nCAACCTGAACCTGCCA-1,1,25,123,5517,11248\nAGTCCCTCGCAGAAAG-1,0,24,124,5397,11317\nTGTATACGGATGATGA-1,0,25,125,5517,11386\nTTGTGGTATAGGTATG-1,0,24,126,5397,11454\nTCTGCACCATTAGTAA-1,0,25,127,5516,11523\nAAATGTATCTTATCCC-1,1,26,0,5658,2787\nACTCTAAACCTGGGAT-1,1,27,1,5778,2856\nGCTGGCAGGTGCCGTG-1,0,26,2,5658,2925\nCTCATTCGTGAACATC-1,1,27,3,5777,2994\nTCGCCGGATGGGCAAG-1,0,26,4,5658,3062\nGGACTAAGTCAGGAGT-1,1,27,5,5777,3132\nTATCAAAGGTCTGTAA-1,1,26,6,5657,3200\nTTCAGTTTGTGGCAGC-1,1,27,7,5777,3269\nTGTTCATAAATGTGCT-1,1,26,8,5657,3338\nCTTAGCCCGGATAGTG-1,1,27,9,5776,3407\nGATGCGAATGGTATTA-1,1,26,10,5657,3475\nTCTAACTGTATGTAAA-1,1,27,11,5776,3544\nTTAAACCTGGTTCCTT-1,1,26,12,5656,3613\nGCTAAGTAAAGGCGAT-1,1,27,13,5776,3682\nAGTTACCGCACATGGT-1,1,26,14,5656,3750\nGACTGCGGCACGTGTA-1,1,27,15,5775,3819\nTGGTGATCGTATTTGT-1,1,26,16,5655,3888\nTTATCGCCTGCGAAGC-1,1,27,17,5775,3957\nTGGAATTAGACGCTTT-1,1,26,18,5655,4026\nTCGTCACACTGTTAGC-1,1,27,19,5775,4095\nTTATGTTTGCGATAGA-1,1,26,20,5655,4163\nGGTGCTGATCACAAAG-1,1,27,21,5774,4232\nCATAGCCGCCCGGGAT-1,1,26,22,5654,4301\nGGTTAGTTACGGCGCC-1,1,27,23,5774,4370\nATTCCCACATAAACAA-1,1,26,24,5654,4438\nATTCAGTAGCAGGGTC-1,1,27,25,5774,4507\nCAGTTCCGCGGGTCGA-1,1,26,26,5654,4576\nAAGAGATGAATCGGTA-1,1,27,27,5773,4645\nCGCAATTCTACAATAA-1,1,26,28,5653,4713\nTAACGCTTTGAGAGCG-1,1,27,29,5773,4783\nAGGCTATGGTTAGCTT-1,1,26,30,5653,4851\nGAGGAATGGAGAGGTT-1,1,27,31,5773,4920\nTCCTCTACGAGATGGC-1,1,26,32,5653,4989\nTTGATTATGCAGATGA-1,1,27,33,5772,5058\nTCAGTACTGACCCGCG-1,1,26,34,5652,5126\nTTATGACAAACTGGAT-1,1,27,35,5772,5195\nGTAAGTAGGGTATACC-1,1,26,36,5652,5264\nCGCAAACACGAGTTAC-1,1,27,37,5771,5333\nTGGCCGTATATTGACC-1,1,26,38,5652,5401\nACTGTAGCACTTTGGA-1,1,27,39,5771,5470\nGCTCTATGTTACGTGC-1,1,26,40,5651,5539\nTGCGCGATTAACGGAG-1,1,27,41,5771,5608\nGAATCGACATGGTCAC-1,1,26,42,5651,5676\nGACTAAGTAGGCTCAC-1,1,27,43,5770,5746\nATCTTGACCTGCAACG-1,1,26,44,5651,5814\nATGCACTACCGCATTG-1,1,27,45,5770,5883\nCAGATACTAACATAGT-1,1,26,46,5650,5952\nGATCGACACTATCTGA-1,1,27,47,5770,6021\nATAGAGTACTGGGACA-1,1,26,48,5650,6089\nCCTACTGCTTACACTT-1,1,27,49,5769,6158\nCCTGCTATTTGAGAAG-1,1,26,50,5650,6227\nCGCGTTCATGAAATAC-1,1,27,51,5769,6296\nCATTATGCTTGTTGTG-1,1,26,52,5649,6364\nCCAGGGACGTGGCCTC-1,1,27,53,5769,6433\nTATGGATGTGCTACGC-1,1,26,54,5649,6502\nGTACTAAGATTTGGAG-1,1,27,55,5768,6571\nAGACCCGCCCTCCTCG-1,1,26,56,5648,6640\nCGCATTAGCTAATAGG-1,1,27,57,5768,6709\nGCTCTCGGGTACCGAA-1,1,26,58,5648,6777\nCACCGCCAGAAGGTTT-1,1,27,59,5768,6846\nTCCCAAAGACGAAGGA-1,1,26,60,5648,6915\nATGGATTGACCAAACG-1,1,27,61,5767,6984\nGTCATGGACATGACTA-1,1,26,62,5647,7052\nCTACTGCCACCTGACC-1,1,27,63,5767,7121\nTTATATTTGGCAATCC-1,1,26,64,5647,7190\nAGCACCAGTACTCACG-1,1,27,65,5767,7259\nCATGGTCTAGATACCG-1,1,26,66,5647,7327\nTCTACCGTCCACAAGC-1,1,27,67,5766,7397\nCTAGTTGGGCCCGGTA-1,1,26,68,5646,7465\nTCCCGCGTACTCCTGG-1,1,27,69,5766,7534\nCAGAGCATGAGCTTGC-1,1,26,70,5646,7603\nACACGGGAACTTAGGG-1,1,27,71,5766,7672\nGGCTCTGCTCCAACGC-1,1,26,72,5646,7740\nAGAACGTGGTACATTC-1,1,27,73,5765,7809\nCAATAAACCTTGGCCC-1,1,26,74,5645,7878\nACTTCGCCATACGCAC-1,1,27,75,5765,7947\nATCTGGTTAAGACTGT-1,1,26,76,5645,8015\nTCGTAAGACGACATTG-1,1,27,77,5764,8084\nGTGTACCTTGGCTACG-1,1,26,78,5645,8153\nGCCCGTAATACCTTCT-1,1,27,79,5764,8222\nCGGTCAAGTGGGAACC-1,1,26,80,5644,8291\nTTGTAAGGCCAGTTGG-1,1,27,81,5764,8360\nGGAGCACCAAGAACTA-1,1,26,82,5644,8428\nTAATAGTGACGACCAG-1,1,27,83,5763,8497\nCTAAATCCTATTCCGG-1,1,26,84,5644,8566\nCGAGTTCTGTCCCACC-1,1,27,85,5763,8635\nAGGCAGATGCGTAAAC-1,1,26,86,5643,8703\nAAGGATGAGGGACCTC-1,1,27,87,5763,8772\nAGAGAACCGTCTAGGA-1,1,26,88,5643,8841\nGAGGGCGCAGCTCTGC-1,1,27,89,5762,8910\nAAGATTGGCGGAACGT-1,1,26,90,5643,8978\nCCAGTAGTCTGATCCA-1,1,27,91,5762,9048\nAAGGGACAGATTCTGT-1,1,26,92,5642,9116\nATAGAGTTATCAACTT-1,1,27,93,5762,9185\nAAATTACCTATCGATG-1,1,26,94,5642,9254\nGATCCTAAATCGGGAC-1,1,27,95,5761,9323\nTTACAGACCTAAATGA-1,1,26,96,5641,9391\nCCTCACCTTAGCATCG-1,1,27,97,5761,9460\nCATGCGACCAGTTTAA-1,1,26,98,5641,9529\nAACATATCAACTGGTG-1,1,27,99,5761,9598\nCTATAAGAGCCAATCG-1,1,26,100,5641,9666\nAATATCGAGGGTTCTC-1,1,27,101,5760,9735\nGTACTCCTGGGTATGC-1,1,26,102,5640,9804\nATAAGTAGGATTCAGA-1,1,27,103,5760,9873\nAGGTCGCGGAGTTACT-1,1,26,104,5640,9941\nCTAATTCTCAGATATT-1,1,27,105,5760,10011\nGCCAACCATTTCCGGA-1,1,26,106,5640,10079\nTGATCCCAGCATTAGT-1,1,27,107,5759,10148\nCGTTGTAAGATTGATT-1,1,26,108,5639,10217\nGAAACCATGGTGCGCT-1,1,27,109,5759,10286\nAATCTATGCCGGAGCC-1,1,26,110,5639,10354\nGACTCCCAGAATAAGG-1,1,27,111,5759,10423\nTATGATCCGGCACGCC-1,1,26,112,5639,10492\nCCGCTTGCTGACATGG-1,1,27,113,5758,10561\nTGGTTAAGGGCGCTGG-1,1,26,114,5638,10629\nTTGATAGTCAATACAT-1,1,27,115,5758,10698\nGGTTTAATTGAGCAGG-1,1,26,116,5638,10767\nCATTACATAGATTGTG-1,1,27,117,5757,10836\nGGTACACCAGATTTAT-1,1,26,118,5638,10905\nGGCCCGTATACCATGC-1,1,27,119,5757,10974\nATCTTTCGTATAACCA-1,1,26,120,5637,11042\nGAGATGACAATCCTTA-1,1,27,121,5757,11111\nAAAGCTTGCCTACATA-1,1,26,122,5637,11180\nGAACGATAAGTTAAAG-1,0,27,123,5756,11249\nTAATAGCTAAATGATG-1,0,26,124,5637,11317\nTATGGCTAGGCTAATT-1,0,27,125,5756,11386\nAGGAGAGTCTGGCTAC-1,0,26,126,5636,11455\nTGCTCTGCCGGTTCAC-1,0,27,127,5756,11524\nCCAATAGATTTCATCT-1,1,28,0,5898,2788\nGGGCACGAATTGGCCG-1,1,29,1,6017,2857\nTCGTTGACAGGGTCCC-1,1,28,2,5897,2925\nATCGTATTCCGAGAAC-1,1,29,3,6017,2995\nGGGAATTCTGTCCAGT-1,1,28,4,5897,3063\nACGCGTTTCTTAAGAG-1,1,29,5,6016,3132\nGAGAGCGCAGTCCCTG-1,1,28,6,5897,3201\nGTCCTATTGTTGTGGT-1,1,29,7,6016,3270\nCATCTGCAGGATCATT-1,1,28,8,5896,3338\nGAGTCGACAGACCCTC-1,1,29,9,6016,3407\nAAGTGCAAAGGTAGAC-1,1,28,10,5896,3476\nAGGGTGGATAGTGCAT-1,1,29,11,6015,3545\nTGATAGCGGGATTCTA-1,1,28,12,5896,3613\nGTCAGTTTGGTAGTCG-1,1,29,13,6015,3682\nGCATTCGAAATGAACA-1,1,28,14,5895,3751\nAAAGACTGGGCGCTTT-1,1,29,15,6015,3820\nTAACAATATTTGTTGC-1,1,28,16,5895,3889\nCCAGCTTCCGCCCGCA-1,1,29,17,6014,3958\nGATATGGATTACGCGG-1,1,28,18,5894,4026\nAGAGCAGTTATGAGAC-1,1,29,19,6014,4095\nTCACATCTTATCTGAT-1,1,28,20,5894,4164\nTATGAAGACAGGTGCG-1,1,29,21,6014,4233\nTACCTGCTGCACTGTG-1,1,28,22,5894,4301\nTAGGTCCAAGTAAGGA-1,1,29,23,6013,4370\nGAAACTCGTGCGATGC-1,1,28,24,5893,4439\nAACAATTACTCTACGC-1,1,29,25,6013,4508\nCCGCACGTGACCTCGG-1,1,28,26,5893,4576\nAACTTGCCCGTATGCA-1,1,29,27,6013,4646\nGGGTATGTATGCACTT-1,1,28,28,5893,4714\nTTCGTACTCCAGAACG-1,1,29,29,6012,4783\nGAATTTCTCGCTGCAG-1,1,28,30,5892,4852\nAACAGGATGGGCCGCG-1,1,29,31,6012,4921\nGACGTGTAGGGATTAT-1,1,28,32,5892,4989\nTAGGTGAGCCCTACTC-1,1,29,33,6012,5058\nCTAATTCGCACGCGCT-1,1,28,34,5892,5127\nGAAGCTTGCTGACCGC-1,1,29,35,6011,5196\nGGTTAGGCTTGGAGAA-1,1,28,36,5891,5264\nACAAGGACAAGAGGTT-1,1,29,37,6011,5333\nAGGCCACCCGTTATGA-1,1,28,38,5891,5402\nGTGGGCTTAGACACAC-1,1,29,39,6011,5471\nCGTGTCCCATTCGCGA-1,1,28,40,5891,5540\nTGGAGTGATGCGATGA-1,1,29,41,6010,5609\nAACAACTGGTAGTTGC-1,1,28,42,5890,5677\nCCTGGCTAGACCCGCC-1,1,29,43,6010,5746\nCGCAATTAGGGTAATA-1,1,28,44,5890,5815\nTCGAAATTTAGGACCA-1,1,29,45,6009,5884\nAGACTAGCCTTCCAGA-1,1,28,46,5890,5952\nTTGATCTAACTTTGTC-1,1,29,47,6009,6021\nAAGGAGCGGTTGGTGC-1,1,28,48,5889,6090\nACTTGGGACCCGGTGG-1,1,29,49,6009,6159\nTGATCTCCGGCGCCAG-1,1,28,50,5889,6227\nCAGTTCAAATTGACAC-1,1,29,51,6008,6297\nGTCCGGCTGAATTGCG-1,1,28,52,5889,6365\nCTGGAAATGGATGCTT-1,1,29,53,6008,6434\nTGATCGGTTTGACCCT-1,1,28,54,5888,6503\nTAGAGTCTAAGCGAAC-1,1,29,55,6008,6572\nGAGACTGATGGGTAGA-1,1,28,56,5888,6640\nTAGCTAAGTCCGGGAG-1,1,29,57,6007,6709\nGGGCGATATGTGTGAA-1,1,28,58,5888,6778\nCTCGAGGTCGAACAGT-1,1,29,59,6007,6847\nGATCCCTTTATACTGC-1,1,28,60,5887,6915\nGTCATGCACCTCCGTT-1,1,29,61,6007,6984\nACTTTCCTATAGCTTC-1,1,28,62,5887,7053\nTCGCTCGATATATTCC-1,1,29,63,6006,7122\nATAGGTTGGGCAGATG-1,1,28,64,5886,7190\nCAATTAAGGGTGATGA-1,1,29,65,6006,7260\nACCGACTGAGTCCCAC-1,1,28,66,5886,7328\nCCTGTCACCCGGGCTC-1,1,29,67,6006,7397\nGATCGGTGGCCATAAC-1,1,28,68,5886,7466\nCCTATGGGTTACCGTC-1,1,29,69,6005,7535\nTTGGGACACTGCCCGC-1,1,28,70,5885,7603\nCGAGGCTAAATATGGC-1,1,29,71,6005,7672\nTCAGGGTGTAACGTAA-1,1,28,72,5885,7741\nCGAGAGATGTGAACCT-1,1,29,73,6005,7810\nTCGCTGGGCGGATTGT-1,1,28,74,5885,7878\nAGATCTCAGGTGTGAT-1,1,29,75,6004,7947\nTGGCCAAACTGAAGTA-1,1,28,76,5884,8016\nGCTTCCGTCCCTAGAC-1,1,29,77,6004,8085\nCAGCAGCCCGTTCCTT-1,1,28,78,5884,8154\nTGTATAACAGATCCTG-1,1,29,79,6004,8223\nCGCGGGAATTAGGCAG-1,1,28,80,5884,8291\nTGCATGTGGTAATCTA-1,1,29,81,6003,8360\nACAATTTGAGCAGTGG-1,1,28,82,5883,8429\nGAGCTAAGGGCATATC-1,1,29,83,6003,8498\nCCAGATAGTTGAGTGA-1,1,28,84,5883,8566\nCCACAATGTACGTCTT-1,1,29,85,6002,8635\nCAATGGATCTCTACCA-1,1,28,86,5883,8704\nTGTGGCAAAGCGTATG-1,1,29,87,6002,8773\nTAAAGCGTTAGGAGAA-1,1,28,88,5882,8841\nTCCGTTTAGCCTTGAA-1,1,29,89,6002,8911\nCAGCTCGACAAGTTAA-1,1,28,90,5882,8979\nGCCTATAGTGTCAGGG-1,1,29,91,6001,9048\nATAGACAACGGGACCT-1,1,28,92,5882,9117\nCTACTATCTTTCAGAG-1,1,29,93,6001,9186\nGCGCTGCTTTGCATTT-1,1,28,94,5881,9254\nGCGCATCCAGTCAGCA-1,1,29,95,6001,9323\nGACTCGCGGGAATGAC-1,1,28,96,5881,9392\nCTGGTAACGAGCTCTT-1,1,29,97,6000,9461\nTCCGGCCTAGCGTACA-1,1,28,98,5881,9529\nTCTAGGTGGCGACGCT-1,1,29,99,6000,9598\nACGCTAGTGATACACT-1,1,28,100,5880,9667\nATCTGCACCTCTGCGA-1,1,29,101,6000,9736\nCCTCACCTGAGGGAGC-1,1,28,102,5880,9804\nAGTGAGCCTCGCCGCC-1,1,29,103,5999,9874\nACGAGTACGGATGCCC-1,1,28,104,5879,9942\nGGTACCATTAAGACGG-1,1,29,105,5999,10011\nTTCTGCTAGACTCCAA-1,1,28,106,5879,10080\nTAACTATTACGCCAAA-1,1,29,107,5999,10149\nGCATTCAAGGCAACGC-1,1,28,108,5879,10217\nAGTACATCATTTATCA-1,1,29,109,5998,10286\nGTCGTGTCTGGTCATC-1,1,28,110,5878,10355\nAGTCTAAAGTATACTC-1,1,29,111,5998,10424\nCGGCCCAACCTGTAGT-1,1,28,112,5878,10492\nAGGGAGACATACTTCG-1,1,29,113,5998,10561\nTCCCTAGATCAATAGG-1,1,28,114,5878,10630\nTCCCGTCGCGTCATAG-1,1,29,115,5997,10699\nCGCATCCATCAGCCAG-1,1,28,116,5877,10768\nCTGCACCTAGTCCACA-1,1,29,117,5997,10837\nCGAGGATCGGGAACGA-1,1,28,118,5877,10905\nCAATGAGGTTCGACTA-1,1,29,119,5997,10974\nTCTGACGGGCTAACCC-1,1,28,120,5877,11043\nTTCTATGCCTTTCGCA-1,1,29,121,5996,11112\nAGAGTATAGTGTTACG-1,0,28,122,5876,11180\nCCATTGTTTCCTCCAT-1,1,29,123,5996,11249\nCTCATCACTTAGTGAT-1,0,28,124,5876,11318\nCCGAAGGGCGTACCGC-1,0,29,125,5995,11387\nTCAAGCTGCCTTGAAA-1,0,28,126,5876,11455\nCTCATTAACGTTGCCC-1,0,29,127,5995,11525\nGTCTTCCTCACCTAAG-1,1,30,0,6137,2789\nGGTGATGAAGGAAGTG-1,1,31,1,6257,2858\nTCAATACAATTGCTGC-1,1,30,2,6137,2926\nGCAACCCAAGTTGTTT-1,1,31,3,6256,2995\nATGAAGTGGACCCAGC-1,1,30,4,6136,3064\nGAGAATCTCACGATCA-1,1,31,5,6256,3133\nTATATCATTGATCAGT-1,1,30,6,6136,3201\nAACTTTACGGGAGCTT-1,1,31,7,6256,3270\nTTCTTGTGTCCATCAG-1,1,30,8,6136,3339\nACAATTTAGGAGGCTC-1,1,31,9,6255,3408\nATACTTGTTCTCGAGC-1,1,30,10,6135,3476\nCACGGGATTGAGGGTT-1,1,31,11,6255,3546\nGTTAATGTCTATCTTA-1,1,30,12,6135,3614\nGCGTTATATTTGGAAC-1,1,31,13,6254,3683\nCGTCAAGGCTATAAAT-1,1,30,14,6135,3752\nTTAGCTCTGTAATCCG-1,1,31,15,6254,3821\nAATGGTCCACCGTTCA-1,1,30,16,6134,3889\nGTCATTGCATTGACCC-1,1,31,17,6254,3958\nTGTCCGTGGCGCCTTT-1,1,30,18,6134,4027\nTCAACTAACGTATAAC-1,1,31,19,6253,4096\nTCCTCTCCAGTTGTCC-1,1,30,20,6134,4164\nTGTGTTCGTATCCAAG-1,1,31,21,6253,4233\nCCGCGTAGGTAAGGGC-1,1,30,22,6133,4302\nCTGCGGGTGAAATGTT-1,1,31,23,6253,4371\nTATCTACAGAGGTAAT-1,1,30,24,6133,4439\nCTACTCTAGGCCCGGC-1,1,31,25,6252,4509\nACAAGCAGTGCCTAGC-1,1,30,26,6132,4577\nTACAAGTCTCGTGCAT-1,1,31,27,6252,4646\nTCGGAATGCGCTCTGA-1,1,30,28,6132,4715\nTCGCGTCCAGAAGGTC-1,1,31,29,6252,4784\nTATGGCCCGGCCTCGC-1,1,30,30,6132,4852\nGCTGGCATATTCACCT-1,1,31,31,6251,4921\nGTCAGAATAGTCTATG-1,1,30,32,6131,4990\nGGCGTCCTATCCGCTG-1,1,31,33,6251,5059\nCGGAGTTTGAGAGACA-1,1,30,34,6131,5127\nAGCACTTAAGGACGCC-1,1,31,35,6251,5196\nTCCACAATGGTTTACG-1,1,30,36,6131,5265\nCCAACGATGCACTGAT-1,1,31,37,6250,5334\nATTTACAGTTTACTGG-1,1,30,38,6130,5403\nCCCTGAAATGAGTTGA-1,1,31,39,6250,5472\nCAAACGGTCGCACTTT-1,1,30,40,6130,5540\nTGATTCGTCTATCACT-1,1,31,41,6250,5609\nTCAGGTTCTTTGAGAA-1,1,30,42,6130,5678\nCACGCAGCGAGGCTTT-1,1,31,43,6249,5747\nTTAAGCGCCTGACCCA-1,1,30,44,6129,5815\nCTTACACGGTATTCCA-1,1,31,45,6249,5884\nAAGGCTGTGCTCATCG-1,1,30,46,6129,5953\nGACCAGAGCCCTGTAG-1,1,31,47,6249,6022\nTCCCAGGCTTAGCTAA-1,1,30,48,6129,6090\nATTGAAGATCTTAGTG-1,1,31,49,6248,6160\nAGTTCCTACAGAATTA-1,1,30,50,6128,6228\nGGGCTGGTTAGTCGCG-1,1,31,51,6248,6297\nGAAATGGCGGTGTTAG-1,1,30,52,6128,6366\nTACGAACACGACTTCA-1,1,31,53,6247,6435\nACCACAAGTTTCTATC-1,1,30,54,6128,6503\nATATTTAACCCTCAAG-1,1,31,55,6247,6572\nGATCATTCCAAACATT-1,1,30,56,6127,6641\nTCCAGGCGAGTACGGT-1,1,31,57,6247,6710\nGTTTGACCAAATCCTA-1,1,30,58,6127,6778\nCACAGCACCCACGGCA-1,1,31,59,6246,6847\nTGCAAGAATGACGTAA-1,1,30,60,6127,6916\nGCGAAGCCATACCCGT-1,1,31,61,6246,6985\nTCCTTTCTTACGCTTA-1,1,30,62,6126,7053\nGCTGCTCTCCGGACAC-1,1,31,63,6246,7123\nACTGTCTTCTTTAGAA-1,1,30,64,6126,7191\nTCAAACTTAGATTGTT-1,1,31,65,6245,7260\nCTATGTCACTAGCCCA-1,1,30,66,6125,7329\nTGCGCAAAGCATTTGG-1,1,31,67,6245,7398\nTTAATGTAGACCAGGT-1,1,30,68,6125,7466\nGGCGGTAGGATCATTG-1,1,31,69,6245,7535\nGGCAATAGTCAATGAG-1,1,30,70,6125,7604\nACACGAGACTCCTTCT-1,1,31,71,6244,7673\nGACACAAGGGAAGAAA-1,1,30,72,6124,7741\nTCAGCAAATGCATCTC-1,1,31,73,6244,7810\nGAGATCTGTCACTCCG-1,1,30,74,6124,7879\nATGCCGGTCTTGCATA-1,1,31,75,6244,7948\nTTGGGCGGCGGTTGCC-1,1,30,76,6124,8017\nTTGTTGTGTGTCAAGA-1,1,31,77,6243,8086\nACTGTACGATACACAT-1,1,30,78,6123,8154\nTCCACTTTATCTAGGT-1,1,31,79,6243,8223\nGGTCTGAGAATCTGGA-1,1,30,80,6123,8292\nTAGAAAGGTGGCGCTA-1,1,31,81,6243,8361\nTATGTCTCATTGTGCC-1,1,30,82,6123,8429\nGGATTTCACTTCTATA-1,1,31,83,6242,8498\nTGAGTGGTCCGTGACG-1,1,30,84,6122,8567\nCGCTTTCTTGCATTCG-1,1,31,85,6242,8636\nACCCAACGCCCGTGGC-1,1,30,86,6122,8704\nGAACGTCTCATGGTCG-1,1,31,87,6242,8774\nAGGGTTCCCTTTGGTT-1,1,30,88,6122,8842\nGTAGCTTCCTCTTGTT-1,1,31,89,6241,8911\nGCATGAGGGACGCGGC-1,1,30,90,6121,8980\nCTACCCTAAGGTCATA-1,1,31,91,6241,9049\nTCACCGCTCGGCACTC-1,1,30,92,6121,9117\nGGCTCGCGTTGAGGTA-1,1,31,93,6240,9186\nCTAACGAAACTTGCTG-1,1,30,94,6121,9255\nTTAAACTCGAATTCAT-1,1,31,95,6240,9324\nTACTTTACTGAGCCGG-1,1,30,96,6120,9392\nGCTTGGATCGATTAGG-1,1,31,97,6240,9461\nCGGTTATCCAACAGTG-1,1,30,98,6120,9530\nCAGACCTGTAAGTGTT-1,1,31,99,6239,9599\nGACGGTCAATAGAAGC-1,1,30,100,6120,9668\nCTGACTGCGCAGCTCG-1,1,31,101,6239,9737\nCCATACCTTTACTTGT-1,1,30,102,6119,9805\nGTAATAAAGGGCTCCC-1,1,31,103,6239,9874\nGTGAACTCCCATTCGA-1,1,30,104,6119,9943\nGTGGTTACTTCTTTCG-1,1,31,105,6238,10012\nTCAGAACCTCCACAGG-1,1,30,106,6118,10080\nTCCCACTCTCTTCCGG-1,1,31,107,6238,10149\nATCTTGACTTGTCCAA-1,1,30,108,6118,10218\nTCGGGAACGTGCCTAG-1,1,31,109,6238,10287\nGTTAGCCGTAAATCAA-1,1,30,110,6118,10355\nATTTACTAAGTCCATT-1,1,31,111,6237,10425\nGGGTGCATATGAAAGC-1,1,30,112,6117,10493\nTCCGAATGGTCCTGAG-1,1,31,113,6237,10562\nTGATGGCTGTTTCTGA-1,1,30,114,6117,10631\nAAATAAGGTAGTGCCC-1,1,31,115,6237,10700\nCCACTATCCGGGTCAC-1,1,30,116,6117,10768\nACACCACATAATTAGC-1,1,31,117,6236,10837\nCGCGGTAAGTCTAGCT-1,1,30,118,6116,10906\nGCGGGCATTACGATGC-1,1,31,119,6236,10975\nAGGATTGCTTACGACA-1,1,30,120,6116,11043\nCTCGGGATAACACCTA-1,1,31,121,6236,11112\nGTCGTCTGGTTGGCTA-1,1,30,122,6116,11181\nGCAATTAGTCGCACCG-1,1,31,123,6235,11250\nGTGACCTAAAGAATAA-1,1,30,124,6115,11318\nCTGAGCGAGACTTATT-1,0,31,125,6235,11388\nCAAGACTCAGAAGCGC-1,0,30,126,6115,11456\nACTTCGCTAGCGAGTG-1,0,31,127,6235,11525\nCCATACTCGCCTCTCC-1,1,32,0,6376,2789\nACGATACCTATCCTGA-1,1,33,1,6496,2858\nCTCACCAGTACAAGTG-1,1,32,2,6376,2927\nCGAAGACGGTGAGTGC-1,1,33,3,6496,2996\nAAATTAATAAGCGCGA-1,1,32,4,6376,3064\nGGGCCCTTATCTATAC-1,1,33,5,6495,3133\nCTGCCCACGAAGCGTT-1,1,32,6,6375,3202\nGGACAAGTTGCAGTGA-1,1,33,7,6495,3271\nGTCCGAGAGCAATCAT-1,1,32,8,6375,3339\nATGGCAGCCGAGAAAC-1,1,33,9,6495,3409\nCCTCGGATGCTACCTG-1,1,32,10,6375,3477\nATACGGTGAAGATGCA-1,1,33,11,6494,3546\nACATCAGCTGGGACGC-1,1,32,12,6374,3615\nGGTTGTGTAGCCTGGC-1,1,33,13,6494,3684\nCCTGCGTTCTACGCTT-1,1,32,14,6374,3752\nGAACGTTAGGAAGACG-1,1,33,15,6493,3821\nCTGATAGTGTATCTCA-1,1,32,16,6374,3890\nTTCTGCGAGCGCCCTT-1,1,33,17,6493,3959\nCTTTGGCGCTTTATAC-1,1,32,18,6373,4027\nTCCGAACTTGGCTTAC-1,1,33,19,6493,4096\nTAGATTCCTGGTTATT-1,1,32,20,6373,4165\nCCGACAAAGGGAGTGC-1,1,33,21,6492,4234\nCCATGGCCCTTGTACC-1,1,32,22,6373,4303\nGAAATATCACCATCAG-1,1,33,23,6492,4372\nACGAAATGGGCGGCAC-1,1,32,24,6372,4440\nGTGAGCGTGCTGCACT-1,1,33,25,6492,4509\nCCGCGGGTACGAAGAA-1,1,32,26,6372,4578\nTCCCTGGCTCGCTGGA-1,1,33,27,6491,4647\nCAGCTTAGTAGGTAGC-1,1,32,28,6372,4715\nCACGAAAGTTAGTCCC-1,1,33,29,6491,4784\nACCTAATCGACTTCCT-1,1,32,30,6371,4853\nAAGTAGTGACGCGAGG-1,1,33,31,6491,4922\nTCCGATTACATTGCCG-1,1,32,32,6371,4990\nCCTCCGACAATTCAAG-1,1,33,33,6490,5059\nGTTCACAGGAGTCTAG-1,1,32,34,6370,5128\nCGAAGTTGCTCTGTGT-1,1,33,35,6490,5197\nGTCGGATATCTCAGAC-1,1,32,36,6370,5266\nCGCTCTCCGTAGATTA-1,1,33,37,6490,5335\nCAAGCAACGTCGGAGT-1,1,32,38,6370,5403\nCCATTCCCTGCCCACA-1,1,33,39,6489,5472\nCTTTGGCTTTAGTAAA-1,1,32,40,6369,5541\nGGCTATTAAGTTGTAT-1,1,33,41,6489,5610\nCCATTAGCGATAATCC-1,1,32,42,6369,5678\nTGTTCTTCCATTGACT-1,1,33,43,6489,5747\nAGATAACTTCAGGGCC-1,1,32,44,6369,5816\nATAGACGAAGAGAAAG-1,1,33,45,6488,5885\nGGCGGAGTAATATTAG-1,1,32,46,6368,5953\nTGACCCACGTTAGACA-1,1,33,47,6488,6023\nTCACAGGTTATTGGGC-1,1,32,48,6368,6091\nTCACGCATTGTAGATC-1,1,33,49,6488,6160\nTTGAAGAATTCCCAGG-1,1,32,50,6368,6229\nAAATGGTCAATGTGCC-1,1,33,51,6487,6298\nTAGTGCCCTCCAGAGT-1,1,32,52,6367,6366\nGGTATTGCCGAGTTTA-1,1,33,53,6487,6435\nCGTATTAAGAGATCTA-1,1,32,54,6367,6504\nACTGTCCAGGATTATA-1,1,33,55,6487,6573\nCGGGCAGCTAAACCGC-1,1,32,56,6367,6641\nTTGCTGATCATGTTCG-1,1,33,57,6486,6710\nTATGGGTACGTATCGT-1,1,32,58,6366,6779\nCAGCTCACTGAGACAT-1,1,33,59,6486,6848\nGGGACTGCATAGATAG-1,1,32,60,6366,6917\nACGCATTCGTGAGTAC-1,1,33,61,6485,6986\nCTCTGGACGCCTGGTG-1,1,32,62,6366,7054\nAGGGTTTAGTTCGGGA-1,1,33,63,6485,7123\nGGGAGAACTCACAGTA-1,1,32,64,6365,7192\nATCAATCTGGGCTGCA-1,1,33,65,6485,7261\nTCTTCGATACCAATAA-1,1,32,66,6365,7329\nACGTAGATTGCTGATG-1,1,33,67,6484,7398\nTCTTGATGCGTAGCGA-1,1,32,68,6365,7467\nGGGCTGCCTAGGGCGA-1,1,33,69,6484,7536\nCTCTCACAATCGATGA-1,1,32,70,6364,7604\nCCAAGCGTAACTCGTA-1,1,33,71,6484,7674\nACAACAGCATGAGCTA-1,1,32,72,6364,7742\nGTCCCAACGTAAAGTA-1,1,33,73,6483,7811\nTCGGAGTACATGAGTA-1,1,32,74,6363,7880\nGGGAGTTAATGAGGCG-1,1,33,75,6483,7949\nCCGGGCGGTCTCGTCA-1,1,32,76,6363,8017\nCCGTAAGTTGGTCCCA-1,1,33,77,6483,8086\nGGAGGGCTTGGTTGGC-1,1,32,78,6363,8155\nTCGGACGCCCAGCCCA-1,1,33,79,6482,8224\nTCTGTGCCATCATAGT-1,1,32,80,6362,8292\nGTACTGGAGTTAGACC-1,1,33,81,6482,8361\nGGAATGCGCTAGCGTG-1,1,32,82,6362,8430\nGTGTGAATAACTTAGG-1,1,33,83,6482,8499\nGGTCGGCCAGGAGCTT-1,1,32,84,6362,8567\nTAGCCGGCGGTCAGCG-1,1,33,85,6481,8637\nCGGGTGTACCCATTTA-1,1,32,86,6361,8705\nAGTGATTCAAGCAGGA-1,1,33,87,6481,8774\nGTTGGATTGAGAACAC-1,1,32,88,6361,8843\nCACACGCGCTGTCTTA-1,1,33,89,6481,8912\nTAGACGCCCGTACCGG-1,1,32,90,6361,8980\nGGTTTCAATCGGTCAG-1,1,33,91,6480,9049\nAATCTGCGTTGGGACG-1,1,32,92,6360,9118\nTTACGGATGGTTCGAG-1,1,33,93,6480,9187\nCGGCAGGGTCGGGTTG-1,1,32,94,6360,9255\nGCTTTCAGAGGAGGTG-1,1,33,95,6480,9324\nTCTTCCCATGGGCACA-1,1,32,96,6360,9393\nTACCGCGGACTTGCAG-1,1,33,97,6479,9462\nAGAATTATGGATTCGA-1,1,32,98,6359,9531\nATTGATGAGTCCTAAC-1,1,33,99,6479,9600\nTAGGTCGCCGGAACTG-1,1,32,100,6359,9668\nTAACCTACCGTCCGAG-1,1,33,101,6478,9737\nCTTAGTAGGCCTACAG-1,1,32,102,6359,9806\nCTAGATGTGAGTGTAA-1,1,33,103,6478,9875\nACTCCCGAATTCGTTT-1,1,32,104,6358,9943\nGTTCATCGTTTGGCTG-1,1,33,105,6478,10012\nACTTTACCCTCATGAA-1,1,32,106,6358,10081\nGCGAGAGTTGCGTCCA-1,1,33,107,6477,10150\nGTTCGGGCGTACCATT-1,1,32,108,6358,10218\nCGACTTTGTATAGCCT-1,1,33,109,6477,10288\nGCCATCGATGCTGCAT-1,1,32,110,6357,10356\nGCATTTCCAAGGCTCC-1,1,33,111,6477,10425\nATGTAAGGCTGCTCTT-1,1,32,112,6357,10494\nACGTTCGCAATCAATT-1,1,33,113,6476,10563\nGTGACGAGGGTGACCC-1,1,32,114,6356,10631\nATTATAGCTACTTTAC-1,1,33,115,6476,10700\nCGTGTGTTAAACCCTG-1,1,32,116,6356,10769\nTTGGTATGGCTTGTGT-1,1,33,117,6476,10838\nCATTCCCATTCCGTCG-1,1,32,118,6356,10906\nTGCCGAAAGCGTATTC-1,1,33,119,6475,10975\nCAACACATCTCCTGCC-1,1,32,120,6355,11044\nCTGCCTCATATGCAAC-1,1,33,121,6475,11113\nTCCCGCCTATGTGCGT-1,1,32,122,6355,11182\nGGTTACCCGACACTTT-1,1,33,123,6475,11251\nCCAGCGGGATCACCAG-1,0,32,124,6355,11319\nATGTTTCGGCCCGGAG-1,0,33,125,6474,11388\nGCGTCTAACCTCCTAA-1,0,32,126,6354,11457\nATCAGGTAGCTGACAG-1,0,33,127,6474,11526\nGGTATGAAAGAACTGA-1,1,34,0,6616,2790\nGTGGCCTAATATCATT-1,1,35,1,6735,2859\nCCTGTGAAACCGTAAC-1,1,34,2,6615,2927\nGGCAGAGAGATCGGGA-1,1,35,3,6735,2996\nTAGCGTCGAATATTGA-1,1,34,4,6615,3065\nCGCCGACTATTCGCTA-1,1,35,5,6735,3134\nTCTGGCGCAAGCCGGG-1,1,34,6,6615,3202\nAGTGGTTGCGTATAGG-1,1,35,7,6734,3272\nATCGGTTACCTAGTAA-1,1,34,8,6614,3340\nCCTGCCCGTTGTCTAG-1,1,35,9,6734,3409\nGCACACGCCCATGGTC-1,1,34,10,6614,3478\nAGTACGGCCCGTATCG-1,1,35,11,6734,3547\nTATCTAGCCTAAAGGA-1,1,34,12,6614,3615\nCACTCGGTTAGGAGGA-1,1,35,13,6733,3684\nATGTTCGTCGACCCAC-1,1,34,14,6613,3753\nTTCCTCTGCCCGAATA-1,1,35,15,6733,3822\nTTACTATCGGCTTCTC-1,1,34,16,6613,3890\nGCCGCATTAGTCCGGC-1,1,35,17,6733,3959\nTAAGGGCTGGGAGAGG-1,1,34,18,6613,4028\nTAAGCAGGCGACACGC-1,1,35,19,6732,4097\nAGCACTACCGGCCTGT-1,1,34,20,6612,4166\nGAAAGCCCTTTGGACC-1,1,35,21,6732,4235\nGACCGACTGAAGCGTC-1,1,34,22,6612,4303\nCGGTGAAGACTAAAGT-1,1,35,23,6731,4372\nCCCTGCGCTACGCATA-1,1,34,24,6612,4441\nTACTGGACAGCTCGGC-1,1,35,25,6731,4510\nTTAGTAGGGCGGCGGG-1,1,34,26,6611,4578\nGAGGCTATCAAAGTCG-1,1,35,27,6731,4647\nTTACCATTGATTACCC-1,1,34,28,6611,4716\nATACCACGGGCAACTT-1,1,35,29,6730,4785\nTGTCCTAAGTCACCGC-1,1,34,30,6611,4853\nAGGTAGGTACAAAGCT-1,1,35,31,6730,4923\nGGCATACAGGTAGCGG-1,1,34,32,6610,4991\nTGTAGTGATCTATAAT-1,1,35,33,6730,5060\nTCCCGGGTGTGCTGCT-1,1,34,34,6610,5129\nTACGATGTTGATCATC-1,1,35,35,6729,5198\nCCTCTCTCCCATCTAG-1,1,34,36,6610,5266\nGCAGGACTATAGAATA-1,1,35,37,6729,5335\nCTAGTGAAGGACAGGA-1,1,34,38,6609,5404\nTACGAGAACTTCACGT-1,1,35,39,6729,5473\nCGTTGTTTCAATTCCC-1,1,34,40,6609,5541\nGCAAATATTACGCTTT-1,1,35,41,6728,5610\nCCAATAGTGCCGTCGA-1,1,34,42,6608,5679\nATTGCTGCTCCTCCAT-1,1,35,43,6728,5748\nGAGATCTGCTTGGCAT-1,1,34,44,6608,5816\nGCCGAAATTCCTACGT-1,1,35,45,6728,5886\nGGCACTCCACTGGGCA-1,1,34,46,6608,5954\nGGGTCACCGTGACGGT-1,1,35,47,6727,6023\nCACTTAATCAGACGGA-1,1,34,48,6607,6092\nCGTTTCGCTCATTACA-1,1,35,49,6727,6161\nATAAAGGCTCGGTCGT-1,1,34,50,6607,6229\nCACTAAAGTTGCCTAT-1,1,35,51,6727,6298\nGTGCTCAAGTACTGTC-1,1,34,52,6607,6367\nCCATGCCTGTTTAGTA-1,1,35,53,6726,6436\nTCTAGTTATCAGAAGA-1,1,34,54,6606,6504\nTTGTAATCCGTACTCG-1,1,35,55,6726,6573\nTCCCAGCTTTAGTCTG-1,1,34,56,6606,6642\nCTACGCACGGAGTACC-1,1,35,57,6726,6711\nAAATTAACGGGTAGCT-1,1,34,58,6606,6780\nCGGCCACGCACAAAGT-1,1,35,59,6725,6849\nGAAGCGTGAGGAATTT-1,1,34,60,6605,6917\nATATCTTAGGGCCTTC-1,1,35,61,6725,6986\nACGCGGGCCAAGGACA-1,1,34,62,6605,7055\nGCGAGTTCTGCAAAGA-1,1,35,63,6724,7124\nTATTCGTGCCAGAATA-1,1,34,64,6605,7192\nAGGGCTGCAGTTACAG-1,1,35,65,6724,7261\nCTAGCATAGTATAATG-1,1,34,66,6604,7330\nTAGGTTCGAGTTCGTC-1,1,35,67,6724,7399\nGAATTATAGTGAAAGG-1,1,34,68,6604,7467\nCTATCGGGTCTCAACA-1,1,35,69,6723,7537\nGCGCTAATTGAATAGA-1,1,34,70,6604,7605\nATGCGACAGTCCCATT-1,1,35,71,6723,7674\nGGTAGTGCTCGCACCA-1,1,34,72,6603,7743\nAAGCTCGTGCCAAGTC-1,1,35,73,6723,7812\nTATTCAATTCTAATCC-1,1,34,74,6603,7880\nTTCAAAGTCTCTAGCC-1,1,35,75,6722,7949\nTTGAATATGGACTTTC-1,1,34,76,6603,8018\nAAGAGCTCTTTATCGG-1,1,35,77,6722,8087\nTTACTCCGGCCGGGAA-1,1,34,78,6602,8155\nAAACGAGACGGTTGAT-1,1,35,79,6722,8224\nGCTAAGTAGTTTCTCT-1,1,34,80,6602,8293\nATAACGCCGGAGGGTC-1,1,35,81,6721,8362\nGGATCCGGAATATACT-1,1,34,82,6601,8431\nTGAAAGGACCTGACTC-1,1,35,83,6721,8500\nTCCGCGGCAGCATCTG-1,1,34,84,6601,8568\nTGCATATGTCTGTCAC-1,1,35,85,6721,8637\nTGTAGGAGAAATTTCC-1,1,34,86,6601,8706\nAGTGAGACTTCCAGTA-1,1,35,87,6720,8775\nCCCAAACATGCTGCTC-1,1,34,88,6600,8843\nGCTTATGAAGCAGGAA-1,1,35,89,6720,8912\nTTCTAACCGAAGCTTA-1,1,34,90,6600,8981\nGGATGTCCTTACCGCA-1,1,35,91,6720,9050\nAGGGTGCTCTCGAGGG-1,1,34,92,6600,9118\nAACTCTCAATAGAGCG-1,1,35,93,6719,9188\nTCTGAATTCCGTACAA-1,1,34,94,6599,9256\nGCGTGGTACTGGGTTA-1,1,35,95,6719,9325\nCGTCGGATAGTGTTGA-1,1,34,96,6599,9394\nATATGTCTCCCTAGCC-1,1,35,97,6719,9463\nTCTTTAAGACTATGAA-1,1,34,98,6599,9531\nTCATTTAAGTCTCCGA-1,1,35,99,6718,9600\nGATATTGAGATTGGCG-1,1,34,100,6598,9669\nTGACATCGAGCGGACC-1,1,35,101,6718,9738\nGCGTAAATGGCCATAA-1,1,34,102,6598,9806\nATTGTACAACTCGGCT-1,1,35,103,6717,9875\nTACGCTATAGAAACCT-1,1,34,104,6598,9944\nCACCCAAATCTTATGT-1,1,35,105,6717,10013\nAGATGATGGAGTCTGG-1,1,34,106,6597,10081\nCCACGGTGCCCGGTAG-1,1,35,107,6717,10151\nTCAAGAAATACTAGCT-1,1,34,108,6597,10219\nAGGTATAATTGATAGT-1,1,35,109,6716,10288\nCAAGGTCCTATAGGCT-1,1,34,110,6597,10357\nCCGGCACGACCGTTTC-1,1,35,111,6716,10426\nACCTCCGTTATTCACC-1,1,34,112,6596,10494\nGCAGCCTATATCACAT-1,1,35,113,6716,10563\nGGTATAGTGACACATA-1,1,34,114,6596,10632\nAAATTCCAGGTCCAAA-1,1,35,115,6715,10701\nTCTTTAGCAGGCGAAC-1,1,34,116,6596,10769\nTATTGACATTTCTGCC-1,1,35,117,6715,10838\nTCTGATCGGGTGCTAG-1,1,34,118,6595,10907\nGGCCCGGAGCATGTCT-1,1,35,119,6715,10976\nGGGCGCAGCGTTACTC-1,1,34,120,6595,11045\nTTGGCGATCCGAATAT-1,1,35,121,6714,11114\nCCACGTAAATTAGACT-1,1,34,122,6594,11182\nTCTGATTGGAAATGGA-1,1,35,123,6714,11251\nATGGCGGAATAGTCGC-1,0,34,124,6594,11320\nATCGCTTTACGTCTCA-1,0,35,125,6714,11389\nTACGTGCAAGGTTCCT-1,0,34,126,6594,11457\nCAGGACAGCTGCCCTT-1,0,35,127,6713,11526\nCAAACCAGGTCTGCAT-1,1,36,0,6855,2790\nACAAGCTATATGGAAG-1,1,37,1,6975,2859\nTCGCCCACTGCGAGAG-1,1,36,2,6855,2928\nAGCCGCAAATTCAAAT-1,1,37,3,6974,2997\nTTAACGTTAAAGCCTG-1,1,36,4,6855,3065\nCAGCGCCAACACGATA-1,1,37,5,6974,3135\nATCCAATGGTACCGAA-1,1,36,6,6854,3203\nGTGCTGCAGATAAGGA-1,1,37,7,6974,3272\nGGCCTTTGCAACTGGC-1,1,36,8,6854,3341\nGTCGTACCTACGATTG-1,1,37,9,6973,3410\nTAGAAATTCACGTATA-1,1,36,10,6853,3478\nAGAATAAATCTTCAGG-1,1,37,11,6973,3547\nCATTGCGAAATGGGCG-1,1,36,12,6853,3616\nGTCTACTCAATTACAA-1,1,37,13,6973,3685\nTGTAATGACCACAATA-1,1,36,14,6853,3753\nAAAGTCGACCCTCAGT-1,1,37,15,6972,3822\nTACTCGGCACGCCGGG-1,1,36,16,6852,3891\nAGGTGTATCGCCATGA-1,1,37,17,6972,3960\nTGTGCTTTACGTAAGA-1,1,36,18,6852,4029\nAAACCTCATGAAGTTG-1,1,37,19,6972,4098\nTATAGGGTACTCATGA-1,1,36,20,6852,4166\nCCAGCTGATGGTACTT-1,1,37,21,6971,4235\nAATATTGGAGTATTGA-1,1,36,22,6851,4304\nGGCCCTCACCCACTTA-1,1,37,23,6971,4373\nAACCAAGACTTCTCTG-1,1,36,24,6851,4441\nTCGTATTACCCATTGC-1,1,37,25,6971,4510\nATTCGACGCCGGGCCT-1,1,36,26,6851,4579\nGGCGCAGGACATCTTC-1,1,37,27,6970,4648\nGTACTCCCTTATCGCT-1,1,36,28,6850,4716\nTGGTCTGTTGGGCGTA-1,1,37,29,6970,4786\nAATAACAACGCTCGGC-1,1,36,30,6850,4854\nCATACCCGTACCCAGT-1,1,37,31,6969,4923\nACAATCCATTTAAACC-1,1,36,32,6850,4992\nGTTACAATTGGTGACG-1,1,37,33,6969,5061\nTTGCCCTGATCACGGG-1,1,36,34,6849,5129\nCTAACCGCGCGCCCGT-1,1,37,35,6969,5198\nCTAAAGAATGCCTACT-1,1,36,36,6849,5267\nACCCATCTTGAGGGTA-1,1,37,37,6968,5336\nGATCTTTGCAGGGTAT-1,1,36,38,6849,5404\nGGGTACTTCATGAACT-1,1,37,39,6968,5473\nGCCGCTTGTGAGAAAC-1,1,36,40,6848,5542\nCCTGTAAGACATGATA-1,1,37,41,6968,5611\nCGACAGTTCGCGTTAT-1,1,36,42,6848,5680\nACGATGCATATGTTAT-1,1,37,43,6967,5749\nTGTTCCGCTTCCATGA-1,1,36,44,6848,5817\nGGATGACGCGAGTTTA-1,1,37,45,6967,5886\nGAAGTTTCCACTCAAT-1,1,36,46,6847,5955\nGCGAGGCCCGAGCAGA-1,1,37,47,6967,6024\nCATACTATGTAATTGT-1,1,36,48,6847,6092\nCCAATGTCACAGCAAG-1,1,37,49,6966,6161\nGTTGGATTTGCGTTGG-1,1,36,50,6846,6230\nGGGAGGATGCCCGAAA-1,1,37,51,6966,6299\nGATCGCGGGCTCTCCA-1,1,36,52,6846,6367\nGTTCGCCATAAGTGCC-1,1,37,53,6966,6437\nAGATTATAGGACGTTT-1,1,36,54,6846,6505\nTCGAGACCAACACCGT-1,1,37,55,6965,6574\nTATGGGACCGAGCAGG-1,1,36,56,6845,6643\nGATGCGTCCTGCATTC-1,1,37,57,6965,6712\nTATGGTCTGAGTAACA-1,1,36,58,6845,6780\nGCATAGAGCACTCAGG-1,1,37,59,6965,6849\nCTTCATTGTCAGTGGA-1,1,36,60,6845,6918\nGCAGATTAGGGATATC-1,1,37,61,6964,6987\nCCTGTCGCCCGTAAAT-1,1,36,62,6844,7055\nCAATTTCGTATAAGGG-1,1,37,63,6964,7124\nGTACACTTACCTGAAG-1,1,36,64,6844,7193\nCCAGCCTGGACCAATA-1,1,37,65,6964,7262\nATGGAGCAGGCCGTGA-1,1,36,66,6844,7330\nGTCATTAGAGCGAACG-1,1,37,67,6963,7400\nAAGACTGCAAGCTACT-1,1,36,68,6843,7468\nCTAGTCACGTCTTAAG-1,1,37,69,6963,7537\nACTCTTGTATAGTAAC-1,1,36,70,6843,7606\nATTAGGCGATGCTTTC-1,1,37,71,6962,7675\nTTCGGGACTAATCGCG-1,1,36,72,6843,7743\nTGGACTGTTCGCTCAA-1,1,37,73,6962,7812\nAACGTGCGAAAGTCTC-1,1,36,74,6842,7881\nCCACCCAAGGAAAGTG-1,1,37,75,6962,7950\nCCGCACAAAGACCAAC-1,1,36,76,6842,8018\nGCGATTGTTAACGTTA-1,1,37,77,6961,8087\nACTCGTCAGTAATCCC-1,1,36,78,6842,8156\nGGTGATAAGGAGCAGT-1,1,37,79,6961,8225\nAAGAGGCATGGATCGC-1,1,36,80,6841,8294\nCACGTTCGTGCTCTAG-1,1,37,81,6961,8363\nCTATTTGGTTACGGAT-1,1,36,82,6841,8431\nGTACAGAGGCAAGGGT-1,1,37,83,6960,8500\nGGGCCGGCCGAAGTAC-1,1,36,84,6841,8569\nCCTGAACGATATATTC-1,1,37,85,6960,8638\nCCGGGCTGCTCCATAC-1,1,36,86,6840,8706\nTACTTGTTAGTAGTCC-1,1,37,87,6960,8775\nCCTAGGCGTAGCGATC-1,1,36,88,6840,8844\nCTGGCGCACAGGTCTG-1,1,37,89,6959,8913\nACTTATACTTACCCGG-1,1,36,90,6839,8981\nGAAGTCTCCCTAGCGA-1,1,37,91,6959,9051\nACCGATGGTAGCATCG-1,1,36,92,6839,9119\nCGAGTTTATCGGACTG-1,1,37,93,6959,9188\nCATAACGGACAGTCGT-1,1,36,94,6839,9257\nTGACGATGCACTAGAA-1,1,37,95,6958,9326\nTAGGGAGCTTGGGATG-1,1,36,96,6838,9394\nAGGGTCGATGCGAACT-1,1,37,97,6958,9463\nTATATCCCTGGGAGGA-1,1,36,98,6838,9532\nCATCTTACACCACCTC-1,1,37,99,6958,9601\nGTGCGACAGGGAGTGT-1,1,36,100,6838,9669\nCCGATCTCAACCTTAT-1,1,37,101,6957,9738\nACGATCATCTTGTAAA-1,1,36,102,6837,9807\nGAAAGAACAGCGTTAT-1,1,37,103,6957,9876\nCTAGGTCTGAAGGAAT-1,1,36,104,6837,9945\nATATCAACCTACAGAG-1,1,37,105,6957,10014\nAAATAGCTTAGACTTT-1,1,36,106,6837,10082\nGCGACATGTAAACATC-1,1,37,107,6956,10151\nATAAGTAGGGCGACTC-1,1,36,108,6836,10220\nGCGAGCGCATGCTCCC-1,1,37,109,6956,10289\nAGGGACCGGCTGCGTT-1,1,36,110,6836,10357\nCCTATGAAGTGGTGCC-1,1,37,111,6955,10426\nGCTTACGTAGTTAGTA-1,1,36,112,6836,10495\nCATACTTAGGCAATAC-1,1,37,113,6955,10564\nCCTGTCCCTCACGTTA-1,1,36,114,6835,10632\nCAATGTGCCAACCCTT-1,1,37,115,6955,10702\nGTTAAGTTAGAGTGGG-1,1,36,116,6835,10770\nCTGGGATACGCTACCC-1,1,37,117,6954,10839\nAACCTGTCACGGAATT-1,1,36,118,6835,10908\nACTGCGGACACACCGT-1,1,37,119,6954,10977\nCCGTGAGGCATTCATG-1,1,36,120,6834,11045\nGCCCAGATGCTGGAGA-1,1,37,121,6954,11114\nTCTGGCCGTTCAAGTT-1,1,36,122,6834,11183\nATACGAAGGCTTTCCA-1,1,37,123,6953,11252\nGATCCGAATATAAGTG-1,0,36,124,6834,11320\nGTGGAGCATGTCGGCC-1,0,37,125,6953,11389\nACTCTTCAGCTCCCGC-1,0,36,126,6833,11458\nCCCGATAGCCTCGCCT-1,0,37,127,6953,11527\nACAGGTGTGTTGTTGC-1,1,38,0,7095,2791\nTGAACTGCTATGACTT-1,1,39,1,7214,2860\nTGACATATATGACGAT-1,1,38,2,7094,2929\nAAGTCTTCTGTGGCCT-1,1,39,3,7214,2998\nACCAGACCATAACAAC-1,1,38,4,7094,3066\nTGAGACGTACCTCTCA-1,1,39,5,7213,3135\nGACCGTTACATGCGAC-1,1,38,6,7094,3204\nGGTTCGGATTATACTA-1,1,39,7,7213,3273\nCCTCCTGAGCCCACAT-1,1,38,8,7093,3341\nCCGCGATTTGGTAGGT-1,1,39,9,7213,3410\nAATCTCTACTGTGGTT-1,1,38,10,7093,3479\nACTTTGACTGCATCCT-1,1,39,11,7212,3548\nCCCTGACTAACAAATT-1,1,38,12,7092,3616\nACGCTGTGAGGCGTAG-1,1,39,13,7212,3686\nGCATACGAGGTCTTTA-1,1,38,14,7092,3754\nTCGGGATTCAAACATA-1,1,39,15,7212,3823\nGGGCCCTACGAAAGGG-1,1,38,16,7092,3892\nCGCCAAGAAGCCGAGT-1,1,39,17,7211,3961\nGGTACATCTGGGACGA-1,1,38,18,7091,4029\nGGATGCTGGCGTTCCT-1,1,39,19,7211,4098\nCCGATTCGAGGGACCC-1,1,38,20,7091,4167\nCCACTGGTGGCTGGTT-1,1,39,21,7211,4236\nGACAGGCACACACTAT-1,1,38,22,7091,4304\nTCAACGCGACCGGCAG-1,1,39,23,7210,4373\nCTACGACTAGCTATAA-1,1,38,24,7090,4442\nCGGTTGACCTGGCATA-1,1,39,25,7210,4511\nATCCTGAATCGCTGCG-1,1,38,26,7090,4579\nGTTTCATATCGTCGCT-1,1,39,27,7210,4649\nATAAATATTAGCAGCT-1,1,38,28,7090,4717\nAAGAGGATGTACGCGA-1,1,39,29,7209,4786\nTCCTGCGTTGATACTC-1,1,38,30,7089,4855\nCGTGCATTGTCGACGC-1,1,39,31,7209,4924\nCCGGGTTCGAGGTTAC-1,1,38,32,7089,4992\nCCCAATTTCACAACTT-1,1,39,33,7209,5061\nTGATTTATTAGCTGTG-1,1,38,34,7089,5130\nTGGAAGAAGGGAACGT-1,1,39,35,7208,5199\nGACGCTTGCTTCTAAA-1,1,38,36,7088,5267\nGGGAACGGGAGGTTAG-1,1,39,37,7208,5336\nGCGGCTCTGACGTACC-1,1,38,38,7088,5405\nACGTTAGATTTGCCCG-1,1,39,39,7207,5474\nGAGAGGGCGCGAGGTT-1,1,38,40,7088,5543\nGCGTCTCTGCATTGGG-1,1,39,41,7207,5612\nGCAGCACACAGCCCAG-1,1,38,42,7087,5680\nCAGGCCGTTTGGGTGT-1,1,39,43,7207,5749\nAACTCAAGTTAATTGC-1,1,38,44,7087,5818\nCTTCGTAGATAGGTGA-1,1,39,45,7206,5887\nTGCAGAGTACCGAGCA-1,1,38,46,7087,5955\nGAAGTGATTTATCGTG-1,1,39,47,7206,6024\nCGCTACGGGACATTTA-1,1,38,48,7086,6093\nCCACACTGAGATATTA-1,1,39,49,7206,6162\nCGATCCGACCCAGTGC-1,1,38,50,7086,6230\nCTGTACTTCTTAGCAT-1,1,39,51,7205,6300\nACTTATTAGGATCGGT-1,1,38,52,7086,6368\nTAGTCCGCAGAGAATG-1,1,39,53,7205,6437\nTTCACGAAAGGATCAC-1,1,38,54,7085,6506\nTACATTTCTAACGTGC-1,1,39,55,7205,6575\nACCATATCCGCAATAA-1,1,38,56,7085,6643\nCACTCAAGAGCTATGG-1,1,39,57,7204,6712\nTGTACGAACAAATCCG-1,1,38,58,7084,6781\nATCATCCAATATTTGT-1,1,39,59,7204,6850\nCGCTATTCTTAGGCTC-1,1,38,60,7084,6918\nTGGCAGCAGTAATAGT-1,1,39,61,7204,6987\nTCACGTGCCCGATTCA-1,1,38,62,7084,7056\nCATACGGCGTCTGGGC-1,1,39,63,7203,7125\nCACATGATTCAGCAAC-1,1,38,64,7083,7194\nGCTAGTAGAGCTTGTA-1,1,39,65,7203,7263\nTGCTGTTGAAGAACTC-1,1,38,66,7083,7331\nCGGAGCATGGCGATCC-1,1,39,67,7203,7400\nTAGCGTTGGGTCTTAC-1,1,38,68,7083,7469\nGTAGCGGCTATACACT-1,1,39,69,7202,7538\nTAACATACAATGTGGG-1,1,38,70,7082,7606\nTCTTCGAATAGACGTT-1,1,39,71,7202,7675\nGATCGTGACTGATATC-1,1,38,72,7082,7744\nGATCCGGGAATTAACA-1,1,39,73,7202,7813\nTTATATACGCTGTCAC-1,1,38,74,7082,7881\nGTCGCGTAACCCGTTG-1,1,39,75,7201,7951\nAGCTCTAGACGTTCCA-1,1,38,76,7081,8019\nGTCAAGCGGACTCGGG-1,1,39,77,7201,8088\nCGAGGGACTGCGGTCG-1,1,38,78,7081,8157\nAATCGCCTCAGCGCCA-1,1,39,79,7200,8226\nCTTGTTGCTGAGTCAA-1,1,38,80,7081,8294\nGATATGAGACACTAAC-1,1,39,81,7200,8363\nTTATGATCTTAACGAA-1,1,38,82,7080,8432\nCGCCGCCCATGCCTGT-1,1,39,83,7200,8501\nCTGGGATAAATAATGG-1,1,38,84,7080,8569\nGTGCCCGTTCGGATTC-1,1,39,85,7199,8638\nTTCAATACTCTGAATC-1,1,38,86,7080,8707\nCGCACATGTCCACTAC-1,1,39,87,7199,8776\nAGAAGAGCGCCGTTCC-1,1,38,88,7079,8844\nGATAACTCGCACTGTG-1,1,39,89,7199,8914\nAGTCGACGGTCTCAAG-1,1,38,90,7079,8982\nGTGACCGCACACTACG-1,1,39,91,7198,9051\nGTATGTGGGTCTAGTT-1,1,38,92,7079,9120\nCTTGAGTTAGGGTAAT-1,1,39,93,7198,9189\nTTAGCTGATTTGCCGT-1,1,38,94,7078,9257\nGCTGTTGCTACCGAAC-1,1,39,95,7198,9326\nTATTACCATCCTGCTT-1,1,38,96,7078,9395\nTTGAATTCACGTGAGG-1,1,39,97,7197,9464\nCCATCTCACCAGTGAA-1,1,38,98,7077,9532\nCGCACGTGCGCTATCA-1,1,39,99,7197,9601\nACCCGGATGACGCATC-1,1,38,100,7077,9670\nCGCTAGAGACCGCTGC-1,1,39,101,7197,9739\nATAGTTCCACCCACTC-1,1,38,102,7077,9808\nGCAGACCCAGCACGTA-1,1,39,103,7196,9877\nTAGACTACCTAGCGTT-1,1,38,104,7076,9945\nGGTTCTACTCGTCTGA-1,1,39,105,7196,10014\nGACTCACCCACGTGAG-1,1,38,106,7076,10083\nAGCTCTTCGTAACCTT-1,1,39,107,7196,10152\nACTATCCAGGGCATGG-1,1,38,108,7076,10220\nAAGGATCGATCGCTTG-1,1,39,109,7195,10289\nATATCGGTAGGGAGAT-1,1,38,110,7075,10358\nTTCCAGACGAGATTTA-1,1,39,111,7195,10427\nGACCGACGTGAAAGCA-1,1,38,112,7075,10495\nCCTGGAAACGTTCTGC-1,1,39,113,7195,10565\nCCGGTAATGGCTAGTC-1,1,38,114,7075,10633\nGCCGTGGAAGAAATGT-1,0,39,115,7194,10702\nGTCTTGAGGAGCAGTG-1,1,38,116,7074,10771\nTCCCAAAGCCCTAAAT-1,1,39,117,7194,10840\nTTGAGCGCCACGTGAT-1,1,38,118,7074,10908\nTTGAGTCCCGCTGCTG-1,1,39,119,7193,10977\nATGGAACCTTTGCACA-1,1,38,120,7074,11046\nGCTAGCACCTGGGCCA-1,1,39,121,7193,11115\nCGCCGTCTACCCATCG-1,1,38,122,7073,11183\nGATAGGTGTCCCGGGC-1,1,39,123,7193,11252\nAGGTATGCGGACATTA-1,0,38,124,7073,11321\nTTGGTTCGCTCAAAGG-1,0,39,125,7192,11390\nTCTGGAGCGTAAGAGT-1,0,38,126,7073,11459\nTGCCTAAATTTAATAG-1,0,39,127,7192,11528\nTAATTTCCGTCCAGTA-1,1,40,0,7334,2792\nTCCTTCAATCCCTACG-1,1,41,1,7454,2861\nTACCTATCCCTAGAGG-1,1,40,2,7334,2929\nGCATGGGTACTGACGC-1,1,41,3,7453,2998\nGTCGGGAACATGGTAG-1,1,40,4,7333,3067\nGCAAATGAGGACACTT-1,1,41,5,7453,3136\nGAATGGGCTTATCGAC-1,1,40,6,7333,3204\nTGGTCGTGCAAGGCAA-1,1,41,7,7452,3273\nCACCACGCCACACAGA-1,1,40,8,7333,3342\nGAACCTCGACCTACAC-1,1,41,9,7452,3411\nCAATACGCTCTGAGGC-1,1,40,10,7332,3479\nTGGTAAGCAGGATTGA-1,1,41,11,7452,3549\nAGTGGCTCCGTCGGCC-1,1,40,12,7332,3617\nGATCGGATAGAACCAT-1,1,41,13,7451,3686\nGCTACAGTACGGACCG-1,1,40,14,7332,3755\nTCTATTACTAGAGGAT-1,1,41,15,7451,3824\nTTCAGGCGTCAAAGCC-1,1,40,16,7331,3892\nAGACCGGGAAACCCTG-1,1,41,17,7451,3961\nAGAGATCTCTAAAGCG-1,1,40,18,7331,4030\nCCCTGCCCAATCCGCT-1,1,41,19,7450,4099\nGTGGCGGTCCCAGCGT-1,1,40,20,7330,4167\nGCATTGTAATTCATAT-1,1,41,21,7450,4236\nCCGTTCCGAATCTCGG-1,1,40,22,7330,4305\nAGCTTGATCTTAACTT-1,1,41,23,7450,4374\nCCTGTACTCACGCCCA-1,1,40,24,7330,4443\nAAGTGACGACCGAATT-1,1,41,25,7449,4512\nCTCACTTGGCTGGTAA-1,1,40,26,7329,4580\nCGCCTGGCCTACGTAA-1,1,41,27,7449,4649\nCCCGTAAGTCTAGGCC-1,1,40,28,7329,4718\nTTGGACATGTGGCTTA-1,1,41,29,7449,4787\nATTACGCGCTGGCAGG-1,1,40,30,7329,4855\nACGCGCTACACAGGGT-1,1,41,31,7448,4924\nTACGTTTACCGGCAAT-1,1,40,32,7328,4993\nCGAAACGCAATTCATG-1,1,41,33,7448,5062\nTAGTCTAACAACGAGA-1,1,40,34,7328,5130\nTTGCATGCTGATCACG-1,1,41,35,7448,5200\nTCTGGGTAGCGCTCAT-1,1,40,36,7328,5268\nACATCGGTCAGCCGCG-1,1,41,37,7447,5337\nAGATACCGGTGTTCAC-1,1,40,38,7327,5406\nGATTACTGAATTTGGG-1,1,41,39,7447,5475\nTCCAACTTTAAATTCT-1,1,40,40,7327,5543\nTCCTAGCAAAGAAGCT-1,1,41,41,7447,5612\nGTCTATCTGAGTTTCT-1,1,40,42,7327,5681\nGATGTTCAATCCACGA-1,1,41,43,7446,5750\nAGTTAAACACTTGCGA-1,1,40,44,7326,5818\nAGCTCTTTACTCAGTT-1,1,41,45,7446,5887\nATCCAGGATTCGTGAA-1,1,40,46,7326,5956\nAGTCAACACCACCATC-1,1,41,47,7445,6025\nCGATACCTCGCGGACA-1,1,40,48,7326,6093\nTACAACGCACAACTCA-1,1,41,49,7445,6163\nAATTAAAGGTCGGCGT-1,1,40,50,7325,6231\nTACGCAGTTCTTTCCT-1,1,41,51,7445,6300\nGACCGTGCTGACGGTG-1,1,40,52,7325,6369\nGGCAAATTACTTTACT-1,1,41,53,7444,6438\nGGTACAAACATGCTAT-1,1,40,54,7325,6506\nCGGGCCTTCTTTGTAA-1,1,41,55,7444,6575\nCGTGAAGTTAATTCAC-1,1,40,56,7324,6644\nATAGTGAAGCGTTCTC-1,1,41,57,7444,6713\nTACGCCATATTCTAAT-1,1,40,58,7324,6781\nGCCGGGTTAGGGTCGC-1,1,41,59,7443,6850\nTACATAGGCATACACC-1,1,40,60,7323,6919\nGCCGATTGGCCAAGCT-1,1,41,61,7443,6988\nCTGCCATGCATCACAT-1,1,40,62,7323,7057\nTTATGAATGAAAGGGA-1,1,41,63,7443,7126\nGCTGAGGCGTGAGTAT-1,1,40,64,7323,7194\nGCGCCGTTCCACGATA-1,1,41,65,7442,7263\nCGCATGGTGCGATGCT-1,1,40,66,7322,7332\nAGGTTTCACACACCTT-1,1,41,67,7442,7401\nCAAGGATCGCATGTTC-1,1,40,68,7322,7469\nACGTTAATGTCGAAGA-1,1,41,69,7442,7538\nTCCAGAGCACCGGTTC-1,1,40,70,7322,7607\nGATTCGACGGTTCACG-1,1,41,71,7441,7676\nGTTTCTGCAGTCTCCC-1,1,40,72,7321,7744\nGCTGCACGGTTTCTTA-1,1,41,73,7441,7814\nCGTGCAGACTGGGACA-1,1,40,74,7321,7882\nGTGTTACTATGCGTCC-1,1,41,75,7441,7951\nTCCTCGGGCTGGGCTT-1,1,40,76,7321,8020\nGTGAGGACACTTAAGG-1,1,41,77,7440,8089\nATACGCCGGCGAAACC-1,1,40,78,7320,8157\nTCTGCCAGAAACTGCA-1,1,41,79,7440,8226\nTTCTGCGGGTTAGCGG-1,1,40,80,7320,8295\nCTCGGTACCACTGCTC-1,1,41,81,7440,8364\nGTAAGTAACAGTCTGG-1,1,40,82,7320,8432\nGTGCGTGTATATGAGC-1,1,41,83,7439,8501\nATTTGTCTTGGGAGCT-1,1,40,84,7319,8570\nCCTCGGACCGGGATAG-1,1,41,85,7439,8639\nTAGGTGCTCGCCTAGC-1,1,40,86,7319,8708\nCTTTAGGAACACTGTT-1,1,41,87,7438,8777\nTCGGGCCGTCGTGGTA-1,1,40,88,7319,8845\nAGTGCTTGCACGAATA-1,1,41,89,7438,8914\nTGCAGTTTCCTCCCAT-1,1,40,90,7318,8983\nTGAGAGATTTACCACG-1,1,41,91,7438,9052\nGAAACAGATGACCACC-1,1,40,92,7318,9120\nAGCAACATATCTTATT-1,1,41,93,7437,9189\nCAAGTGTGGTTGCAAA-1,1,40,94,7318,9258\nGCCTCATCTGGAAATA-1,1,41,95,7437,9327\nAACCCTACTGTCAATA-1,1,40,96,7317,9395\nACGTATTACTCCGATC-1,1,41,97,7437,9465\nTCTGGGAACCTTTGAA-1,1,40,98,7317,9533\nGCTCGCTCATGTCCAA-1,1,41,99,7436,9602\nGCGCAAGAGCGCGCTG-1,1,40,100,7316,9671\nTTGACGCTCCATGAGC-1,1,41,101,7436,9740\nTATAGATGGTCGCAGT-1,1,40,102,7316,9808\nTTACATGCCACAACTA-1,1,41,103,7436,9877\nACATGGCGCCAAAGTA-1,1,40,104,7316,9946\nTATGGTTAGTGGGAGA-1,1,41,105,7435,10015\nCATGACTTCGCTGAAT-1,1,40,106,7315,10083\nACCACCAATGTAACAA-1,1,41,107,7435,10152\nTCTTAGAGCTCCAATT-1,1,40,108,7315,10221\nCCACGAATTTAACCTC-1,1,41,109,7435,10290\nTTCTTGCTAGCATCTC-1,1,40,110,7315,10358\nACACCTTACTACTTGC-1,1,41,111,7434,10428\nAGTCGGTTGCGTGAGA-1,1,40,112,7314,10496\nACCTACAGTATGTGGT-1,1,41,113,7434,10565\nGAGGATAAACAGTGCT-1,0,40,114,7314,10634\nTTCCGGTATCTGTGTC-1,0,41,115,7434,10703\nGGAGGCCGAAGTCGTC-1,0,40,116,7314,10771\nTTCGCTAGGAAGTTGT-1,1,41,117,7433,10840\nTAAAGACAACCCTTTA-1,1,40,118,7313,10909\nGTTGCGCTAACATTAC-1,1,41,119,7433,10978\nGCTGAACTCTCCAGGG-1,1,40,120,7313,11046\nAGCCCTGTCGCACCGT-1,1,41,121,7433,11115\nAGCGCTAGAGCGATGT-1,1,40,122,7313,11184\nTGCGCCGTTAATAACG-1,1,41,123,7432,11253\nTGACAACGCATGTCGC-1,1,40,124,7312,11322\nCGTCTTGAGTGTGACG-1,0,41,125,7432,11391\nGATAGGATTAATTACA-1,0,40,126,7312,11459\nGATACCGTGTCGGAGT-1,0,41,127,7431,11528\nTGTGTCGAAGTCGAGG-1,1,42,0,7573,2792\nGGATCAGAGCCATCAG-1,1,43,1,7693,2861\nGTCCCAATCATCCCGC-1,1,42,2,7573,2930\nTGCCACACTAGAGGAA-1,1,43,3,7693,2999\nTGAATTTCACTTGCCT-1,1,42,4,7573,3067\nCTCAGATTGTGATAAG-1,1,43,5,7692,3136\nCGGACGTTACTTGAAG-1,1,42,6,7572,3205\nCCGCCTTGCGATGTCG-1,1,43,7,7692,3274\nCCAGTCAAATCTCTTA-1,1,42,8,7572,3342\nAAACAGCTTTCAGAAG-1,1,43,9,7691,3412\nTAGCTGATGTGAAGCG-1,1,42,10,7572,3480\nCTTATGCGCTCAGGGC-1,1,43,11,7691,3549\nTGGCTCTTGTCGCGTA-1,1,42,12,7571,3618\nTTGTGAGGCATGACGC-1,1,43,13,7691,3687\nCATAAGCTCTCCGTCT-1,1,42,14,7571,3755\nCCTTCTCAGCGTTCCT-1,1,43,15,7690,3824\nTGCAGCTACGTACTTC-1,1,42,16,7571,3893\nCTCTACACTGGCGATT-1,1,43,17,7690,3962\nTGCAGATCGTCCTAGG-1,1,42,18,7570,4030\nTGCCAAAGTCAGACTT-1,1,43,19,7690,4099\nACAATAGTCGTACGTT-1,1,42,20,7570,4168\nCATGCCAACTCGCAAA-1,1,43,21,7689,4237\nTGCAGAACTATATCGT-1,1,42,22,7570,4306\nTCGTGTCACGCTGACA-1,1,43,23,7689,4375\nTGATCAGGGAACTGCT-1,1,42,24,7569,4443\nCTTGCAACCGCCTCCT-1,1,43,25,7689,4512\nAGCCCATACATGTAAG-1,1,42,26,7569,4581\nATCCAATGGAGGGTCC-1,1,43,27,7688,4650\nAAACCGGGTAGGTACC-1,1,42,28,7568,4718\nGCAACCACGGCCGCGT-1,1,43,29,7688,4787\nCGCTTATTCCCGGTCG-1,1,42,30,7568,4856\nTTACTCTGGTACGTAC-1,1,43,31,7688,4925\nGTGGTTTCCGCCTTTC-1,1,42,32,7568,4993\nATAGGCGGCTATAGAA-1,1,43,33,7687,5063\nGTGCCATCACACGGTG-1,1,42,34,7567,5131\nCCCAACATACGTCGCG-1,1,43,35,7687,5200\nCGGTGCAGATAGAACG-1,1,42,36,7567,5269\nGGGCGGGTTCCCTACG-1,1,43,37,7687,5338\nTGAGCCATACAGTCTC-1,1,42,38,7567,5406\nCTCCGCCCACATGAGG-1,1,43,39,7686,5475\nGTTGAACCGGTTCCAT-1,1,42,40,7566,5544\nTTGACTACCATATGGT-1,1,43,41,7686,5613\nACCATCGTATATGGTA-1,1,42,42,7566,5681\nTGCGTAAGAACCTGAT-1,1,43,43,7686,5750\nAGAAGGTTGCCGAATT-1,1,42,44,7566,5819\nAGGACGACCCATTAGA-1,1,43,45,7685,5888\nGGTGCTGGTACACATT-1,1,42,46,7565,5957\nGCGCTATGCCGAGGCA-1,1,43,47,7685,6026\nACCCGGTTACACTTCC-1,1,42,48,7565,6094\nTAACTCATCCGCGCGG-1,1,43,49,7685,6163\nCACAATGAGCTGCTAT-1,1,42,50,7565,6232\nGTTACTTTGGGCCTAG-1,1,43,51,7684,6301\nGGGCCCGTCTTAAACA-1,1,42,52,7564,6369\nGAAATTGTCTCTATAA-1,1,43,53,7684,6438\nGGCGCATGAATTGATG-1,1,42,54,7564,6507\nCATATAGGTACAGTCA-1,1,43,55,7683,6576\nTCAACGAGGAGACAAA-1,1,42,56,7564,6644\nTTGCACAATTCAGAAA-1,1,43,57,7683,6714\nCATCGGACGGGTTAAT-1,1,42,58,7563,6782\nATTAAACATGCGGACC-1,1,43,59,7683,6851\nTATCTACCACAGCGGG-1,1,42,60,7563,6920\nCGAGACCCTAGAGTGT-1,1,43,61,7682,6989\nACATCGATCGTTTACC-1,1,42,62,7563,7057\nATCGACCCAATACAGA-1,1,43,63,7682,7126\nGAATCTGAACATTCTC-1,1,42,64,7562,7195\nAGTTCCTATTTATGTT-1,1,43,65,7682,7264\nCAGTCTGTATACTGGG-1,1,42,66,7562,7332\nGGAGACGACACCTTTG-1,1,43,67,7681,7401\nCCTAAATTAACGGTTC-1,1,42,68,7561,7470\nGCTACGACTTATTGGG-1,1,43,69,7681,7539\nCTGTGCAGGGTAGGTC-1,1,42,70,7561,7607\nACGCCGCTAGACGACC-1,1,43,71,7681,7677\nACTTGACTCCCTCTTT-1,1,42,72,7561,7745\nCGCCTCCCTCCTCTAT-1,1,43,73,7680,7814\nCTAGATTTACGACGGC-1,1,42,74,7560,7883\nGATGCTGTATTTCATC-1,1,43,75,7680,7952\nTGCGTTTGTTGACACT-1,1,42,76,7560,8020\nCGTGCCCTCCCGAAGA-1,1,43,77,7680,8089\nACTCTCTGACTTAGGT-1,1,42,78,7560,8158\nCTGGGTAGGCAGTTAA-1,1,43,79,7679,8227\nGTTTGGCCCAAGTTAT-1,1,42,80,7559,8295\nGAATGTGGTCCGGATT-1,1,43,81,7679,8364\nCCCAGTTAAGGCGCCG-1,1,42,82,7559,8433\nCGGTACTAGAATCAAA-1,1,43,83,7679,8502\nGCTTAATGTAACTAAC-1,1,42,84,7559,8571\nAGGACAGTCGAATCCC-1,1,43,85,7678,8640\nGACAGCCAGACCTGAC-1,1,42,86,7558,8708\nGGCGAAATCTAACTTG-1,1,43,87,7678,8777\nCTGGTAACACATAGAA-1,1,42,88,7558,8846\nTAGCCATTTCAAAGTC-1,1,43,89,7678,8915\nGGGATTTACCGCACCT-1,1,42,90,7558,8983\nATGCCATTTGCGACCA-1,1,43,91,7677,9052\nGAATAGACGCGACCCA-1,1,42,92,7557,9121\nTGTATGGCGCAGACAG-1,1,43,93,7677,9190\nGGATGAAGATCGCTGA-1,1,42,94,7557,9258\nCGACAATTTGATCTAA-1,1,43,95,7676,9328\nAAAGTTGACTCCCGTA-1,1,42,96,7557,9396\nCGCGGCTCAACTTGAA-1,1,43,97,7676,9465\nCACGCGGAACTGTTGC-1,1,42,98,7556,9534\nTCTTAGAGTGAACTCT-1,1,43,99,7676,9603\nAGTCCATTGGCTGATG-1,1,42,100,7556,9671\nTTAAGATAGGATTGAC-1,1,43,101,7675,9740\nACATCCTGGTAACTGT-1,1,42,102,7556,9809\nCACCTTGGCGCCTTTG-1,1,43,103,7675,9878\nGCTAGACCGTCTACTG-1,1,42,104,7555,9946\nCGGCCCAGGTATATCC-1,1,43,105,7675,10015\nGTCCATTACTGCTACG-1,1,42,106,7555,10084\nGGGTTTAGGATAGGAT-1,1,43,107,7674,10153\nTAATTAGGACATCCGT-1,1,42,108,7554,10221\nGCTCCCAGTCGGTCCA-1,1,43,109,7674,10291\nATTCGTGCTATCTCTT-1,1,42,110,7554,10359\nGTTAACTATGTTGTCA-1,1,43,111,7674,10428\nGCTCCGCTCGCTTCAG-1,1,42,112,7554,10497\nGCAACGGCTAGTTATG-1,1,43,113,7673,10566\nAATCGCGCAGAGGACT-1,0,42,114,7553,10634\nAGGTTACACCATGCCG-1,1,43,115,7673,10703\nCGAGATTTCGCTCGGG-1,1,42,116,7553,10772\nCGATAATACTCAGGTT-1,1,43,117,7673,10841\nAAGGCAGGCTGTCTCC-1,1,42,118,7553,10909\nGTAAGTCCACACTCTA-1,1,43,119,7672,10978\nATGAGGGCAGCGGCTA-1,1,42,120,7552,11047\nGCCGCACTCCGTTTCA-1,1,43,121,7672,11116\nGAGCACGGCGCCTCTT-1,1,42,122,7552,11185\nACAATTTGGCCATATT-1,1,43,123,7672,11254\nCTGGTTCGCGAGCTAC-1,1,42,124,7552,11322\nGACGGTCCTAGGGTGT-1,0,43,125,7671,11391\nACTCGCGATCTGACGC-1,0,42,126,7551,11460\nGTGTACGAACCGTTCC-1,0,43,127,7671,11529\nCTTTGACGTCGCTTCT-1,1,44,0,7813,2793\nCGTTATCATACTTCCA-1,1,45,1,7932,2862\nGCTATGCCAGCTTATG-1,1,44,2,7812,2930\nCAGTCGGCCTAGATAT-1,1,45,3,7932,2999\nCCCGTGAGGGCGGTGA-1,1,44,4,7812,3068\nTCTCGTGTTACGAGGA-1,1,45,5,7932,3137\nACGTCTCGTTCCGGGA-1,1,44,6,7812,3206\nCGAGAGCGCGTAGATA-1,1,45,7,7931,3275\nGACAGATTTCTGGCTC-1,1,44,8,7811,3343\nGGGCCTAAATGGGCTA-1,1,45,9,7931,3412\nACTTGTAGTCCCTTCA-1,1,44,10,7811,3481\nCCCGAAGTTTCGCGAA-1,1,45,11,7931,3550\nACCATCCGCCAACTAG-1,1,44,12,7811,3618\nTGCGAATATGGGATTT-1,1,45,13,7930,3687\nTACATCCCTATCCCTG-1,1,44,14,7810,3756\nGTGGGAAGACTGAATC-1,1,45,15,7930,3825\nTCAACATCGACCGAGA-1,1,44,16,7810,3893\nCTATGTGAGTCACGGC-1,1,45,17,7929,3963\nCCGAACACTGGGCCTC-1,1,44,18,7810,4031\nAAACTTGCAAACGTAT-1,1,45,19,7929,4100\nAGGGCGAGCAGCTGAT-1,1,44,20,7809,4169\nAACACGAGACGCGGCC-1,1,45,21,7929,4238\nTGACGAATATTTCCCT-1,1,44,22,7809,4306\nTCGGAGAGTATCGGGA-1,1,45,23,7928,4375\nCAAATCTCTCACAAGG-1,1,44,24,7809,4444\nAGGCCCTAGAACGCCA-1,1,45,25,7928,4513\nTAGAGATCATGCAACT-1,1,44,26,7808,4581\nTTGTTTCCATACAACT-1,1,45,27,7928,4650\nGAGAGGTGCATTCTGG-1,1,44,28,7808,4719\nGTGGACCAACCCGATT-1,1,45,29,7927,4788\nCTGGGCCTGCTATATC-1,1,44,30,7808,4856\nCATAGTCCACAAGAAC-1,1,45,31,7927,4926\nTTGACATGAACGTGGA-1,1,44,32,7807,4994\nGGTTACCACCCTCGGG-1,1,45,33,7927,5063\nTACCGGTCGTTTCCAT-1,1,44,34,7807,5132\nCGAGTACTAAAGAGGA-1,1,45,35,7926,5201\nGCAAGAATTCCTTGGC-1,1,44,36,7806,5269\nTCGCCGAAGTTGCGTC-1,1,45,37,7926,5338\nTTGAGAGTACTGCTAA-1,1,44,38,7806,5407\nGCCACAATTTAAGGAC-1,1,45,39,7926,5476\nATATTCAGTTAAACCT-1,1,44,40,7806,5544\nTGAGTGCCTCTTAAAT-1,1,45,41,7925,5613\nATCAGACGGCACGCCG-1,1,44,42,7805,5682\nGTGCGAAATCGAACAC-1,1,45,43,7925,5751\nGTGCCGCTTCAAAGGT-1,1,44,44,7805,5820\nGATACGATGGGAGTCA-1,1,45,45,7925,5889\nGACACTGAGTTCAGTG-1,1,44,46,7805,5957\nATCCTGCGTGGAATGG-1,1,45,47,7924,6026\nATCCTACCTAAGCTCT-1,1,44,48,7804,6095\nAGTGATATGAGTAGTT-1,1,45,49,7924,6164\nATGATGCAATGGTACA-1,1,44,50,7804,6232\nGAAACCGAATTACCTT-1,1,45,51,7924,6301\nAGTGACCTACTTTACG-1,1,44,52,7804,6370\nCAAATGTCCTTCCGTG-1,1,45,53,7923,6439\nTTACTGGGATATTTCA-1,1,44,54,7803,6507\nCTTGCCCAGGCTCTAC-1,1,45,55,7923,6577\nAAATCGTGTACCACAA-1,1,44,56,7803,6645\nGTGATCATAGATCTGC-1,1,45,57,7922,6714\nTGGCAGATTACGATCA-1,1,44,58,7803,6783\nTCACCCTCTTAAGATT-1,1,45,59,7922,6852\nCAGGATATATCGTTGT-1,1,44,60,7802,6920\nCCTGACCACCGATGGT-1,1,45,61,7922,6989\nCTAAAGGGAAATAGGA-1,1,44,62,7802,7058\nCCGCTATCAGCACCAG-1,1,45,63,7921,7127\nCTTTAGTGCTATTATT-1,1,44,64,7802,7195\nCGGGAATTTATGTAAA-1,1,45,65,7921,7264\nTACGACTGCCTCTTAG-1,1,44,66,7801,7333\nAAACTGCTGGCTCCAA-1,1,45,67,7921,7402\nGTACGTTTGCCCGTCA-1,1,44,68,7801,7471\nGGCAAGGCGAAATAGC-1,1,45,69,7920,7540\nGATCTTGGAGGGCATA-1,1,44,70,7801,7608\nAGCGTGGTATTCTACT-1,1,45,71,7920,7677\nCTAAGGGAATGATTGG-1,1,44,72,7800,7746\nCATGGTAAGTAGCGTT-1,1,45,73,7920,7815\nCGTTGAGCGACCGTCG-1,1,44,74,7800,7883\nTGCCCGTACCGTTAAA-1,1,45,75,7919,7952\nACAAGGGCAGGCTCTG-1,1,44,76,7799,8021\nGAGATCTTCCATGACA-1,1,45,77,7919,8090\nAATGACGTAGGATGTC-1,1,44,78,7799,8158\nGTGGTGGCCAAGTGAA-1,1,45,79,7919,8227\nTCCCGTGTGCAATTTG-1,1,44,80,7799,8296\nACATCGTATGCAATGG-1,1,45,81,7918,8365\nGCGAAACTTAACTGGA-1,1,44,82,7798,8434\nAATTGAACGCTCTGGT-1,1,45,83,7918,8503\nACAAATGGTAGTGTTT-1,1,44,84,7798,8571\nATGGTCGCGTGGTTTC-1,1,45,85,7918,8640\nTGTTATTGTATGTGGC-1,1,44,86,7798,8709\nTTCCGGTTACCCACTT-1,1,45,87,7917,8778\nGAGTGTGCGGTACCCA-1,1,44,88,7797,8846\nCAAGATATTATAACGT-1,1,45,89,7917,8915\nACACACCAGGACCAGT-1,1,44,90,7797,8984\nATGGGCCTCGGCCTCT-1,1,45,91,7917,9053\nAAGGTGATAAACCAGC-1,1,44,92,7797,9121\nTCTTACTTATGCCTCT-1,1,45,93,7916,9191\nAAAGTGTGATTTATCT-1,1,44,94,7796,9259\nTGCTCCACAGTTCTTA-1,1,45,95,7916,9328\nCTGGCTGATTCATCCT-1,1,44,96,7796,9397\nTAAGGCTGAATCCCTC-1,1,45,97,7915,9466\nTCTAGTGATATCGTGG-1,1,44,98,7796,9534\nTCGAAGAACCGAGCAC-1,1,45,99,7915,9603\nGACAAACATATGCAGG-1,1,44,100,7795,9672\nAAGTCAATTGTCGTCA-1,1,45,101,7915,9741\nAGTGAACAAACTTCTC-1,1,44,102,7795,9809\nCATGATGGAAGTTAGC-1,1,45,103,7914,9878\nAAGTGCCTTGACTGTA-1,1,44,104,7795,9947\nATCGCCAGTCAACATT-1,1,45,105,7914,10016\nACCGCGGTGGAAGTCG-1,1,44,106,7794,10085\nTCTTCTATAACCCGCC-1,1,45,107,7914,10154\nCACATTTCTTGTCAGA-1,1,44,108,7794,10222\nTAGCGTCCCTCGATTG-1,1,45,109,7913,10291\nGTTCGGATCGGGAACA-1,1,44,110,7794,10360\nCAAACTCGCGACGCCG-1,1,45,111,7913,10429\nGTCTTGTAGCTATTCA-1,1,44,112,7793,10497\nTCTCGACGTATCGCCG-1,1,45,113,7913,10566\nTTGCCAAGCAGAACCC-1,1,44,114,7793,10635\nAAACCCGAACGAAATC-1,1,45,115,7912,10704\nTTGAGCAGCCCACGGT-1,1,44,116,7792,10772\nCGCCTTTAGCATGCTC-1,1,45,117,7912,10842\nTGTGGCTCCCACCAAC-1,1,44,118,7792,10910\nCCGCCGTTGAGGATAA-1,1,45,119,7912,10979\nCAATACGAGAGTCTGA-1,1,44,120,7792,11048\nCATCTAGTGAAGGGAA-1,1,45,121,7911,11117\nGGTGGAGGTTGATACG-1,1,44,122,7791,11185\nCCGCACACGAACGTGT-1,1,45,123,7911,11254\nAGAACCCAGCGTGACA-1,1,44,124,7791,11323\nGCGCTCGATCACCTGT-1,0,45,125,7911,11392\nATCATGGACTACCGAC-1,0,44,126,7791,11460\nTACGCCGCCTCAGAAG-1,0,45,127,7910,11529\nCGACCTACTAGACAAT-1,1,46,0,8052,2793\nGAGTCTTGTAAAGGAC-1,1,47,1,8172,2862\nAATATCCTAGCAAACT-1,1,46,2,8052,2931\nCCCTAGGCAACAAGAG-1,1,47,3,8171,3000\nACAAAGAAGGTAGGCC-1,1,46,4,8051,3069\nCCCTGGCTGTTCCTTC-1,1,47,5,8171,3138\nTCGCCGCACCGCGTGA-1,1,46,6,8051,3206\nTATAGCGCACGTTATC-1,1,47,7,8171,3275\nTTATCTGACATTAGGA-1,1,46,8,8051,3344\nAGTGGTGTTACCCGTG-1,1,47,9,8170,3413\nGCCAAGAATACTTCTG-1,1,46,10,8050,3481\nCCGGCGTGAGACTCTG-1,1,47,11,8170,3550\nTTCCCGGCGCCAATAG-1,1,46,12,8050,3619\nAAACAGGGTCTATATT-1,1,47,13,8170,3688\nACAGTAATACAACTTG-1,1,46,14,8050,3756\nCGAACGGCCGGACAAC-1,1,47,15,8169,3826\nGCAACACACTAGAACT-1,1,46,16,8049,3894\nACTCCCATTCCTAAAG-1,1,47,17,8169,3963\nACCTGCGTGTCATGTT-1,1,46,18,8049,4032\nTACTTTCCGCACGCCA-1,1,47,19,8169,4101\nAGGTCAGGTGAGAGTG-1,1,46,20,8049,4169\nTCCTCCTAAGACATTC-1,1,47,21,8168,4238\nATGTGAAAGCCTAATG-1,1,46,22,8048,4307\nAGTCGGCTCAACTTTA-1,1,47,23,8168,4376\nCGATCTGTTGGAGGAC-1,1,46,24,8048,4444\nACGGGAGTGTCGGCCC-1,1,47,25,8167,4513\nTTAACTTCAGGTAGGA-1,1,46,26,8048,4582\nCCACGGAGCCATAAGA-1,1,47,27,8167,4651\nCTTCTATGTTGAAGTA-1,1,46,28,8047,4720\nCACCGTTGCGCGATAT-1,1,47,29,8167,4789\nTCTAGCAATCTCCGCC-1,1,46,30,8047,4857\nAGTTTGGCCAGACCTA-1,1,47,31,8166,4926\nTTGTAAGGACCTAAGT-1,1,46,32,8047,4995\nAAATTTGCGGGTGTGG-1,1,47,33,8166,5064\nAAGTTCGGCCAACAGG-1,1,46,34,8046,5132\nCCGCTTACCTCACTCT-1,1,47,35,8166,5201\nATCACGTGCTAATTAA-1,1,46,36,8046,5270\nGGTGAAGTACAGGGAT-1,1,47,37,8165,5339\nGCTGTATTACTGGCCC-1,1,46,38,8046,5407\nAACGGCCATCTCCGGT-1,1,47,39,8165,5476\nTAAGTAACATCTTGAC-1,1,46,40,8045,5545\nTTCTTGAGCCGCGCTA-1,1,47,41,8165,5614\nAGTGCGTAGCTCGTAA-1,1,46,42,8045,5683\nGGGATGGTCGTAACCG-1,1,47,43,8164,5752\nGTCTGGGCGGTCGAGA-1,1,46,44,8044,5820\nCGGAACGTAAACATAG-1,1,47,45,8164,5889\nTGCGACACCCTAGTGC-1,1,46,46,8044,5958\nCAAACGAGTATCGCAG-1,1,47,47,8164,6027\nTCAGTAGGGACTATAA-1,1,46,48,8044,6095\nGCGGTTCCCTATCATG-1,1,47,49,8163,6164\nGTGACTTCAGTAGTGC-1,1,46,50,8043,6233\nCGTCACGTCCATTGGT-1,1,47,51,8163,6302\nATAAGTTACCGCGACG-1,1,46,52,8043,6370\nCTGAATCCGAGACCTC-1,1,47,53,8163,6440\nTACGACGCTTGCTGCG-1,1,46,54,8043,6508\nGAATTCACCCGGGTGT-1,1,47,55,8162,6577\nGGGTGACACCTTAACT-1,1,46,56,8042,6646\nTGAGTAAATTAGCGTA-1,1,47,57,8162,6715\nGAGTCCGCTTACCGGA-1,1,46,58,8042,6783\nGGCGCGTTCGAGTTTA-1,1,47,59,8162,6852\nATTCATATACTGTCCA-1,1,46,60,8042,6921\nATAAACGGACCCGTAA-1,1,47,61,8161,6990\nGTCGTACCATCTCGGG-1,1,46,62,8041,7058\nATAATAGCTGTTGAAT-1,1,47,63,8161,7127\nTTCCGCGTGAGGCGAT-1,1,46,64,8041,7196\nTACCGGCTCACTGCCC-1,1,47,65,8160,7265\nGTATCTTTCATAACCA-1,1,46,66,8041,7334\nGTTACCTACAACTTGC-1,1,47,67,8160,7403\nTGACTATAATCCTTTC-1,1,46,68,8040,7471\nAAATACCTATAAGCAT-1,1,47,69,8160,7540\nTTATTAGAGCGTGTTC-1,1,46,70,8040,7609\nATGTGGACATCTTGAT-1,1,47,71,8159,7678\nGTACGCTTCATTGCAC-1,1,46,72,8040,7746\nCACATATTAGCAGGAT-1,1,47,73,8159,7815\nCAAGAGGGCGGAGTAC-1,1,46,74,8039,7884\nTCGCAAAGATGCATTT-1,1,47,75,8159,7953\nGTCGTTATTCGCTTAT-1,1,46,76,8039,8021\nTCGTGTACTATGGATG-1,1,47,77,8158,8091\nGCGGAGAGGGAGAACG-1,1,46,78,8039,8159\nTCACAGGGAATCGCAA-1,1,47,79,8158,8228\nGACTGCACCAGCCCAG-1,1,46,80,8038,8297\nTCCGCTGTCATCCCGG-1,1,47,81,8158,8366\nAGATACTCAAGATCGA-1,1,46,82,8038,8434\nACTGAATGGCGAAAGT-1,1,47,83,8157,8503\nTCCTACATCCACGGCC-1,1,46,84,8037,8572\nTTCAGCCCTGGTCCAC-1,1,47,85,8157,8641\nTTAGAGTTTAGAAGGA-1,1,46,86,8037,8709\nGGTTAGCTATATGTCT-1,1,47,87,8157,8778\nCTGGTTTCGAGCAAGA-1,1,46,88,8037,8847\nTGGGCCCATACTAATT-1,1,47,89,8156,8916\nACTTATTTATGTGCCA-1,1,46,90,8036,8984\nTTAGTTATTCGTGGCA-1,1,47,91,8156,9054\nAACCGCTAAGGGATGC-1,1,46,92,8036,9122\nATACGTCCACTCCTGT-1,1,47,93,8156,9191\nCGACACGCTCCGACAG-1,1,46,94,8036,9260\nTCTGAACTCGTACCCG-1,1,47,95,8155,9329\nAGGATATAGGGATTTA-1,1,46,96,8035,9397\nGTGATCACTAACGCCT-1,1,47,97,8155,9466\nACCGGGCCTTTGTTGA-1,1,46,98,8035,9535\nTTGACAGGAGCTCCCG-1,1,47,99,8155,9604\nCTCTTCTATTGACTGG-1,1,46,100,8035,9672\nCGTGGCCGAATATCTA-1,1,47,101,8154,9741\nGGTTTAGCCTTTCTTG-1,1,46,102,8034,9810\nTCAAGGTTACTACACC-1,1,47,103,8154,9879\nTTGACTATTGTCCGGC-1,1,46,104,8034,9948\nCGGGAATATAGTATAC-1,1,47,105,8153,10017\nGCAACAGCAGTATGCG-1,1,46,106,8034,10085\nATCAAACACTGTTCCA-1,1,47,107,8153,10154\nTGGGCAATAGTTGGGT-1,1,46,108,8033,10223\nTGAATGTCAGCCGGCC-1,1,47,109,8153,10292\nCGGCTCTAAAGCTGCA-1,1,46,110,8033,10360\nGGACACAAGTTTACAC-1,1,47,111,8152,10429\nCCGAGAAGTCGCATAA-1,1,46,112,8033,10498\nAACTACCCGTTTGTCA-1,1,47,113,8152,10567\nGCACAAACGAGGCGTG-1,1,46,114,8032,10635\nCACACAGCTTGCGCTC-1,1,47,115,8152,10705\nGTGCTTACATCAGCGC-1,1,46,116,8032,10773\nGCAAACGTCGCCAGGT-1,1,47,117,8151,10842\nGAATAGCATTTAGGGT-1,1,46,118,8032,10911\nTACTGTTTCTCTGGTA-1,1,47,119,8151,10980\nTCCTATCATAGGTAAC-1,1,46,120,8031,11048\nTTCGGTGGAGACGCCC-1,1,47,121,8151,11117\nTGTTGCCGTCGTCCCA-1,1,46,122,8031,11186\nCCAACTTGATAGATCC-1,1,47,123,8150,11255\nCAAAGGTCATCTGAAA-1,1,46,124,8030,11323\nCTAGCTTAATGGTCCC-1,1,47,125,8150,11392\nAGGCCACATTGGTTAC-1,0,46,126,8030,11461\nTTAGGATGGGAGGGTA-1,0,47,127,8150,11530\nTATGACCTTGCGCTGG-1,1,48,0,8292,2794\nATATACATGTATGGTA-1,1,49,1,8411,2863\nTGTGTGACCATGAATC-1,1,48,2,8291,2932\nCCCACTCCACGGTATC-1,1,49,3,8411,3001\nGGGCAGACGTCACTGC-1,1,48,4,8291,3069\nTCAAAGAGCTATCTGT-1,1,49,5,8410,3138\nAGATGCATCCTGTGTC-1,1,48,6,8290,3207\nGTGAAGTCACGACTCG-1,1,49,7,8410,3276\nTGATTCAGGTCCCGCG-1,1,48,8,8290,3344\nCGCCACAGGTCGCGAT-1,1,49,9,8410,3413\nTTGAATCGTTGTATAA-1,1,48,10,8290,3482\nTGGCCAATTTGGTACT-1,1,49,11,8409,3551\nCGACCCTTAACGCCGG-1,1,48,12,8289,3619\nTCGGCGTACTGCACAA-1,1,49,13,8409,3689\nGAGTAAGGCCACGGGA-1,1,48,14,8289,3757\nATCAGCTCGTCCACTA-1,1,49,15,8409,3826\nAGGATCACGCGATCTG-1,1,48,16,8289,3895\nAAATGGCCCGTGCCCT-1,1,49,17,8408,3964\nCAGTACCAGTTTACGT-1,1,48,18,8288,4032\nGGTCGTAAGCTCGCAC-1,1,49,19,8408,4101\nCGTTCATGGTGCGCGT-1,1,48,20,8288,4170\nGACTACAAAGCGGTGG-1,1,49,21,8408,4239\nCAGCTGGCGTAACCGT-1,1,48,22,8288,4307\nACTCGTAACCCGTCCT-1,1,49,23,8407,4376\nCATATGTCAGGCTACG-1,1,48,24,8287,4445\nGCCCGACTTCTTCCCG-1,1,49,25,8407,4514\nCTGGCATCCGAATGAG-1,1,48,26,8287,4583\nAAGTGCGTTAGAATCT-1,1,49,27,8407,4652\nAGAAGTGATTCGTGAT-1,1,48,28,8287,4720\nTACCTTAAGATTTCCC-1,1,49,29,8406,4789\nAATGACAGCAATGTCT-1,1,48,30,8286,4858\nTCAACGCAGGAAATAA-1,1,49,31,8406,4927\nTATTCCTCCGCCCACT-1,1,48,32,8286,4995\nATCAAACGAAGGTTTG-1,1,49,33,8405,5064\nTTGATTAGCTGTTTCT-1,1,48,34,8286,5133\nCTGCTGAGGCCACGAA-1,1,49,35,8405,5202\nCCGTGTTAAATTCCAT-1,1,48,36,8285,5270\nCACGTCGGCAACCTCT-1,1,49,37,8405,5340\nGTATTCTGAGAAACGA-1,1,48,38,8285,5408\nCGGTGCGCGTTGGTCC-1,1,49,39,8404,5477\nGTGATTCGCCGCTCAA-1,1,48,40,8285,5546\nGCCCGCGCGTAAACGG-1,1,49,41,8404,5615\nAACGTACTGTGGGTAC-1,1,48,42,8284,5683\nGAGTATGCCCGCCTTG-1,1,49,43,8404,5752\nTGATTCTGTCGCCGGT-1,1,48,44,8284,5821\nGGGCGGCAAATGAATT-1,1,49,45,8403,5890\nTTCCAATCTGGCTATC-1,1,48,46,8284,5958\nGGAACCGTGTAAATTG-1,1,49,47,8403,6027\nTTGGTCACACTCGTAA-1,1,48,48,8283,6096\nCGAACCCGCATGCGTC-1,1,49,49,8403,6165\nGGGCGTCACCACGTAA-1,1,48,50,8283,6233\nTACCAGAAGTAGGTTC-1,1,49,51,8402,6303\nTTGGATATCGTCTACG-1,1,48,52,8282,6371\nTAGATATGGACTGGAA-1,1,49,53,8402,6440\nGCGACGATAGTTGTAC-1,1,48,54,8282,6509\nCTGGATTTACACTTGA-1,1,49,55,8402,6578\nGTTATATTATCTCCCT-1,1,48,56,8282,6646\nTATTCCACTCAGCTCG-1,1,49,57,8401,6715\nCTGTTACCCAATCTAG-1,1,48,58,8281,6784\nACGCGAAGTCAGACGA-1,1,49,59,8401,6853\nGATAGGTAACGTTGAC-1,1,48,60,8281,6921\nTGTTCGTATTGCGGTG-1,1,49,61,8401,6990\nGGTCAGTGGGTCCCAC-1,1,48,62,8281,7059\nAATTCTAGAGTTAGGC-1,1,49,63,8400,7128\nGTTTACGTTCCATCTG-1,1,48,64,8280,7197\nAGACCCACCGCTGATC-1,1,49,65,8400,7266\nTCTAGCATGCCCAGAA-1,1,48,66,8280,7334\nCCCGCAGCGCGAACTA-1,1,49,67,8400,7403\nCTCTATTTGGCTGCAG-1,1,48,68,8280,7472\nTTCTGCCGCGCCTAGA-1,1,49,69,8399,7541\nACTCCCTAGAATAGTA-1,1,48,70,8279,7609\nCCGAACCTTCCCGGCC-1,1,49,71,8399,7678\nTCAGACGCTATAGAAG-1,1,48,72,8279,7747\nCAGGCGCCATGCTAGG-1,1,49,73,8398,7816\nCACTCGAGCTGAACAA-1,1,48,74,8279,7884\nACAAGGAAATCCGCCC-1,1,49,75,8398,7954\nACCACGTGCAGCTATA-1,1,48,76,8278,8022\nACGGCGACGATGGGAA-1,1,49,77,8398,8091\nTTACTAAAGGACTTTA-1,1,48,78,8278,8160\nGGCTGAAATAGCAAAG-1,1,49,79,8397,8229\nTTAGACGAGTCACCTC-1,1,48,80,8278,8297\nGTAGTCGCGGGAATCA-1,1,49,81,8397,8366\nGCAAACCCTACATTAT-1,1,48,82,8277,8435\nGTTAAAGTAGGACTGG-1,1,49,83,8397,8504\nACCCTCCCTTGCTATT-1,1,48,84,8277,8572\nCATGTCTCATTTATGG-1,1,49,85,8396,8641\nAATAGAATCTGTTTCA-1,1,48,86,8277,8710\nTCTCATGAGATAGGGT-1,1,49,87,8396,8779\nATGACTATGCGACATT-1,1,48,88,8276,8848\nCCTAACCCAAACAAGT-1,1,49,89,8396,8917\nAGTCAAGATGACACTT-1,1,48,90,8276,8985\nCGGGAGCTTCAGTGTA-1,1,49,91,8395,9054\nCCACAGTACCCATCCT-1,1,48,92,8275,9123\nGAGCGCGCACGAGTAG-1,1,49,93,8395,9192\nACCAGTGCGGGAGACG-1,1,48,94,8275,9260\nTCCTTACGACGGTCCG-1,1,49,95,8395,9329\nTAGTCGATCACGGGTT-1,1,48,96,8275,9398\nAGACGAAGTGCCGGTC-1,1,49,97,8394,9467\nCGCTGGTGACTACCCT-1,1,48,98,8274,9535\nAGGCATTGTCGTAGGG-1,1,49,99,8394,9605\nTGGTAGAATATATGGG-1,1,48,100,8274,9673\nTTCTTATCCGCTGGGT-1,1,49,101,8394,9742\nAGTTTGGCACGGGTTG-1,1,48,102,8274,9811\nGCATCGGCCGTGTAGG-1,1,49,103,8393,9880\nATGCGAGTCCCACCAC-1,1,48,104,8273,9948\nTCGGCGAACCCAAACC-1,1,49,105,8393,10017\nAGAGTAAACTTCACTA-1,1,48,106,8273,10086\nAAGCCGAAGCGGTTTA-1,1,49,107,8393,10155\nCCCGAGTTTCTCCGTA-1,1,48,108,8273,10223\nAATATCGAATCAATGC-1,1,49,109,8392,10292\nGCGCCTCCCACTCCGA-1,1,48,110,8272,10361\nCGTTAAACTAGTTAGG-1,1,49,111,8392,10430\nTGAGTTAAAGACATTC-1,1,48,112,8272,10498\nAACAGGTAGTATGGAT-1,1,49,113,8391,10568\nGAACGCGGGTCACACG-1,1,48,114,8272,10636\nTGTTTGAGATCGTCAG-1,1,49,115,8391,10705\nATCTCGTGAGCGAAAC-1,1,48,116,8271,10774\nCTGAGAAAGTTCGGCG-1,1,49,117,8391,10843\nATTACCACACTGCCTG-1,1,48,118,8271,10911\nCTGCTGTCTAACGAGC-1,1,49,119,8390,10980\nCGTGGAAGCCTCGTAC-1,1,48,120,8271,11049\nGGACTCACAAATTAGG-1,1,49,121,8390,11118\nCTTTGCTGTCATGGAT-1,1,48,122,8270,11186\nGGTGTTGGGCGTCTTA-1,1,49,123,8390,11255\nTACCATGTATTGATTT-1,1,48,124,8270,11324\nGGTCTCCAAGTAGTGC-1,1,49,125,8389,11393\nATCCTTCTGAAAGAAC-1,0,48,126,8270,11462\nCCGGAAGTTATCAGTC-1,0,49,127,8389,11531\nTTAGAGGGATATACAG-1,1,50,0,8531,2795\nTTGTACACCTCGAACA-1,1,51,1,8650,2864\nGTGGGTACTGAGCGTA-1,1,50,2,8531,2932\nCTTAAGCAGCGAGCCG-1,1,51,3,8650,3001\nGCATTGACTTGCGGAA-1,1,50,4,8530,3070\nCCATAACCTGTGCAGT-1,1,51,5,8650,3139\nGGGCTACTATTTCGTG-1,1,50,6,8530,3207\nGGCGAGCGAAACGGCA-1,1,51,7,8649,3276\nGGAGACCATCTACATA-1,1,50,8,8530,3345\nAGACCAAACCACACCT-1,1,51,9,8649,3414\nCGACGCATCCGTACCT-1,1,50,10,8529,3482\nCCTAGGTAAAGGTAGC-1,1,51,11,8649,3552\nAGCGGCGGTTAGCGGT-1,1,50,12,8529,3620\nCTGGACGCAGTCCGGC-1,1,51,13,8648,3689\nAGCCTAATACCCACGT-1,1,50,14,8528,3758\nAATCTGGCTTTCTAGT-1,1,51,15,8648,3827\nCACCCTTGGTGAGACC-1,1,50,16,8528,3895\nGGATCTTGACTCAACC-1,1,51,17,8648,3964\nACGAGGATACCACTCT-1,1,50,18,8528,4033\nGAGTTGATGGCAATTT-1,1,51,19,8647,4102\nGTGGCAAACAGCGGCA-1,1,50,20,8527,4170\nGATGTAACGAACCACC-1,1,51,21,8647,4239\nAGCCGTGGCTAAATGT-1,1,50,22,8527,4308\nCTCCCTCCTTTCGATC-1,1,51,23,8647,4377\nTTGGAAGAATACAGTC-1,1,50,24,8527,4446\nAGTTAAGTCAACCGCT-1,1,51,25,8646,4515\nCTCATGGTAATTTGCG-1,1,50,26,8526,4583\nAAAGTAGCATTGCTCA-1,1,51,27,8646,4652\nTTGTGGTAGGAGGGAT-1,1,50,28,8526,4721\nAGTCGTGGGCATTACG-1,1,51,29,8646,4790\nACCTAAGTACCTTTCA-1,1,50,30,8526,4858\nGTAGAGGGAGACAAGT-1,1,51,31,8645,4927\nGATCCTCGACACTGGC-1,1,50,32,8525,4996\nCCTACATTCACAGACG-1,1,51,33,8645,5065\nTTGACCGTGTTAATGA-1,1,50,34,8525,5133\nTCTGTTACCCAGCATA-1,1,51,35,8645,5203\nCTAACTGGTCCGGTTC-1,1,50,36,8525,5271\nAGCGACAGGAACGGTC-1,1,51,37,8644,5340\nTAATAGAACAGAGTTA-1,1,50,38,8524,5409\nACAGGTGGAGGTGAGG-1,1,51,39,8644,5478\nTGCGAGAATATTACCC-1,1,50,40,8524,5546\nTTGCGTCGGCCAACCG-1,1,51,41,8643,5615\nAGGCTTCCCGAAGAAG-1,1,50,42,8524,5684\nGCGGACCGCGTTGTGG-1,1,51,43,8643,5753\nGTAATCTGATTCTTCG-1,1,50,44,8523,5821\nCCGCGGAATGCGTCAC-1,1,51,45,8643,5890\nTTCCACACAGATTTGA-1,1,50,46,8523,5959\nCTTCTATTAATGCTAG-1,1,51,47,8642,6028\nCATTTGAGTGGTACGT-1,1,50,48,8523,6097\nTCACAGCAAACTCGAA-1,1,51,49,8642,6166\nCAGACGAACCTGATAC-1,1,50,50,8522,6234\nTAGCTAGAAGGCATGA-1,1,51,51,8642,6303\nATCCAGAGCAACAACC-1,1,50,52,8522,6372\nTCCGGTTCGTCCGGTC-1,1,51,53,8641,6441\nCGCGCATGTTTGATTG-1,1,50,54,8521,6509\nTGGCGATCAAGTTATG-1,1,51,55,8641,6578\nCCCTTTGACAGGTCTT-1,1,50,56,8521,6647\nCAGAGACGGTCACCCA-1,1,51,57,8641,6716\nTAGCTCGCCTGATAAC-1,1,50,58,8521,6784\nTTGTGTTTCCCGAAAG-1,1,51,59,8640,6854\nTATACACAGACGCCTT-1,1,50,60,8520,6922\nACATTAGTTTATATCC-1,1,51,61,8640,6991\nCCATCGCAGTTAAACT-1,1,50,62,8520,7060\nAATTAGCGCTGCAGCG-1,1,51,63,8640,7129\nTCCGCTTATCCCATTA-1,1,50,64,8520,7197\nGTTTGGGCTTGTGAGC-1,1,51,65,8639,7266\nCTTGTCAACATTCGAG-1,1,50,66,8519,7335\nGGACCAACAGGATAAC-1,1,51,67,8639,7404\nAAGCTAGATCGAGTAA-1,1,50,68,8519,7472\nTACCGTGCCTCGGACC-1,1,51,69,8639,7541\nGTAGCCAAACATGGGA-1,1,50,70,8519,7610\nTGCCAGTACGTGGAGA-1,1,51,71,8638,7679\nATAAGGTGGAGAACAT-1,1,50,72,8518,7747\nCTTTAATATTGGTCGA-1,1,51,73,8638,7817\nTGGTTCAACGGGTAAT-1,1,50,74,8518,7885\nGCTGCTACTGCGTAGC-1,1,51,75,8638,7954\nCTGCACAACTACATAT-1,1,50,76,8518,8023\nATATTCCACATAGTGA-1,1,51,77,8637,8092\nTGGCAAACTAAATTAC-1,1,50,78,8517,8160\nACCGCAATAACTGCCT-1,1,51,79,8637,8229\nTCGCACCAGGAGGCAG-1,1,50,80,8517,8298\nACTTCAGGCTGATCCC-1,1,51,81,8636,8367\nACAAATCGCACCGAAT-1,1,50,82,8517,8435\nTCTGATGTATTCTGTC-1,1,51,83,8636,8504\nCTTTACCGAATAGTAG-1,1,50,84,8516,8573\nGCAGATCCATAAGACT-1,1,51,85,8636,8642\nTTCCTCGGACTAACCA-1,1,50,86,8516,8711\nTTATCCGGGATCTATA-1,1,51,87,8635,8780\nCTGGAAGACACGGTGG-1,1,50,88,8516,8848\nGTTCGTCTAAAGAACT-1,1,51,89,8635,8917\nGTCTATTGGTTCCGGT-1,1,50,90,8515,8986\nCTAGGCGCCCTATCAG-1,1,51,91,8635,9055\nAACGGACGTACGTATA-1,1,50,92,8515,9123\nAACACACGCTCGCCGC-1,1,51,93,8634,9192\nGCTACTATAGTAGAGT-1,1,50,94,8514,9261\nAGCATATCAATATGCT-1,1,51,95,8634,9330\nCGAACAGTATGGGCGT-1,1,50,96,8514,9398\nACAATTGTGTCTCTTT-1,1,51,97,8634,9468\nGATCGCTACCCGATTT-1,1,50,98,8514,9536\nATTGCGATCAGTAACT-1,1,51,99,8633,9605\nCAGCAGTCCAGACTAT-1,1,50,100,8513,9674\nAGAGGCTTCGGAAACC-1,1,51,101,8633,9743\nAAACAAGTATCTCCCA-1,1,50,102,8513,9811\nGGCAGCAAACCTATGC-1,1,51,103,8633,9880\nACTAGTTGCGATCGTC-1,1,50,104,8513,9949\nTTGGACCATCTGGCAA-1,1,51,105,8632,10018\nCCCTCCTCGCTCGTAT-1,1,50,106,8512,10086\nGAGCGCAAATACTCCG-1,1,51,107,8632,10155\nATTACATGTCAGTCTT-1,1,50,108,8512,10224\nTTGGGACGTAAGAGTT-1,1,51,109,8632,10293\nCTTCGGCCAATTGTTT-1,1,50,110,8512,10362\nAGACCGCTCCGCGGTT-1,1,51,111,8631,10431\nGACCGCGTCTGACGTG-1,1,50,112,8511,10499\nCCAAATAACAAGATTC-1,1,51,113,8631,10568\nTCTTTAGAGTCTAACA-1,1,50,114,8511,10637\nCTCCCAATGAGTCGCG-1,1,51,115,8631,10706\nTAGTCTTTCCGAATTG-1,1,50,116,8511,10774\nGAGTAAACCGGAAAGT-1,1,51,117,8630,10843\nGATCTCGACGCTGTGG-1,1,50,118,8510,10912\nCTGACATAGAAATAGA-1,1,51,119,8630,10981\nTATACCGAGTGCCACA-1,1,50,120,8510,11049\nTCCGAACGTTGCCGCT-1,1,51,121,8629,11119\nGCAGAAGGTAATCTCC-1,1,50,122,8510,11187\nGGTACTAAGTGCTTTG-1,1,51,123,8629,11256\nCAGAATATTCGTTATC-1,1,50,124,8509,11325\nTAATCAACCAAATGGG-1,1,51,125,8629,11394\nTCTGCTTAGAACAAGC-1,0,50,126,8509,11462\nAACTGAGTTATACTGA-1,0,51,127,8628,11531\nCCTCAACGATCGCTGT-1,1,52,0,8770,2795\nCGGTACGGCAAACCCA-1,1,53,1,8890,2864\nCAGATGTTTGTCCCAA-1,1,52,2,8770,2933\nGTTTGTTAGCCAAGTA-1,1,53,3,8889,3002\nACGTGACAAAGTAAGT-1,1,52,4,8770,3070\nTTGTCACCGCGGTATC-1,1,53,5,8889,3139\nGATCTAACCGTATTCA-1,1,52,6,8769,3208\nATGGAAATTTAAGGAG-1,1,53,7,8889,3277\nATTGATCACCACATTT-1,1,52,8,8769,3346\nACTGCTCGGAAGGATG-1,1,53,9,8888,3415\nTCTAAAGAACAGTCTC-1,1,52,10,8769,3483\nCTGGGTTGAGTTAAAG-1,1,53,11,8888,3552\nCCCAGTAAACTTGGGA-1,1,52,12,8768,3621\nAGATTCACAACCGATA-1,1,53,13,8888,3690\nAGAAGGTACACTTCAC-1,1,52,14,8768,3758\nGCAGGAACTTAGATCT-1,1,53,15,8887,3827\nAATCTAGGTTTACTTG-1,1,52,16,8768,3896\nCCCGGTGTATCGGAAT-1,1,53,17,8887,3965\nTTATCCTCAAGGAATA-1,1,52,18,8767,4033\nAGCATCATTTCGAAAG-1,1,53,19,8887,4103\nCGCGAGTCTGCCGGGT-1,1,52,20,8767,4171\nTGCGTCATGACTGAGC-1,1,53,21,8886,4240\nGTATGAAATTTCACTC-1,1,52,22,8766,4309\nTTGGTTGCGGTGCGCG-1,1,53,23,8886,4378\nACAGAACTGAGAACAA-1,1,52,24,8766,4446\nCTACTATCATAGGTTT-1,1,53,25,8886,4515\nCCTCTATCGATTAGCA-1,1,52,26,8766,4584\nCCGTATCTCGTCGTAG-1,1,53,27,8885,4653\nTCACGATGTCCGTGGA-1,1,52,28,8765,4721\nTCAACAAAGATAATTC-1,1,53,29,8885,4790\nATGACGCGTTCTATCC-1,1,52,30,8765,4859\nATTTGCGCGAGTAGCT-1,1,53,31,8885,4928\nTTCTTGGACGATCTGC-1,1,52,32,8765,4996\nAGACGACGATGCCGCT-1,1,53,33,8884,5066\nGGTCTCTGAATGGACT-1,1,52,34,8764,5134\nGCTCAATCCGTTTATT-1,1,53,35,8884,5203\nCCAGAAAGCAACTCAT-1,1,52,36,8764,5272\nCACCGTTAGGGATCAC-1,1,53,37,8884,5341\nAATAACACTAGAACAA-1,1,52,38,8764,5409\nCACCCGGTTTGTGACT-1,1,53,39,8883,5478\nCCACCAACTTTACTGT-1,1,52,40,8763,5547\nAACTCTCAGTGTGCTC-1,1,53,41,8883,5616\nAAACCGTTCGTCCAGG-1,1,52,42,8763,5684\nCAGCGATTCCCTTCAA-1,1,53,43,8882,5753\nGCTAGCTTGAATAGCT-1,1,52,44,8763,5822\nTTGCGGCATCAGAAAG-1,1,53,45,8882,5891\nAATAGAACAGAGTGGC-1,1,52,46,8762,5960\nGCCATCGAGCTGCGTG-1,1,53,47,8882,6029\nACACTGATCAAGGTGT-1,1,52,48,8762,6097\nACCAACGCTTATTTAT-1,1,53,49,8881,6166\nGACATCGATTTATAAC-1,1,52,50,8762,6235\nCAGACACCGATCGCTG-1,1,53,51,8881,6304\nCCAAGAAAGTGGGCGA-1,1,52,52,8761,6372\nGGTAGACCGTTGGGCG-1,1,53,53,8881,6441\nTCGTTAGGAGTCCCTA-1,1,52,54,8761,6510\nACGGCCAACATGGACT-1,1,53,55,8880,6579\nGTGAGTGGTACAACGC-1,1,52,56,8761,6647\nGAGACTTCGCGACCGA-1,1,53,57,8880,6717\nGAAGTCAGTTGCACTA-1,1,52,58,8760,6785\nGTTTGGTAGGGTCAAC-1,1,53,59,8880,6854\nACGGCACTTGCTTGGG-1,1,52,60,8760,6923\nCCTCTGTACTATTCTA-1,1,53,61,8879,6992\nCTATTCATGTGTCCCA-1,1,52,62,8759,7060\nGCCCGATCTGTGGTCG-1,1,53,63,8879,7129\nTTAGAAGAACATGACT-1,1,52,64,8759,7198\nAGGATAAAGTCGGGAT-1,1,53,65,8879,7267\nCGGCAAACATCGTGCG-1,1,52,66,8759,7335\nCTAGTTACAACCCGGT-1,1,53,67,8878,7404\nTTCGACAGAGCCCGTG-1,1,52,68,8758,7473\nAAGCATACTCTCCTGA-1,1,53,69,8878,7542\nGACGACGATCCGCGTT-1,1,52,70,8758,7611\nGGTAGAAGACCGCCTG-1,1,53,71,8878,7680\nGAGATGGGAGTCGACA-1,1,52,72,8758,7748\nAACGATAATGCCGTAG-1,1,53,73,8877,7817\nTGGTCCCACGCTACGG-1,1,52,74,8757,7886\nCTGCTTGGCGATAGCT-1,1,53,75,8877,7955\nATCGGAGACAGACGGC-1,1,52,76,8757,8023\nTAGCGTCCGGTGTGGT-1,1,53,77,8877,8092\nGTTGAGTCCCGCCGGT-1,1,52,78,8757,8161\nAAATAGGGTGCTATTG-1,1,53,79,8876,8230\nAAGTGTTTGGAGACGG-1,1,52,80,8756,8298\nAGCACTACCTCACCAG-1,1,53,81,8876,8368\nTGCCTTGGCCAGGCAA-1,1,52,82,8756,8436\nTCGTCTTAGGCGTTAA-1,1,53,83,8876,8505\nTTCTGACCGGGCTCAA-1,1,52,84,8756,8574\nCAGAACTTAGCCCTCT-1,1,53,85,8875,8643\nAGCTCCTTCGCACATC-1,1,52,86,8755,8711\nACAGGCTTGCCCGACT-1,1,53,87,8875,8780\nGCTATACGTCTCGGAC-1,1,52,88,8755,8849\nGAGCCAGCTACCTGTG-1,1,53,89,8874,8918\nTGCTAAGTGTCTATTT-1,1,52,90,8755,8986\nGTCCTACGAATAGTCT-1,1,53,91,8874,9055\nCAGTGTCGGCTGGCCC-1,1,52,92,8754,9124\nCTATCGACGAAATACA-1,1,53,93,8874,9193\nCATCATTACCCTGAGG-1,1,52,94,8754,9261\nTAAGTTGCGACGTAGG-1,1,53,95,8873,9331\nAGTGCACGCTTAAGAA-1,1,52,96,8754,9399\nTGTGCCGGTGCCGGAA-1,1,53,97,8873,9468\nAGGAGGCCTTCGCGCG-1,1,52,98,8753,9537\nTCCGCGGCCCAATGAA-1,1,53,99,8873,9606\nTCCGTTAAGCTAATAT-1,1,52,100,8753,9674\nAAATCTAGCCCTGCTA-1,1,53,101,8872,9743\nCGCAGGCGATCCAAAC-1,1,52,102,8752,9812\nCCATATGGAAACTATA-1,1,53,103,8872,9881\nCACCGTATCCCATCCG-1,1,52,104,8752,9949\nGGTCAAGACTACTTCG-1,1,53,105,8872,10018\nCCCGTTTCGCAGATGT-1,1,52,106,8752,10087\nGTTATAATACGGTGAA-1,1,53,107,8871,10156\nACTTGTGGATGGAACG-1,1,52,108,8751,10225\nGTTCGCTGAGACGTCT-1,1,53,109,8871,10294\nGACACTGGAACCCGAT-1,1,52,110,8751,10362\nCCCAGGTCTGAAGGCT-1,1,53,111,8871,10431\nTCTCGAGGAGGTTCGC-1,1,52,112,8751,10500\nACTATCTGCCCGCGTA-1,1,53,113,8870,10569\nCCTAAATTGTATCCTA-1,1,52,114,8750,10637\nAGAAATTATGACTCGC-1,1,53,115,8870,10706\nTCCAGCGCTATAAGCG-1,1,52,116,8750,10775\nTCGTGTATTGGTCACG-1,1,53,117,8870,10844\nCCACATACTGCACCCA-1,1,52,118,8750,10912\nGTTGCGGACGGTCAGG-1,1,53,119,8869,10982\nGTATTTAATGGCATAA-1,1,52,120,8749,11050\nGTCGATAGGTGACTTT-1,1,53,121,8869,11119\nAATAATCTTCGTATCG-1,1,52,122,8749,11188\nATTGTTCAACGATCCG-1,1,53,123,8869,11257\nGTGCAGCGTAGAGTAG-1,1,52,124,8749,11325\nTGTGGTAGGGTGCCTT-1,1,53,125,8868,11394\nTGTGGACTATCTACGT-1,0,52,126,8748,11463\nGGCGGGCTCTAAGAGT-1,0,53,127,8868,11532\nTGTGCCAGAGGCAAAG-1,1,54,0,9010,2796\nTGGCTTATGTATAATG-1,1,55,1,9129,2865\nGCAAGCTGGAAACCGC-1,1,54,2,9009,2933\nGATATCAAGCAGGAGC-1,1,55,3,9129,3002\nCACCAATCATCCGTCT-1,1,54,4,9009,3071\nCCACATGGCTCTTTAT-1,1,55,5,9129,3140\nGTCCTACTCTACGGGC-1,1,54,6,9009,3209\nCTCGAGACATACGATA-1,1,55,7,9128,3278\nTATCAGTGGCGTAGTC-1,1,54,8,9008,3346\nTGACAGGACAAGTCCA-1,1,55,9,9128,3415\nTATTAACACCAAAGCA-1,1,54,10,9008,3484\nTAGGCGATGAGGTCTC-1,1,55,11,9127,3553\nCCGCCGGAACTTCTCG-1,1,54,12,9008,3621\nGCCATTAGCCTCAAAC-1,1,55,13,9127,3690\nGTTAGGCTACCCGTTT-1,1,54,14,9007,3759\nGGGTATTCTAGCAAAC-1,1,55,15,9127,3828\nGCCCTAGCCGTCGCGA-1,1,54,16,9007,3896\nCAGATCCTGGTTTGAA-1,1,55,17,9126,3966\nCTTCAGTGGTCGCCTA-1,1,54,18,9007,4034\nGGGCAACCGCACGTGC-1,1,55,19,9126,4103\nGACCCAATTATGATAC-1,1,54,20,9006,4172\nGAAGCTCGGACCCGTC-1,1,55,21,9126,4241\nCGTCGGGTCTAAGCGC-1,1,54,22,9006,4309\nGAGGCCCGACTCCGCA-1,1,55,23,9125,4378\nTTGCCATAGCCCGCTC-1,1,54,24,9006,4447\nTAACATACACGCGATC-1,1,55,25,9125,4516\nCCGACGGGCATGAGGT-1,1,54,26,9005,4584\nAATGTGCCCGAGGTGT-1,1,55,27,9125,4653\nAGCTGTAACCTCAATC-1,1,54,28,9005,4722\nCGAGTGAAGGTACCAG-1,1,55,29,9124,4791\nAGTCTCACAAGACTAC-1,1,54,30,9004,4860\nAAATTGATAGTCCTTT-1,1,55,31,9124,4929\nTAAGTCGCCGAGTATC-1,1,54,32,9004,4997\nGCGGAGAAACTTCGCA-1,1,55,33,9124,5066\nGGCAAAGGCGCCAATA-1,1,54,34,9004,5135\nATTTCCGGGTTCTGCG-1,1,55,35,9123,5204\nTAAACCCAGGAGGGCA-1,1,54,36,9003,5272\nTTCGGGCGCTAGTCTT-1,1,55,37,9123,5341\nGTGGAGTCGGCGGTTG-1,1,54,38,9003,5410\nGCACGCCTACTTAGAT-1,1,55,39,9123,5479\nCAATATTCTTGACCTA-1,1,54,40,9003,5547\nCGTTTGTGTAGAGGGT-1,1,55,41,9122,5617\nCATAGCGTTGCCCACC-1,1,54,42,9002,5685\nTGATACATTTAGCCGT-1,1,55,43,9122,5754\nTTCCGGCCTTGAGGCT-1,1,54,44,9002,5823\nCCTATACCGTCCTGTC-1,1,55,45,9122,5892\nTCGTTGCTATCCGGTC-1,1,54,46,9002,5960\nTGGCGACTGCTCCAAA-1,1,55,47,9121,6029\nCAGAGGCGATGCATGA-1,1,54,48,9001,6098\nTCACAGGAGAATAAGA-1,1,55,49,9121,6167\nGGGTTAACATTTGAGT-1,1,54,50,9001,6235\nGCGGGCGAGCCTTACC-1,1,55,51,9120,6304\nTGCCAATGGGTACTCT-1,1,54,52,9001,6373\nAGGGACTCTACGCGAC-1,1,55,53,9120,6442\nGACGCCGTAAAGGCTA-1,1,54,54,9000,6510\nAAAGGCTCTCGCGCCG-1,1,55,55,9120,6580\nCTGGCTGGTTGTCAGT-1,1,54,56,9000,6648\nACATCCCGGCCATACG-1,1,55,57,9119,6717\nCTCTTGTCCCGCTTGG-1,1,54,58,9000,6786\nTACGCCTCCATTCCGA-1,1,55,59,9119,6855\nCGGAGCAATTTAATCG-1,1,54,60,8999,6923\nAACTAGGCTTGGGTGT-1,1,55,61,9119,6992\nGCGTCGCCAGGGTGAT-1,1,54,62,8999,7061\nGTTGGTCATGCTATCC-1,1,55,63,9118,7130\nGCTGGCGGCGCATGCT-1,1,54,64,8999,7198\nTGAGCGGAAAGTGTTC-1,1,55,65,9118,7267\nCTTTCTGTGCGGGCTT-1,1,54,66,8998,7336\nATCTAATATCCTACGG-1,1,55,67,9118,7405\nAACCTTTAAATACGGT-1,1,54,68,8998,7474\nTTCACTCGAGCACCTA-1,1,55,69,9117,7543\nCTGCACTCCAGTACAG-1,1,54,70,8997,7611\nCCATTTCTACCTATTA-1,1,55,71,9117,7680\nCAAGCGGCACATAATT-1,1,54,72,8997,7749\nGTTAACATCACTTAAA-1,1,55,73,9117,7818\nATATAAATGTAGCTGC-1,1,54,74,8997,7886\nGCCTCTATACATAGCA-1,1,55,75,9116,7955\nGTGTCGTATAGCGTTC-1,1,54,76,8996,8024\nATACGTACTTAGCCAC-1,1,55,77,9116,8093\nGCTATCATACTCATGG-1,1,54,78,8996,8161\nTATCCAATTGGTTATC-1,1,55,79,9116,8231\nTCTCCCTGGGCAGCGT-1,1,54,80,8996,8299\nCCTATAATGAGTGCCC-1,1,55,81,9115,8368\nCAATGCGAGAAGTATC-1,1,54,82,8995,8437\nTACCAATAAAGTACCA-1,1,55,83,9115,8506\nGCTAGGCACCACGGAG-1,1,54,84,8995,8574\nGGTGGACTGCTCTGGC-1,1,55,85,9115,8643\nTATTCCGAGCTGTTAT-1,1,54,86,8995,8712\nTTGAGAAGTTTAGCAT-1,1,55,87,9114,8781\nGTCCGGACCTGAAATT-1,1,54,88,8994,8849\nCTGTGGTCGGGAGATA-1,1,55,89,9114,8918\nCATGTAAGAGACATTT-1,1,54,90,8994,8987\nCACGTTTCGTACACAC-1,1,55,91,9113,9056\nCATCCTCTCAAAGATC-1,1,54,92,8994,9125\nGATTAACCGAAAGCCC-1,1,55,93,9113,9194\nAACGCTGTTGCTGAAA-1,1,54,94,8993,9262\nTTCAGCTGGCGTGCCC-1,1,55,95,9113,9331\nTAGAGGTTCTACTTGT-1,1,54,96,8993,9400\nGTATCAAACGTTAGCT-1,1,55,97,9112,9469\nAATCATGTAAAGACTC-1,1,54,98,8993,9537\nAGTCACTAGCTCTCGA-1,1,55,99,9112,9606\nGCGAAACGATCGGGAG-1,1,54,100,8992,9675\nCATGGATTGTCTTCCG-1,1,55,101,9112,9744\nATTGGATTACAGCGTA-1,1,54,102,8992,9812\nCACACACGCTAACGAG-1,1,55,103,9111,9882\nCTATGGGAAGCGGAAT-1,1,54,104,8992,9950\nTGCCTAATTGAAGATT-1,1,55,105,9111,10019\nTGGCAATGGGACGGCG-1,1,54,106,8991,10088\nACCTCCGCCCTCGCTG-1,1,55,107,9111,10157\nCGGCAATAAGATCGCC-1,1,54,108,8991,10225\nCCTTGACCACTTTATT-1,1,55,109,9110,10294\nTCATTTAGAAGTGTGA-1,1,54,110,8990,10363\nCAGAATAACACACGGA-1,1,55,111,9110,10432\nTCTGAGCAATTGACTG-1,1,54,112,8990,10500\nCAACGTGGTGGAGTCT-1,1,55,113,9110,10569\nCACAGTTCGCTTCCCA-1,1,54,114,8990,10638\nTCTCTCGCCGCACATA-1,1,55,115,9109,10707\nTGGGCAGGCCACCGCA-1,1,54,116,8989,10775\nCTCCTGTTCAAGGCAG-1,1,55,117,9109,10845\nTATTTGATTTGCACAG-1,1,54,118,8989,10913\nGCCACTCCTTACGGTA-1,1,55,119,9109,10982\nACCGATATTTAATCAT-1,1,54,120,8989,11051\nTTATTATCTGGAAGGC-1,1,55,121,9108,11120\nTACATGCCGGAATTGT-1,1,54,122,8988,11188\nGGCACTCAGCCGACCC-1,1,55,123,9108,11257\nAAACCGGAAATGTTAA-1,1,54,124,8988,11326\nAGACAGGCATCTCAGC-1,0,55,125,9108,11395\nGTGGTGATGGTTTGTG-1,0,54,126,8988,11463\nCCGATATGACGTAAGG-1,0,55,127,9107,11532\nCATACACAAAGTCAGC-1,1,56,0,9249,2796\nTGCCGTTCTTAATCGG-1,1,57,1,9369,2866\nTCGAGTCTACGATTCG-1,1,56,2,9249,2934\nACTTCCAGTGGAAGCT-1,1,57,3,9368,3003\nCAATTGGGCCGCACTC-1,1,56,4,9248,3072\nAAATCGCGGAAGGAGT-1,1,57,5,9368,3141\nATATCAATTCCAGCCT-1,1,56,6,9248,3209\nCTTTGTCGAATGCTCC-1,1,57,7,9368,3278\nTGAGGCATGTACTGTG-1,1,56,8,9248,3347\nTTCGCCGCTCGCGCTA-1,1,57,9,9367,3416\nGACTCCTTCCAATTGA-1,1,56,10,9247,3484\nACCGAAGAGTCTGGTT-1,1,57,11,9367,3553\nTGGCATGAAGTTTGGG-1,1,56,12,9247,3622\nACGAGGTTTACAACGT-1,1,57,13,9367,3691\nAACCCATCCCATGATC-1,1,56,14,9247,3759\nTCGACAACTGAACCCG-1,1,57,15,9366,3829\nTATCTTGCAATACAAC-1,1,56,16,9246,3897\nTATTATGTTTGCCTGC-1,1,57,17,9366,3966\nGTCAAAGAAGTGGTGT-1,1,56,18,9246,4035\nGGGACAGAGTTACTCC-1,1,57,19,9365,4104\nAGCAACCGAAAGTAAT-1,1,56,20,9246,4172\nCTCTCTAACTGCCTAG-1,1,57,21,9365,4241\nATACGGAACGTCGTTT-1,1,56,22,9245,4310\nTCGGTCCCGACAATAG-1,1,57,23,9365,4379\nCGCGCAAATGTCCAGA-1,1,56,24,9245,4447\nTCTAATACTGCCTCAG-1,1,57,25,9364,4516\nCTCGGTTGTCGGCCCT-1,1,56,26,9245,4585\nGGTAACCGGGAGGATA-1,1,57,27,9364,4654\nCAGGCGCACGGTGGTC-1,1,56,28,9244,4723\nTCATCGATGGTCCCAA-1,1,57,29,9364,4792\nCAAAGATTATTGGGCC-1,1,56,30,9244,4860\nACAATGAATACGGAGA-1,1,57,31,9363,4929\nGCTCCTGACATACTGG-1,1,56,32,9244,4998\nGATGACAAGTAGGGCA-1,1,57,33,9363,5067\nTACGCCGAGGGTACCC-1,1,56,34,9243,5135\nAAGCGTCCCTCATCGA-1,1,57,35,9363,5204\nCACTCCTATGTAAGAT-1,1,56,36,9243,5273\nTCGAGCCAGGCAGGCC-1,1,57,37,9362,5342\nTCGTCAAGTACGCGCA-1,1,56,38,9242,5410\nCCCGTAGCTGGGAAGA-1,1,57,39,9362,5480\nACCCTATGCCATATCG-1,1,56,40,9242,5548\nGTGGTATAGTCTGCCG-1,1,57,41,9362,5617\nCCGGTTTGTAATTGTG-1,1,56,42,9242,5686\nGTCGCCGTTGTGTGTT-1,1,57,43,9361,5755\nTACTGAACAGATTTAG-1,1,56,44,9241,5823\nTATTAACCTGACCGCG-1,1,57,45,9361,5892\nCTGTAGCCATCTCACT-1,1,56,46,9241,5961\nTACTATGGTTCCTCAG-1,1,57,47,9361,6030\nGAGTCAGACCAGAATC-1,1,56,48,9241,6098\nTAAGGCATAACATCAA-1,1,57,49,9360,6167\nCTTCCGCTCCGTGAAG-1,1,56,50,9240,6236\nGGGCTATGATCGATGG-1,1,57,51,9360,6305\nGGTCGGTCGTCCACAG-1,1,56,52,9240,6374\nGGGACCCGTATATCTT-1,1,57,53,9360,6443\nGAAAGCAGTGCACTTT-1,1,56,54,9240,6511\nAGCGCATAATGAATCG-1,1,57,55,9359,6580\nGCTGCTAAGTAGTCGA-1,1,56,56,9239,6649\nAATAGTCCGTCCCGAC-1,1,57,57,9359,6718\nAGCAGCCAGATGAATA-1,1,56,58,9239,6786\nCCTCGCGCTGTGCGAT-1,1,57,59,9358,6855\nTTGTGTATGCCACCAA-1,1,56,60,9239,6924\nTGCTCGGCGAAACCCA-1,1,57,61,9358,6993\nCGATTAAATATCTCCT-1,1,56,62,9238,7061\nTGCCGTGGATCGTCCT-1,1,57,63,9358,7131\nGTGGACGCATTTGTCC-1,1,56,64,9238,7199\nACCCGGAAACTCCCAG-1,1,57,65,9357,7268\nCCGCATGTGGTACGAT-1,1,56,66,9238,7337\nGCAAACCTTGGCCATA-1,1,57,67,9357,7406\nATTCACTGATGTTGGA-1,1,56,68,9237,7474\nACTCCCTAATGCTAAA-1,1,57,69,9357,7543\nTCCTAAATTGGGAAGC-1,1,56,70,9237,7612\nGAACAGATTACTAAAT-1,1,57,71,9356,7681\nATACAGGCCCTCCAAT-1,1,56,72,9237,7749\nCGATTCGCCTGGCTGC-1,1,57,73,9356,7818\nTCCGCCTGTCTACAAG-1,1,56,74,9236,7887\nCACATCTCACCGACGA-1,1,57,75,9356,7956\nCGTCGTCCTTCGCGAA-1,1,56,76,9236,8024\nCGTTGAATACCGCGCT-1,1,57,77,9355,8094\nACCAATATGCAAGTTA-1,1,56,78,9235,8162\nACGGATGGTGCGGATA-1,1,57,79,9355,8231\nCACCCTAACAAGATCT-1,1,56,80,9235,8300\nAAGCTCTTTCATGGTG-1,1,57,81,9355,8369\nCATGGGTATGCCTTAT-1,1,56,82,9235,8437\nGCGCGTCATTGGTACA-1,1,57,83,9354,8506\nTCTGAAGCACGTGGTC-1,1,56,84,9234,8575\nAAGTTCACTCCAAGCT-1,1,57,85,9354,8644\nTTGGACCTATAACAGT-1,1,56,86,9234,8712\nATCAGCCTCATGCTGC-1,1,57,87,9354,8781\nAGTACTCTTATGCCCA-1,1,56,88,9234,8850\nGCGCTTAAATAATTGG-1,1,57,89,9353,8919\nTTGTGAACCTAATCCG-1,1,56,90,9233,8988\nAGTCGTCGACCACCAA-1,1,57,91,9353,9057\nTCCTTTAAATCCGCTT-1,1,56,92,9233,9125\nACTGAAACGCCGTTAG-1,1,57,93,9353,9194\nGCACACACTGGTAGCC-1,1,56,94,9233,9263\nTAAGGGCCTGTCCGAT-1,1,57,95,9352,9332\nGCTAGAGTAGAGATGT-1,1,56,96,9232,9400\nCACAGGGCCATATAGT-1,1,57,97,9352,9469\nCCATGCCCTAGATTTC-1,1,56,98,9232,9538\nTCGCGTAGCAGTGTCC-1,1,57,99,9351,9607\nGACGAGGCTAATAAAC-1,1,56,100,9232,9675\nCTACACTCGCAGATGG-1,1,57,101,9351,9745\nACGCATACGTTTACTA-1,1,56,102,9231,9813\nGGAGCAACATTTCAAG-1,1,57,103,9351,9882\nCGAGAGGGTAGCCGCG-1,1,56,104,9231,9951\nTCGTGTTCGACCACAA-1,1,57,105,9350,10020\nTCTACCCAATAGAGAG-1,1,56,106,9231,10088\nATAAACGTTGCACCAC-1,1,57,107,9350,10157\nTCAACTGCAGAGTCAG-1,1,56,108,9230,10226\nAGCTCCATATATGTTC-1,1,57,109,9350,10295\nGTAACATCTAAGATAA-1,1,56,110,9230,10363\nCCGACAATAGGCCGCC-1,1,57,111,9349,10432\nATGCTTAGGAGTTGAT-1,1,56,112,9230,10501\nAACCCGAGCAGAATCG-1,1,57,113,9349,10570\nCGGGTTTGTTAGGGCT-1,1,56,114,9229,10639\nAGGCACGTGACTGTCC-1,1,57,115,9349,10708\nGACAAGAGATGAGATT-1,1,56,116,9229,10776\nAGCCCTTGGACATCCC-1,1,57,117,9348,10845\nGCAGTTCGATCCGAGC-1,1,56,118,9228,10914\nTCGAGGGCAACAGACG-1,1,57,119,9348,10983\nACTTTGTCGACGCACT-1,1,56,120,9228,11051\nGAGTATCAAAGTTACA-1,1,57,121,9348,11120\nCCAGGCTGGCGTCTGA-1,1,56,122,9228,11189\nGGACTCTTCCGGTTGA-1,1,57,123,9347,11258\nTCCCAGCACACGACAT-1,1,56,124,9227,11326\nGCTCCATGAGTGCAGA-1,0,57,125,9347,11395\nGCAGTGCGGGCGGATG-1,0,56,126,9227,11464\nGCGTTCTGACTAAGCG-1,0,57,127,9347,11533\nATTTGGAGATTGCGGT-1,1,58,0,9488,2797\nAAGGGTTTGATTTCAG-1,1,59,1,9608,2866\nCCGGCCGCGAGCATAT-1,1,58,2,9488,2935\nGAGTTCTGTGGGTGCT-1,1,59,3,9608,3004\nAAACGAAGATGGAGTA-1,1,58,4,9488,3072\nAATTACGAGACCCATC-1,1,59,5,9607,3141\nCCTTTGAATTATGGCT-1,1,58,6,9487,3210\nCATGGTATTAGTTTGT-1,1,59,7,9607,3279\nAACGAAAGTCGTCCCA-1,1,58,8,9487,3347\nACATAAGTCGTGGTGA-1,1,59,9,9607,3416\nGCACAACCTCGGGCGT-1,1,58,10,9487,3485\nATGCACGCGCTGTTCA-1,1,59,11,9606,3554\nCGATGTTGTTATCTAC-1,1,58,12,9486,3623\nGTCTCCTGCCAGTATG-1,1,59,13,9606,3692\nGATGCCTTCTGCGGCA-1,1,58,14,9486,3760\nACCGGTCAGGTACACC-1,1,59,15,9606,3829\nGTGGAGCGTTTACCGA-1,1,58,16,9486,3898\nAGGCGGTTTGTCCCGC-1,1,59,17,9605,3967\nTCCCTGGCGTATTAAC-1,1,58,18,9485,4035\nAAACACCAATAACTGC-1,1,59,19,9605,4104\nTGGACGCAATCCAGCC-1,1,58,20,9485,4173\nGGAACCTTGACTCTGC-1,1,59,21,9605,4242\nGTATCTCAGTCTTGAC-1,1,58,22,9485,4310\nGCTGAATCTTCCAATC-1,1,59,23,9604,4380\nAGATGCAAGACGTGCA-1,1,58,24,9484,4448\nTTAAGGCCCGTACTTT-1,1,59,25,9604,4517\nAGTATGCTGGAGACCA-1,1,58,26,9484,4586\nGCGGCAAAGTATTGCC-1,1,59,27,9603,4655\nGCGCAAATATATTCAA-1,1,58,28,9484,4723\nGAAGAAACGATATTGT-1,1,59,29,9603,4792\nGGACCTCTAGGCCGCC-1,1,58,30,9483,4861\nAACAGCTGTGTGGCAA-1,1,59,31,9603,4930\nTCGAATATCCCGCAGG-1,1,58,32,9483,4998\nGCGACCCAACCATCTG-1,1,59,33,9602,5067\nAGAGAAGGAGTACAAT-1,1,58,34,9483,5136\nGGCTCCTCCACCTGTT-1,1,59,35,9602,5205\nTCAGTGTATACGTCAT-1,1,58,36,9482,5273\nATTATTATGTCCGTCA-1,1,59,37,9602,5343\nTTAGGTCATAACCGAC-1,1,58,38,9482,5411\nAAGATGGCACCGGACC-1,1,59,39,9601,5480\nCTACAAGAAATAACCC-1,1,58,40,9481,5549\nACTATTTCCGGGCCCA-1,1,59,41,9601,5618\nTTGTTTCACATCCAGG-1,1,58,42,9481,5686\nCGCTGTGTGGATGTTG-1,1,59,43,9601,5755\nTCAACATAGCGCCCTA-1,1,58,44,9481,5824\nGGCCGTTTGGGTTTCA-1,1,59,45,9600,5893\nTACTCTTTCGTCTTCA-1,1,58,46,9480,5961\nTTCGCGCGCCATACGA-1,1,59,47,9600,6030\nCGGAAAGAATCAAACG-1,1,58,48,9480,6099\nAGGCAGGGAGCGTACT-1,1,59,49,9600,6168\nCCTTTAAGGGAGCACT-1,1,58,50,9480,6237\nCGCCCAGCACGCCTAG-1,1,59,51,9599,6306\nAGTATTTGGCACGACC-1,1,58,52,9479,6374\nTATCCGCACCGTCGGG-1,1,59,53,9599,6443\nACCCGTGTCATCAGTA-1,1,58,54,9479,6512\nCAACTGCTCATCCGAT-1,1,59,55,9599,6581\nCCGGAGCGTACTTTCT-1,1,58,56,9479,6649\nTACCGTAGGTTAACTA-1,1,59,57,9598,6718\nTGAGGAGTGCCAGCTT-1,1,58,58,9478,6787\nCCAGTCTTGTCATAGA-1,1,59,59,9598,6856\nGGGCAGAGCAATCGTT-1,1,58,60,9478,6924\nTACCAAATAGCCCAGA-1,1,59,61,9598,6994\nTACCTACTCCCAGTAT-1,1,58,62,9478,7062\nTGTGAGACTAGCCCAA-1,1,59,63,9597,7131\nATACCTAACCAAGAAA-1,1,58,64,9477,7200\nGTCGGGAAGCAGAAAC-1,1,59,65,9597,7269\nAATTAACGGATTTCCA-1,1,58,66,9477,7337\nGAACCCTCTGTGTTCT-1,1,59,67,9596,7406\nATATAAAGCGCTCGTG-1,1,58,68,9477,7475\nCCTGGTCGAATGTGGG-1,1,59,69,9596,7544\nCTAGGTTCGGACGTGA-1,1,58,70,9476,7612\nTCGGGAGACAGCGTAC-1,1,59,71,9596,7681\nCATAAGAAGCTTGGCT-1,1,58,72,9476,7750\nAAGCACCCTGCGTATC-1,1,59,73,9595,7819\nAGTTTGCACCTGCCTC-1,1,58,74,9476,7888\nGACTGCAAATCGAGCT-1,1,59,75,9595,7957\nGTGTATATCAGCGGGC-1,1,58,76,9475,8025\nCTTCAACTCCACTTGG-1,1,59,77,9595,8094\nACTTACGCATCCACGC-1,1,58,78,9475,8163\nCATGAGATGCACTCTC-1,1,59,79,9594,8232\nGAGCACCTGTGTCCAG-1,1,58,80,9475,8300\nCGCGACCGCGACAGAT-1,1,59,81,9594,8369\nGTCCCGCGACGTTATG-1,1,58,82,9474,8438\nGCAGTGTGGCTATAGG-1,1,59,83,9594,8507\nATACTGCCTTACACCG-1,1,58,84,9474,8575\nTACAGAAACGGTGGGC-1,1,59,85,9593,8644\nGGCGCGGAGATCTTTC-1,1,58,86,9473,8713\nCGTTAATGTCCCGACG-1,1,59,87,9593,8782\nGCTAACTGAAGTCTGA-1,1,58,88,9473,8851\nAGTGATAACCTGCGCG-1,1,59,89,9593,8920\nTATGATCTTCTCTTTA-1,1,58,90,9473,8988\nTGTACCTACACGAGGG-1,1,59,91,9592,9057\nATATCGTTCCTCGAAC-1,1,58,92,9472,9126\nCGGCGCCATCAATCCC-1,1,59,93,9592,9195\nCCTGTTTGAAGACACG-1,1,58,94,9472,9263\nGCCACTCAGAGCGCGA-1,1,59,95,9592,9332\nTCGGCTTGTATCGACG-1,1,58,96,9472,9401\nTAGATGGTTCCTTACT-1,1,59,97,9591,9470\nATCGTTAGCTAGCGGA-1,1,58,98,9471,9538\nACAGCGACATTCTCAT-1,1,59,99,9591,9608\nATTCAACCATTTAAGG-1,1,58,100,9471,9676\nCACTCTTCTGCTAGCC-1,1,59,101,9591,9745\nCTAACTGATAATCGCC-1,1,58,102,9471,9814\nTGATGTCAATTAAGTG-1,1,59,103,9590,9883\nCGCCGTTCAGCATAGT-1,1,58,104,9470,9951\nATACTACCCGTACCAC-1,1,59,105,9590,10020\nCCACTGTTTGGATTAA-1,1,58,106,9470,10089\nCACTACGGGAGCTGCC-1,1,59,107,9589,10158\nGACCAAACGTTGACTG-1,1,58,108,9470,10226\nAAGAGGCCCTTTGGAA-1,1,59,109,9589,10295\nACACGTAGGCCACAAG-1,1,58,110,9469,10364\nTACTCTTACTTTACTG-1,1,59,111,9589,10433\nAGAGCCGCCGAGATTT-1,1,58,112,9469,10502\nGGCCTGCTTCTCCCGA-1,1,59,113,9588,10571\nTCAAGCGCGGACGGTA-1,1,58,114,9469,10639\nACACTGGGACAGTCGT-1,1,59,115,9588,10708\nTGTGTAGTAGCACGTG-1,1,58,116,9468,10777\nCTAGCCACAGGCGAGC-1,1,59,117,9588,10846\nCTATCGCGTAGAGAAC-1,1,58,118,9468,10914\nGTTCCCGTAAACATAT-1,1,59,119,9587,10983\nTCTTTCCTTCGAGATA-1,1,58,120,9468,11052\nATATTGGAGAGGCCTT-1,1,59,121,9587,11121\nAGGCGATAACTGGCGT-1,1,58,122,9467,11189\nACCCGGAGCACCACAA-1,1,59,123,9587,11259\nTGAGCTCAACTGTATA-1,1,58,124,9467,11327\nACGACCCATGAGTTGC-1,0,59,125,9586,11396\nCGGTCAGTCCATATTT-1,0,58,126,9466,11465\nTCCGGCTGTCGGGTCG-1,0,59,127,9586,11534\nATTCCCGAAGGTACAG-1,1,60,0,9728,2798\nCTAACAGCACAATAAC-1,1,61,1,9847,2867\nTTGTGCGGAAGCGGAT-1,1,60,2,9728,2935\nGGGCACTATTGACCAT-1,1,61,3,9847,3004\nCCATAGAGGCTGCCAG-1,1,60,4,9727,3073\nCCGAAGCATTGACCAA-1,1,61,5,9847,3142\nTACTGAGGGAAGAAAG-1,1,60,6,9727,3210\nTATGTAGAAACCCGGC-1,1,61,7,9846,3279\nTGGTTAACTTACATTT-1,1,60,8,9726,3348\nAGAACCCTCAATTGGG-1,1,61,9,9846,3417\nCCTATATCGTGTCACG-1,1,60,10,9726,3486\nACGGCTGGATGTAGAA-1,1,61,11,9846,3555\nCTCGCATTGCATAGCC-1,1,60,12,9726,3623\nACTTTGGTCGTGCTCC-1,1,61,13,9845,3692\nGCTGGTTTAGGCCATA-1,1,60,14,9725,3761\nTGTCGTTATCACATAT-1,1,61,15,9845,3830\nACACATGATCAAATCT-1,1,60,16,9725,3898\nTATCCATCTCGGTTAG-1,1,61,17,9845,3967\nCTCGTCGAGGGCTCAT-1,1,60,18,9725,4036\nATCAATGCCGTGGCTG-1,1,61,19,9844,4105\nGAAACATAGGAAACAG-1,1,60,20,9724,4173\nTAATAGGTCACCAGAA-1,1,61,21,9844,4243\nCACCAGTCAGCATGCA-1,1,60,22,9724,4311\nGAGCGCTGTTAGGTAA-1,1,61,23,9844,4380\nCCCTATGTAGAGCAGA-1,1,60,24,9724,4449\nGATCGCTATATCTCAG-1,1,61,25,9843,4518\nAAGGGACTATGCATTC-1,1,60,26,9723,4586\nCTCGGGTTCTCTGGCC-1,1,61,27,9843,4655\nTATATATCGAGAAATG-1,1,60,28,9723,4724\nACCCGAGCGAAATTAC-1,1,61,29,9843,4793\nTTGTTTCATTAGTCTA-1,1,60,30,9723,4861\nATGGGACCTGCTGAAC-1,1,61,31,9842,4930\nTGCGACGGCCGAACGT-1,1,60,32,9722,4999\nAGCTAACAAGCAATGT-1,1,61,33,9842,5068\nCGTGTCTCGTTACGAC-1,1,60,34,9722,5137\nCTAAATCCGGTGTACA-1,1,61,35,9841,5206\nGAATAGTGCTCGATTA-1,1,60,36,9722,5274\nAACAATACATTGTCGA-1,1,61,37,9841,5343\nCACATCGTGCACGCGC-1,1,60,38,9721,5412\nACCGTGACCACGTGGG-1,1,61,39,9841,5481\nCCCAATGAGATTTGCA-1,1,60,40,9721,5549\nTCAGCAGTAGGCCCTG-1,1,61,41,9840,5618\nCTACTCAAGGTATAGT-1,1,60,42,9721,5687\nTAGTAGCTTATACCAG-1,1,61,43,9840,5756\nACGTTCGTTCAGGAAA-1,1,60,44,9720,5824\nTTAGAATAAGGGTCGG-1,1,61,45,9840,5893\nTTCTAGAAAGTCTTAT-1,1,60,46,9720,5962\nCCTTCAGTTAAAGTGA-1,1,61,47,9839,6031\nTAGCTAGTGATGATGG-1,1,60,48,9719,6100\nAGCCCGGTAGCCTGTA-1,1,61,49,9839,6169\nTACCTCAGTTGTCTGT-1,1,60,50,9719,6237\nGACGGGTTGGCCCGTA-1,1,61,51,9839,6306\nGAAACCTATACAAATG-1,1,60,52,9719,6375\nTCCACCTCTAGCCTTT-1,1,61,53,9838,6444\nAAGACCCAACTGAACA-1,1,60,54,9718,6512\nCATACACGGTTCCCAC-1,1,61,55,9838,6581\nACTACGCGTTAGAATT-1,1,60,56,9718,6650\nCTTGGCCAAGCTGGGA-1,1,61,57,9838,6719\nTTACCCTAGGGATTGG-1,1,60,58,9718,6787\nTTCGCACTGTACGACA-1,1,61,59,9837,6857\nTAGGTGTTCCACAGAT-1,1,60,60,9717,6925\nGAAGCCTGCACATTCC-1,1,61,61,9837,6994\nGGCTTTCAATAAGGGT-1,1,60,62,9717,7063\nTCCAGGGTATATACGA-1,1,61,63,9837,7132\nCTCGCCGAATGTAGGG-1,1,60,64,9717,7200\nAGGAAGCTGTCCGCCG-1,1,61,65,9836,7269\nGTCTCAAGGCCCGGCT-1,1,60,66,9716,7338\nACCGACACATCTCCCA-1,1,61,67,9836,7407\nTATACGCGTCATCACT-1,1,60,68,9716,7475\nATCTGGGCTGTTCTTG-1,1,61,69,9836,7544\nTCAAATTTGAGACTCA-1,1,60,70,9716,7613\nGTTAAGGGTGCGATGT-1,1,61,71,9835,7682\nAGCAAAGGCCGCTAGT-1,1,60,72,9715,7751\nTTATAGGTAATTGTCT-1,1,61,73,9835,7820\nTTGTGCAGCCACGTCA-1,1,60,74,9715,7888\nACGGACTCTCAAAGCG-1,1,61,75,9834,7957\nATGCTCTGGCGCGGTA-1,1,60,76,9715,8026\nATAATCTTGGAGAACC-1,1,61,77,9834,8095\nCTCATTGCTCTAACAA-1,1,60,78,9714,8163\nTAGGTGACGATAACCT-1,1,61,79,9834,8232\nCGGATCCTCAAGGACT-1,1,60,80,9714,8301\nGGGCGTGGTTTCCCAG-1,1,61,81,9833,8370\nTGGCTATGTGACATAC-1,1,60,82,9714,8438\nCTTAGTGTAGTAGCAT-1,1,61,83,9833,8508\nGGATTGCTGTGACTCC-1,1,60,84,9713,8576\nACTTCCATGCGGGACA-1,1,61,85,9833,8645\nACTCAAGTGCAAGGCT-1,1,60,86,9713,8714\nACAAGGATGCTTTAGG-1,1,61,87,9832,8783\nTTGAACGACGTGCTGA-1,1,60,88,9712,8851\nATTCATCGTTGAGGCA-1,1,61,89,9832,8920\nTGCAACCCATCTGCGG-1,1,60,90,9712,8989\nAGTGCTAAACACAGCA-1,1,61,91,9832,9058\nAACTCCAGAGCGTGTT-1,1,60,92,9712,9126\nTCTCCACAAGTTGAAT-1,1,61,93,9831,9195\nCTTTGCATCGCTCTTG-1,1,60,94,9711,9264\nTCACAAACCGAGGTAC-1,1,61,95,9831,9333\nCCGTAGGAAATCCCTG-1,1,60,96,9711,9401\nAAACATTTCCCGGATT-1,1,61,97,9831,9471\nGCCTCCGACAATTCAC-1,1,60,98,9711,9539\nTCTGCATACCTTGCTT-1,1,61,99,9830,9608\nACCCATTTGTCCCTCT-1,1,60,100,9710,9677\nCTGCAGAGAATCAGAG-1,1,61,101,9830,9746\nGGAAACTAAATGGGCC-1,1,60,102,9710,9814\nCAACCTACCGAGCAGT-1,1,61,103,9830,9883\nCCGTAGGGTTGTTTAC-1,1,60,104,9710,9952\nGGTTTGTGACCTGAGG-1,1,61,105,9829,10021\nTAACCTAGGGAGTCCA-1,1,60,106,9709,10089\nTTCCATCGACAGCGTG-1,1,61,107,9829,10158\nATCTCCCACGGAATAT-1,1,60,108,9709,10227\nGTTTCAAACGAGTTGT-1,1,61,109,9829,10296\nGGAACTTTGGCGATTA-1,1,60,110,9709,10365\nCTGCAAGCACGTTCCG-1,1,61,111,9828,10434\nCACAGTCCCGCTTCGC-1,1,60,112,9708,10502\nTACGCTCGGTATTGGA-1,1,61,113,9828,10571\nCTGGTAAAGTGTGGGC-1,1,60,114,9708,10640\nCAACCAGTGGCCTACC-1,1,61,115,9827,10709\nTCTTCGCGGTGAGAGG-1,1,60,116,9708,10777\nTTCGGGTTGCCACGGG-1,1,61,117,9827,10846\nACCGATAGGCATAACC-1,1,60,118,9707,10915\nCTTATAGATGGCTGTT-1,1,61,119,9827,10984\nAGACGCCCACTTCGCC-1,1,60,120,9707,11052\nAGGTCGCCTCCCAACA-1,1,61,121,9826,11122\nCCTCACGTCAGCTAAT-1,1,60,122,9707,11190\nCCACCTGTATGGAATA-1,1,61,123,9826,11259\nGGACTCGAAGCTTTCA-1,1,60,124,9706,11328\nAAGCCGGAGAGCAGGA-1,1,61,125,9826,11397\nGCCTATCAGGTAAGAT-1,0,60,126,9706,11465\nTCAGAGTGTGAGCATG-1,0,61,127,9825,11534\nAAACATGGTGAGAGGA-1,1,62,0,9967,2798\nATCATTGTACCGCATT-1,0,63,1,10087,2867\nTCAGCTTGAGCTTTCG-1,1,62,2,9967,2936\nCCGCGCAAGATACCCA-1,1,63,3,10086,3005\nCTATACTTAAAGCGAG-1,1,62,4,9967,3073\nACCGTCCACTGGGCCC-1,1,63,5,10086,3143\nAAGACCAAATAACTCA-1,1,62,6,9966,3211\nTGTTTCTGAAGCGTGC-1,1,63,7,10086,3280\nTTCTCTTACAGGTGAT-1,1,62,8,9966,3349\nAATTCATTGTCATGCA-1,1,63,9,10085,3418\nGCCCGAGAGTCTAAAT-1,1,62,10,9966,3486\nATATAGAGTATTGGTC-1,1,63,11,10085,3555\nTTGAAGGATGGGCGCC-1,1,62,12,9965,3624\nTAGTACCACAACTTTC-1,1,63,13,10085,3693\nATCCCATCCACAGCGC-1,1,62,14,9965,3761\nCTTAGGTATAGACCAG-1,1,63,15,10084,3830\nTGACTCCGAATCATAC-1,1,62,16,9964,3899\nACTACATCCCGACAAG-1,1,63,17,10084,3968\nTTGCCGCAGACCTACA-1,1,62,18,9964,4036\nTAAGAGGGACAGGGAC-1,1,63,19,10084,4106\nCGAATGAAGTCATTGC-1,1,62,20,9964,4174\nAAGGGAACGACTGGCT-1,1,63,21,10083,4243\nCGAATCTGCTCGACGC-1,1,62,22,9963,4312\nATCGTGGAAAGTCTGG-1,1,63,23,10083,4381\nCCTATGGCTCCTAGTG-1,1,62,24,9963,4449\nCGTCGCTTGGTTATAC-1,1,63,25,10083,4518\nTGCTTAGAGAGAATGC-1,1,62,26,9963,4587\nGATTGCTCCAGTTGCA-1,1,63,27,10082,4656\nCGTGCTTCTACCTAAA-1,1,62,28,9962,4724\nACTCCAATATCATCAT-1,1,63,29,10082,4793\nGCCGACCCACGACTGC-1,1,62,30,9962,4862\nGAGGGTAGTAACAAAG-1,1,63,31,10082,4931\nGGCCGAGACTCTGGTG-1,1,62,32,9962,5000\nTATATAGGGCTTTACG-1,1,63,33,10081,5069\nTGCCGGATGTACGAGC-1,1,62,34,9961,5137\nAGGTAGATCGAGATAT-1,1,63,35,10081,5206\nGATCAACATAAAGGGA-1,1,62,36,9961,5275\nTAACAGCGTTTGTGCT-1,1,63,37,10080,5344\nCTATAGGCGTTGATGT-1,1,62,38,9961,5412\nGGTTTAATTACCATCG-1,1,63,39,10080,5481\nCTCTTCTGGAAGTTAG-1,1,62,40,9960,5550\nATCTGACATGGAAGGA-1,1,63,41,10080,5619\nTACGTACTAGTGCTGA-1,1,62,42,9960,5687\nATGACTATCAGCTGTG-1,1,63,43,10079,5757\nCTTTGGGATTGTTGCA-1,1,62,44,9960,5825\nAGACTGTTACCGGGTC-1,1,63,45,10079,5894\nTCCTAACCGTCGGGCA-1,1,62,46,9959,5963\nACGCAAACTAATAGAT-1,1,63,47,10079,6032\nAAGTGAGTCGGGTTTA-1,1,62,48,9959,6100\nTAAAGCGGTATTTCCA-1,1,63,49,10078,6169\nCTCCTCCAGCTCACAC-1,1,62,50,9959,6238\nCGGTCCGTCGCAAGCC-1,1,63,51,10078,6307\nCCTATATTTGTCCTGG-1,1,62,52,9958,6375\nTCCAATAAAGGCTACC-1,1,63,53,10078,6444\nAAAGGCTACGGACCAT-1,1,62,54,9958,6513\nCGTGACCAGTCCTCTG-1,1,63,55,10077,6582\nGATCTGCTATCTAAGG-1,1,62,56,9957,6650\nCGAATGACGCATAATG-1,1,63,57,10077,6720\nGCTCAATGTAATACCG-1,1,62,58,9957,6788\nCTATGCCCGAATGCAA-1,1,63,59,10077,6857\nCACAGGGCCGTTGTCA-1,1,62,60,9957,6926\nTCAATACGCCGTCATG-1,1,63,61,10076,6995\nAGCGGACACTTCGTAG-1,1,62,62,9956,7063\nATACGTTATGCACGGA-1,1,63,63,10076,7132\nGAGTAGATACTAGTTG-1,1,62,64,9956,7201\nCCAGCTCGAACGCATT-1,1,63,65,10076,7270\nTGCTTCCCAAGCAGTA-1,1,62,66,9956,7338\nGTAGACACGCCTGACT-1,1,63,67,10075,7407\nCTCACATTTACTAAAT-1,1,62,68,9955,7476\nGAATGCGAATCGGTTC-1,1,63,69,10075,7545\nTAAGGCCCGTCACCCT-1,1,62,70,9955,7614\nGAGTATGCGCGTGCAT-1,1,63,71,10075,7683\nATCCACATCGACAGAA-1,1,62,72,9955,7751\nTAGACGAAACGCCAAT-1,1,63,73,10074,7820\nCAGCAGTCTGTGCTGC-1,1,62,74,9954,7889\nCTCACTGTGATACTTA-1,1,63,75,10074,7958\nTTAATTGCTTTGGGTG-1,1,62,76,9954,8026\nGTAGTCTACGATATTG-1,1,63,77,10074,8095\nTTGCCGCTTTCTAGTA-1,1,62,78,9954,8164\nGAACCTTTAACGATCC-1,1,63,79,10073,8233\nTGGACCAATCTAAGAT-1,1,62,80,9953,8301\nCTAGTAGAAAGGGATT-1,1,63,81,10073,8371\nAACCTCGCTTTAGCCC-1,1,62,82,9953,8439\nGAGTGTCAACCAGAAA-1,1,63,83,10072,8508\nGTGATCCTTGTCATGA-1,1,62,84,9953,8577\nCTCAACTAACCCGGAT-1,1,63,85,10072,8646\nAGTGGCGTCTGAAGGT-1,1,62,86,9952,8714\nGGGAAGGGCTTTCTCA-1,1,63,87,10072,8783\nAGGCCTGAGAATCTCG-1,1,62,88,9952,8852\nAGCCACAGGTTACCCG-1,1,63,89,10071,8921\nTGCAATCTAACACGGT-1,1,62,90,9952,8989\nGTTTGGGTTTCGCCCG-1,1,63,91,10071,9058\nCCTATGTCCACTCCAC-1,1,62,92,9951,9127\nCTGCCTTTCTAGTAAA-1,1,63,93,10071,9196\nTTAGTAAACCTGCTCT-1,1,62,94,9951,9265\nTTGTGGTGGTACTAAG-1,1,63,95,10070,9334\nCCTCCTGTTGTGTCGT-1,1,62,96,9950,9402\nACAAAGCATGACCTAG-1,1,63,97,10070,9471\nATAGCAACTAGGGAAG-1,1,62,98,9950,9540\nCCGCTCTTCCGAACTA-1,1,63,99,10070,9609\nTAGAATAGCCGATGAA-1,1,62,100,9950,9677\nGGTCTCCGTCCAGGTT-1,1,63,101,10069,9746\nTAATTACGTCAGTAGA-1,1,62,102,9949,9815\nTACGTAAAGCGGAGTG-1,1,63,103,10069,9884\nGCTACAATCGAGGATA-1,1,62,104,9949,9952\nTCAGCCAATCCGTAAA-1,1,63,105,10069,10022\nATCATAGCCCTATGTA-1,1,62,106,9949,10090\nGAGGCCTGTTGATACA-1,1,63,107,10068,10159\nATATACGCTCGTGACG-1,1,62,108,9948,10228\nCCCTTTAATGGAGTTC-1,1,63,109,10068,10297\nTTACACGATCTGCGAC-1,1,62,110,9948,10365\nTCGCATAAAGGGCGCA-1,1,63,111,10068,10434\nTGTAGCCATCCCATTC-1,1,62,112,9948,10503\nGGGTCTATCGCTTTCC-1,1,63,113,10067,10572\nGGAGGGTCAAGTAAGA-1,1,62,114,9947,10640\nTACGGCATGGACGCTA-1,1,63,115,10067,10709\nCACGTCAATCAATGGA-1,1,62,116,9947,10778\nCGCTTGGACGGATAGA-1,1,63,117,10067,10847\nCGCGGTACGGTATACA-1,1,62,118,9947,10915\nCCTACTCAACACGATT-1,1,63,119,10066,10985\nCGTTTGGTGTTGTGGG-1,1,62,120,9946,11053\nGTTGTTGCAAGATGGC-1,1,63,121,10066,11122\nCATTCGTCGTAGCGGT-1,1,62,122,9946,11191\nCAGCCTCGATAGCGGT-1,1,63,123,10065,11260\nTCATAAGTCCAAGAAG-1,1,62,124,9946,11328\nAAAGTGCCATCAATTA-1,0,63,125,10065,11397\nGTTGCAGTCGACAACA-1,0,62,126,9945,11466\nTAAATGGGCTACTGAG-1,0,63,127,10065,11535\nGGACAAGCCATGATCG-1,0,64,0,10207,2799\nTCCAATGCGTCGCCGC-1,0,65,1,10326,2868\nAAAGAATGACCTTAGA-1,0,64,2,10206,2936\nCCTATTATTCCGGCCG-1,0,65,3,10326,3006\nCAGTAAGGGACGTCTC-1,1,64,4,10206,3074\nGGGATGGACCCGCGTC-1,1,65,5,10325,3143\nGACGCATACCCGTCGG-1,1,64,6,10206,3212\nGGGCAGGATTTCTGTG-1,1,65,7,10325,3281\nTATCTGAGCCGATATT-1,1,64,8,10205,3349\nAATGTATGGCACTGTA-1,1,65,9,10325,3418\nGCCTTTGTCAGTGGAC-1,1,64,10,10205,3487\nAACCATAGGGTTGAAC-1,1,65,11,10324,3556\nAAACTTAATTGCACGC-1,1,64,12,10205,3624\nTAGGCGGCTGCAAAGA-1,1,65,13,10324,3693\nTCCATCAATACTAATC-1,1,64,14,10204,3762\nACCTCGAACTTATGCT-1,1,65,15,10324,3831\nACAAACCATGCGTCCT-1,1,64,16,10204,3899\nACACCGGTCTGACCGC-1,1,65,17,10323,3969\nATGGCAGCATTACGAT-1,1,64,18,10204,4037\nCTGGTGAATGGGCCTA-1,1,65,19,10323,4106\nCACAGAGACGAGGACG-1,1,64,20,10203,4175\nGCACAAACCCTAGATG-1,1,65,21,10323,4244\nACTATGTCCAGCTGCC-1,1,64,22,10203,4312\nAAGCCGCCGTGAAATC-1,1,65,23,10322,4381\nATGGGTGTATACCTCC-1,1,64,24,10202,4450\nTAAGCGCGAATCAAAT-1,1,65,25,10322,4519\nAGCCCGGCATTAGAGG-1,1,64,26,10202,4587\nGTGATCAAGCGTGCAC-1,1,65,27,10322,4656\nCTGCCTTTAATACCTT-1,1,64,28,10202,4725\nTACCTCCACACCAATG-1,1,65,29,10321,4794\nGAGCATCAACAACTTG-1,1,64,30,10201,4863\nCATACGAACTAGCTGG-1,1,65,31,10321,4932\nGGGAACGGTTTCAGAT-1,1,64,32,10201,5000\nAGGGTGCCGTTCTTTA-1,1,65,33,10321,5069\nACGACTGGTCATACTC-1,1,64,34,10201,5138\nGGATCTTACTGCCCTT-1,1,65,35,10320,5207\nTACTCCTCTAGTTGAC-1,1,64,36,10200,5275\nGTAGCTCCGGGAGGCT-1,1,65,37,10320,5344\nGGACTCGTGAGTGGTC-1,1,64,38,10200,5413\nGTAGTTCGAAGGCGAA-1,1,65,39,10320,5482\nCTCAAACCACTGCCCG-1,1,64,40,10200,5550\nGAGGGAGTCAGATCGC-1,1,65,41,10319,5620\nGCGTCCCTAAGACATA-1,1,64,42,10199,5688\nTCTCTTACCGCGAACC-1,1,65,43,10319,5757\nGGCTACTATACACTCC-1,1,64,44,10199,5826\nCGTTGCCCGCGTGGGA-1,1,65,45,10318,5895\nTATAGAGTCGCTTGAA-1,1,64,46,10199,5963\nATGTTGTAGTCTGTTT-1,1,65,47,10318,6032\nAGTATACACAGCGACA-1,1,64,48,10198,6101\nTGTGGTTGCTAAAGCT-1,1,65,49,10318,6170\nTGATTCCCGGTTACCT-1,1,64,50,10198,6238\nCACTGACGATTGTGGA-1,1,65,51,10317,6307\nAACATTGTGACTCGAG-1,1,64,52,10198,6376\nGTCCAATATTTAGCCT-1,1,65,53,10317,6445\nGCTCTTTCCGCTAGTG-1,1,64,54,10197,6514\nGGGTCAGGAGCTAGAT-1,1,65,55,10317,6583\nAGGTGGTGACCTTCGC-1,1,64,56,10197,6651\nCCCGGGTCGTTCAGGG-1,1,65,57,10316,6720\nAGCCAAGCTTTGTGTC-1,1,64,58,10197,6789\nTGTGGCGGGCTTCTGG-1,1,65,59,10316,6858\nCGAGGGTATCCAGGTG-1,1,64,60,10196,6926\nCGTTCAGACCCGCGAA-1,1,65,61,10316,6995\nTCATATGAGCTTTGTT-1,1,64,62,10196,7064\nAGTTGACGGTCCTTGC-1,1,65,63,10315,7133\nATAATTAGCTAAGTAG-1,1,64,64,10195,7201\nGGTCTTGAGCGCTCTT-1,1,65,65,10315,7271\nCGTTGTCGGCAATTGA-1,1,64,66,10195,7339\nTCTTGCTCCCGATACT-1,1,65,67,10315,7408\nAGCAGAAGGAGAAAGA-1,1,64,68,10195,7477\nGAAGCCACTGATTATG-1,1,65,69,10314,7546\nTGCTCTTGAGAGTTTG-1,1,64,70,10194,7614\nGACATCCGTCGAACTG-1,1,65,71,10314,7683\nAGATATAATACGACTA-1,1,64,72,10194,7752\nTCGCTACTGGCTTTGA-1,1,65,73,10314,7821\nGTGACAGCTTCCCACT-1,1,64,74,10194,7889\nGTGACGCAGGTTTCAT-1,1,65,75,10313,7958\nGGGAAGACGGTCTGTC-1,1,64,76,10193,8027\nCGTCGCATGTGAGCCA-1,1,65,77,10313,8096\nAGAAGGTTGTAGGTCG-1,1,64,78,10193,8164\nGGCCGGCGTCTGCTAT-1,1,65,79,10313,8234\nTTGACCAGGAACAACT-1,1,64,80,10193,8302\nAGTCGTATAAAGCAGA-1,1,65,81,10312,8371\nGGCGTAGGGAAAGCTG-1,1,64,82,10192,8440\nAAACCTAAGCAGCCGG-1,1,65,83,10312,8509\nATTGCCTTTATGTTTG-1,1,64,84,10192,8577\nCAGGCAGTCTTACCAG-1,1,65,85,10311,8646\nAGCTAAGTACGCAGGC-1,1,64,86,10192,8715\nAAATCCGATACACGCC-1,1,65,87,10311,8784\nCGAGAGCTTTCACTAC-1,1,64,88,10191,8852\nCGCCCAGCGTTTCACG-1,1,65,89,10311,8921\nTTAAGTATTGTTATCC-1,1,64,90,10191,8990\nAAACGGGCGTACGGGT-1,1,65,91,10310,9059\nGTATCCTTTGGTAACC-1,1,64,92,10191,9128\nCATAGTCAAATACATA-1,1,65,93,10310,9197\nTCTATGCTATAACGAC-1,1,64,94,10190,9265\nACGTTCTGTACAAGTC-1,1,65,95,10310,9334\nCGCGAAGTGGCATACT-1,1,64,96,10190,9403\nACCAACCGCACTCCAC-1,1,65,97,10309,9472\nGTCATGCGCGAGGGCT-1,1,64,98,10190,9540\nTAGGCTAAAGTGGCAC-1,1,65,99,10309,9609\nTGGCTACACTCTACCT-1,1,64,100,10189,9678\nTGTCCACGGCTCAACT-1,1,65,101,10309,9747\nAATTGCAGCAATCGAC-1,1,64,102,10189,9815\nGTCAACCAGGCCTATA-1,1,65,103,10308,9885\nATCTTATCGCACACCC-1,1,64,104,10188,9953\nGTCTATTGCATGCTCG-1,1,65,105,10308,10022\nGCACGTGGTTTACTTA-1,1,64,106,10188,10091\nATTCGCGCCTTGAGAG-1,1,65,107,10308,10160\nAGCGATGCGCCTAATA-1,1,64,108,10188,10228\nTCTATAGGTGGGTAAT-1,1,65,109,10307,10297\nAGATACCAATAGAACC-1,1,64,110,10187,10366\nCGTGTATGGGAGCTGA-1,1,65,111,10307,10435\nATGTACATGCGGTGAG-1,1,64,112,10187,10503\nTTGGCTCAATATGTGT-1,1,65,113,10307,10572\nGTTCGGAGCACTCAAC-1,1,64,114,10187,10641\nTCCCTCTTCTCAAGGG-1,1,65,115,10306,10710\nTTACGTATCTATGACA-1,1,64,116,10186,10779\nCCCATATAGGTCGATT-1,1,65,117,10306,10848\nGTCCACGTCGCATTCG-1,1,64,118,10186,10916\nAATTGTGGTTGCCAAA-1,1,65,119,10306,10985\nAGGAACGAACGACTTC-1,1,64,120,10186,11054\nTAATTCCAATGCTTCA-1,1,65,121,10305,11123\nAGGGCCCTAATGTTCT-1,1,64,122,10185,11191\nAGGTGTTGCCGACCAC-1,1,65,123,10305,11260\nCTGAGAGTAGAAATAC-1,1,64,124,10185,11329\nTAGAAGAAGGGTTACA-1,1,65,125,10304,11398\nGGTGACTGATAGAGAG-1,0,64,126,10185,11466\nTACCCTCGGTAACCCT-1,0,65,127,10304,11536\nTGTAAGACTGATAAGA-1,0,66,0,10446,2799\nGTCACTCATGAGCGAT-1,0,67,1,10566,2869\nCGGTTTACTGAACATT-1,0,66,2,10446,2937\nGAGATCGATCTTACTC-1,0,67,3,10565,3006\nTCTTTCATCCGTCCTT-1,0,66,4,10445,3075\nTCGGATCTGGATGACC-1,0,67,5,10565,3144\nGTCCTAGGATACCTTA-1,1,66,6,10445,3212\nTGTACCCGACCCTAAT-1,1,67,7,10565,3281\nAGGTTCTCCTTTCCGG-1,1,66,8,10445,3350\nAATTGCGTGGATTACA-1,1,67,9,10564,3419\nCGATAGCGATACAGTG-1,1,66,10,10444,3487\nCGCCCTAATTGTTCAA-1,1,67,11,10564,3556\nATTATGAGACCCAATT-1,1,66,12,10444,3625\nCCTCCTAGCTAGAGTC-1,1,67,13,10563,3694\nCGAAGGGTTTCAGATT-1,1,66,14,10444,3763\nCACAAGCTAAGAAAGG-1,1,67,15,10563,3832\nCACCGGTAGAGACATT-1,1,66,16,10443,3900\nACTCATGGCAGCCTTC-1,1,67,17,10563,3969\nCATCCAATATAGTTTG-1,1,66,18,10443,4038\nCAGATATGAAGATGAC-1,1,67,19,10562,4107\nGCAAAGGGCGTTAGCC-1,1,66,20,10443,4175\nCTGTTGTTCAGTCGTA-1,1,67,21,10562,4244\nAAAGGCCCTATAATAC-1,1,66,22,10442,4313\nAACATTGAAGTTGATC-1,1,67,23,10562,4382\nGCTAGCGATAGGTCTT-1,1,66,24,10442,4450\nATTGCCTATTAGACCG-1,1,67,25,10561,4520\nGGGCGTTTACATTCAT-1,1,66,26,10442,4588\nCCTCTACAAGGATTGG-1,1,67,27,10561,4657\nAGTGACTGTGACACAA-1,1,66,28,10441,4726\nAGCCCTCCCTGGTGGC-1,1,67,29,10561,4795\nCGGTTGAGTATCCTTC-1,1,66,30,10441,4863\nCATTATCCCATTAACG-1,1,67,31,10560,4932\nAATGTGCTAATCTGAG-1,1,66,32,10440,5001\nCTTAGTTGAGGAATCG-1,1,67,33,10560,5070\nCTTCTGGGCGTACCTA-1,1,66,34,10440,5138\nAGTCAAATGATGTGAT-1,1,67,35,10560,5207\nTGAACAAGCAGGGACT-1,1,66,36,10440,5276\nAATACCGGAGGGCTGT-1,1,67,37,10559,5345\nGTTTGGTGATCGGTGC-1,1,66,38,10439,5413\nGACTGCTGGTGAGAAA-1,1,67,39,10559,5483\nCGATCCTCGCAACATA-1,1,66,40,10439,5551\nCCAACCTTATGTAACT-1,1,67,41,10559,5620\nCGTGGTACCCAAAGGC-1,1,66,42,10439,5689\nAACCCGACAACCCGTG-1,1,67,43,10558,5758\nTATGTAAAGTGCTTAA-1,1,66,44,10438,5826\nGCAAGTGTAAAGCATG-1,1,67,45,10558,5895\nGTAACTGCCCAAGGAG-1,1,66,46,10438,5964\nGGTGCATAAATGATTA-1,1,67,47,10558,6033\nTAGTCCCGGAGACCAC-1,1,66,48,10438,6101\nCGGCCAGAGCGACCAT-1,1,67,49,10557,6170\nTGAATGAGTGTTTCCC-1,1,66,50,10437,6239\nTTCACTTCCTAGAACG-1,1,67,51,10557,6308\nCGCTTAGTATTGATAC-1,1,66,52,10437,6377\nAACGCGACCTTGGGCG-1,1,67,53,10556,6446\nGTGCACCAGCTTCAAC-1,1,66,54,10437,6514\nTTCTATTAAACGCAGC-1,1,67,55,10556,6583\nTGGCAAGCACAAGTCG-1,1,66,56,10436,6652\nATCTGCTGTTATCGCC-1,1,67,57,10556,6721\nAATTCCAAGCATGTAC-1,1,66,58,10436,6789\nAAACGGTTGCGAACTG-1,1,67,59,10555,6858\nGAAGGAGTCGAGTGCG-1,1,66,60,10436,6927\nGGCAAGCCCATAGTGG-1,1,67,61,10555,6996\nTGACCAAATCTTAAAC-1,1,66,62,10435,7064\nTTCGGTACTGTAGAGG-1,1,67,63,10555,7134\nATGTACGATGACGTCG-1,1,66,64,10435,7202\nATCTAGCTTGTGAGGG-1,1,67,65,10554,7271\nAGCGGGTCTGACACTC-1,1,66,66,10435,7340\nGAGGTACGCGTGTCCC-1,1,67,67,10554,7409\nGCACTGCCTACCTTTA-1,1,66,68,10434,7477\nGGAGCGAGGCCTACTT-1,1,67,69,10554,7546\nAAACTCGGTTCGCAAT-1,1,66,70,10434,7615\nGGCGCTTCATTCCCTG-1,1,67,71,10553,7684\nTGCTCGGTGGGTCACC-1,1,66,72,10433,7752\nAAGCGCAGGGCTTTGA-1,1,67,73,10553,7821\nGACCTTCCACGTCTAC-1,1,66,74,10433,7890\nGTTAGCCCATGACATC-1,1,67,75,10553,7959\nAATTCGATTCGAGGAT-1,1,66,76,10433,8028\nAGTCTTTAAAGTGTCC-1,1,67,77,10552,8097\nGACTAGGCCGTTAGGT-1,1,66,78,10432,8165\nTCCAAGCCTAGACACA-1,1,67,79,10552,8234\nGATTGGGAAAGGTTGT-1,1,66,80,10432,8303\nGATGAGGAACCTTCGG-1,1,67,81,10552,8372\nTGAATACCGACGCGTA-1,1,66,82,10432,8440\nAATCGTGAGCCGAGCA-1,1,67,83,10551,8509\nGCGAAGAATCTGACGG-1,1,66,84,10431,8578\nGAACAACTGGGATGAA-1,1,67,85,10551,8647\nTCGCCGACATATTCGC-1,1,66,86,10431,8715\nCGTAGAGAGTAATTAT-1,1,67,87,10551,8785\nCGATAGTCGTACTGCA-1,1,66,88,10431,8853\nCGCATGCCGAATGCGT-1,1,67,89,10550,8922\nCTCTAGCCCTCGGAAA-1,1,66,90,10430,8991\nTCCAACTCAGCTATCT-1,1,67,91,10550,9060\nGTCTAGTGAGCCGCTT-1,1,66,92,10430,9128\nTACCGAATAATTGTAA-1,1,67,93,10549,9197\nATATAACACGGGCGCA-1,1,66,94,10430,9266\nGCATGCTAATAACGAT-1,1,67,95,10549,9335\nTCCGGGCTTGACGGGA-1,1,66,96,10429,9403\nATTATTCAGAGTCACT-1,1,67,97,10549,9472\nTAGCAGTATGACTAAA-1,1,66,98,10429,9541\nTATGAAGAATTAAGGT-1,1,67,99,10548,9610\nCAAGGCCAGTGGTGCA-1,1,66,100,10429,9678\nTGCGTACGGCTAATTG-1,1,67,101,10548,9748\nCAGATAATGGGCGGGT-1,1,66,102,10428,9816\nGAGAACGGTTCTGACT-1,1,67,103,10548,9885\nCGCTTCGGTCTAAGAC-1,1,66,104,10428,9954\nAGCATTACGAGGCAAG-1,1,67,105,10547,10023\nAAGTTGTGATGTTATA-1,1,66,106,10428,10091\nTTGCTCATAGTACGTG-1,1,67,107,10547,10160\nTACTTAAACATGTACA-1,1,66,108,10427,10229\nAGTGAGTCGAATTAAC-1,1,67,109,10547,10298\nTAGGCCTATATAGTCT-1,1,66,110,10427,10366\nACTCGCCGTTCGATAA-1,1,67,111,10546,10435\nGGTACGTTGCGGCCGG-1,1,66,112,10426,10504\nTATTGCCGGGCTTGTA-1,1,67,113,10546,10573\nGGTAAACTCTGCGCTG-1,1,66,114,10426,10642\nTCCCTTGTCTGAAACT-1,1,67,115,10546,10711\nCATGATCGCTTTGAGA-1,1,66,116,10426,10779\nGGACGGGCGACCAACC-1,1,67,117,10545,10848\nGAGAGGCCTATGTGTA-1,1,66,118,10425,10917\nTCGTAACTCCCAAGAC-1,1,67,119,10545,10986\nCGTATCTAGAACTAAG-1,1,66,120,10425,11054\nGCACCTTCCCGAAGGT-1,1,67,121,10545,11123\nTACCTCTTTACCATCC-1,1,66,122,10425,11192\nTGACTCTAACTGGTAA-1,1,67,123,10544,11261\nGAGAAACTGGATCCCA-1,1,66,124,10424,11329\nATCCGGACCAGCCTGA-1,1,67,125,10544,11399\nTTATTTAGGTTCCTTA-1,0,66,126,10424,11467\nCGAATTCCCGGTTCAA-1,0,67,127,10544,11536\nTAGTTTGATCGGTCGC-1,0,68,0,10685,2800\nAGTCGAAACGATTCAG-1,0,69,1,10805,2869\nACGTCGGGCAACTCGG-1,0,68,2,10685,2938\nACTTGCTCTATCTACC-1,0,69,3,10805,3007\nATTACGGGCTACGGTT-1,0,68,4,10685,3075\nGCAAACGTAAGCGACC-1,0,69,5,10804,3144\nTTCGGACTGGGCATGG-1,0,68,6,10684,3213\nGGGCCATTCGTGCTGG-1,0,69,7,10804,3282\nGAGTTGTCACCAGTCT-1,1,68,8,10684,3350\nAGTAATTTGCAAGCGT-1,1,69,9,10804,3419\nGTGGCCGGTTTCTCGG-1,1,68,10,10684,3488\nATCTTAGGGCATTAAT-1,1,69,11,10803,3557\nCTACATATCGCGGGAC-1,1,68,12,10683,3626\nATGGTATTTACTGATT-1,1,69,13,10803,3695\nCCGTCAACCTCTGGCG-1,1,68,14,10683,3763\nGCTTCGACGTTCAATC-1,1,69,15,10803,3832\nGACCGATTAAATATGT-1,1,68,16,10683,3901\nGTCGAATTTGGGCGCT-1,1,69,17,10802,3970\nTCACGGCCCAAGAGAG-1,1,68,18,10682,4038\nCTACGATCCTATCCTA-1,1,69,19,10802,4107\nTGCAGTGGTAGGGAAC-1,1,68,20,10682,4176\nTTGCCTAATCCAAAGT-1,1,69,21,10801,4245\nAGCGAGACGTGAAGGC-1,1,68,22,10682,4313\nTTCTACCTCAATCGGT-1,1,69,23,10801,4383\nTACGCTGCTGTGTTAA-1,1,68,24,10681,4451\nTAGTCCTGCACTAAGC-1,1,69,25,10801,4520\nAGCACACGTTTAGACT-1,1,68,26,10681,4589\nGTTCGACAATTGTATA-1,1,69,27,10800,4658\nCCATGTTCATCTATAT-1,1,68,28,10681,4726\nCCCTCGATAATACACA-1,1,69,29,10800,4795\nACTGCCCGCCATTCTC-1,1,68,30,10680,4864\nCTATATCCAGCCTGGC-1,1,69,31,10800,4933\nCGCACGTCTGTTTATG-1,1,68,32,10680,5001\nAATCAGGTTTCATTTA-1,1,69,33,10799,5070\nAGAGAAACACCAGAAA-1,1,68,34,10679,5139\nACCCGATTGGTTCCGA-1,1,69,35,10799,5208\nTACCAGGAATCCCGTC-1,1,68,36,10679,5277\nAAGTGCTTCTCTATTG-1,1,69,37,10799,5346\nTGCATGTGACCCATAG-1,1,68,38,10679,5414\nTAGATTCAAAGTGCGG-1,1,69,39,10798,5483\nAGGGTCAGAGCACTCG-1,1,68,40,10678,5552\nGTGAGATAACCTTATA-1,1,69,41,10798,5621\nGTGTCAGTGTACGTGG-1,1,68,42,10678,5689\nCTGTAGTGAGGATCGA-1,1,69,43,10798,5758\nGATAGTGCGAGTAAGT-1,1,68,44,10678,5827\nAGGGTTCAGACGGTCC-1,1,69,45,10797,5896\nGGACTCGACAGCGCAT-1,1,68,46,10677,5964\nCACTCTCAAGCATCGA-1,1,69,47,10797,6034\nGAGCGGAATGCGGTGT-1,1,68,48,10677,6102\nCAGTTCGAGGACCCGA-1,1,69,49,10797,6171\nCGTAACTTCGACACTT-1,1,68,50,10677,6240\nTGATCACCACACTGAC-1,1,69,51,10796,6309\nCATACTTCTTTCTCCG-1,1,68,52,10676,6377\nCAAGTAAGTGATAGAC-1,1,69,53,10796,6446\nAGCCCGCAACAAGCAG-1,1,68,54,10676,6515\nGCTCCATGCAAAGCAA-1,1,69,55,10796,6584\nTCGTACCGACGTCAAG-1,1,68,56,10676,6652\nATTATCGGAATGTACG-1,1,69,57,10795,6721\nACCTACTATAAATCTA-1,1,68,58,10675,6790\nTATCCTATCAACTGGT-1,1,69,59,10795,6859\nGCGCGGTCTAGTAACT-1,1,68,60,10675,6927\nCCTTCGTATAGAATCC-1,1,69,61,10794,6997\nTGGGCGATACAATAAG-1,1,68,62,10675,7065\nTGACATGTAACGTGAC-1,1,69,63,10794,7134\nTTGGGAAGACGAGCCG-1,1,68,64,10674,7203\nGGTGTAAATCGATTGT-1,1,69,65,10794,7272\nGAACTGTGGAGAGACA-1,1,68,66,10674,7340\nTCCACATCGTATATTG-1,1,69,67,10793,7409\nCCTATGGTCAAAGCTG-1,1,68,68,10674,7478\nTCCTTGTCCTTTAATT-1,1,69,69,10793,7547\nAAGTTTATGGGCCCAA-1,1,68,70,10673,7615\nGTCCGGGTTCACATTA-1,1,69,71,10793,7684\nATGTAGCGCGCGTAGG-1,1,68,72,10673,7753\nTACGGAAGCCAAACCA-1,1,69,73,10792,7822\nGCCCTGAGGATGGGCT-1,1,68,74,10673,7891\nCGGGCGATGGATCACG-1,1,69,75,10792,7960\nTGCGGACTTGACTCCG-1,1,68,76,10672,8028\nTAATACTAGAACAGAC-1,1,69,77,10792,8097\nTCAAATTGTTGTGCCG-1,1,68,78,10672,8166\nGGGCAGTCAACGCCAA-1,1,69,79,10791,8235\nTCGCTGCCAATGCTGT-1,1,68,80,10671,8303\nTACGTGGGCCCAGGGC-1,1,69,81,10791,8372\nGCCTACGTTCTGTGCA-1,1,68,82,10671,8441\nTTGCGTGTGTAGGCAT-1,1,69,83,10791,8510\nTCCTAAAGATTCAGAC-1,1,68,84,10671,8578\nTGGGTGCACAAGCCAT-1,1,69,85,10790,8648\nGGGAGCGACCGTAGTG-1,1,68,86,10670,8716\nCATAGAGGAGATACTA-1,1,69,87,10790,8785\nCTTCGATTGCGCAAGC-1,1,68,88,10670,8854\nGTGCCTGAGACCAAAC-1,1,69,89,10790,8923\nTCATCCTCAGCTGCTT-1,1,68,90,10670,8991\nTGCAACTACTGGTTGA-1,1,69,91,10789,9060\nATGTTGATTAGAGACT-1,1,68,92,10669,9129\nGTTTCTAGAGGCGCGG-1,1,69,93,10789,9198\nTACAAGGGCTTCTTTA-1,1,68,94,10669,9266\nACAGGCACGGATCCTT-1,1,69,95,10789,9335\nCCCAAGTCATTACACT-1,1,68,96,10669,9404\nACTCTGACCTAATAGA-1,1,69,97,10788,9473\nGGATGGCTTGAAGTAT-1,1,68,98,10668,9542\nACCCTCCCGTCAGGGC-1,1,69,99,10788,9611\nAGCTTCTTCTCGAGCA-1,1,68,100,10668,9679\nTTGCGCTTGATCAATA-1,1,69,101,10787,9748\nTGGGTGTAATAGATTT-1,1,68,102,10668,9817\nCATGGTTTATTAATCA-1,1,69,103,10787,9886\nTTGCCTTCTCGCCGGG-1,1,68,104,10667,9954\nTTAAAGTAAGTCGCCA-1,1,69,105,10787,10023\nAGTTGGCAAGGCTAGA-1,1,68,106,10667,10092\nCCGCACTTGCAATGAC-1,1,69,107,10786,10161\nAATGTTGTCGTGAGAC-1,1,68,108,10667,10229\nCCAAGGTTGCCCTTTC-1,1,69,109,10786,10299\nTGTGACTAGAGTTTGC-1,1,68,110,10666,10367\nTTGTGATCTGTTCAGT-1,1,69,111,10786,10436\nGGAGTTGATTCTGTGT-1,1,68,112,10666,10505\nGTAGTGAGCAACCTCA-1,1,69,113,10785,10574\nAAGACATACGTGGTTT-1,1,68,114,10666,10642\nCATCCCGAGATTCATA-1,1,69,115,10785,10711\nGGCCGCAGGAACCGCA-1,1,68,116,10665,10780\nCCAGAAACTGATGCGA-1,1,69,117,10785,10849\nACCCGTAGTCTAGTTG-1,1,68,118,10665,10917\nAGGGTCTGGACGCAGT-1,1,69,119,10784,10986\nTTGAAACCCTCATTCC-1,1,68,120,10664,11055\nTCAATCCCGCGCCAAA-1,1,69,121,10784,11124\nAGGACATCGGCACACT-1,1,68,122,10664,11192\nGACAAGACGCCCGTGC-1,1,69,123,10784,11262\nGAGTAAATTAAGAACC-1,1,68,124,10664,11330\nTAAGTCGGTGAGCTAG-1,1,69,125,10783,11399\nCTTACTGACTCCTCTG-1,0,68,126,10663,11468\nTTAACTGATCGTTTGG-1,0,69,127,10783,11537\nGGGCGATCCATAGGCC-1,0,70,0,10925,2801\nCGTATTGTTTGGCGCC-1,0,71,1,11044,2870\nCGCGTGGGCCTGTGTT-1,0,70,2,10924,2938\nAAATCTGCCCGCGTCC-1,0,71,3,11044,3007\nGCTGAAGGGTTCTTGG-1,0,70,4,10924,3076\nTAATTGCGCTGATTAC-1,0,71,5,11044,3145\nTCGCGCGTTTACATGA-1,0,70,6,10924,3213\nTAACTGAAATACGCCT-1,0,71,7,11043,3283\nCCTGTCGTGTATGAAG-1,0,70,8,10923,3351\nCGGGAACGCCCTGCAT-1,0,71,9,11043,3420\nAGTTGCGGTCCTCAAC-1,1,70,10,10923,3489\nGGGTCCTTGGAAGAAG-1,1,71,11,11043,3558\nCGGGAAGTACCGTGGC-1,1,70,12,10923,3626\nACGGTCACCGAGAACA-1,1,71,13,11042,3695\nTAGACTCAGTTGGCCT-1,1,70,14,10922,3764\nCCTTCCGCAACGCTGC-1,1,71,15,11042,3833\nCAGTGTTAATCTCTCA-1,1,70,16,10922,3901\nCAGGCCAGTACCACCT-1,1,71,17,11042,3970\nGATCGCTGTGGTGCGT-1,1,70,18,10922,4039\nGTATCTCGGGCGCTTT-1,1,71,19,11041,4108\nCCGGGCTAAGAATTTC-1,1,70,20,10921,4176\nCTTCACGCCCTGGTAC-1,1,71,21,11041,4246\nCGGATGAATGCTGTGA-1,1,70,22,10921,4314\nCTCTGCAGGCATTCTT-1,1,71,23,11041,4383\nTCAAAGTCACGGCGTC-1,1,70,24,10921,4452\nTGGGTTTCGGGCGTAC-1,1,71,25,11040,4521\nTGCTATGGCAAAGGGA-1,1,70,26,10920,4589\nTTCACGGTCGTCACGT-1,1,71,27,11040,4658\nCGTAAAGCAAGAAATC-1,1,70,28,10920,4727\nCTGATCCCTTTATGCA-1,1,71,29,11039,4796\nGAGTATACCCTAATCA-1,1,70,30,10920,4864\nCCACGGCAGGTGTAGG-1,1,71,31,11039,4933\nACGTACTTTGGCACGG-1,1,70,32,10919,5002\nCCTGAGAATAAATGCA-1,1,71,33,11039,5071\nTTGATGCCGCTCGTCG-1,1,70,34,10919,5140\nGCCAGGAAAGAACACT-1,1,71,35,11038,5209\nTCCGACCGCTAATCAA-1,1,70,36,10919,5277\nTATATCAAAGTGATCT-1,1,71,37,11038,5346\nCCTCGGTTTCCTTGCC-1,1,70,38,10918,5415\nTAAATTTAGTAACACC-1,1,71,39,11038,5484\nCTCGTTACGGCTACCA-1,1,70,40,10918,5552\nGCCGTCGGTTTCGGGC-1,1,71,41,11037,5621\nGTTCCGTCCGCCTGCA-1,1,70,42,10917,5690\nCTAGGGATAGGGACAA-1,1,71,43,11037,5759\nGTCCAGGCACGTGTGC-1,1,70,44,10917,5827\nGAGGAATATCTCTTTG-1,1,71,45,11037,5897\nCGTTCAAGGAAACGGA-1,1,70,46,10917,5965\nGTGGGAACAAACCGGG-1,1,71,47,11036,6034\nGGGCACGTAGTACTGT-1,1,70,48,10916,6103\nACTACAAAGAGAGGTG-1,1,71,49,11036,6172\nGGTGTAGGTAAGTAAA-1,1,70,50,10916,6240\nCCTGTTCAACCTCGGG-1,1,71,51,11036,6309\nTGAGGTGTGTGGCGGA-1,1,70,52,10916,6378\nATTGTCTGTTTCATGT-1,1,71,53,11035,6447\nGCCGGTCGTATCTCTC-1,1,70,54,10915,6515\nATATGGGATAGCAACT-1,1,71,55,11035,6584\nGATAGAACCCGCTAGG-1,1,70,56,10915,6653\nATCCCATTTCCGTGGG-1,1,71,57,11035,6722\nGATGCTACCTTCGATG-1,1,70,58,10915,6791\nTGCAAAGTTCGTCTGT-1,1,71,59,11034,6860\nGTTCAGTCGCCAAATG-1,1,70,60,10914,6928\nGTCTCCGCCTCAATAC-1,1,71,61,11034,6997\nAGTAGAAGGCGCCTCA-1,1,70,62,10914,7066\nGTTGTAGATTTATGAG-1,1,71,63,11034,7135\nCTATTTGCTTGGAGGA-1,1,70,64,10914,7203\nAGCCATATAGTATGTG-1,1,71,65,11033,7272\nTGTTCCGGCCTGAGCT-1,1,70,66,10913,7341\nTGCCGTGGGACCCAAT-1,1,71,67,11033,7410\nTTGCAAGAAGACTCCT-1,1,70,68,10913,7478\nCCGCTCCAGGGCGATC-1,1,71,69,11032,7548\nCAAACCCTCCGGCGGG-1,1,70,70,10913,7616\nCGGACCTTTACGTCCC-1,1,71,71,11032,7685\nACGTAGGAGAGTCGCT-1,1,70,72,10912,7754\nCTGCGACCTCGCCGAA-1,1,71,73,11032,7823\nGGCCCAGCTGGTTTGC-1,1,70,74,10912,7891\nGGATTAATCATGGACC-1,1,71,75,11031,7960\nGAGATGGCTTTAATCA-1,1,70,76,10912,8029\nAGTACAGAAGCTTATA-1,1,71,77,11031,8098\nGCGGCTTTAGCAAGTT-1,1,70,78,10911,8166\nACCGAGTCTCCTTATT-1,1,71,79,11031,8235\nGACCACACTTCCCTTT-1,1,70,80,10911,8304\nTGTAGCCAATTCCGTT-1,1,71,81,11030,8373\nCTCGTCTGTGCCTTCG-1,1,70,82,10910,8441\nGGACTCTTTGACTAAG-1,1,71,83,11030,8511\nTCATCGACGACCGTCG-1,1,70,84,10910,8579\nTACTACGTGCAATGCG-1,1,71,85,11030,8648\nCCCTACTTGAACAATG-1,1,70,86,10910,8717\nGCAGCTGTCAACGCAT-1,1,71,87,11029,8786\nATCTACCATCTGCTCC-1,1,70,88,10909,8854\nTCTCAAATCAATCGGG-1,1,71,89,11029,8923\nACTCCGGCCGACCACT-1,1,70,90,10909,8992\nTACACCTCTTCGAATC-1,1,71,91,11029,9061\nGTGGCTGTTTCTGTTC-1,1,70,92,10909,9129\nACGACTCTAGGGCCGA-1,1,71,93,11028,9198\nTCGTTTACGCGACCCT-1,1,70,94,10908,9267\nAGGATATCCGACTGCA-1,1,71,95,11028,9336\nGACACTTCCAATTACC-1,1,70,96,10908,9405\nACGGTACAGTTCAATG-1,1,71,97,11028,9474\nCTTGATGACCATCCAG-1,1,70,98,10908,9542\nCTGGTAAAGACTTACA-1,0,71,99,11027,9611\nCTTGCCCACCCACGCA-1,1,70,100,10907,9680\nCATTACGCAGGAAGGG-1,1,71,101,11027,9749\nGGACAACCATGAAGCC-1,1,70,102,10907,9817\nGTTATCAAGCTATCGA-1,1,71,103,11027,9886\nAGGTGCACGTCCACAT-1,1,70,104,10907,9955\nAAAGAATGTGGACTAA-1,1,71,105,11026,10024\nACCAAGTCATCGGCAG-1,1,70,106,10906,10092\nAGTAACTATAGCAGCC-1,0,71,107,11026,10162\nGGGAAAGAATGCCAAC-1,1,70,108,10906,10230\nAATCGGTATAGCCCTC-1,1,71,109,11025,10299\nGGCACTGCGGTGGTTT-1,1,70,110,10906,10368\nCGTCCTCATCGCGTGC-1,1,71,111,11025,10437\nCTTAACTTACAGTATA-1,1,70,112,10905,10505\nGTGAGTCGACTAATAG-1,1,71,113,11025,10574\nTGTACTTCCGGGCATG-1,1,70,114,10905,10643\nCACAAACCGCAGAACT-1,0,71,115,11024,10712\nGCTTTACACAACTGGG-1,1,70,116,10905,10780\nCACCGATGATGGGTAC-1,0,71,117,11024,10849\nTCAATATACAGGAGGC-1,1,70,118,10904,10918\nAACGACCTCCTAGCCG-1,0,71,119,11024,10987\nGTTACCAAGGCGTACG-1,1,70,120,10904,11056\nTAGCTCACTGTGTTTG-1,0,71,121,11023,11125\nTCGGCAGGGTTAAGGG-1,1,70,122,10903,11193\nTGTTAACAAAGTGACT-1,1,71,123,11023,11262\nCAGAGCGATGGATGCT-1,1,70,124,10903,11331\nCATAGTGGGCACGCCT-1,0,71,125,11023,11400\nAAACGCTGGGCACGAC-1,0,70,126,10903,11468\nTGTTCTCATACTATAG-1,0,71,127,11022,11537\nTCCGCTGGGTCGATCG-1,0,72,0,11164,2801\nTTAGCATCCCTCACGT-1,0,73,1,11284,2870\nAGCCTTGTCACTGATA-1,0,72,2,11164,2939\nGTTTCTTGTTAGAGCT-1,0,73,3,11283,3008\nTCGGAGTCCTGGTTGC-1,0,72,4,11164,3076\nACGGGTTGTGACCTGT-1,0,73,5,11283,3146\nTGACAGAAATCTTGCT-1,0,72,6,11163,3214\nTAGCCATGATTGCCTA-1,0,73,7,11283,3283\nCGAAGCCACAGCATGG-1,0,72,8,11163,3352\nACCCGTAGCAGAGAAT-1,0,73,9,11282,3421\nAGCTTGATCAGGGTAG-1,0,72,10,11162,3489\nCTATAAGTAGGGTTTG-1,0,73,11,11282,3558\nTATAGTTAGGTGTACT-1,1,72,12,11162,3627\nTACGCTGATAGTTGTA-1,0,73,13,11282,3696\nTTCTCAATTGCTACAA-1,1,72,14,11162,3764\nGTTGCCCTAACGGGTG-1,0,73,15,11281,3833\nGACATTTCGCCCAGCC-1,1,72,16,11161,3902\nGTAACTACGTAGACCT-1,0,73,17,11281,3971\nCCAGTGTACAGACCGA-1,1,72,18,11161,4040\nGCCCGATGCCCAGTTC-1,0,73,19,11281,4109\nCGAACCTCTTTCCTAG-1,1,72,20,11161,4177\nCATGCACATGAGAGGC-1,0,73,21,11280,4246\nCCAAATCAAAGGGCAA-1,1,72,22,11160,4315\nGTAGTTTAAGCACACG-1,0,73,23,11280,4384\nTAAGTGAATAGTCTAC-1,1,72,24,11160,4452\nGGTCCACGTCTATTTG-1,0,73,25,11280,4521\nCCCTCAGATCGAGAAC-1,1,72,26,11160,4590\nACATTCGCGCGGAATA-1,0,73,27,11279,4659\nCTATGAACACCTTGCC-1,1,72,28,11159,4727\nCGAATGGTAGGTCGTC-1,0,73,29,11279,4797\nTCGACATAGCGTAGCG-1,1,72,30,11159,4865\nGACGTTGCTCGGCGGC-1,0,73,31,11278,4934\nCTCGAGGCAAGTTTCA-1,1,72,32,11159,5003\nGCTGGGAGCGCGTCAA-1,0,73,33,11278,5072\nGACGATATCACTGGGT-1,1,72,34,11158,5140\nCAGCTCTGGGCTCACT-1,0,73,35,11278,5209\nTAGTCTGCGGCACATT-1,1,72,36,11158,5278\nTGGTCGATATACCTCT-1,0,73,37,11277,5347\nTTCAGTTCAAGAGGAG-1,1,72,38,11158,5415\nGTCTGTAGGTTGAACA-1,0,73,39,11277,5484\nGAGAGTCTCGGGAGAG-1,1,72,40,11157,5553\nTTGTTTGTATTACACG-1,0,73,41,11277,5622\nGCACAGCACGGGCCGA-1,1,72,42,11157,5690\nAAACAGTGTTCCTGGG-1,0,73,43,11276,5760\nTGGAGAATAATCGTCC-1,0,72,44,11157,5828\nCGCCGCGTTCTGAACG-1,0,73,45,11276,5897\nCATGCGTTGAGAGGAG-1,1,72,46,11156,5966\nTACGCTCCTAGAACTG-1,0,73,47,11276,6035\nCCTTGTGAACGTGGTT-1,1,72,48,11156,6103\nTTAAAGGCGATGCTCG-1,0,73,49,11275,6172\nTAAAGTGCACGTCTCG-1,0,72,50,11155,6241\nTGTCCGCAAACAATTC-1,0,73,51,11275,6310\nAGGCCTATCATACCAA-1,0,72,52,11155,6378\nACTGGGATGCCAGTGC-1,0,73,53,11275,6447\nCGATCCACCATTGTTG-1,0,72,54,11155,6516\nGGGCATGCATGTCGAG-1,0,73,55,11274,6585\nTCCAGGCAGGACGATC-1,1,72,56,11154,6654\nCAATGACCCTTAATTT-1,0,73,57,11274,6723\nATCAAGATCCCAGGAC-1,0,72,58,11154,6791\nCTCAGAGCTAATGTCG-1,0,73,59,11274,6860\nCTGCGATTTCGAGATT-1,1,72,60,11154,6929\nATGGTATTGGGAACCG-1,0,73,61,11273,6998\nTAGGTTCTGCTGAGAA-1,0,72,62,11153,7066\nGCCTGTCCCGGTGCAT-1,0,73,63,11273,7135\nGGCCTGCTCTGATGTT-1,1,72,64,11153,7204\nTAGAGGGAGTTTATCT-1,0,73,65,11273,7273\nCTCTACATCCTGCGTG-1,0,72,66,11153,7341\nACACATTTCCGTAGAC-1,0,73,67,11272,7411\nTGGGTGTTAAGTAGAA-1,0,72,68,11152,7479\nGAGCTCAACATGAGCG-1,0,73,69,11272,7548\nGCACACAGCTATTACC-1,1,72,70,11152,7617\nGCGACGGTAGTCTCCT-1,0,73,71,11272,7686\nGTATCAAGGTACTTCC-1,0,72,72,11152,7754\nGGAAAGGGAATTGAGC-1,0,73,73,11271,7823\nTACACAGCCGTGGTGC-1,0,72,74,11151,7892\nCGGTTGGGCAGGGTCC-1,0,73,75,11271,7961\nGCACGCCGATTCCCGC-1,0,72,76,11151,8029\nCGCTTTCCGCCAAGGT-1,0,73,77,11270,8098\nTGCCCGATAGTTAGAA-1,0,72,78,11151,8167\nTCAGAACGGCGGTAAT-1,0,73,79,11270,8236\nCCACTCAGATCCGCAA-1,0,72,80,11150,8305\nGAAGGGTCATTAAGAC-1,0,73,81,11270,8374\nGCGGTCTTGCTTTCAC-1,0,72,82,11150,8442\nGGAAGGACACCGTATA-1,0,73,83,11269,8511\nCGCGGGAATTCCTTTC-1,0,72,84,11150,8580\nTCTATCCGATTGCACA-1,0,73,85,11269,8649\nTACTCGTTTGAATCAA-1,0,72,86,11149,8717\nGTCTGCCGACTCGACG-1,0,73,87,11269,8786\nTACGGGTAATAACATA-1,0,72,88,11149,8855\nCCAGTTCGGTAACTCA-1,0,73,89,11268,8924\nCGTGCACACCACTGTA-1,0,72,90,11148,8992\nCTTCAGTTGGACAACG-1,0,73,91,11268,9061\nTGATCTATCACACTCT-1,0,72,92,11148,9130\nTAGCAGATACTTAGGG-1,0,73,93,11268,9199\nCACCGGGCATCACAAG-1,0,72,94,11148,9268\nAGTGGCCCGCAAATGG-1,0,73,95,11267,9337\nCTCGCTAGGTAAGCGA-1,0,72,96,11147,9405\nGATAGATAGTACAGTC-1,0,73,97,11267,9474\nCGAGACTACTGCTGCT-1,0,72,98,11147,9543\nACATTTGAAACCTAAC-1,0,73,99,11267,9612\nGACACAGCCGGGACTG-1,0,72,100,11147,9680\nACGGAACACGAGTGCC-1,0,73,101,11266,9749\nACGCCAGTGCGTTTGC-1,0,72,102,11146,9818\nGCACCTAGGCGAGTCC-1,0,73,103,11266,9887\nTCACTATCCCTTCGGT-1,0,72,104,11146,9955\nTGAGTGTAACAACGGG-1,0,73,105,11266,10025\nATTGGTTGTGCATTAC-1,0,72,106,11146,10093\nCTGGCGATTTACATGT-1,0,73,107,11265,10162\nTCCACCAAGACATAGG-1,0,72,108,11145,10231\nAACGTCGCTGCACTTC-1,0,73,109,11265,10300\nGTCTCCCGAGTCCCGT-1,0,72,110,11145,10368\nAGCCCGCACTACAATG-1,0,73,111,11265,10437\nGGCACGCTGCTACAGT-1,0,72,112,11145,10506\nGCTTGAGTGACCTCTG-1,0,73,113,11264,10575\nATAGCCATAACAGTCA-1,0,72,114,11144,10643\nTAGGAGGCTCGAGAAC-1,0,73,115,11264,10712\nGTCGGGTGAAGTACCG-1,0,72,116,11144,10781\nCACCCGCGTTTGACAC-1,0,73,117,11263,10850\nTCTCAGGCTACTCGCT-1,0,72,118,11144,10919\nTAGATTCTCTAGCAAA-1,0,73,119,11263,10988\nCCCATTATTGTATCCT-1,0,72,120,11143,11056\nGTTAATAGCGTCATTA-1,0,73,121,11263,11125\nTACTCATTGACGCATC-1,0,72,122,11143,11194\nTAGTGACAAGCTCTAC-1,0,73,123,11262,11263\nATCGCCGTGGTTCATG-1,0,72,124,11143,11331\nCTGCTCTCAACACACC-1,0,73,125,11262,11400\nGCATGACACAAAGGAA-1,0,72,126,11142,11469\nCAACGATCGATCCAAT-1,0,73,127,11262,11538\nATACAGCGTCCACTGA-1,0,74,0,11404,2802\nTCACGATTAATACGTT-1,0,75,1,11523,2871\nCGTGAACTGACCCGAT-1,0,74,2,11403,2939\nTCGATGTTACGGCCGT-1,0,75,3,11523,3009\nGACCGGTGATACTCTC-1,0,74,4,11403,3077\nTCTGTTTAGATTGTTC-1,0,75,5,11522,3146\nGCTACTCGGACGCAGA-1,0,74,6,11403,3215\nAAGAAAGTTTGATGGG-1,0,75,7,11522,3284\nCGCCAGTAGTACCTTG-1,0,74,8,11402,3352\nCTCCCTTGTATCAAGG-1,0,75,9,11522,3421\nACTTTATACACCACTT-1,0,74,10,11402,3490\nAGATCTGGAGAGGATA-1,0,75,11,11521,3559\nTCATCCATCTGATCAC-1,0,74,12,11402,3627\nAGAACACGGCGATGGT-1,0,75,13,11521,3696\nCCGCTCCGGATAAGCT-1,0,74,14,11401,3765\nTGGTCGGGTACAGGGC-1,0,75,15,11521,3834\nACGGACGCAGCGACAA-1,0,74,16,11401,3903\nACATGCTTACGGCAGC-1,0,75,17,11520,3972\nCAGGGAGATAGGCCAG-1,0,74,18,11400,4040\nTCGCCACCCGGATTAC-1,0,75,19,11520,4109\nTCTTGGTCAATGATAC-1,0,74,20,11400,4178\nCATAGGGACACTTGTG-1,0,75,21,11520,4247\nCACGAGCAAACCAGAC-1,0,74,22,11400,4315\nGGGATATTGATCGCCA-1,0,75,23,11519,4384\nTAATGTCGGTTCATGG-1,0,74,24,11399,4453\nCGATTTGTCATTAATG-1,0,75,25,11519,4522\nTTAAGATACCCAGAGA-1,0,74,26,11399,4590\nCGACCGTTGGTATTCG-1,0,75,27,11519,4660\nACTACCATCCGAGGGC-1,0,74,28,11399,4728\nTCTACGCACGATCTCC-1,0,75,29,11518,4797\nAAGCCACTTGCAGGTA-1,0,74,30,11398,4866\nACGTTGTCGTTGAAAG-1,0,75,31,11518,4935\nACCGCTAGTCATTGGT-1,0,74,32,11398,5003\nCATGAGTCCATCACGG-1,0,75,33,11518,5072\nATGTCTTGTTTGACTC-1,0,74,34,11398,5141\nTCCCAAACATCCTCTA-1,0,75,35,11517,5210\nCAACGCGATGAGCCAA-1,0,74,36,11397,5278\nGACACCCAAAGACGCG-1,0,75,37,11517,5347\nACGTTCACTATGCCGC-1,0,74,38,11397,5416\nATGGACTGCTTAGTTG-1,0,75,39,11516,5485\nCCATGGCAAACGCTCA-1,0,74,40,11397,5554\nTTGTCTCGGCAAGATG-1,0,75,41,11516,5623\nTTCGAACGAAACATGC-1,0,74,42,11396,5691\nGGAAAGTCTTGATTGT-1,0,75,43,11516,5760\nATCATAGATCGACGAG-1,0,74,44,11396,5829\nGATTAGAAACAAGCGT-1,0,75,45,11515,5898\nGCTGCGAAGAATTATT-1,0,74,46,11396,5966\nTTCCAGTGGGTTTCGT-1,0,75,47,11515,6035\nGCTCATTGATCATATC-1,0,74,48,11395,6104\nATACGAGGTTTGTAAG-1,0,75,49,11515,6173\nGTCTATACACGCATGG-1,0,74,50,11395,6241\nGATTTGCGCTAACACC-1,0,75,51,11514,6311\nTGGTGCCCTGCCTTAC-1,0,74,52,11395,6379\nTCGAATCGCAGGGTAG-1,0,75,53,11514,6448\nGGTAGGCCAATATCAC-1,0,74,54,11394,6517\nCGTAAATAACAAAGGG-1,0,75,55,11514,6586\nGACTCTAGAGTTCCAA-1,0,74,56,11394,6654\nACTTGTTACCGGATCA-1,0,75,57,11513,6723\nGGCATTGAACATCTCA-1,0,74,58,11393,6792\nAACCCGCTGTATTCCA-1,0,75,59,11513,6861\nTAACAGGTTCCCTTAG-1,0,74,60,11393,6929\nGTTCTTGTAACTCAAT-1,0,75,61,11513,6998\nGATTCTGTTAATGAGT-1,0,74,62,11393,7067\nGGACACCTCGGTGTTG-1,0,75,63,11512,7136\nCGATCGAGAAGCACCA-1,0,74,64,11392,7204\nCTGCTCTGACGGCAAA-1,0,75,65,11512,7274\nCCCAGGAAGAATTCGA-1,0,74,66,11392,7342\nTGTCGTGGGTATAGGC-1,0,75,67,11512,7411\nATATGTCTAGAGCGTG-1,0,74,68,11392,7480\nGCGGGCAGACGGGTGA-1,0,75,69,11511,7549\nCGAAGATCAGTTTCAT-1,0,74,70,11391,7617\nACGCCACTCGAAACAG-1,0,75,71,11511,7686\nCTTAGATGTTTCATCC-1,0,74,72,11391,7755\nGCGAATGGACTAGCGA-1,0,75,73,11511,7824\nTAAATTGTGGGTAAAG-1,0,74,74,11391,7892\nATTTATACTGGTAAAG-1,0,75,75,11510,7961\nGCGTCGTAACATGGTC-1,0,74,76,11390,8030\nCGGTATGGGCACTCTG-1,0,75,77,11510,8099\nCTATCACAACGCTGGA-1,0,74,78,11390,8168\nACCACACGGTTGATGG-1,0,75,79,11509,8237\nTCAGCGCACGCCGTTT-1,0,74,80,11390,8305\nAATGTTAAGACCCTGA-1,0,75,81,11509,8374\nCTTACATAGATTTCTT-1,0,74,82,11389,8443\nAACATAGCGTGTATCG-1,0,75,83,11509,8512\nAGCAGTCGAAGCATGC-1,0,74,84,11389,8580\nGTACTACGGCCTCGTT-1,0,75,85,11508,8649\nTTGGCCAAATTGTATC-1,0,74,86,11389,8718\nTCAGAGGACGCGTTAG-1,0,75,87,11508,8787\nAGTAATCTAAGGGTGG-1,0,74,88,11388,8855\nAATTATACCCAGCAAG-1,0,75,89,11508,8925\nCCCTACCCACACCCAG-1,0,74,90,11388,8993\nCTCTGTCCATGCACCA-1,0,75,91,11507,9062\nTCGCCTCCTTCGGCTC-1,0,74,92,11388,9131\nCCTGATTCGCGAAGAA-1,0,75,93,11507,9200\nGACTTCAACGCATCAA-1,0,74,94,11387,9268\nGCTTCCCGTAAGCTCC-1,0,75,95,11507,9337\nAGAACTGTACTTTGTA-1,0,74,96,11387,9406\nCCTTAAGTACGCAATT-1,0,75,97,11506,9475\nGGCCAATTGTATAGAC-1,0,74,98,11386,9543\nAGAATACAGGCTATCC-1,0,75,99,11506,9612\nAATACCTGATGTGAAC-1,0,74,100,11386,9681\nTAGAGTGTTCCGGGTA-1,0,75,101,11506,9750\nGCTTCCATGTAACCGC-1,0,74,102,11386,9818\nGGTTCGCATTTGCCGT-1,0,75,103,11505,9888\nTCAATCCGGGAAGTTT-1,0,74,104,11385,9956\nCCGGAAGTGCAATATG-1,0,75,105,11505,10025\nCTCATAAATGTGTATA-1,0,74,106,11385,10094\nGACGGGCATCGAATTT-1,0,75,107,11505,10163\nGCTAAGCCCAGTATGC-1,0,74,108,11385,10231\nGCTATAAGGGCCAGGA-1,0,75,109,11504,10300\nGCGGATTACTTGTTCT-1,0,74,110,11384,10369\nTACAACAGCGCATACA-1,0,75,111,11504,10438\nTCTAACCTAGCCTGCG-1,0,74,112,11384,10506\nTCTACCCGCATCATTT-1,0,75,113,11504,10575\nGTCTTACCACGCCAAG-1,0,74,114,11384,10644\nAATAACGTCGCGCCCA-1,0,75,115,11503,10713\nCGTTTCGGTTATATGC-1,0,74,116,11383,10782\nTACGCTGCACGGTCGT-1,0,75,117,11503,10851\nCGTTAAATACGACCAG-1,0,74,118,11383,10919\nTCGACTGACGATGGCT-1,0,75,119,11502,10988\nACGCTACTGAATGGGC-1,0,74,120,11383,11057\nAAGGGTTAGCCATGCG-1,0,75,121,11502,11126\nACGTTTCGGTGCACTT-1,0,74,122,11382,11194\nTTAACCCGAGGCGTGT-1,0,75,123,11502,11263\nGCCCGTCAAGCCCAAT-1,0,74,124,11382,11332\nATTCGTCCCGAGGTTA-1,0,75,125,11501,11401\nTCAAATTATGTTCGAC-1,0,74,126,11382,11469\nAACGTTCTACCATTGT-1,0,75,127,11501,11539\nCTGTCCTGCGCACTAC-1,0,76,0,11643,2803\nGCCCAGCGACACAAAG-1,0,77,1,11763,2872\nTATAGCTATTATCTCT-1,0,76,2,11643,2940\nAAGCTATGGATTGACC-1,0,77,3,11762,3009\nCCGAAACACGACCTCT-1,0,76,4,11642,3078\nATCAGAAGCTGGTTGC-1,0,77,5,11762,3147\nTGAACCTGAATGTGAG-1,0,76,6,11642,3215\nTCAGAGCATGTCAACG-1,0,77,7,11761,3284\nATACCCTCCCGGCCAA-1,0,76,8,11642,3353\nGCCACCTTATTCGCGA-1,0,77,9,11761,3422\nCAACGAGCTTATTATG-1,0,76,10,11641,3490\nGATCCAACCTTTAAAC-1,0,77,11,11761,3560\nTTGTGGAGACAGCCGG-1,0,76,12,11641,3628\nAGATAATCACACCTAT-1,0,77,13,11760,3697\nGGTTAGGGATGCTAAT-1,0,76,14,11641,3766\nTCATGAAGCGCTGCAT-1,0,77,15,11760,3835\nCCAGGCGAGATGGTCT-1,0,76,16,11640,3903\nCGACTTGCCGGGAAAT-1,0,77,17,11760,3972\nGTGCCAAACGTTTCGA-1,0,76,18,11640,4041\nTGCTCAAAGGATGCAC-1,0,77,19,11759,4110\nGCCTCAGGTACCGGTC-1,0,76,20,11640,4178\nACAAGTAATTGTAAGG-1,0,77,21,11759,4247\nTTAGAGTATTGTCGAG-1,0,76,22,11639,4316\nGTCATCCCAAACTCAC-1,0,77,23,11759,4385\nCTCCTAGTAATCGTGA-1,0,76,24,11639,4453\nCCCTCATCACAGAGTA-1,0,77,25,11758,4523\nGGAAACAGAGCTTGGG-1,0,76,26,11638,4591\nACATCGCAATATTCGG-1,0,77,27,11758,4660\nTTAGCCATAGGGCTCG-1,0,76,28,11638,4729\nCGGATTCTGCCTTATG-1,0,77,29,11758,4798\nTTGTTGGCAATGACTG-1,0,76,30,11638,4866\nGGATCTACCGTTCGTC-1,0,77,31,11757,4935\nGGTGGGATTAGGTCCC-1,0,76,32,11637,5004\nAATAGGCACGACCCTT-1,0,77,33,11757,5073\nGAGGTCCGTTCGCTGT-1,0,76,34,11637,5141\nTGACAACTTAAAGGTG-1,0,77,35,11757,5210\nAGCTAAGCTCCGTCCG-1,0,76,36,11637,5279\nCTAGCCCGGGAGACGA-1,0,77,37,11756,5348\nCCCAGATTCCCGTGAC-1,0,76,38,11636,5417\nGCCCGTTCACACAATT-1,0,77,39,11756,5486\nACAAACTCCATCAGAG-1,0,76,40,11636,5554\nGGTTCCGTACGACTAA-1,0,77,41,11756,5623\nTTGCGGAAAGCTGCCC-1,0,76,42,11636,5692\nGGAACTCGTGAATACG-1,0,77,43,11755,5761\nGGTTACCGCTCCCTAC-1,0,76,44,11635,5829\nAGCGACTTTGAAGACA-1,0,77,45,11755,5898\nCCGTTTCCTTTCCGTG-1,0,76,46,11635,5967\nTGGATGGCATCTTGGA-1,0,77,47,11754,6036\nCAAGTCGTTGAAATCT-1,0,76,48,11635,6104\nTGCAAACGTACTAGTT-1,0,77,49,11754,6174\nGGCTGTCCTACTGCGG-1,0,76,50,11634,6242\nTACTGCATGATTAAAT-1,0,77,51,11754,6311\nCGGACCTCTGTAGTTA-1,0,76,52,11634,6380\nCCCTAGCTCTAAGGTC-1,0,77,53,11753,6449\nGTAACAGGTTAACGGC-1,0,76,54,11634,6517\nCCCGCAAATAATCATC-1,0,77,55,11753,6586\nAATCCCGCTCAGAGCC-1,0,76,56,11633,6655\nAACGTCATCCGGCTTG-1,0,77,57,11753,6724\nCTCGTGGCACTGAAAG-1,0,76,58,11633,6792\nCTTTATCCGACGCATG-1,0,77,59,11752,6861\nAAATGACTGATCAAAC-1,0,76,60,11633,6930\nTTCGTGCATGTTATAG-1,0,77,61,11752,6999\nATCAAAGAGCCGTGGT-1,0,76,62,11632,7067\nATGTCATAATAAACGA-1,0,77,63,11752,7137\nCTTAATCGACTTAGTA-1,0,76,64,11632,7205\nCTAAAGGATGAGATAC-1,0,77,65,11751,7274\nGTGCTATCCAGCTGGA-1,0,76,66,11631,7343\nGACAGAGGTCTTCAGT-1,0,77,67,11751,7412\nTCGATTTACGAAACGA-1,0,76,68,11631,7480\nAGGCTTAAGTTGCACA-1,0,77,69,11751,7549\nTAGGATCTTAACCGCA-1,0,76,70,11631,7618\nGACTTTCGAGCGGTTC-1,0,77,71,11750,7687\nCCCTGGGAGGGATCCT-1,0,76,72,11630,7755\nTAGTAGTTGCCGGACA-1,0,77,73,11750,7824\nGCCCTAAGTGCAGGAT-1,0,76,74,11630,7893\nACTTGGGCTTTCGCCA-1,0,77,75,11750,7962\nCATGATACGGTGAAAC-1,0,76,76,11630,8031\nTCCTGCAGCCGCCAAT-1,0,77,77,11749,8100\nGTCCGTTAGAGGGCCT-1,0,76,78,11629,8168\nGCTTTATTAAGTTACC-1,0,77,79,11749,8237\nAATCGAGGTCTCAAGG-1,0,76,80,11629,8306\nGTGCTGTTAGAACATA-1,0,77,81,11749,8375\nGGCTTCTCGTGGGTGG-1,0,76,82,11629,8443\nCCTACAGTTGAGGGAG-1,0,77,83,11748,8512\nTATCCCTCGATCTGCA-1,0,76,84,11628,8581\nCCGGTATCTGGCGACT-1,0,77,85,11748,8650\nCTCTAACACCGGCAGC-1,0,76,86,11628,8718\nCTTCATCACCAGGGCT-1,0,77,87,11747,8788\nGGCTGGCAATCCCACG-1,0,76,88,11628,8856\nCAAACCATAAGCGTAT-1,0,77,89,11747,8925\nCATGTGGGCTCATCAC-1,0,76,90,11627,8994\nCCCGTGACAGTGCCTT-1,0,77,91,11747,9063\nGATTATCTTGCATTAT-1,0,76,92,11627,9131\nGAGTCCACCAGGTTTA-1,0,77,93,11746,9200\nTGTACTATCGCTCGTT-1,0,76,94,11627,9269\nTTAGCGAATAGATAGG-1,0,77,95,11746,9338\nACATACAATCAAGCGG-1,0,76,96,11626,9406\nATTCGTGTACCCATTC-1,0,77,97,11746,9475\nGCTAGCAACGCACCTA-1,0,76,98,11626,9544\nACCTCAGCGAGGCGCA-1,0,77,99,11745,9613\nTCCGGGCCACTAACGG-1,0,76,100,11626,9682\nTCCCAATATCGACGAC-1,0,77,101,11745,9751\nGGATTGAAGTAGCCTC-1,0,76,102,11625,9819\nGGCGCACAGTTTACCT-1,0,77,103,11745,9888\nCCCGGCACGTGTCAGG-1,0,76,104,11625,9957\nAGTAAGGGACAGAATC-1,0,77,105,11744,10026\nTTGCGCACAACCACGT-1,0,76,106,11624,10094\nCCTGCTACAACCATAC-1,0,77,107,11744,10163\nCTGGGCTACTGGAGAG-1,0,76,108,11624,10232\nAGCTACGAATGGTGGT-1,0,77,109,11744,10301\nATTCGTTTATCGTATT-1,0,76,110,11624,10369\nGGCATTCCCTCCCTCG-1,0,77,111,11743,10439\nAGTTATTCAGACTGTG-1,0,76,112,11623,10507\nCCACTTTCCTTCTAGG-1,0,77,113,11743,10576\nCGCAGTTCTATCTTTC-1,0,76,114,11623,10645\nAGGAGCGTTTATTATC-1,0,77,115,11743,10714\nGACAGGTAATCCGTGT-1,0,76,116,11623,10782\nTTCCCAAAGTACTGAT-1,0,77,117,11742,10851\nCTGCAGGGTGACGCTC-1,0,76,118,11622,10920\nTCATGGAGGCCTTTGT-1,0,77,119,11742,10989\nATGGCCCGAAAGGTTA-1,0,76,120,11622,11057\nCGTAATATGGCCCTTG-1,0,77,121,11742,11126\nAGAGTCTTAATGAAAG-1,0,76,122,11622,11195\nGAACGTTTGTATCCAC-1,0,77,123,11741,11264\nATTGAATTCCCTGTAG-1,0,76,124,11621,11332\nTACCTCACCAATTGTA-1,0,77,125,11741,11402\nAGTCGAATTAGCGTAA-1,0,76,126,11621,11470\nTTGAAGTGCATCTACA-1,0,77,127,11740,11539\n\n\n{\"spot_diameter_fullres\": 89.42751063343188, \"tissue_hires_scalef\": 0.150015, \"fiducial_diameter_fullres\": 144.45982486939, \"tissue_lowres_scalef\": 0.045004502}\n\n# PAGA for hematopoiesis in mouse [(Paul *et al.*, 2015)](https://doi.org/10.1016/j.cell.2015.11.013)\n# Hematopoiesis: trace myeloid and erythroid differentiation for data of [Paul *et al.* (2015)](https://doi.org/10.1016/j.cell.2015.11.013).\n#\n# This is the subsampled notebook for testing.\nfrom __future__ import annotations\n\nfrom functools import partial\nfrom pathlib import Path\n\nimport numpy as np\nimport pytest\nfrom matplotlib.testing import setup\n\nimport scanpy as sc\nfrom testing.scanpy._helpers.data import paul15\nfrom testing.scanpy._pytest.marks import needs\n\nHERE: Path = Path(__file__).parent\nROOT = HERE / \"_images_paga_paul15_subsampled\"\n\n\n@pytest.mark.skip(reason=\"Broken, needs fixing\")\n@needs.igraph\n@needs.louvain\ndef test_paga_paul15_subsampled(image_comparer, plt):\n setup()\n save_and_compare_images = partial(image_comparer, ROOT, tol=25)\n\n adata = paul15()\n sc.pp.subsample(adata, n_obs=200)\n del adata.uns[\"iroot\"]\n adata.X = adata.X.astype(\"float64\")\n\n # Preprocessing and Visualization\n sc.pp.recipe_zheng17(adata)\n sc.pp.pca(adata, svd_solver=\"arpack\")\n sc.pp.neighbors(adata, n_neighbors=4, n_pcs=20)\n sc.tl.draw_graph(adata)\n sc.pl.draw_graph(adata, color=\"paul15_clusters\", legend_loc=\"on data\")\n\n sc.tl.diffmap(adata)\n sc.tl.diffmap(adata) # See #1262\n sc.pp.neighbors(adata, n_neighbors=10, use_rep=\"X_diffmap\")\n sc.tl.draw_graph(adata)\n\n sc.pl.draw_graph(adata, color=\"paul15_clusters\", legend_loc=\"on data\")\n\n # TODO: currently needs skip if louvain isn't installed, do major rework\n\n # Clustering and PAGA\n sc.tl.louvain(adata, resolution=1.0)\n sc.tl.paga(adata, groups=\"louvain\")\n # sc.pl.paga(adata, color=['louvain', 'Hba-a2', 'Elane', 'Irf8'])\n # sc.pl.paga(adata, color=['louvain', 'Itga2b', 'Prss34'])\n\n adata.obs[\"louvain_anno\"] = adata.obs[\"louvain\"]\n sc.tl.paga(adata, groups=\"louvain_anno\")\n\n PAGA_CONNECTIVITIES = np.array(\n [\n [0.0, 0.128553, 0.0, 0.07825, 0.0, 0.0, 0.238741, 0.0, 0.0, 0.657049],\n [\n *[0.128553, 0.0, 0.480676, 0.257505, 0.533036],\n *[0.043871, 0.0, 0.032903, 0.0, 0.087743],\n ],\n ]\n )\n\n assert np.allclose(\n adata.uns[\"paga\"][\"connectivities\"].toarray()[:2],\n PAGA_CONNECTIVITIES,\n atol=1e-4,\n )\n\n sc.pl.paga(adata, threshold=0.03)\n\n # !!!! no clue why it doesn't produce images with the same shape\n # save_and_compare_images('paga')\n\n sc.tl.draw_graph(adata, init_pos=\"paga\")\n sc.pl.paga_compare(\n adata,\n threshold=0.03,\n title=\"\",\n right_margin=0.2,\n size=10,\n edge_width_scale=0.5,\n legend_fontsize=12,\n fontsize=12,\n frameon=False,\n edges=True,\n )\n\n # slight deviations because of graph drawing\n # save_and_compare_images('paga_compare')\n\n adata.uns[\"iroot\"] = np.flatnonzero(adata.obs[\"louvain_anno\"] == \"3\")[0]\n sc.tl.dpt(adata)\n gene_names = [\n \"Gata2\",\n \"Gata1\",\n \"Klf1\",\n \"Hba-a2\", # erythroid\n \"Elane\",\n \"Cebpe\", # neutrophil\n \"Irf8\",\n ] # monocyte\n\n paths = [\n (\"erythrocytes\", [3, 9, 0, 6]),\n (\"neutrophils\", [3, 1, 2]),\n (\"monocytes\", [3, 1, 4, 5]),\n ]\n\n adata.obs[\"distance\"] = adata.obs[\"dpt_pseudotime\"]\n\n _, axs = plt.subplots(\n ncols=3, figsize=(6, 2.5), gridspec_kw={\"wspace\": 0.05, \"left\": 0.12}\n )\n plt.subplots_adjust(left=0.05, right=0.98, top=0.82, bottom=0.2)\n for ipath, (descr, path) in enumerate(paths):\n _, data = sc.pl.paga_path(\n adata,\n path,\n gene_names,\n show_node_names=False,\n ax=axs[ipath],\n ytick_fontsize=12,\n left_margin=0.15,\n n_avg=50,\n annotations=[\"distance\"],\n show_yticks=ipath == 0,\n show_colorbar=False,\n color_map=\"Greys\",\n color_maps_annotations={\"distance\": \"viridis\"},\n title=f\"{descr} path\",\n return_data=True,\n show=False,\n )\n # add a test for this at some point\n # data.to_csv('./write/paga_path_{}.csv'.format(descr))\n\n save_and_compare_images(\"paga_path\")\n\n\n# *First compiled on May 5, 2017. Updated August 14, 2018.*\n# # Clustering 3k PBMCs following a Seurat Tutorial\n#\n# This started out with a demonstration that Scanpy would allow to reproduce most of Seurat's\n# ([Satija *et al.*, 2015](https://doi.org/10.1038/nbt.3192)) clustering tutorial as described on\n# https://satijalab.org/seurat/articles/pbmc3k_tutorial.html (July 26, 2017), which we gratefully acknowledge.\n# In the meanwhile, we have added and removed several pieces.\n#\n# The data consists in *3k PBMCs from a Healthy Donor* and is freely available from 10x Genomics\n# ([here](https://cf.10xgenomics.com/samples/cell-exp/1.1.0/pbmc3k/pbmc3k_filtered_gene_bc_matrices.tar.gz)\n# from this [webpage](https://support.10xgenomics.com/single-cell-gene-expression/datasets/1.1.0/pbmc3k)).\nfrom __future__ import annotations\n\nfrom functools import partial\nfrom pathlib import Path\n\nimport numpy as np\nfrom matplotlib.testing import setup\n\nsetup()\n\nimport scanpy as sc\nfrom testing.scanpy._pytest.marks import needs\n\nHERE: Path = Path(__file__).parent\nROOT = HERE / \"_images_pbmc3k\"\n\n\n@needs.leidenalg\ndef test_pbmc3k(image_comparer):\n # ensure violin plots and other non-determinstic plots have deterministic behavior\n np.random.seed(0)\n save_and_compare_images = partial(image_comparer, ROOT, tol=20)\n adata = sc.datasets.pbmc3k()\n\n # Preprocessing\n\n sc.pl.highest_expr_genes(adata, n_top=20, show=False)\n save_and_compare_images(\"highest_expr_genes\")\n\n sc.pp.filter_cells(adata, min_genes=200)\n sc.pp.filter_genes(adata, min_cells=3)\n\n mito_genes = [name for name in adata.var_names if name.startswith(\"MT-\")]\n # for each cell compute fraction of counts in mito genes vs. all genes\n # the `.A1` is only necessary as X is sparse to transform to a dense array after summing\n adata.obs[\"percent_mito\"] = (\n np.sum(adata[:, mito_genes].X, axis=1).A1 / np.sum(adata.X, axis=1).A1\n )\n # add the total counts per cell as observations-annotation to adata\n adata.obs[\"n_counts\"] = adata.X.sum(axis=1).A1\n\n sc.pl.violin(\n adata,\n [\"n_genes\", \"n_counts\", \"percent_mito\"],\n jitter=False,\n multi_panel=True,\n show=False,\n )\n save_and_compare_images(\"violin\")\n\n sc.pl.scatter(adata, x=\"n_counts\", y=\"percent_mito\", show=False)\n save_and_compare_images(\"scatter_1\")\n sc.pl.scatter(adata, x=\"n_counts\", y=\"n_genes\", show=False)\n save_and_compare_images(\"scatter_2\")\n\n adata = adata[adata.obs[\"n_genes\"] < 2500, :]\n adata = adata[adata.obs[\"percent_mito\"] < 0.05, :]\n\n adata.raw = sc.pp.log1p(adata, copy=True)\n\n sc.pp.normalize_per_cell(adata, counts_per_cell_after=1e4)\n\n filter_result = sc.pp.filter_genes_dispersion(\n adata.X,\n min_mean=0.0125,\n max_mean=3,\n min_disp=0.5,\n )\n sc.pl.filter_genes_dispersion(filter_result, show=False)\n save_and_compare_images(\"filter_genes_dispersion\")\n\n adata = adata[:, filter_result.gene_subset]\n sc.pp.log1p(adata)\n sc.pp.regress_out(adata, [\"n_counts\", \"percent_mito\"])\n sc.pp.scale(adata, max_value=10)\n\n # PCA\n\n sc.pp.pca(adata, svd_solver=\"arpack\")\n sc.pl.pca(adata, color=\"CST3\", show=False)\n save_and_compare_images(\"pca\")\n\n sc.pl.pca_variance_ratio(adata, log=True, show=False)\n save_and_compare_images(\"pca_variance_ratio\")\n\n # UMAP\n\n sc.pp.neighbors(adata, n_neighbors=10, n_pcs=40)\n # sc.tl.umap(adata) # umaps lead to slight variations\n\n # sc.pl.umap(adata, color=['CST3', 'NKG7', 'PPBP'], use_raw=False, show=False)\n # save_and_compare_images('umap_1')\n\n # Clustering the graph\n\n sc.tl.leiden(\n adata,\n resolution=0.9,\n random_state=0,\n directed=False,\n n_iterations=2,\n flavor=\"igraph\",\n )\n\n # sc.pl.umap(adata, color=[\"leiden\", \"CST3\", \"NKG7\"], show=False)\n # save_and_compare_images(\"umap_2\")\n sc.pl.scatter(adata, \"CST3\", \"NKG7\", color=\"leiden\", show=False)\n save_and_compare_images(\"scatter_3\")\n\n # Finding marker genes\n # Due to incosistency with our test runner vs local, these clusters need to\n # be pre-annotated as the numbers for each cluster are not consistent.\n marker_genes = [\n \"RP11-18H21.1\",\n \"GZMK\",\n \"CD79A\",\n \"FCGR3A\",\n \"GNLY\",\n \"S100A8\",\n \"FCER1A\",\n \"PPBP\",\n ]\n new_labels = [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\"]\n data_df = adata[:, marker_genes].to_df()\n data_df[\"leiden\"] = adata.obs[\"leiden\"]\n max_idxs = data_df.groupby(\"leiden\", observed=True).mean().idxmax()\n leiden_relabel = {}\n for marker_gene, new_label in zip(marker_genes, new_labels):\n leiden_relabel[max_idxs[marker_gene]] = new_label\n adata.obs[\"leiden_old\"] = adata.obs[\"leiden\"].copy()\n adata.rename_categories(\n \"leiden\", [leiden_relabel[key] for key in sorted(leiden_relabel.keys())]\n )\n # ensure that the column can be sorted for consistent plotting since it is by default unordered\n adata.obs[\"leiden\"] = adata.obs[\"leiden\"].cat.reorder_categories(\n list(map(str, range(len(adata.obs[\"leiden\"].cat.categories)))), ordered=True\n )\n\n sc.tl.rank_genes_groups(adata, \"leiden\")\n sc.pl.rank_genes_groups(adata, n_genes=20, sharey=False, show=False)\n save_and_compare_images(\"rank_genes_groups_1\")\n\n sc.tl.rank_genes_groups(adata, \"leiden\", method=\"logreg\")\n sc.pl.rank_genes_groups(adata, n_genes=20, sharey=False, show=False)\n save_and_compare_images(\"rank_genes_groups_2\")\n\n sc.tl.rank_genes_groups(adata, \"leiden\", groups=[\"0\"], reference=\"1\")\n sc.pl.rank_genes_groups(adata, groups=\"0\", n_genes=20, show=False)\n save_and_compare_images(\"rank_genes_groups_3\")\n\n # gives a strange error, probably due to jitter or something\n # sc.pl.rank_genes_groups_violin(adata, groups='0', n_genes=8)\n # save_and_compare_images('rank_genes_groups_4')\n\n new_cluster_names = [\n \"CD4 T cells\",\n \"CD8 T cells\",\n \"B cells\",\n \"NK cells\",\n \"FCGR3A+ Monocytes\",\n \"CD14+ Monocytes\",\n \"Dendritic cells\",\n \"Megakaryocytes\",\n ]\n adata.rename_categories(\"leiden\", new_cluster_names)\n\n # sc.pl.umap(adata, color='leiden', legend_loc='on data', title='', frameon=False, show=False)\n # save_and_compare_images('umap_3')\n sc.pl.violin(\n adata, [\"CST3\", \"NKG7\", \"PPBP\"], groupby=\"leiden\", rotation=90, show=False\n )\n save_and_compare_images(\"violin_2\")\n\n\nfrom __future__ import annotations\n\nimport scanpy as sc\nimport scanpy.external as sce\nfrom testing.scanpy._helpers.data import pbmc3k\nfrom testing.scanpy._pytest.marks import needs\n\npytestmark = [needs.wishbone]\n\n\ndef test_run_wishbone():\n adata = pbmc3k()\n sc.pp.normalize_per_cell(adata)\n sc.pp.neighbors(adata, n_pcs=15, n_neighbors=10)\n sc.pp.pca(adata)\n sc.tl.tsne(adata=adata, n_pcs=5, perplexity=30)\n sc.tl.diffmap(adata, n_comps=10)\n\n sce.tl.wishbone(\n adata=adata,\n start_cell=\"ACAAGAGACTTATC-1\",\n components=[2, 3],\n num_waypoints=150,\n )\n assert all(\n [k in adata.obs for k in [\"trajectory_wishbone\", \"branch_wishbone\"]]\n ), \"Run Wishbone Error!\"\n\n\nfrom __future__ import annotations\n\nimport sys\n\nimport pytest\n\nimport scanpy as sc\nimport scanpy.external as sce\nfrom testing.scanpy._helpers.data import pbmc68k_reduced\nfrom testing.scanpy._pytest.marks import needs\n\npytestmark = [\n needs.scanorama,\n pytest.mark.skipif(sys.version_info < (3, 10), reason=\"annoy is unstable on 3.9\"),\n]\n\n\ndef test_scanorama_integrate():\n \"\"\"\n Test that Scanorama integration works.\n\n This is a very simple test that just checks to see if the Scanorama\n integrate wrapper succesfully added a new field to ``adata.obsm``\n and makes sure it has the same dimensions as the original PCA table.\n \"\"\"\n adata = pbmc68k_reduced()\n sc.pp.pca(adata)\n adata.obs[\"batch\"] = 350 * [\"a\"] + 350 * [\"b\"]\n sce.pp.scanorama_integrate(adata, \"batch\", approx=False)\n assert adata.obsm[\"X_scanorama\"].shape == adata.obsm[\"X_pca\"].shape\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\nfrom anndata import AnnData\n\nimport scanpy as sc\nimport scanpy.external as sce\nfrom testing.scanpy._pytest.marks import needs\n\npytestmark = [needs.phenograph]\n\n\ndef test_phenograph():\n df = np.random.rand(1000, 40)\n dframe = pd.DataFrame(df)\n dframe.index, dframe.columns = (map(str, dframe.index), map(str, dframe.columns))\n adata = AnnData(dframe)\n sc.pp.pca(adata, n_comps=20)\n sce.tl.phenograph(adata, clustering_algo=\"leiden\", k=50)\n assert adata.obs[\"pheno_leiden\"].shape[0], \"phenograph_Community Detection Error!\"\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nfrom anndata import AnnData\n\nimport scanpy as sc\nfrom testing.scanpy._pytest.marks import needs\n\npytestmark = [needs.magic]\n\nA_list = [\n [0, 0, 7, 0, 0],\n [8, 5, 0, 2, 0],\n [6, 0, 0, 2, 5],\n [0, 0, 0, 1, 0],\n [8, 8, 2, 1, 0],\n [0, 0, 0, 4, 5],\n]\n\n\ndef test_magic_default():\n A = np.array(A_list, dtype=\"float32\")\n adata = AnnData(A)\n sc.external.pp.magic(adata, knn=1)\n # check raw unchanged\n np.testing.assert_array_equal(adata.raw.X, A)\n # check .X changed\n assert not np.all(adata.X == A)\n # check .X shape unchanged\n assert adata.X.shape == A.shape\n\n\ndef test_magic_pca_only():\n A = np.array(A_list, dtype=\"float32\")\n # pca only\n adata = AnnData(A)\n n_pca = 3\n sc.external.pp.magic(adata, knn=1, name_list=\"pca_only\", n_pca=n_pca)\n # check raw unchanged\n np.testing.assert_array_equal(adata.X, A)\n # check .X shape consistent with n_pca\n assert adata.obsm[\"X_magic\"].shape == (A.shape[0], n_pca)\n\n\ndef test_magic_copy():\n A = np.array(A_list, dtype=\"float32\")\n adata = AnnData(A)\n adata_copy = sc.external.pp.magic(adata, knn=1, copy=True)\n # check adata unchanged\n np.testing.assert_array_equal(adata.X, A)\n # check copy raw unchanged\n np.testing.assert_array_equal(adata_copy.raw.X, A)\n # check .X changed\n assert not np.all(adata_copy.X == A)\n # check .X shape unchanged\n assert adata_copy.X.shape == A.shape\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nfrom anndata import AnnData\n\nimport scanpy.external as sce\n\n\ndef test_cell_demultiplexing():\n import random\n\n from scipy import stats\n\n random.seed(52)\n signal = stats.poisson.rvs(1000, 1, 990)\n doublet_signal = stats.poisson.rvs(1000, 1, 10)\n x = np.reshape(stats.poisson.rvs(500, 1, 10000), (1000, 10))\n for idx, signal_count in enumerate(signal):\n col_pos = idx % 10\n x[idx, col_pos] = signal_count\n\n for idx, signal_count in enumerate(doublet_signal):\n col_pos = (idx % 10) - 1\n x[idx, col_pos] = signal_count\n\n test_data = AnnData(np.random.randint(0, 100, size=x.shape), obs=x)\n sce.pp.hashsolo(test_data, test_data.obs.columns)\n\n doublets = [\"Doublet\"] * 10\n classes = list(\n np.repeat(np.arange(10), 98).reshape(98, 10, order=\"F\").ravel().astype(str)\n )\n negatives = [\"Negative\"] * 10\n classification = doublets + classes + negatives\n assert test_data.obs[\"Classification\"].astype(str).tolist() == classification\n\n\nfrom __future__ import annotations\n\nimport numpy as np\n\nimport scanpy as sc\nimport scanpy.external as sce\nfrom testing.scanpy._helpers.data import pbmc3k\nfrom testing.scanpy._pytest.marks import needs\n\npytestmark = [needs.samalg]\n\n\ndef test_sam():\n adata_ref = pbmc3k()\n ix = np.random.choice(adata_ref.shape[0], size=200, replace=False)\n adata = adata_ref[ix, :].copy()\n sc.pp.normalize_total(adata, target_sum=10000)\n sc.pp.log1p(adata)\n sce.tl.sam(adata, inplace=True)\n uns_keys = list(adata.uns.keys())\n obsm_keys = list(adata.obsm.keys())\n assert all([\"sam\" in uns_keys, \"X_umap\" in obsm_keys, \"neighbors\" in uns_keys])\n\n\nfrom __future__ import annotations\n\nfrom itertools import product\n\nfrom anndata import AnnData\n\nimport scanpy as sc\nimport scanpy.external as sce\nfrom testing.scanpy._helpers.data import pbmc3k\nfrom testing.scanpy._pytest.marks import needs\n\npytestmark = [needs.harmony]\n\n\ndef test_load_timepoints_from_anndata_list():\n adata_ref = pbmc3k()\n start = [596, 615, 1682, 1663, 1409, 1432]\n adata = AnnData.concatenate(\n *(adata_ref[i : i + 1000] for i in start),\n join=\"outer\",\n batch_key=\"sample\",\n batch_categories=[f\"sa{i}_Rep{j}\" for i, j in product((1, 2, 3), (1, 2))],\n )\n adata.obs[\"time_points\"] = adata.obs[\"sample\"].str.split(\"_\", expand=True)[0]\n adata.obs[\"time_points\"] = adata.obs[\"time_points\"].astype(\"category\")\n sc.pp.normalize_total(adata, target_sum=10000)\n sc.pp.log1p(adata)\n sc.pp.highly_variable_genes(adata, n_top_genes=1000, subset=True)\n\n sce.tl.harmony_timeseries(adata=adata, tp=\"time_points\", n_components=None)\n assert all(\n [adata.obsp[\"harmony_aff\"].shape[0], adata.obsp[\"harmony_aff_aug\"].shape[0]]\n ), \"harmony_timeseries augmented affinity matrix Error!\"\n\n\nfrom __future__ import annotations\n\nimport scanpy.external as sce\nfrom testing.scanpy._helpers.data import pbmc3k_processed\nfrom testing.scanpy._pytest.marks import needs\n\npytestmark = [needs.palantir]\n\n\ndef test_palantir_core():\n adata = pbmc3k_processed()\n\n sce.tl.palantir(adata=adata, n_components=5, knn=30)\n assert adata.layers[\"palantir_imp\"].shape[0], \"palantir_imp matrix Error!\"\n\n\nfrom __future__ import annotations\n\nimport scanpy as sc\nimport scanpy.external as sce\nfrom testing.scanpy._helpers.data import pbmc3k\nfrom testing.scanpy._pytest.marks import needs\n\npytestmark = [needs.harmonypy]\n\n\ndef test_harmony_integrate():\n \"\"\"\n Test that Harmony integrate works.\n\n This is a very simple test that just checks to see if the Harmony\n integrate wrapper succesfully added a new field to ``adata.obsm``\n and makes sure it has the same dimensions as the original PCA table.\n \"\"\"\n adata = pbmc3k()\n sc.pp.recipe_zheng17(adata)\n sc.pp.pca(adata)\n adata.obs[\"batch\"] = 1350 * [\"a\"] + 1350 * [\"b\"]\n sce.pp.harmony_integrate(adata, \"batch\")\n assert adata.obsm[\"X_pca_harmony\"].shape == adata.obsm[\"X_pca\"].shape\n\n\n# Usage Principles\n\nImport Scanpy as:\n\n```\nimport scanpy as sc\n```\n\n## Workflow\n\nThe typical workflow consists of subsequent calls of data analysis tools\nin `sc.tl`, e.g.:\n\n```\nsc.tl.umap(adata, **tool_params) # embed a neighborhood graph of the data using UMAP\n```\n\nwhere `adata` is an {class}`~anndata.AnnData` object.\nEach of these calls adds annotation to an expression matrix *X*,\nwhich stores *n_obs* observations (cells) of *n_vars* variables (genes).\nFor each tool, there typically is an associated plotting function in `sc.pl`:\n\n```\nsc.pl.umap(adata, **plotting_params)\n```\n\nIf you pass `show=False`, a {class}`~matplotlib.axes.Axes` instance is returned\nand you have all of matplotlib's detailed configuration possibilities.\n\nTo facilitate writing memory-efficient pipelines, by default,\nScanpy tools operate *inplace* on `adata` and return `None` –\nthis also allows to easily transition to [out-of-memory pipelines].\nIf you want to return a copy of the {class}`~anndata.AnnData` object\nand leave the passed `adata` unchanged, pass `copy=True` or `inplace=False`.\n\n## AnnData\n\nScanpy is based on {mod}`anndata`, which provides the {class}`~anndata.AnnData` class.\n\n```{image} https://falexwolf.de/img/scanpy/anndata.svg\n:width: 300px\n```\n\nAt the most basic level, an {class}`~anndata.AnnData` object `adata` stores\na data matrix `adata.X`, annotation of observations\n`adata.obs` and variables `adata.var` as `pd.DataFrame` and unstructured\nannotation `adata.uns` as `dict`. Names of observations and\nvariables can be accessed via `adata.obs_names` and `adata.var_names`,\nrespectively. {class}`~anndata.AnnData` objects can be sliced like\ndataframes, for example, `adata_subset = adata[:, list_of_gene_names]`.\nFor more, see this [blog post].\n\nTo read a data file to an {class}`~anndata.AnnData` object, call:\n\n```\nadata = sc.read(filename)\n```\n\nto initialize an {class}`~anndata.AnnData` object. Possibly add further annotation using, e.g., `pd.read_csv`:\n\n```\nimport pandas as pd\nanno = pd.read_csv(filename_sample_annotation)\nadata.obs['cell_groups'] = anno['cell_groups'] # categorical annotation of type pandas.Categorical\nadata.obs['time'] = anno['time'] # numerical annotation of type float\n# alternatively, you could also set the whole dataframe\n# adata.obs = anno\n```\n\nTo write, use:\n\n```\nadata.write(filename)\nadata.write_csvs(filename)\nadata.write_loom(filename)\n```\n\n[blog post]: https://falexwolf.de/blog/171223_AnnData_indexing_views_HDF5-backing/\n[matplotlib]: https://matplotlib.org/\n[out-of-memory pipelines]: https://falexwolf.de/blog/171223_AnnData_indexing_views_HDF5-backing/\n[seaborn]: https://seaborn.pydata.org/\n\n\n---\norphan: true\n---\n\nThis file has moved to .\n\n\n# Community\n\nScanpy is a community driven project. There are multiple channels for users and developers to communicate and connect.\n\n## [Discourse](https://discourse.scverse.org)\n\nThe scverse Discourse forum is place to go to ask usage questions and for longer form discussions around the project.\n\n## [Github Issue Tracker](https://github.com/scverse/scanpy/issues)\n\nThe [Scanpy](https://github.com/scverse/scanpy/issues) and [anndata](https://github.com/scverse/anndata/issues) issue trackers are for reports and discussion of:\n\n- Bug reports\n- Documentation issues\n- Feature requests\n\n## [Developer Chat](https://scverse.zulipchat.com/)\n\nZulip chat instance for synchronous discussion of scanpy, anndata, and other scverse packages.\n\n\nfrom __future__ import annotations\n\nimport sys\nfrom datetime import datetime\nfrom functools import partial\nfrom pathlib import Path, PurePosixPath\nfrom typing import TYPE_CHECKING\n\nimport matplotlib # noqa\nfrom docutils import nodes\nfrom packaging.version import Version\n\n# Don’t use tkinter agg when importing scanpy → … → matplotlib\nmatplotlib.use(\"agg\")\n\nHERE = Path(__file__).parent\nsys.path[:0] = [str(HERE.parent), str(HERE / \"extensions\")]\nimport scanpy # noqa\n\nif TYPE_CHECKING:\n from sphinx.application import Sphinx\n\n\n# -- General configuration ------------------------------------------------\n\nnitpicky = True # Warn about broken links. This is here for a reason: Do not change.\nneeds_sphinx = \"4.0\" # Nicer param docs\nsuppress_warnings = [\n \"myst.header\", # https://github.com/executablebooks/MyST-Parser/issues/262\n]\n\n# General information\nproject = \"Scanpy\"\nauthor = \"Scanpy development team\"\nrepository_url = \"https://github.com/scverse/scanpy\"\ncopyright = f\"{datetime.now():%Y}, the Scanpy development team\"\nversion = scanpy.__version__.replace(\".dirty\", \"\")\n\n# Bumping the version updates all docs, so don't do that\nif Version(version).is_devrelease:\n parsed = Version(version)\n version = f\"{parsed.major}.{parsed.minor}.{parsed.micro}.dev\"\n\nrelease = version\n\n# Bibliography settings\nbibtex_bibfiles = [\"references.bib\"]\nbibtex_reference_style = \"author_year\"\n\n\n# default settings\ntemplates_path = [\"_templates\"]\nmaster_doc = \"index\"\ndefault_role = \"literal\"\nexclude_patterns = [\n \"_build\",\n \"Thumbs.db\",\n \".DS_Store\",\n \"**.ipynb_checkpoints\",\n # exclude all 0.x.y.md files, but not index.md\n \"release-notes/[!i]*.md\",\n]\n\nextensions = [\n \"myst_nb\",\n \"sphinx_copybutton\",\n \"sphinx.ext.autodoc\",\n \"sphinx.ext.intersphinx\",\n \"sphinx.ext.doctest\",\n \"sphinx.ext.coverage\",\n \"sphinx.ext.mathjax\",\n \"sphinx.ext.napoleon\",\n \"sphinx.ext.autosummary\",\n \"sphinx.ext.extlinks\",\n \"sphinxcontrib.bibtex\",\n \"matplotlib.sphinxext.plot_directive\",\n \"sphinx_autodoc_typehints\", # needs to be after napoleon\n \"git_ref\", # needs to be before scanpydoc.rtd_github_links\n \"scanpydoc\", # needs to be before sphinx.ext.linkcode\n \"sphinx.ext.linkcode\",\n \"sphinx_design\",\n \"sphinx_tabs.tabs\",\n \"sphinx_search.extension\",\n \"sphinxext.opengraph\",\n *[p.stem for p in (HERE / \"extensions\").glob(\"*.py\") if p.stem not in {\"git_ref\"}],\n]\n\n# Generate the API documentation when building\nautosummary_generate = True\nautodoc_member_order = \"bysource\"\n# autodoc_default_flags = ['members']\nnapoleon_google_docstring = False\nnapoleon_numpy_docstring = True\nnapoleon_include_init_with_doc = False\nnapoleon_use_rtype = True # having a separate entry generally helps readability\nnapoleon_use_param = True\nnapoleon_custom_sections = [(\"Params\", \"Parameters\")]\ntodo_include_todos = False\napi_dir = HERE / \"api\" # function_images\nmyst_enable_extensions = [\n \"amsmath\",\n \"colon_fence\",\n \"deflist\",\n \"dollarmath\",\n \"html_image\",\n \"html_admonition\",\n]\nmyst_url_schemes = (\"http\", \"https\", \"mailto\", \"ftp\")\nmyst_heading_anchors = 3\nnb_output_stderr = \"remove\"\nnb_execution_mode = \"off\"\nnb_merge_streams = True\n\n\nogp_site_url = \"https://scanpy.readthedocs.io/en/stable/\"\nogp_image = \"https://scanpy.readthedocs.io/en/stable/_static/Scanpy_Logo_BrightFG.svg\"\n\ntypehints_defaults = \"braces\"\n\npygments_style = \"default\"\npygments_dark_style = \"native\"\n\nintersphinx_mapping = dict(\n anndata=(\"https://anndata.readthedocs.io/en/stable/\", None),\n bbknn=(\"https://bbknn.readthedocs.io/en/latest/\", None),\n cuml=(\"https://docs.rapids.ai/api/cuml/stable/\", None),\n cycler=(\"https://matplotlib.org/cycler/\", None),\n dask=(\"https://docs.dask.org/en/stable/\", None),\n dask_ml=(\"https://ml.dask.org/\", None),\n h5py=(\"https://docs.h5py.org/en/stable/\", None),\n ipython=(\"https://ipython.readthedocs.io/en/stable/\", None),\n igraph=(\"https://python.igraph.org/en/stable/api/\", None),\n leidenalg=(\"https://leidenalg.readthedocs.io/en/latest/\", None),\n louvain=(\"https://louvain-igraph.readthedocs.io/en/latest/\", None),\n matplotlib=(\"https://matplotlib.org/stable/\", None),\n networkx=(\"https://networkx.org/documentation/stable/\", None),\n numpy=(\"https://numpy.org/doc/stable/\", None),\n pandas=(\"https://pandas.pydata.org/pandas-docs/stable/\", None),\n pynndescent=(\"https://pynndescent.readthedocs.io/en/latest/\", None),\n pytest=(\"https://docs.pytest.org/en/latest/\", None),\n python=(\"https://docs.python.org/3\", None),\n rapids_singlecell=(\"https://rapids-singlecell.readthedocs.io/en/latest/\", None),\n scipy=(\"https://docs.scipy.org/doc/scipy/\", None),\n seaborn=(\"https://seaborn.pydata.org/\", None),\n sklearn=(\"https://scikit-learn.org/stable/\", None),\n)\n\n\n# -- Options for HTML output ----------------------------------------------\n\n# The theme is sphinx-book-theme, with patches for readthedocs-sphinx-search\nhtml_theme = \"scanpydoc\"\nhtml_theme_options = {\n \"repository_url\": repository_url,\n \"use_repository_button\": True,\n}\nhtml_static_path = [\"_static\"]\nhtml_show_sphinx = False\nhtml_logo = \"_static/img/Scanpy_Logo_BrightFG.svg\"\nhtml_title = \"scanpy\"\n\n\ndef setup(app: Sphinx):\n \"\"\"App setup hook.\"\"\"\n app.add_generic_role(\"small\", partial(nodes.inline, classes=[\"small\"]))\n app.add_generic_role(\"smaller\", partial(nodes.inline, classes=[\"smaller\"]))\n app.add_config_value(\n \"recommonmark_config\",\n {\n \"auto_toc_tree_section\": \"Contents\",\n \"enable_auto_toc_tree\": True,\n \"enable_math\": True,\n \"enable_inline_math\": False,\n \"enable_eval_rst\": True,\n },\n True, # noqa: FBT003\n )\n\n\n# -- Options for other output formats ------------------------------------------\n\nhtmlhelp_basename = f\"{project}doc\"\ndoc_title = f\"{project} Documentation\"\nlatex_documents = [(master_doc, f\"{project}.tex\", doc_title, author, \"manual\")]\nman_pages = [(master_doc, project, doc_title, [author], 1)]\ntexinfo_documents = [\n (\n master_doc,\n project,\n doc_title,\n author,\n project,\n \"One line description of project.\",\n \"Miscellaneous\",\n )\n]\n\n\n# -- Suppress link warnings ----------------------------------------------------\n\nqualname_overrides = {\n \"sklearn.neighbors._dist_metrics.DistanceMetric\": \"sklearn.metrics.DistanceMetric\",\n \"scanpy.plotting._matrixplot.MatrixPlot\": \"scanpy.pl.MatrixPlot\",\n \"scanpy.plotting._dotplot.DotPlot\": \"scanpy.pl.DotPlot\",\n \"scanpy.plotting._stacked_violin.StackedViolin\": \"scanpy.pl.StackedViolin\",\n \"pandas.core.series.Series\": \"pandas.Series\",\n \"numpy.bool_\": \"numpy.bool\", # Since numpy 2, numpy.bool is the canonical dtype\n}\n\nnitpick_ignore = [\n # Technical issues\n (\"py:class\", \"numpy.int64\"), # documented as “attribute”\n (\"py:class\", \"numpy._typing._dtype_like._SupportsDType\"),\n (\"py:class\", \"numpy._typing._dtype_like._DTypeDict\"),\n # Will probably be documented\n (\"py:class\", \"scanpy._settings.Verbosity\"),\n (\"py:class\", \"scanpy.neighbors.OnFlySymMatrix\"),\n (\"py:class\", \"scanpy.plotting._baseplot_class.BasePlot\"),\n # Currently undocumented\n # https://github.com/mwaskom/seaborn/issues/1810\n (\"py:class\", \"seaborn.matrix.ClusterGrid\"),\n (\"py:class\", \"samalg.SAM\"),\n # Won’t be documented\n (\"py:class\", \"scanpy.plotting._utils._AxesSubplot\"),\n (\"py:class\", \"scanpy._utils.Empty\"),\n (\"py:class\", \"numpy.random.mtrand.RandomState\"),\n (\"py:class\", \"scanpy.neighbors._types.KnnTransformerLike\"),\n # Will work once scipy 1.8 is released\n (\"py:class\", \"scipy.sparse.base.spmatrix\"),\n (\"py:class\", \"scipy.sparse.csr.csr_matrix\"),\n]\n\n# Options for plot examples\n\nplot_include_source = True\nplot_formats = [(\"png\", 90)]\nplot_html_show_formats = False\nplot_html_show_source_link = False\nplot_working_directory = HERE.parent # Project root\n\n# link config\nextlinks = {\n \"issue\": (\"https://github.com/scverse/scanpy/issues/%s\", \"issue%s\"),\n \"pr\": (\"https://github.com/scverse/scanpy/pull/%s\", \"pr%s\"),\n}\nrtd_links_prefix = PurePosixPath(\"src\")\n\n\n# Installation\n\nTo use `scanpy` from another project, install it using your favourite environment manager:\n\n::::{tabs}\n\n:::{group-tab} Hatch (recommended)\nAdding `scanpy[leiden]` to your dependencies is enough.\nSee below for how to use Scanpy’s {ref}`dev-install-instructions`.\n:::\n\n:::{group-tab} Pip/PyPI\nIf you prefer to exclusively use PyPI run:\n\n```console\n$ pip install 'scanpy[leiden]'\n```\n:::\n\n:::{group-tab} Conda\nAfter installing installing e.g. [Miniconda][], run:\n\n```console\n$ conda install -c conda-forge scanpy python-igraph leidenalg\n```\n\nPull Scanpy [from PyPI][] (consider using `pip3` to access Python 3):\n\n```console\n$ pip install scanpy\n```\n\n[miniconda]: https://docs.anaconda.com/miniconda/miniconda-install/\n[from pypi]: https://pypi.org/project/scanpy\n:::\n\n::::\n\nIf you use Hatch or pip, the extra `[leiden]` installs two packages that are needed for popular\nparts of scanpy but aren't requirements: [igraph][] {cite:p}`Csardi2006` and [leiden][] {cite:p}`Traag2019`.\nIf you use conda, you should to add these dependencies to your environment individually.\n\n[igraph]: https://python.igraph.org/en/stable/\n[leiden]: https://leidenalg.readthedocs.io\n\n(dev-install-instructions)=\n\n## Development Version\n\nTo work with the latest version [on GitHub][]: clone the repository and `cd` into its root directory.\n\n```console\n$ gh repo clone scverse/scanpy\n$ cd scanpy\n```\n\n::::{tabs}\n\n:::{group-tab} Hatch (recommended)\nTo use one of the predefined [Hatch environments][] in {file}`hatch.toml`,\nrun either `hatch test [args]` or `hatch run [env:]command [...args]`, e.g.:\n\n```console\n$ hatch test -p # run tests in parallel\n$ hatch run docs:build # build docs\n$ hatch run towncrier:create # create changelog entry\n```\n\n[hatch environments]: https://hatch.pypa.io/latest/tutorials/environment/basic-usage/\n:::\n\n:::{group-tab} Pip/PyPI\nIf you are using `pip>=21.3`, an editable install can be made:\n\n```console\n$ python -m venv .venv\n$ source .venv/bin/activate\n$ pip install -e '.[dev,test]'\n```\n:::\n\n:::{group-tab} Conda\nIf you want to let `conda` handle the installations of dependencies, do:\n\n```console\n$ pipx install beni\n$ beni pyproject.toml > environment.yml\n$ conda env create -f environment.yml\n$ conda activate scanpy\n$ pip install -e '.[dev,doc,test]'\n```\n\nFor instructions on how to work with the code, see the {ref}`contribution guide `.\n:::\n\n::::\n\n[on github]: https://github.com/scverse/scanpy\n\n## Docker\n\nIf you're using [Docker][], you can use e.g. the image [gcfntnu/scanpy][] from Docker Hub.\n\n[docker]: https://en.wikipedia.org/wiki/Docker_(software)\n[gcfntnu/scanpy]: https://hub.docker.com/r/gcfntnu/scanpy\n\n## Troubleshooting\n\nIf you get a `Permission denied` error, never use `sudo pip`. Instead, use virtual environments or:\n\n```console\n$ pip install --user scanpy\n```\n\n\n# Contributors\n\n[anndata graph](https://github.com/scverse/anndata/graphs/contributors>) | [scanpy graph](https://github.com/scverse/scanpy/graphs/contributors)| ☀ = maintainer\n## Current developers\n\n- [Isaac Virshup](https://github.com/ivirshup), lead developer since 2019 ☀\n- [Gökcen Eraslan](https://twitter.com/gokcen), developer, diverse contributions ☀\n- [Sergei Rybakov](https://github.com/Koncopd), developer, diverse contributions ☀\n- [Fidel Ramirez](https://github.com/fidelram) developer, plotting ☀\n- [Giovanni Palla](https://twitter.com/g_palla1), developer, spatial data\n- [Malte Luecken](https://twitter.com/MDLuecken), developer, community & forum\n- [Lukas Heumos](https://twitter.com/LukasHeumos), developer, diverse contributions\n- [Philipp Angerer](https://github.com/flying-sheep), developer, software quality, initial anndata conception ☀\n\n## Other roles\n\n- [Alex Wolf](https://twitter.com/falexwolf): lead developer 2016-2019, initial anndata & scanpy conception\n- [Fabian Theis](https://twitter.com/fabian_theis) & lab: enabling guidance, support and environment\n\n## Former developers\n\n- Tom White: developer 2018-2019, distributed computing\n\n\n# Ecosystem\n\n```{warning}\nWe are no longer accepting new tools on this page.\nInstead, please submit your tool to the [scverse ecosystem package listing](https://scverse.org/packages/#ecosystem).\n```\n\n## Viewers\n\nInteractive manifold viewers.\n\n- [cellxgene](https://github.com/chanzuckerberg/cellxgene) via direct reading of `.h5ad` {small}`CZI`\n- [cirrocumulus](https://cirrocumulus.readthedocs.io/) via direct reading of `.h5ad` {small}`Broad Inst.`\n- [cell browser](https://cells.ucsc.edu/) via exporing through {func}`~scanpy.external.exporting.cellbrowser` {small}`UCSC`\n- [SPRING](https://github.com/AllonKleinLab/SPRING) via exporting through {func}`~scanpy.external.exporting.spring_project` {small}`Harvard Med`\n- [vitessce](https://github.com/vitessce/vitessce#readme) for purely browser based viewing of zarr formatted AnnData files {smaller}`Harvard Med`\n\n## Portals\n\n- the [Gene Expression Analysis Resource](https://umgear.org/) {small}`U Maryland`\n- the [Galaxy Project](https://humancellatlas.usegalaxy.eu) for the Human Cell Atlas [\\[tweet\\]](https://twitter.com/ExpressionAtlas/status/1151797848469626881) {small}`U Freiburg`\n- the [Expression Atlas](https://www.ebi.ac.uk/gxa/sc/help.html) {small}`EMBL-EBI`\n\n## Modalities\n\n### RNA velocity\n\n- [scVelo](https://scvelo.org) {small}`Helmholtz Munich`\n\n### Spatial Transcriptomics Tools\n\n- [squidpy](https://squidpy.readthedocs.io/en/stable/) {small}`Helmholtz Munich`\n\n > Squidpy is a comprehensive toolkit for working with spatial single cell omics data.\n\n- [PASTE](https://github.com/raphael-group/paste) {small}`Princeton`\n\n > PASTE is a computational method to align and integrate spatial transcriptomics data across adjacent tissue slices by leveraging both gene expression similarity and spatial distances between spots.\n\n- [bento](https://bento-tools.readthedocs.io/en/latest/) 🍱 {small}`UC San Diego`\n\n > Bento is an accessible Python toolkit for performing subcellular analysis of spatial transcriptomics data.\n\n### Multimodal integration\n\n- [MUON](https://muon.readthedocs.io/en/latest/) and [MuData](https://mudata.readthedocs.io/en/latest/) {small}`EMBL/ DKFZ`\n\n > MUON, and it's associated data structure MuData are designed to organise, analyse, visualise, and exchange multimodal data.\n > MUON enables a range of analyses for ATAC and CITE-seq, from data preprocessing to flexible multi-omics alignment.\n\n### Adaptive immune receptor repertoire (AIRR)\n\n- [scirpy](https://github.com/icbi-lab/scirpy) {small}`Medical University of Innsbruck`\n\n > scirpy is a scanpy extension to expore single-cell T-cell receptor (TCR) and B-cell receptor (BCR) repertoires.\n\n- [dandelion](https://github.com/zktuong/dandelion) {small}`University of Cambridge`\n\n > dandelion is a single-cell BCR-seq network analysis package that integrates with transcriptomic data analyzed via scanpy.\n\n### Long reads\n\n- [Swan](https://freese.gitbook.io/swan/tutorials/data_processing) {small}`UC Irvine`\n\n > Swan is a Python library designed for the analysis and visualization of transcriptomes, especially with long-read transcriptomes in mind.\n > Users can add transcriptomes from different datasets and explore distinct splicing and expression patterns across datasets.\n\n## Analysis methods\n\n### scvi-tools\n\n- [scvi-tools](https://github.com/YosefLab/scvi-tools) {small}`Berkeley`\n\n > scvi-tools hosts deep generative models (DGM) for end-to-end analysis of single-cell\n > omics data (e.g., scVI, scANVI, totalVI). It also contains several primitives to build novel DGMs.\n\n### Fate mapping\n\n- [CellRank](https://cellrank.org) {small}`Helmholtz Munich`\n\n > CellRank is a framework to uncover cellular dynamics based on single-cell data.\n > It incorporates modalities such as RNA velocity, pseudotime, developmental potential, real-time information, etc.\n\n### Differential expression\n\n- [diffxpy](https://github.com/theislab/diffxpy) {small}`Helmholtz Munich`\n\n### Data integration\n\n- [scanaroma](https://github.com/brianhie/scanorama) {small}`MIT`\n\n### Modeling perturbations\n\n- [scGen](https://github.com/theislab/scgen) / [trVAE](https://github.com/theislab/trvae) {small}`Helmholtz Munich`\n\n### Feature selection\n\n- [triku 🦔](https://gitlab.com/alexmascension/triku) {small}`Biodonostia Health Research Institute`\n- [CIARA](https://github.com/ScialdoneLab/CIARA_python) {small}`Helmholtz Munich`\n\n > CIARA is an algorithm for feature selection, that aims for the identification of rare cell types via scRNA-Seq data in scanpy.\n\n### Annotation/ Enrichment Analysis\n\nAnalyses using curated prior knowledge\n\n- [decoupler](https://github.com/saezlab/decoupler-py) is a collection of footprint enrichment methods that allows to infer transcription factor or pathway activities. {small}`Institute for Computational Biomedicine, Heidelberg University`\n- [Cubé](https://github.com/connerlambden/Cube) {small}`Harvard University`\n\n > Intuitive Nonparametric Gene Network Search Algorithm that learns from existing biological pathways & multiplicative gene interference patterns.\n\n\n```{include} ../README.md\n:end-before: '## Citation'\n```\n\n::::{grid} 1 2 3 3\n:gutter: 2\n\n:::{grid-item-card} Installation {octicon}`plug;1em;`\n:link: installation\n:link-type: doc\n\nNew to *scanpy*? Check out the installation guide.\n:::\n\n:::{grid-item-card} Tutorials {octicon}`play;1em;`\n:link: tutorials/index\n:link-type: doc\n\nThe tutorials walk you through real-world applications of scanpy.\n:::\n\n:::{grid-item-card} API reference {octicon}`book;1em;`\n:link: api/index\n:link-type: doc\n\nThe API reference contains a detailed description of\nthe scanpy API.\n:::\n\n:::{grid-item-card} Discussion {octicon}`megaphone;1em;`\n:link: https://discourse.scverse.org\n\nNeed help? Reach out on our forum to get your questions answered!\n:::\n\n:::{grid-item-card} GitHub {octicon}`mark-github;1em;`\n:link: https://github.com/scverse/scanpy\n\nFind a bug? Interested in improving scanpy? Checkout our GitHub for the latest developments.\n:::\n::::\n\n**Other resources**\n\n* Follow changes in the {ref}`release notes `.\n* Find tools that harmonize well with anndata & Scanpy at [scverse.org/packages/](https://scverse.org/packages/)\n* Check out our {ref}`contribution guide ` for development practices.\n* Consider citing [Genome Biology (2018)] along with original {doc}`references `.\n\n## News\n\n```{include} news.md\n:start-after: ''\n:end-before: ''\n```\n\n{ref}`(past news) `\n\n% put references first so all references are resolved\n\n% NO! there is a particular meaning to this sequence\n\n```{toctree}\n:hidden: true\n:maxdepth: 1\n\ninstallation\ntutorials/index\nusage-principles\nhow-to/index\napi/index\nexternal/index\necosystem\nrelease-notes/index\ncommunity\nnews\ndev/index\ncontributors\nreferences\n```\n\n[contribution guide]: dev/index.md\n[genome biology (2018)]: https://doi.org/10.1186/s13059-017-1382-0\n[github]: https://github.com/scverse/scanpy\n\n\n(News)=\n## News\n\n\n\n### `rapids-singlecell` brings scanpy to the GPU! {small}`2024-03-18`\n\n{doc}`rapids-singlecell ` by Severin Dicks provides a scanpy-like API with accelerated operations implemented on GPU.\n\n### Scanpy hits 100 contributors! {small}`2022-03-31`\n\n[100 people have contributed to Scanpy's source code!](https://github.com/scverse/scanpy/graphs/contributors)\n\nOf course, contributions to the project are not limited to direct modification of the source code.\nMany others have improved the project by building on top of it, participating in development discussions, helping others with usage, or by showing off what it's helped them accomplish.\n\nThanks to all our contributors for making this project possible!\n\n### New community channels {small}`2022-03-31`\n\nWe've moved our forums and have a new publicly available chat!\n\n* Our discourse forum has migrated to a joint scverse forum ([discourse.scverse.org](https://discourse.scverse.org)).\n* Our private developer Slack has been replaced by a public Zulip chat ([scverse.zulipchat.com](https://scverse.zulipchat.com)).\n\n### Toolkit for spatial (squidpy) and multimodal (muon) published {small}`2022-02-01`\n\nTwo large toolkits extending our ecosystem to new modalities have had their manuscripts published!\n\n* [Muon](https://muon.readthedocs.io/), a framework for multimodal has been published in [Genome Biology](https://genomebiology.biomedcentral.com/articles/10.1186/s13059-021-02577-8).\n* [Squidpy](https://squidpy.readthedocs.io/) a toolkit for working with spatial single cell data has been published in [Nature Methods](https://www.nature.com/articles/s41592-021-01358-2).\n\n\n\n### scVelo on the cover of Nature Biotechnology {small}`2020-12-01`\n\nScanpy's counterpart for RNA velocity, [scVelo](https://scvelo.org/), made it on the cover of [Nature Biotechnology](https://www.nature.com/nbt/volumes/38/issues/12) \\[[tweet](https://twitter.com/NatureBiotech/status/1334647540030070792)\\].\n\n### Scanpy selected among 20 papers for 20 years of Genome Biology {small}`2020-08-01`\n\n[Genome Biology: Celebrating 20 Years of Genome Biology](https://genomebiology.biomedcentral.com/20years) selected the initial Scanpy paper for the year 2018 among 20 papers for 20 years \\[[tweet](https://twitter.com/falexwolf/status/1295748952504045572)\\].\n\n### COVID-19 datasets distributed as `h5ad` {small}`2020-04-01`\n\nIn a joint initiative, the Wellcome Sanger Institute, the Human Cell Atlas, and the CZI distribute datasets related to COVID-19 via anndata's `h5ad` files: [covid19cellatlas.org](https://www.covid19cellatlas.org/). It wasn't anticipated that the [initial idea](https://falexwolf.de/blog/2017-12-23-anndata-indexing-views-HDF5-backing/) of sharing and backing an on-disk representation of `AnnData` would become so widely adopted. Curious? Read up more on the [format](https://anndata.readthedocs.io/en/latest/fileformat-prose.html).\n\n### Scanpy featured in Nature Biotechnoloogy {small}`2020-02-01`\n\n[Single-cell RNA-seq analysis software providers scramble to offer solutions](https://www.nature.com/articles/s41587-020-0449-8) mentions Scanpy along with Seurat as the two major open source software packages for single-cell analysis \\[[pdf](https://rdcu.be/b2M5l)\\].\n\n### Scanpy has been selected an \"Essential open source software for science\" by CZI {small}`2019-11-14`\n\nScanpy has been selected an [essential open source software for science] by\nCZI among [32 projects], along with giants such as Scipy, Numpy, Pandas,\nMatplotlib, scikit-learn, scikit-image/plotly, pip, jupyterhub/binder,\nBioconda, Seurat, Bioconductor, and others.\n\n### Nature Biotechnology: A comparison of single-cell trajectory inference methods {small}`2019-04-01`\n\n[Nature Biotechnology](https://www.nature.com/articles/s41587-019-0071-9) reviews more than 70 TI tools and ranks PAGA as the best graph-based trajectory inference method, and overall, among the top 3.\n\n### Science “Breakthrough of the Year 2018” {small}`2018-12-01`\n\nThe Science “Breakthrough of the Year 2018”, [Development cell by cell](https://vis.sciencemag.org/breakthrough2018/finalists/#cell-development), mentions the first application of PAGA {cite:p}`Plass2018` among 5 papers.\n\n[32 projects]: https://chanzuckerberg.com/eoss/proposals/\n[essential open source software for science]: https://chanzuckerberg.com/newsroom/chan-zuckerberg-initiative-awards-5-million-for-open-source-software-projects-essential-to-science/\n\n\n# How to\n\nThis section contains short examples on how to perform specific tasks with scanpy.\n\n```{toctree}\nknn-transformers\nplotting-with-marsilea\n```\n\n\n# CI\n\n## Plotting tests\n\nA frequent frustration in testing is the reproducibility of the plots and `matplotlib`'s behaviour in different environments.\nWe have some tooling to help with this.\n\n### Viewing plots from failed tests on Azure pipelines\n\nThe fixtures `check_same_image` and `image_comparer` upload plots from failing tests so you can view them from the azure pipelines test viewer.\nTo find these, navigate to the tests tab for your build\n\n```{image} ../_static/img/ci_plot-view_tests-tab.png\n:width: 750px\n```\n\nSelect your failing test\n\n```{image} ../_static/img/ci_plot-view_select-test.png\n:width: 750px\n```\n\nAnd open the attachments tab\n\n```{image} ../_static/img/ci_plot-view_attachment-tab.png\n:width: 750px\n```\n\nFrom here you can view and download the images which were compared, as well as a diff between them.\n\n### Misc\n\n{func}`matplotlib.testing.setup` tries to establish a consistent environment for creating plots. Make sure it's active!\n\n\n# Making a release\n\nFirst, check out {doc}`versioning` to see which kind of release you want to make.\nThat page also explains concepts like *pre-releases* and applications thereof.\n\n## Preparing the release\n\n1. Switch to the `main` branch for a major/minor release and the respective release series branch for a *patch* release (e.g. `1.8.x` when releasing version 1.8.4).\n2. Run `hatch towncrier:build` to generate a PR that creates a new release notes file. Wait for the PR to be auto-merged.\n3. If it is a *patch* release, merge the backport PR (see {ref}`versioning-tooling`) into the `main` branch.\n\n## Actually making the release\n\n1. Go to GitHub’s [releases][] page.\n2. Click the “Draft a new release” button.\n3. Open the “Choose a tag” dropdown and type the version of the tag you want to release, such as `1.9.6`.\n4. Select the dropdown entry “**+ Create new tag: 1.\\.\\** on publish”.\n5. In the second dropdown “Target:”, select the base branch i.e. `main` for a minor/major release,\n and e.g. `1.9.x` for our example patch release `1.9.6`.\n6. If the version is a *pre-release* version, such as `1.7.0rc1` or `1.10.0a1`, tick the “Set as a pre-release” checkbox.\n\n[releases]: https://github.com/scverse/scanpy/releases\n\n## After making a release\n\nAfter *any* release has been made:\n\n- Create a milestone for the next release (in case you made a bugfix release) or releases (in case of a major/minor release).\n For bugfix releases, this should have `on-merge: backport to 0..x`,\n so the [meeseeksdev][] bot will create a backport PR. See {doc}`versioning` for more info.\n- Clear out and close the milestone you just made a release for.\n\nAfter a *major* or *minor* release has been made:\n\n- Tweet about it! Announce it on Zulip! Announce it on Discourse! Think about making a bot for this! Maybe actually do that?\n- Create a new release notes file for the next minor release. This should only be added to the dev branch.\n- Tag the development branch. If you just released `1.7.0`, this would be `1.8.0.dev0`.\n- Create a new branch for this release series, like `1.7.x`. This should get a new release notes file.\n\n[meeseeksdev]: https://meeseeksbox.github.io\n\n## Debugging the build process\n\nIf you changed something about the build process (e.g. [Hatchling’s build configuration][hatch-build]),\nor something about the package’s structure,\nyou might want to manually check if the build and upload process behaves as expected:\n\n```console\n$ # Clear out old distributions\n$ rm -r dist\n$ # Build source distribution and wheel both\n$ python -m build\n$ # Now check those build artifacts\n$ twine check dist/*\n$ # List the wheel archive’s contents\n$ bsdtar -tf dist/*.whl\n```\n\nYou can also upload the package to ([tutorial][testpypi tutorial])\n```console\n$ twine upload --repository testpypi dist/*\n```\n\nThe above approximates what the [publish workflow][] does automatically for us.\nIf you want to replicate the process more exactly, make sure you are careful,\nand create a version tag before building (make sure you delete it after uploading to TestPyPI!).\n\n[hatch-build]: https://hatch.pypa.io/latest/config/build/\n[testpypi tutorial]: https://packaging.python.org/en/latest/tutorials/packaging-projects/#uploading-the-distribution-archives\n[publish workflow]: https://github.com/scverse/scanpy/tree/main/.github/workflows/publish.yml\n\n\n# Documentation\n\n(building-the-docs)=\n\n## Building the docs\n\nTo build the docs, run `hatch run docs:build`.\nAfterwards, you can run `hatch run docs:open` to open {file}`docs/_build/html/index.html`.\n\nYour browser and Sphinx cache docs which have been built previously.\nSometimes these caches are not invalidated when you've updated the docs.\nIf docs are not updating the way you expect, first try \"force reloading\" your browser page – e.g. reload the page without using the cache.\nNext, if problems persist, clear the sphinx cache (`hatch run docs:clean`) and try building them again.\n\n## Adding to the docs\n\nFor any user-visible changes, please make sure a note has been added to the release notes using [`hatch run towncrier:create`][towncrier create].\nWe recommend waiting on this until your PR is close to done since this can often causes merge conflicts.\n\nOnce you've added a new function to the documentation, you'll need to make sure there is a link somewhere in the documentation site pointing to it.\nThis should be added to `docs/api.md` under a relevant heading.\n\nFor tutorials and more in depth examples, consider adding a notebook to the [scanpy-tutorials][] repository.\n\nThe tutorials are tied to this repository via a submodule.\nTo update the submodule, run `git submodule update --remote` from the root of the repository.\nSubsequently, commit and push the changes in a PR.\nThis should be done before each release to ensure the tutorials are up to date.\n\n[towncrier create]: https://towncrier.readthedocs.io/en/stable/tutorial.html#creating-news-fragments\n[scanpy-tutorials]: https://github.com/scverse/scanpy-tutorials/\n\n## docstrings format\n\nWe use the numpydoc style for writing docstrings.\nWe'd primarily suggest looking at existing docstrings for examples, but the [napolean guide to numpy style docstrings][] is also a great source.\nIf you're unfamiliar with the reStructuredText (rST) markup format, check out the [Sphinx rST primer][].\n\nSome key points:\n\n- We have some custom sphinx extensions activated. When in doubt, try to copy the style of existing docstrings.\n- We autopopulate type information in docstrings when possible, so just add the type information to signatures.\n- When docs exist in the same file as code, line length restrictions still apply. In files which are just docs, go with a sentence per line (for easier `git diff`s).\n- Check that the docs look like what you expect them too! It's easy to forget to add a reference to function, be sure it got added and looks right.\n\nLook at [sc.tl.louvain](https://github.com/scverse/scanpy/blob/a811fee0ef44fcaecbde0cad6336336bce649484/scanpy/tools/_louvain.py#L22-L90) as an example for everything mentioned here.\n\n[napolean guide to numpy style docstrings]: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html#example-numpy\n[sphinx rst primer]: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html\n\n### Plots in docstrings\n\nOne of the most useful things you can include in a docstring is examples of how the function should be used.\nThese are a great way to demonstrate intended usage and give users a template they can copy and modify.\nWe're able to include the plots produced by these snippets in the rendered docs using [matplotlib's plot directive][].\nFor examples of this, see the `Examples` sections of {func}`~scanpy.pl.dotplot` or {func}`~scanpy.pp.calculate_qc_metrics`.\n\nNote that anything in these sections will need to be run when the docs are built, so please keep them computationally light.\n\n- If you need computed features (e.g. an embedding, differential expression results) load data that has this precomputed.\n- Try to re-use datasets, this reduces the amount of data that needs to be downloaded to the CI server.\n\n[matplotlib's plot directive]: https://matplotlib.org/devel/plot_directive.html\n\n### `Params` section\n\nThe `Params` abbreviation is a legit replacement for `Parameters`.\n\nTo document parameter types use type annotations on function parameters.\nThese will automatically populate the docstrings on import, and when the documentation is built.\n\nUse the python standard library types (defined in {mod}`collections.abc` and {mod}`typing` modules) for containers, e.g.\n{class}`~collections.abc.Sequence`s (like `list`),\n{class}`~collections.abc.Iterable`s (like `set`), and\n{class}`~collections.abc.Mapping`s (like `dict`).\nAlways specify what these contain, e.g. `{'a': (1, 2)}` → `Mapping[str, Tuple[int, int]]`.\nIf you can’t use one of those, use a concrete class like `AnnData`.\nIf your parameter only accepts an enumeration of strings, specify them like so: `Literal['elem-1', 'elem-2']`.\n\n### `Returns` section\n\nThere are three types of return sections – prose, tuple, and a mix of both.\n\n1. Prose is for simple cases.\n2. Tuple return sections are formatted like parameters. Other than in numpydoc, each tuple is first characterized by the identifier and *not* by its type. Provide type annotation in the function header.\n3. Mix of prose and tuple is relevant in complicated cases, e.g. when you want to describe that you *added something as annotation to an \\`AnnData\\` object*.\n\n#### Examples\n\nFor simple cases, use prose as in {func}`~scanpy.pp.normalize_total`:\n\n```rst\nReturns\n-------\nReturns dictionary with normalized copies of `adata.X` and `adata.layers`\nor updates `adata` with normalized versions of the original\n`adata.X` and `adata.layers`, depending on `inplace`.\n```\n\nFor tuple return values, you can use the standard numpydoc way of populating it,\ne.g. as in {func}`~scanpy.pp.calculate_qc_metrics`.\nDo not add types in the docstring, but specify them in the function signature:\n\n```python\ndef myfunc(...) -> tuple[int, str]:\n \"\"\"\n ...\n Returns\n -------\n one_identifier\n Description.\n second_identifier\n Description 2.\n \"\"\"\n ...\n```\n\nMany functions also just modify parts of the passed AnnData object, like e.g. {func}`~scanpy.tl.dpt`.\nYou can then combine prose and lists to best describe what happens:\n\n```rst\nReturns\n-------\nDepending on `copy`, returns or updates `adata` with the following fields.\n\nIf `n_branchings==0`, no field `dpt_groups` will be written.\n\ndpt_pseudotime : :class:`~pandas.Series` (`adata.obs`, dtype `float`)\n Array of dim (number of samples) that stores the pseudotime of each\n cell, that is, the DPT distance with respect to the root cell.\ndpt_groups : :class:`pandas.Series` (`adata.obs`, dtype `category`)\n Array of dim (number of samples) that stores the subgroup id ('0',\n '1', ...) for each cell. The groups typically correspond to\n 'progenitor cells', 'undecided cells' or 'branches' of a process.\n```\n\n\n# Contributing code\n\n## Development workflow\n\n1. {ref}`Fork the Scanpy repository ` to your own GitHub account\n2. Create a {ref}`development environment `\n3. {ref}`Create a new branch ` for your PR\n4. Add your feature or bugfix to the codebase\n5. {ref}`Make sure all tests are passing `\n6. {ref}`Build and visually check any changed documentation `\n7. {ref}`Open a PR back to the main repository `\n\n## Code style\n\nCode contributions will be formatted and style checked using [Ruff][].\nIgnored checks are configured in the `tool.ruff.lint` section of {file}`pyproject.toml`.\nTo learn how to ignore checks per line please read about [ignoring errors][].\nAdditionally, we use Scanpy’s [EditorConfig][],\nso using an editor/IDE with support for both is helpful.\n\n[Ruff]: https://docs.astral.sh/ruff/\n[ignoring errors]: https://docs.astral.sh/ruff/tutorial/#ignoring-errors\n[EditorConfig]: https://github.com/scverse/scanpy/blob/main/.editorconfig\n\n\n# Getting set up\n\n## Working with `git`\n\nThis section of the docs covers our practices for working with `git` on our codebase. For more in-depth guides, we can recommend a few sources:\n\nFor a more complete git tutorials we recommend checking out:\n\n[Atlassian's git tutorial](https://www.atlassian.com/git/tutorials)\n: Beginner friendly introductions to the git command line interface\n\n[Setting up git for GitHub](https://docs.github.com/en/free-pro-team@latest/github/getting-started-with-github/set-up-git)\n: Configuring git to work with your GitHub user account\n\n(forking-and-cloning)=\n\n### Forking and cloning\n\nTo get the code, and be able to push changes back to the main project, you'll need to (1) fork the repository on github and (2) clone the repository to your local machine.\n\nThis is very straight forward if you're using [GitHub's CLI][]:\n\n```console\n$ gh repo fork scverse/scanpy --clone --remote\n```\n\nThis will fork the repo to your github account, create a clone of the repo on your current machine, add our repository as a remote, and set the `main` development branch to track our repository.\n\nTo do this manually, first make a fork of the repository by clicking the \"fork\" button on our main github package. Then, on your machine, run:\n\n```console\n$ # Clone your fork of the repository (substitute in your username)\n$ git clone https://github.com/{your-username}/scanpy.git\n$ # Enter the cloned repository\n$ cd scanpy\n$ # Add our repository as a remote\n$ git remote add upstream https://github.com/scverse/scanpy.git\n$ # git branch --set-upstream-to \"upstream/main\"\n```\n\n[GitHub's CLI]: https://cli.github.com\n\n### `pre-commit`\n\nWe use [pre-commit][] to run some styling checks in an automated way.\nWe also test against these checks, so make sure you follow them!\n\nYou can install pre-commit with:\n\n```console\n$ pip install pre-commit\n```\n\nYou can then install it to run while developing here with:\n\n```console\n$ pre-commit install\n```\n\nFrom the root of the repo.\n\nIf you choose not to run the hooks on each commit, you can run them manually with `pre-commit run --files={your files}`.\n\n[pre-commit]: https://pre-commit.com\n\n(creating-a-branch)=\n\n### Creating a branch for your feature\n\nAll development should occur in branches dedicated to the particular work being done.\nAdditionally, unless you are a maintainer, all changes should be directed at the `main` branch.\nYou can create a branch with:\n\n```console\n$ git checkout main # Starting from the main branch\n$ git pull # Syncing with the repo\n$ git switch -c {your-branch-name} # Making and changing to the new branch\n```\n\n(open-a-pr)=\n\n### Open a pull request\n\nWhen you're ready to have your code reviewed, push your changes up to your fork:\n\n```console\n$ # The first time you push the branch, you'll need to tell git where\n$ git push --set-upstream origin {your-branch-name}\n$ # After that, just use\n$ git push\n```\n\nAnd open a pull request by going to the main repo and clicking *New pull request*.\nGitHub is also pretty good about prompting you to open PRs for recently pushed branches.\n\nWe'll try and get back to you soon!\n\n(dev-environments)=\n\n## Development environments\n\nIt's recommended to do development work in an isolated environment.\nThere are number of ways to do this, including virtual environments, conda environments, and virtual machines.\n\nWe think the easiest is probably [Hatch environments][].\nUsing one of the predefined environments in {file}`hatch.toml` is as simple as running `hatch test` or `hatch run docs:build` (they will be created on demand).\nFor an in-depth guide, refer to the {ref}`development install instructions ` of `scanpy`.\n\n[hatch environments]: https://hatch.pypa.io/latest/tutorials/environment/basic-usage/\n\n\n(tests)=\n\n# Tests\n\nPossibly the most important part of contributing to any open source package is the test suite.\nImplementations may change, but the only way we can know the code is working before making a release is the test suite.\n\n## Running the tests\n\nWe use [pytest][] to test scanpy.\nTo run the tests, simply run `hatch test`.\n\nIt can take a while to run the whole test suite. There are a few ways to cut down on this while working on a PR:\n\n1. Only run a subset of the tests.\n This can be done by specifying paths or test name patterns using the `-k` argument (e.g. `hatch test test_plotting.py` or `hatch test -k \"test_umap*\"`)\n2. Run the tests in parallel using the `-n` argument (e.g. `hatch test -n 8`).\n\n[pytest]: https://docs.pytest.org/en/stable/\n\n### Miscellaneous tips\n\n- A lot of warnings can be thrown while running the test suite.\n It's often easier to read the test results with them hidden via the `--disable-pytest-warnings` argument.\n\n## Writing tests\n\nYou can refer to the [existing test suite][] for examples.\nIf you haven't written tests before, Software Carpentry has an [in-depth testing guide][].\n\nWe highly recommend using [Test-Driven Development][] when contributing code.\nThis not only ensures you have tests written, it often makes implementation easier since you start out with a specification for your function.\n\nConsider parameterizing your tests using the `pytest.mark.parameterize` and `pytest.fixture` decorators.\nYou can read more about [fixtures][] in pytest’s documentation, but we’d also recommend searching our test suite for existing usage.\n\n[existing test suite]: https://github.com/scverse/scanpy/tree/main/scanpy/tests\n[in-depth testing guide]: https://katyhuff.github.io/2016-07-11-scipy/testing/\n[test-driven development]: https://en.wikipedia.org/wiki/Test-driven_development\n[fixtures]: https://docs.pytest.org/en/stable/fixture.html\n\n### What to test\n\nIf you're not sure what to tests about your function, some ideas include:\n\n- Are there arguments which conflict with each other? Check that if they are both passed, the function throws an error (see [`pytest.raises`][] docs).\n- Are there input values which should cause your function to error?\n- Did you add a helpful error message that recommends better outputs? Check that that error message is actually thrown.\n- Can you place bounds on the values returned by your function?\n- Are there different input values which should generate equivalent output (e.g. if an array is sparse or dense)?\n- Do you have arguments which should have orthogonal effects on the output? Check that they are independent. For example, if there is a flag for extended output, the base output should remain the same either way.\n- Are you optimizing a method? Check that it's results are the same as a gold standard implementation.\n\n[`pytest.raises`]: https://docs.pytest.org/en/stable/assert.html#assertions-about-expected-exceptions\n\n### Performance\n\nIt's more important that you're accurately testing the code works than it is that test suite runs quickly.\nThat said, it's nice when the test suite runs fast.\n\nYou can check how long tests take to run by passing `--durations=0` argument to `pytest`.\nHopefully your new tests won't show up on top!\nSome approaches to this include:\n\n- Is there a common setup/ computation happening in each test? Consider caching these in a [scoped test fixture][].\n- Is the behaviour you're testing for dependent on the size of the data? If not, consider reducing it.\n\n[scoped test fixture]: https://docs.pytest.org/en/stable/fixture.html#sharing-test-data\n\n### Plotting tests\n\nWhile computational functions will return arrays and values, it can be harder to work with the output of plotting functions.\n\nTo make this easier, we use the `image_comparer` fixture for comparing plotting results (search the test suite for example usage).\nThis is used to check that generated plots look the same as they did previously.\nReference images (the expected output) are stored as `expected.png` to relevant tests directory under `scanpy/tests/_images`.\nWhen run, the test suite will generate `actual.png` files for each check.\nThese files are compared, and if the `actual` plot differs from the reference plot, a `diff` of the images is also generated.\nPaths for all these files will be reported when a test fails, and images for failed plots can be viewed via the :doc:`CI interface `.\n\nA common gotcha here is that plots often change slightly on different machines/ OSs.\n`scanpy`'s test suite sets a number of environment variables to ensure as similar of plots as possible.\nWhen adding new reference plots, the recommended workflow is to write the test as though an expected result already exists, run it once to generate the output, then move that output to the reference directory.\n\n\n# Versioning\n\n```{note}\nWe are currently experimenting with our development practices.\nThese are currently documented on a best effort basis, but may not be completely accurate.\n```\n\n## Semantic versioning\n\nWe try to follow [semantic versioning](https://semver.org) with our versioning scheme.\nThis scheme breaks down a version number into `{major.minor.point}` sections.\nAt a `point` release, there should be no changes beyond bug fixes.\n`minor` releases can include new features.\n`major` releases can break old APIs.\n\n### Version numbers\n\nValid version numbers are described in [PEP 440](https://peps.python.org/pep-0440/).\n\n[Pre-releases](https://peps.python.org/pep-0440/#pre-releases)\n: should have versions like `1.7.0rc1` or `1.7.0rc2`.\n\n[Development versions](https://peps.python.org/pep-0440/#developmental-releases)\n: should look like `1.8.0.dev0`, with a commit hash optionally appended as a local version identifier (e.g. `1.8.0.dev2+g00ad77b`).\n\n(versioning-tooling)=\n## Tooling\n\nTo be sure we can follow this scheme and maintain some agility in development, we use some tooling and development practices.\nWhen a minor release is made, a release branch should be cut and pushed to the main repo (e.g. `1.7.x` for the `1.7` release series).\n\nFor PRs which fix an bug in the most recent minor release, the changes will need to added to both the development and release branches.\nTo accomplish this, PRs which fix bugs are assigned a patch version milestone such as `1.7.4`.\nOnce the PR is approved and merged, the bot will attempt to make a backport and open a PR.\nThis will sometimes require manual intervention due to merge conflicts or test failures.\n\n### Technical details\n\nThe [meeseeks bot][] reacts to commands like this,\ngiven as a comment on the PR, or a label or milestone description:\n\n> @Meeseeksdev backport \\\n\nIn our case, these commands are part of the milestone description,\nwhich causes the merge of a PR assigned to a milestone to trigger the bot.\n\n[meseeks bot]: https://meeseeksbox.github.io\n\n\n(contribution-guide)=\n\n# Contributing\n\nContributions to scanpy are welcome!\nThis section of the docs provides some guidelines and tips to follow when contributing.\n\n```{toctree}\ncode\ngetting-set-up\ntesting\ndocumentation\nci\nversioning\nrelease\n```\n\nParts of the guidelines have been adapted from the [pandas](https://pandas.pydata.org/pandas-docs/stable/development/index.html) and [MDAnalysis](https://userguide.mdanalysis.org/stable/contributing.html) guides.\nThese are both excellent guides and we highly recommend checking them out.\n\n\n## Queries\n\n```{eval-rst}\n.. module:: scanpy.queries\n```\n\n```{eval-rst}\n.. currentmodule:: scanpy\n```\n\nThis module provides useful queries for annotation and enrichment.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n queries.biomart_annotations\n queries.gene_coordinates\n queries.mitochondrial_genes\n queries.enrich\n\n```\n\n\n## Datasets\n\n```{eval-rst}\n.. module:: scanpy.datasets\n```\n\n```{eval-rst}\n.. currentmodule:: scanpy\n```\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n datasets.blobs\n datasets.ebi_expression_atlas\n datasets.krumsiek11\n datasets.moignard15\n datasets.pbmc3k\n datasets.pbmc3k_processed\n datasets.pbmc68k_reduced\n datasets.paul15\n datasets.toggleswitch\n datasets.visium_sge\n\n```\n\n\n## Get object from `AnnData`: `get`\n\n```{eval-rst}\n.. module:: scanpy.get\n```\n\n```{eval-rst}\n.. currentmodule:: scanpy\n```\n\nThe module `sc.get` provides convenience functions for getting values back in\nuseful formats.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n get.obs_df\n get.var_df\n get.rank_genes_groups_df\n get.aggregate\n\n```\n\n\n## Reading\n\n```{eval-rst}\n.. currentmodule:: scanpy\n```\n\n```{note}\nFor reading annotation use {ref}`pandas.read_… `\nand add it to your {class}`anndata.AnnData` object. The following read functions are\nintended for the numeric data in the data matrix `X`.\n```\n\nRead common file formats using\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n read\n```\n\nRead 10x formatted hdf5 files and directories containing `.mtx` files using\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n read_10x_h5\n read_10x_mtx\n read_visium\n```\n\nRead other formats using functions borrowed from {mod}`anndata`\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n read_h5ad\n read_csv\n read_excel\n read_hdf\n read_loom\n read_mtx\n read_text\n read_umi_tools\n\n```\n\n\n## Classes\n\n\n```{eval-rst}\n.. currentmodule:: scanpy\n```\n\n{class}`~anndata.AnnData` is reexported from {mod}`anndata`.\n\nRepresent data as a neighborhood structure, usually a knn graph.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n Neighbors\n\n```\n\n\n## Preprocessing: `pp`\n\n```{eval-rst}\n.. module:: scanpy.pp\n```\n\n```{eval-rst}\n.. currentmodule:: scanpy\n```\n\nFiltering of highly-variable genes, batch-effect correction, per-cell normalization, preprocessing recipes.\n\nAny transformation of the data matrix that is not a *tool*. Other than *tools*, preprocessing steps usually don't return an easily interpretable annotation, but perform a basic transformation on the data matrix.\n\n### Basic Preprocessing\n\nFor visual quality control, see {func}`~scanpy.pl.highest_expr_genes` and\n{func}`~scanpy.pl.filter_genes_dispersion` in {mod}`scanpy.pl`.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n pp.calculate_qc_metrics\n pp.filter_cells\n pp.filter_genes\n pp.highly_variable_genes\n pp.log1p\n pp.pca\n pp.normalize_total\n pp.regress_out\n pp.scale\n pp.subsample\n pp.downsample_counts\n```\n\n### Recipes\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/\n\n pp.recipe_zheng17\n pp.recipe_weinreb17\n pp.recipe_seurat\n```\n\n### Batch effect correction\n\nAlso see [Data integration]. Note that a simple batch correction method is available via {func}`pp.regress_out`. Checkout {mod}`scanpy.external` for more.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/\n\n pp.combat\n```\n\n### Doublet detection\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/\n\n pp.scrublet\n pp.scrublet_simulate_doublets\n```\n\n### Neighbors\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/\n\n pp.neighbors\n\n```\n\n\n\n\n## Experimental\n\n```{eval-rst}\n.. module:: scanpy.experimental\n.. currentmodule:: scanpy\n```\n\nNew methods that are in early development which are not (yet)\nintegrated in Scanpy core.\n\n```{eval-rst}\n.. module:: scanpy.experimental.pp\n.. currentmodule:: scanpy\n```\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n experimental.pp.normalize_pearson_residuals\n experimental.pp.normalize_pearson_residuals_pca\n experimental.pp.highly_variable_genes\n experimental.pp.recipe_pearson_residuals\n```\n\n\n## Metrics\n\n```{eval-rst}\n.. module:: scanpy.metrics\n```\n\n```{eval-rst}\n.. currentmodule:: scanpy\n```\n\nCollections of useful measurements for evaluating results.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n metrics.confusion_matrix\n metrics.gearys_c\n metrics.morans_i\n\n```\n\n\n## Tools: `tl`\n\n```{eval-rst}\n.. module:: scanpy.tl\n```\n\n```{eval-rst}\n.. currentmodule:: scanpy\n```\n\nAny transformation of the data matrix that is not *preprocessing*. In contrast to a *preprocessing* function, a *tool* usually adds an easily interpretable annotation to the data matrix, which can then be visualized with a corresponding plotting function.\n\n### Embeddings\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n pp.pca\n tl.tsne\n tl.umap\n tl.draw_graph\n tl.diffmap\n```\n\nCompute densities on embeddings.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n tl.embedding_density\n```\n\n### Clustering and trajectory inference\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n tl.leiden\n tl.louvain\n tl.dendrogram\n tl.dpt\n tl.paga\n```\n\n### Data integration\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n tl.ingest\n```\n\n### Marker genes\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n tl.rank_genes_groups\n tl.filter_rank_genes_groups\n tl.marker_gene_overlap\n```\n\n### Gene scores, Cell cycle\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n tl.score_genes\n tl.score_genes_cell_cycle\n```\n\n### Simulations\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n tl.sim\n\n```\n\n\n## Plotting: `pl`\n\n```{eval-rst}\n.. module:: scanpy.pl\n```\n\n```{eval-rst}\n.. currentmodule:: scanpy\n```\n\nThe plotting module {mod}`scanpy.pl` largely parallels the `tl.*` and a few of the `pp.*` functions.\nFor most tools and for some preprocessing functions, you'll find a plotting function with the same name.\n\nSee {doc}`/tutorials/plotting/core` for an overview of how to use these functions.\n\n```{note}\nSee the {ref}`settings` section for all important plotting configurations.\n```\n\n(pl-generic)=\n\n### Generic\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n pl.scatter\n pl.heatmap\n pl.dotplot\n pl.tracksplot\n pl.violin\n pl.stacked_violin\n pl.matrixplot\n pl.clustermap\n pl.ranking\n pl.dendrogram\n\n```\n\n### Classes\n\nThese classes allow fine tuning of visual parameters.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/classes\n\n pl.DotPlot\n pl.MatrixPlot\n pl.StackedViolin\n\n```\n\n### Preprocessing\n\nMethods for visualizing quality control and results of preprocessing functions.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/\n\n pl.highest_expr_genes\n pl.filter_genes_dispersion\n pl.highly_variable_genes\n pl.scrublet_score_distribution\n\n```\n\n### Tools\n\nMethods that extract and visualize tool-specific annotation in an\n{class}`~anndata.AnnData` object. For any method in module `tl`, there is\na method with the same name in `pl`.\n\n#### PCA\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/\n\n pl.pca\n pl.pca_loadings\n pl.pca_variance_ratio\n pl.pca_overview\n```\n\n#### Embeddings\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/\n\n pl.tsne\n pl.umap\n pl.diffmap\n pl.draw_graph\n pl.spatial\n pl.embedding\n```\n\nCompute densities on embeddings.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/\n\n pl.embedding_density\n```\n\n#### Branching trajectories and pseudotime, clustering\n\nVisualize clusters using one of the embedding methods passing `color='louvain'`.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/\n\n pl.dpt_groups_pseudotime\n pl.dpt_timeseries\n pl.paga\n pl.paga_path\n pl.paga_compare\n```\n\n#### Marker genes\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/\n\n pl.rank_genes_groups\n pl.rank_genes_groups_violin\n pl.rank_genes_groups_stacked_violin\n pl.rank_genes_groups_heatmap\n pl.rank_genes_groups_dotplot\n pl.rank_genes_groups_matrixplot\n pl.rank_genes_groups_tracksplot\n```\n\n#### Simulations\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: generated/\n\n pl.sim\n\n```\n\n\n## Deprecated functions\n\n```{eval-rst}\n.. currentmodule:: scanpy\n```\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n pp.filter_genes_dispersion\n pp.normalize_per_cell\n```\n\n\n# API\n\nImport Scanpy as:\n\n```\nimport scanpy as sc\n```\n\n```{note}\nAdditional functionality is available in the broader {doc}`ecosystem <../ecosystem>`, with some tools being wrapped in the {mod}`scanpy.external` module.\n```\n\n```{toctree}\n:maxdepth: 2\n\npreprocessing\ntools\nplotting\nreading\nget\nqueries\nmetrics\nexperimental\nclasses\nsettings\ndatasets\ndeprecated\n```\n\n\n(settings)=\n\n## Settings\n\n\n```{eval-rst}\n.. currentmodule:: scanpy\n```\n\nA convenience function for setting some default {obj}`matplotlib.rcParams` and a\nhigh-resolution jupyter display backend useful for use in notebooks.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n set_figure_params\n```\n\nAn instance of the {class}`~scanpy._settings.ScanpyConfig` is available as `scanpy.settings` and allows configuring Scanpy.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n _settings.ScanpyConfig\n```\n\nSome selected settings are discussed in the following.\n\nInfluence the global behavior of plotting functions. In non-interactive scripts,\nyou'd usually want to set `settings.autoshow` to `False`.\n\n% no :toctree: here because they are linked under the class\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n\n ~_settings.ScanpyConfig.autoshow\n ~_settings.ScanpyConfig.autosave\n```\n\nThe default directories for saving figures, caching files and storing datasets.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n\n ~_settings.ScanpyConfig.figdir\n ~_settings.ScanpyConfig.cachedir\n ~_settings.ScanpyConfig.datasetdir\n```\n\nThe verbosity of logging output, where verbosity levels have the following\nmeaning: 0='error', 1='warning', 2='info', 3='hint', 4=more details, 5=even more\ndetails, etc.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n\n ~_settings.ScanpyConfig.verbosity\n```\n\nPrint versions of packages that might influence numerical results.\n\n```{eval-rst}\n.. autosummary::\n :nosignatures:\n :toctree: ../generated/\n\n logging.print_header\n logging.print_versions\n\n```\n\n\n(v0.1.0)=\n### 0.1.0 {small}`2017-05-17`\n\nScanpy computationally outperforms and allows reproducing both the [Cell Ranger\nR kit's](https://github.com/scverse/scanpy_usage/tree/master/170503_zheng17)\nand most of [Seurat’s](https://github.com/scverse/scanpy_usage/tree/master/170505_seurat)\nclustering workflows. {smaller}`A Wolf, P Angerer`\n\n\n(v1.9.3)=\n### 1.9.3 {small}`2023-03-02`\n\n#### Bug fixes\n\n* Variety of fixes against pandas 2.0.0rc0 {pr}`2434` {smaller}`I Virshup`\n\n\n(v0.4.4)=\n### 0.4.4 {small}`2018-02-26`\n\n- embed cells using {func}`~scanpy.tl.umap` {cite:p}`McInnes2018` {pr}`92` {smaller}`G Eraslan`\n- score sets of genes, e.g. for cell cycle, using {func}`~scanpy.tl.score_genes` {cite:p}`Satija2015`:\n [notebook](https://nbviewer.jupyter.org/github/theislab/scanpy_usage/blob/master/180209_cell_cycle/cell_cycle.ipynb)\n\n\n(v1.7.0)=\n### 1.7.0 {small}`2021-02-03`\n\n#### Features\n\n- Add new 10x Visium datasets to {func}`~scanpy.datasets.visium_sge` {pr}`1473` {smaller}`G Palla`\n- Enable download of source image for 10x visium datasets in {func}`~scanpy.datasets.visium_sge` {pr}`1506` {smaller}`H Spitzer`\n- Refactor of {func}`scanpy.pl.spatial`. Better support for plotting without an image, as well as directly providing images {pr}`1512` {smaller}`G Palla`\n- Dict input for {func}`scanpy.queries.enrich` {pr}`1488` {smaller}`G Eraslan`\n- {func}`~scanpy.get.rank_genes_groups_df` can now return fraction of cells in a group expressing a gene, and allows retrieving values for multiple groups at once {pr}`1388` {smaller}`G Eraslan`\n- Color annotations for gene sets in {func}`~scanpy.pl.heatmap` are now matched to color for cluster {pr}`1511` {smaller}`L Sikkema`\n- PCA plots can now annotate axes with variance explained {pr}`1470` {smaller}`bfurtwa`\n- Plots with `groupby` arguments can now group by values in the index by passing the index's name (like `pd.DataFrame.groupby`). {pr}`1583` {smaller}`F Ramirez`\n- Added `na_color` and `na_in_legend` keyword arguments to {func}`~scanpy.pl.embedding` plots. Allows specifying color for missing or filtered values in plots like {func}`~scanpy.pl.umap` or {func}`~scanpy.pl.spatial` {pr}`1356` {smaller}`I Virshup`\n- {func}`~scanpy.pl.embedding` plots now support passing `dict` of `{cluster_name: cluster_color, ...}` for palette argument {pr}`1392` {smaller}`I Virshup`\n\n#### External tools (new)\n\n- Add [Scanorama](https://github.com/brianhie/scanorama) integration to scanpy external API ({func}`~scanpy.external.pp.scanorama_integrate`, {cite:t}`Hie2019`) {pr}`1332` {smaller}`B Hie`\n- Scrublet {cite:p}`Wolock2019` integration: {func}`~scanpy.pp.scrublet`, {func}`~scanpy.pp.scrublet_simulate_doublets`, and plotting method {func}`~scanpy.pl.scrublet_score_distribution` {pr}`1476` {smaller}`J Manning`\n- {func}`~scanpy.external.pp.hashsolo` for HTO demultiplexing {cite:p}`Bernstein2020` {pr}`1432` {smaller}`NJ Bernstein`\n- Added [scirpy](https://github.com/icbi-lab/scirpy) (sc-AIRR analysis) to ecosystem page {pr}`1453` {smaller}`G Sturm`\n- Added [scvi-tools](https://scvi-tools.org) to ecosystem page {pr}`1421` {smaller}`A Gayoso`\n\n#### External tools (changes)\n\n- Updates for {func}`~scanpy.external.tl.palantir` and {func}`~scanpy.external.tl.palantir_results` {pr}`1245` {smaller}`A Mousa`\n- Fixes to {func}`~scanpy.external.tl.harmony_timeseries` docs {pr}`1248` {smaller}`A Mousa`\n- Support for `leiden` clustering by {func}`scanpy.external.tl.phenograph` {pr}`1080` {smaller}`A Mousa`\n- Deprecate `scanpy.external.pp.scvi` {pr}`1554` {smaller}`G Xing`\n- Updated default params of {func}`~scanpy.external.tl.sam` to work with larger data {pr}`1540` {smaller}`A Tarashansky`\n\n#### Documentation\n\n- {ref}`New contribution guide ` {pr}`1544` {smaller}`I Virshup`\n- `zsh` installation instructions {pr}`1444` {smaller}`P Angerer`\n\n#### Performance\n\n- Speed up {func}`~scanpy.read_10x_h5` {pr}`1402` {smaller}`P Weiler`\n- Speed ups for {func}`~scanpy.get.obs_df` {pr}`1499` {smaller}`F Ramirez`\n\n#### Bugfixes\n\n- Consistent fold-change, fractions calculation for filter_rank_genes_groups {pr}`1391` {smaller}`S Rybakov`\n- Fixed bug where `score_genes` would error if one gene was passed {pr}`1398` {smaller}`I Virshup`\n- Fixed `log1p` inplace on integer dense arrays {pr}`1400` {smaller}`I Virshup`\n- Fix docstring formatting for {func}`~scanpy.tl.rank_genes_groups` {pr}`1417` {smaller}`P Weiler`\n- Removed `` PendingDeprecationWarning`s from use of `np.matrix `` {pr}`1424` {smaller}`P Weiler`\n- Fixed indexing byg in `~scanpy.pp.highly_variable_genes` {pr}`1456` {smaller}`V Bergen`\n- Fix default number of genes for marker_genes_overlap {pr}`1464` {smaller}`MD Luecken`\n- Fixed passing `groupby` and `dendrogram_key` to {func}`~scanpy.tl.dendrogram` {pr}`1465` {smaller}`M Varma`\n- Fixed download path of `pbmc3k_processed` {pr}`1472` {smaller}`D Strobl`\n- Better error message when computing DE with a group of size 1 {pr}`1490` {smaller}`J Manning`\n- Update cugraph API usage for v0.16 {pr}`1494` {smaller}`R Ilango`\n- Fixed `marker_gene_overlap` default value for `top_n_markers` {pr}`1464` {smaller}`MD Luecken`\n- Pass `random_state` to RAPIDs UMAP {pr}`1474` {smaller}`C Nolet`\n- Fixed `anndata` version requirement for {func}`~anndata.concat` (re-exported from scanpy as `sc.concat`) {pr}`1491` {smaller}`I Virshup`\n- Fixed the width of the progress bar when downloading data {pr}`1507` {smaller}`M Klein`\n- Updated link for `moignard15` dataset {pr}`1542` {smaller}`I Virshup`\n- Fixed bug where calling `set_figure_params` could block if IPython was installed, but not used. {pr}`1547` {smaller}`I Virshup`\n- {func}`~scanpy.pl.violin` no longer fails if `.raw` not present {pr}`1548` {smaller}`I Virshup`\n- {func}`~scanpy.pl.spatial` refactoring and better handling of spatial data {pr}`1512` {smaller}`G Palla`\n- {func}`~scanpy.pp.pca` works with `chunked=True` again {pr}`1592` {smaller}`I Virshup`\n- {func}`~scanpy.tl.ingest` now works with umap-learn 0.5.0 {pr}`1601` {smaller}`S Rybakov`\n\n\n(v1.3.5)=\n### 1.3.5 {small}`2018-12-09`\n\n- uncountable figure improvements {pr}`369` {smaller}`F Ramirez`\n\n\n(v1.7.2)=\n### 1.7.2 {small}`2021-04-07`\n\n#### Bug fixes\n\n- {func}`scanpy.logging.print_versions` now works when `python<3.8` {pr}`1691` {smaller}`I Virshup`\n- {func}`scanpy.pp.regress_out` now uses `joblib` as the parallel backend, and should stop oversubscribing threads {pr}`1694` {smaller}`I Virshup`\n- {func}`scanpy.pp.highly_variable_genes` with `flavor=\"seurat_v3\"` now returns correct gene means and -variances when used with `batch_key` {pr}`1732` {smaller}`J Lause`\n- {func}`scanpy.pp.highly_variable_genes` now throws a warning instead of an error when non-integer values are passed for method `\"seurat_v3\"`. The check can be skipped by passing `check_values=False`. {pr}`1679` {smaller}`G Palla`\n\n#### Ecosystem\n\n- Added `triku` a feature selection method to the ecosystem page {pr}`1722` {smaller}`AM Ascensión`\n- Added `dorothea` and `progeny` to the ecosystem page {pr}`1767` {smaller}`P Badia-i-Mompel`\n\n\n(v1.8.1)=\n### 1.8.1 {small}`2021-07-07`\n\n#### Bug fixes\n\n- Fixed reproducibility of {func}`scanpy.tl.score_genes`. Calculation and output is now float64 type. {pr}`1890` {smaller}`I Kucinski`\n- Workarounds for some changes/ bugs in pandas 1.3 {pr}`1918` {smaller}`I Virshup`\n- Fixed bug where `sc.pl.paga_compare` could mislabel nodes on the paga graph {pr}`1898` {smaller}`I Virshup`\n- Fixed handling of `use_raw` with {func}`scanpy.tl.rank_genes_groups` {pr}`1934` {smaller}`I Virshup`\n\n\n(v1.4.1)=\n### 1.4.1 {small}`2019-04-26`\n\n#### New functionality\n\n- Scanpy has a command line interface again. Invoking it with `scanpy somecommand [args]` calls `scanpy-somecommand [args]`, except for builtin commands (currently `scanpy settings`) {pr}`604` {smaller}`P Angerer`\n- {func}`~scanpy.datasets.ebi_expression_atlas` allows convenient download of EBI expression atlas {smaller}`I Virshup`\n- {func}`~scanpy.tl.marker_gene_overlap` computes overlaps of marker genes {smaller}`M Luecken`\n- {func}`~scanpy.tl.filter_rank_genes_groups` filters out genes based on fold change and fraction of cells expressing genes {smaller}`F Ramirez`\n- {func}`~scanpy.pp.normalize_total` replaces {func}`~scanpy.pp.normalize_per_cell`, is more efficient and provides a parameter to only normalize using a fraction of expressed genes {smaller}`S Rybakov`\n- {func}`~scanpy.pp.downsample_counts` has been sped up, changed default value of `replace` parameter to `False` {pr}`474` {smaller}`I Virshup`\n- {func}`~scanpy.tl.embedding_density` computes densities on embeddings {pr}`543` {smaller}`M Luecken`\n- {func}`~scanpy.external.tl.palantir` interfaces Palantir {cite:p}`Setty2019` {pr}`493` {smaller}`A Mousa`\n\n#### Code design\n\n- `.layers` support of scatter plots {smaller}`F Ramirez`\n- fix double-logarithmization in compute of log fold change in {func}`~scanpy.tl.rank_genes_groups` {smaller}`A Muñoz-Rojas`\n- fix return sections of docs {smaller}`P Angerer`\n\n\n(v1.0.0)=\n### 1.0.0 {small}`2018-03-30`\n\n#### Major updates\n\n- Scanpy is much faster and more memory efficient: preprocess, cluster and\n visualize 1.3M cells in [6h], 130K cells in [14min], and 68K cells in [3min] {smaller}`A Wolf`\n- the API gained a preprocessing function {func}`~scanpy.pp.neighbors` and a\n class {func}`~scanpy.Neighbors` to which all basic graph computations are\n delegated {smaller}`A Wolf`\n\n```{warning}\n#### Upgrading to 1.0 isn’t fully backwards compatible in the following changes\n\n- the graph-based tools {func}`~scanpy.tl.louvain`\n {func}`~scanpy.tl.dpt` {func}`~scanpy.tl.draw_graph`\n {func}`~scanpy.tl.umap` {func}`~scanpy.tl.diffmap`\n {func}`~scanpy.tl.paga` require prior computation of the graph:\n `sc.pp.neighbors(adata, n_neighbors=5); sc.tl.louvain(adata)` instead of\n previously `sc.tl.louvain(adata, n_neighbors=5)`\n- install `numba` via `conda install numba`, which replaces cython\n- the default connectivity measure (dpt will look different using default\n settings) changed. setting `method='gauss'` in `sc.pp.neighbors` uses\n gauss kernel connectivities and reproduces the previous behavior,\n see, for instance in the example [paul15].\n- namings of returned annotation have changed for less bloated AnnData\n objects, which means that some of the unstructured annotation of old\n AnnData files is not recognized anymore\n- replace occurances of `group_by` with `groupby` (consistency with\n `pandas`)\n- it is worth checking out the notebook examples to see changes, e.g.\n the [seurat] example.\n- upgrading scikit-learn from 0.18 to 0.19 changed the implementation of PCA,\n some results might therefore look slightly different\n```\n\n#### Further updates\n\n- UMAP {cite:p}`McInnes2018` can serve as a first visualization of the data just as tSNE,\n in contrast to tSNE, UMAP directly embeds the single-cell graph and is faster;\n UMAP is also used for measuring connectivities and computing neighbors,\n see {func}`~scanpy.pp.neighbors` {smaller}`A Wolf`\n- graph abstraction: AGA is renamed to [PAGA](https://github.com/theislab/paga): {func}`~scanpy.tl.paga`; now,\n it only measures connectivities between partitions of the single-cell graph,\n pseudotime and clustering need to be computed separately via\n {func}`~scanpy.tl.louvain` and {func}`~scanpy.tl.dpt`, the\n connectivity measure has been improved {smaller}`A Wolf`\n- logistic regression for finding marker genes\n {func}`~scanpy.tl.rank_genes_groups` with parameter `method='logreg'` {smaller}`A Wolf`\n- {func}`~scanpy.tl.louvain` provides a better implementation for\n reclustering via `restrict_to` {smaller}`A Wolf`\n- scanpy no longer modifies rcParams upon import, call\n `settings.set_figure_params` to set the 'scanpy style' {smaller}`A Wolf`\n- default cache directory is `./cache/`, set `settings.cachedir` to change\n this; nested directories in this are avoided {smaller}`A Wolf`\n- show edges in scatter plots based on graph visualization\n {func}`~scanpy.tl.draw_graph` and {func}`~scanpy.tl.umap` by passing `edges=True` {smaller}`A Wolf`\n- {func}`~scanpy.pp.downsample_counts` for downsampling counts {smaller}`MD Luecken`\n- default `'louvain_groups'` are called `'louvain'` {smaller}`A Wolf`\n- `'X_diffmap'` contains the zero component, plotting remains unchanged {smaller}`A Wolf`\n\n[14min]: https://github.com/scverse/scanpy_usage/blob/master/170522_visualizing_one_million_cells/logfile_130K.txt\n[3min]: https://nbviewer.jupyter.org/github/scverse/scanpy_usage/blob/master/170503_zheng17/zheng17.ipynb\n[6h]: https://github.com/scverse/scanpy_usage/blob/master/170522_visualizing_one_million_cells/\n[paul15]: https://nbviewer.jupyter.org/github/scverse/scanpy_usage/blob/master/170502_paul15/paul15.ipynb\n[seurat]: https://nbviewer.jupyter.org/github/scverse/scanpy_usage/blob/master/170505_seurat/seurat.ipynb\n\n\nFix :meth:`scanpy.pl.DotPlot.style`, :meth:`scanpy.pl.MatrixPlot.style`, and :meth:`scanpy.pl.StackedViolin.style` resetting all non-specified parameters {smaller}`P Angerer`\n\n\n(v1.7.1)=\n### 1.7.1 {small}`2021-02-24`\n\n#### Documentation\n\n- More twitter handles for core devs {pr}`1676` {smaller}`G Eraslan`\n\n#### Bug fixes\n\n- {func}`~scanpy.tl.dendrogram` use `1 - correlation` as distance matrix to compute the dendrogram {pr}`1614` {smaller}`F Ramirez`\n- Fixed {func}`~scanpy.get.obs_df`/ {func}`~scanpy.get.var_df` erroring when `keys` not passed {pr}`1637` {smaller}`I Virshup`\n- Fixed argument handling for {func}`scanpy.pp.scrublet` {smaller}`J Manning`\n- Fixed passing of `kwargs` to {func}`scanpy.pl.violin` when `stripplot` was also used {pr}`1655` {smaller}`M van den Beek`\n- Fixed colorbar creation in `scanpy.pl.timeseries_as_heatmap` {pr}`1654` {smaller}`M van den Beek`\n\n\n(v0.2.1)=\n### 0.2.1 {small}`2017-07-24`\n\nScanpy includes preprocessing, visualization, clustering, pseudotime and\ntrajectory inference, differential expression testing and simulation of gene\nregulatory networks. The implementation efficiently deals with [datasets of more\nthan one million cells](https://github.com/scverse/scanpy_usage/tree/master/170522_visualizing_one_million_cells). {smaller}`A Wolf, P Angerer`\n\n\n(v1.10.2)=\n### 1.10.2 {small}`2024-06-25`\n\n#### Development Process\n\n* Add performance benchmarking {pr}`2977` {smaller}`R Shrestha`, {smaller}`P Angerer`\n\n#### Documentation\n\n* Document several missing parameters in docstring {pr}`2888` {smaller}`S Cheney`\n* Fixed incorrect instructions in \"testing\" dev docs {pr}`2994` {smaller}`I Virshup`\n* Update marsilea tutorial to use `group_` methods {pr}`3001` {smaller}`I Virshup`\n* Fixed citations {pr}`3032` {smaller}`P Angerer`\n* Improve dataset documentation {pr}`3060` {smaller}`P Angerer`\n\n#### Bug fixes\n\n* Compatibility with `matplotlib` 3.9 {pr}`2999` {smaller}`I Virshup`\n* Add clear errors where `backed` mode-like matrices (i.e., from `sparse_dataset`) are not supported {pr}`3048` {smaller}`I gold`\n* Write out full pca results when `_choose_representation` is called i.e., {func}`~scanpy.pp.neighbors` without {func}`~scanpy.pp.pca` {pr}`3078` {smaller}`I gold`\n* Fix deprecated use of `.A` with sparse matrices {pr}`3084` {smaller}`P Angerer`\n* Fix zappy support {pr}`3089` {smaller}`P Angerer`\n* Fix dotplot group order with {mod}`pandas` 1.x {pr}`3101` {smaller}`P Angerer`\n\n#### Performance\n\n* `sparse_mean_variance_axis` now uses all cores for the calculations {pr}`3015` {smaller}`S Dicks`\n* `pp.highly_variable_genes` with `flavor=seurat_v3` now uses a numba kernel {pr}`3017` {smaller}`S Dicks`\n* Speed up {func}`~scanpy.pp.scrublet` {pr}`3044` {smaller}`S Dicks` and {pr}`3056` {smaller}`P Angerer`\n* Speed up clipping of array in {func}`~scanpy.pp.scale` {pr}`3100` {smaller}`P Ashish & S Dicks`\n\n\n(v1.8.0)=\n### 1.8.0 {small}`2021-06-28`\n\n#### Metrics module\n\n- Added {mod}`scanpy.metrics` module!\n\n - Added {func}`scanpy.metrics.gearys_c` for spatial autocorrelation {pr}`915` {smaller}`I Virshup`\n - Added {func}`scanpy.metrics.morans_i` for global spatial autocorrelation {pr}`1740` {smaller}`I Virshup, G Palla`\n - Added {func}`scanpy.metrics.confusion_matrix` for comparing labellings {pr}`915` {smaller}`I Virshup`\n\n#### Features\n\n- Added `layer` and `copy` kwargs to {func}`~scanpy.pp.normalize_total` {pr}`1667` {smaller}`I Virshup`\n- Added `vcenter` and `norm` arguments to the plotting functions {pr}`1551` {smaller}`G Eraslan`\n- Standardized and expanded available arguments to the `sc.pl.rank_genes_groups*` family of functions. {pr}`1529` {smaller}`F Ramirez` {smaller}`I Virshup`\n - See examples sections of {func}`~scanpy.pl.rank_genes_groups_dotplot` and {func}`~scanpy.pl.rank_genes_groups_matrixplot` for demonstrations.\n- {func}`scanpy.tl.tsne` now supports the metric argument and records the passed parameters {pr}`1854` {smaller}`I Virshup`\n- {func}`scanpy.pl.scrublet_score_distribution` now uses same API as other scanpy functions for saving/ showing plots {pr}`1741` {smaller}`J Manning`\n\n#### Ecosystem\n\n- Added [Cubé](https://github.com/connerlambden/Cube) to ecosystem page {pr}`1878` {smaller}`C Lambden`\n- Added `triku` a feature selection method to the ecosystem page {pr}`1722` {smaller}`AM Ascensión`\n- Added `dorothea` and `progeny` to the ecosystem page {pr}`1767` {smaller}`P Badia-i-Mompel`\n\n#### Documentation\n\n- Added {doc}`/community` page to docs {pr}`1856` {smaller}`I Virshup`\n- Added rendered examples to many plotting functions {issue}`1664` {smaller}`A Schaar` {smaller}`L Zappia` {smaller}`bio-la` {smaller}`L Hetzel` {smaller}`L Dony` {smaller}`M Buttner` {smaller}`K Hrovatin` {smaller}`F Ramirez` {smaller}`I Virshup` {smaller}`LouisK92` {smaller}`mayarali`\n- Integrated [DocSearch], a find-as-you-type documentation index search. {pr}`1754` {smaller}`P Angerer`\n- Reorganized reference docs {pr}`1753` {smaller}`I Virshup`\n- Clarified docs issues for {func}`~scanpy.pp.neighbors`,\n {func}`~scanpy.tl.diffmap`, {func}`~scanpy.pp.calculate_qc_metrics` {pr}`1680` {smaller}`G Palla`\n- Fixed typos in grouped plot doc-strings {pr}`1877` {smaller}`C Rands`\n- Extended examples for differential expression plotting. {pr}`1529` {smaller}`F Ramirez`\n - See {func}`~scanpy.pl.rank_genes_groups_dotplot` or {func}`~scanpy.pl.rank_genes_groups_matrixplot` for examples.\n\n#### Bug fixes\n\n- Fix {func}`scanpy.pl.paga_path` `TypeError` with recent versions of anndata {pr}`1047` {smaller}`P Angerer`\n- Fix detection of whether IPython is running {pr}`1844` {smaller}`I Virshup`\n- Fixed reproducibility of {func}`scanpy.tl.diffmap` (added random_state) {pr}`1858` {smaller}`I Kucinski`\n- Fixed errors and warnings from embedding plots with small numbers of categories after `sns.set_palette` was called {pr}`1886` {smaller}`I Virshup`\n- Fixed handling of `gene_symbols` argument in a number of `sc.pl.rank_genes_groups*` functions {pr}`1529` {smaller}`F Ramirez` {smaller}`I Virshup`\n- Fixed handling of `use_raw` for `sc.tl.rank_genes_groups` when no `.raw` is present {pr}`1895` {smaller}`I Virshup`\n- {func}`scanpy.pl.rank_genes_groups_violin` now works for `raw=False` {pr}`1669` {smaller}`M van den Beek`\n- {func}`scanpy.pl.dotplot` now uses `smallest_dot` argument correctly {pr}`1771` {smaller}`S Flemming`\n\n#### Development Process\n\n- Switched to [flit] for building and deploying the package, a simple tool with an easy to understand command line interface and metadata {pr}`1527` {smaller}`P Angerer`\n- Use [pre-commit](https://pre-commit.com) for style checks {pr}`1684` {pr}`1848` {smaller}`L Heumos` {smaller}`I Virshup`\n\n#### Deprecations\n\n- Dropped support for Python 3.6. [More details here](https://numpy.org/neps/nep-0029-deprecation_policy.html). {pr}`1897` {smaller}`I Virshup`\n- Deprecated `layers` and `layers_norm` kwargs to {func}`~scanpy.pp.normalize_total` {pr}`1667` {smaller}`I Virshup`\n- Deprecated `MulticoreTSNE` backend for {func}`scanpy.tl.tsne` {pr}`1854` {smaller}`I Virshup`\n\n[docsearch]: https://docsearch.algolia.com/\n[flit]: https://flit.readthedocs.io/en/latest/\n\n\n(v1.10.0)=\n### 1.10.0 {small}`2024-03-26`\n\n`scanpy` 1.10 brings a large amount of new features, performance improvements, and improved documentation.\n\nSome highlights:\n\n* Improved support for out-of-core workflows via `dask`. See new tutorial: {doc}`/tutorials/experimental/dask` demonstrating counts-to-clusters for 1.4 million cells in <10 min.\n* A new {doc}`basic clustering tutorial ` demonstrating an updated workflow.\n* Opt-in increased performance for neighbor search and clustering ({doc}`how to guide `).\n* Ability to `mask` observations or variables from a number of methods (see {doc}`/tutorials/plotting/advanced` for an example with plotting embeddings)\n* A new function {func}`~scanpy.get.aggregate` for computing aggregations of your data, very useful for pseudo bulking!\n\n#### Features\n\n* {func}`~scanpy.pp.scrublet` and {func}`~scanpy.pp.scrublet_simulate_doublets` were moved from {mod}`scanpy.external.pp` to {mod}`scanpy.pp`. The `scrublet` implementation is now maintained as part of scanpy {pr}`2703` {smaller}`P Angerer`\n* {func}`scanpy.pp.pca`, {func}`scanpy.pp.scale`, {func}`scanpy.pl.embedding`, and {func}`scanpy.experimental.pp.normalize_pearson_residuals_pca` now support a `mask` parameter {pr}`2272` {smaller}`C Bright, T Marcella, & P Angerer`\n* Enhanced dask support for some internal utilities, paving the way for more extensive dask support {pr}`2696` {smaller}`P Angerer`\n* {func}`scanpy.pp.highly_variable_genes` supports dask for the default `seurat` and `cell_ranger` flavors {pr}`2809` {smaller}`P Angerer`\n* New function {func}`scanpy.get.aggregate` which allows grouped aggregations over your data. Useful for pseudobulking! {pr}`2590` {smaller}`Isaac Virshup` {smaller}`Ilan Gold` {smaller}`Jon Bloom`\n* {func}`scanpy.pp.neighbors` now has a `transformer` argument allowing the use of different ANN/ KNN libraries {pr}`2536` {smaller}`P Angerer`\n* {func}`scanpy.experimental.pp.highly_variable_genes` using `flavor='pearson_residuals'` now uses numba for variance computation and is faster {pr}`2612` {smaller}`S Dicks & P Angerer`\n* {func}`scanpy.tl.leiden` now offers `igraph`'s implementation of the leiden algorithm via via `flavor` when set to `igraph`. `leidenalg`'s implementation is still default, but discouraged. {pr}`2815` {smaller}`I Gold`\n* {func}`scanpy.pp.highly_variable_genes` has new flavor `seurat_v3_paper` that is in its implementation consistent with the paper description in Stuart et al 2018. {pr}`2792` {smaller}`E Roellin`\n* {func}`scanpy.datasets.blobs` now accepts a `random_state` argument {pr}`2683` {smaller}`E Roellin`\n* {func}`scanpy.pp.pca` and {func}`scanpy.pp.regress_out` now accept a layer argument {pr}`2588` {smaller}`S Dicks`\n* {func}`scanpy.pp.subsample` with `copy=True` can now be called in backed mode {pr}`2624` {smaller}`E Roellin`\n* {func}`scanpy.external.pp.harmony_integrate` now runs with 64 bit floats improving reproducibility {pr}`2655` {smaller}`S Dicks`\n* {func}`scanpy.tl.rank_genes_groups` no longer warns that it's default was changed from t-test_overestim_var to t-test {pr}`2798` {smaller}`L Heumos`\n* `scanpy.pp.calculate_qc_metrics` now allows `qc_vars` to be passed as a string {pr}`2859` {smaller}`N Teyssier`\n* {func}`scanpy.tl.leiden` and {func}`scanpy.tl.louvain` now store clustering parameters in the key provided by the `key_added` parameter instead of always writing to (or overwriting) a default key {pr}`2864` {smaller}`J Fan`\n* {func}`scanpy.pp.scale` now clips `np.ndarray` also at `- max_value` for zero-centering {pr}`2913` {smaller}`S Dicks`\n* Support sparse chunks in dask {func}`~scanpy.pp.scale`, {func}`~scanpy.pp.normalize_total` and {func}`~scanpy.pp.highly_variable_genes` (`seurat` and `cell-ranger` tested) {pr}`2856` {smaller}`ilan-gold`\n\n#### Documentation\n\n* Doc style overhaul {pr}`2220` {smaller}`A Gayoso`\n* Re-add search-as-you-type, this time via `readthedocs-sphinx-search` {pr}`2805` {smaller}`P Angerer`\n* Fixed a lot of broken usage examples {pr}`2605` {smaller}`P Angerer`\n* Improved harmonization of return field of `sc.pp` and `sc.tl` functions {pr}`2742` {smaller}`E Roellin`\n* Improved docs for `percent_top` argument of {func}`~scanpy.pp.calculate_qc_metrics` {pr}`2849` {smaller}`I Virshup`\n* New basic clustering tutorial ({doc}`/tutorials/basics/clustering`), based on one from [scverse-tutorials](https://scverse-tutorials.readthedocs.io/en/latest/notebooks/basic-scrna-tutorial.html) {pr}`2901` {smaller}`I Virshup`\n* Overhauled {doc}`/tutorials/index` page, and added new {doc}`/how-to/index` section to docs {pr}`2901` {smaller}`I Virshup`\n* Added a new tutorial on working with dask ({doc}`/tutorials/experimental/dask`) {pr}`2901` {smaller}`I Gold` {smaller}`I Virshup`\n\n#### Bug fixes\n\n* Updated {func}`~scanpy.read_visium` such that it can read spaceranger 2.0 files {smaller}`L Lehner`\n* Fix {func}`~scanpy.pp.normalize_total` for dask {pr}`2466` {smaller}`P Angerer`\n* Fix setting `sc.settings.verbosity` in some cases {pr}`2605` {smaller}`P Angerer`\n* Fix all remaining pandas warnings {pr}`2789` {smaller}`P Angerer`\n* Fix some annoying plotting warnings around violin plots {pr}`2844` {smaller}`P Angerer`\n* Scanpy now has a test job which tests against the minumum versions of the dependencies. In the process of implementing this, many bugs associated with using older versions of `pandas`, `anndata`, `numpy`, and `matplotlib` were fixed. {pr}`2816` {smaller}`I Virshup`\n* Fix warnings caused by internal usage of `pandas.DataFrame.stack` with `pandas>=2.1` {pr}`2864`{smaller}`I Virshup`\n* {func}`scanpy.get.aggregate` now always returns {class}`numpy.ndarray` {pr}`2893` {smaller}`S Dicks`\n* Removes self from array of neighbors for `use_approx_neighbors = True` in {func}`~scanpy.pp.scrublet` {pr}`2896`{smaller}`S Dicks`\n* Compatibility with scipy 1.13 {pr}`2943` {smaller}`I Virshup`\n* Fix use of {func}`~scanpy.tl.dendrogram` on highly correlated low precision data {pr}`2928` {smaller}`P Angerer`\n* Fix pytest deprecation warning {pr}`2879` {smaller}`P Angerer`\n\n\n#### Development Process\n\n* Scanpy is now tested against python 3.12 {pr}`2863` {smaller}`ivirshup`\n* Fix testing package build {pr}`2468` {smaller}`P Angerer`\n\n#### Deprecations\n\n* Dropped support for Python 3.8. [More details here](https://numpy.org/neps/nep-0029-deprecation_policy.html). {pr}`2695` {smaller}`P Angerer`\n* Deprecated specifying large numbers of function parameters by position as opposed to by name/keyword in all public APIs.\n e.g. prefer `sc.tl.umap(adata, min_dist=0.1, spread=0.8)` over `sc.tl.umap(adata, 0.1, 0.8)` {pr}`2702` {smaller}`P Angerer`\n* Dropped support for `umap<0.5` for performance reasons. {pr}`2870` {smaller}`P Angerer`\n\n\n(v1.3.7)=\n### 1.3.7 {small}`2019-01-02`\n\n- API changed from `import scanpy as sc` to `import scanpy.api as sc`.\n- {func}`~scanpy.external.tl.phenograph` wraps the graph clustering package Phenograph {cite:p}`Levine2015` {smaller}`thanks to A Mousa`\n\n\n(v1.6.0)=\n### 1.6.0 {small}`2020-08-15`\n\nThis release includes an overhaul of {func}`~scanpy.pl.dotplot`, {func}`~scanpy.pl.matrixplot`, and {func}`~scanpy.pl.stacked_violin` ({pr}`1210` {smaller}`F Ramirez`), and of the internals of {func}`~scanpy.tl.rank_genes_groups` ({pr}`1156` {smaller}`S Rybakov`).\n\n#### Overhaul of {func}`~scanpy.pl.dotplot`, {func}`~scanpy.pl.matrixplot`, and {func}`~scanpy.pl.stacked_violin` {pr}`1210` {smaller}`F Ramirez`\n\n- An overhauled tutorial {doc}`/tutorials/plotting/core`.\n\n- New plotting classes can be accessed directly (e.g., {class}`~scanpy.pl.DotPlot`) or using the `return_fig` param.\n\n- It is possible to plot log fold change and p-values in the {func}`~scanpy.pl.rank_genes_groups_dotplot` family of functions.\n\n- Added `ax` parameter which allows embedding the plot in other images.\n\n- Added option to include a bar plot instead of the dendrogram containing the cell/observation totals per category.\n\n- Return a dictionary of axes for further manipulation. This includes the main plot, legend and dendrogram to totals\n\n- Legends can be removed.\n\n- The `groupby` param can take a list of categories, e.g., `groupby=[‘tissue’, ‘cell type’]`.\n\n- Added padding parameter to `dotplot` and `stacked_violin`. {pr}`1270`\n\n- Added title for colorbar and positioned as in dotplot for {func}`~scanpy.pl.matrixplot`.\n\n- {func}`~scanpy.pl.dotplot` changes:\n\n > - Improved the colorbar and size legend for dotplots. Now the colorbar and size have titles, which can be modified using the `colorbar_title` and `size_title` params. They also align at the bottom of the image and do not shrink if the dotplot image is smaller.\n > - Allow plotting genes in rows and categories in columns (`swap_axes`).\n > - Using {class}`~scanpy.pl.DotPlot`, the `dot_edge_color` and line width can be modified, a grid can be added, and other modifications are enabled.\n > - A new style was added in which the dots are replaced by an empty circle and the square behind the circle is colored (like in matrixplots).\n\n- {func}`~scanpy.pl.stacked_violin` changes:\n\n > - Violin colors can be colored based on average gene expression as in dotplots.\n > - The linewidth of the violin plots is thinner.\n > - Removed the tics for the y-axis as they tend to overlap with each other. Using the style method they can be displayed if needed.\n\n#### Additions\n\n- {func}`~anndata.concat` is now exported from scanpy, see {doc}`anndata:concatenation` for more info. {pr}`1338` {smaller}`I Virshup`\n- Added highly variable gene selection strategy from Seurat v3 {pr}`1204` {smaller}`A Gayoso`\n- Added [CellRank](https://github.com/theislab/cellrank/) to scanpy ecosystem {pr}`1304` {smaller}`giovp`\n- Added `backup_url` param to {func}`~scanpy.read_10x_h5` {pr}`1296` {smaller}`A Gayoso`\n- Allow prefix for {func}`~scanpy.read_10x_mtx` {pr}`1250` {smaller}`G Sturm`\n- Optional tie correction for the `'wilcoxon'` method in {func}`~scanpy.tl.rank_genes_groups` {pr}`1330` {smaller}`S Rybakov`\n- Use `sinfo` for {func}`~scanpy.logging.print_versions` and add {func}`~scanpy.logging.print_header` to do what it previously did. {pr}`1338` {smaller}`I Virshup` {pr}`1373`\n\n#### Bug fixes\n\n- Avoid warning in {func}`~scanpy.tl.rank_genes_groups` if 't-test' is passed {pr}`1303` {smaller}`A Wolf`\n- Restrict sphinx version to \\<3.1, >3.0 {pr}`1297` {smaller}`I Virshup`\n- Clean up `_ranks` and fix `dendrogram` for scipy 1.5 {pr}`1290` {smaller}`S Rybakov`\n- Use `.raw` to translate gene symbols if applicable {pr}`1278` {smaller}`E Rice`\n- Fix `diffmap` ({issue}`1262`) {smaller}`G Eraslan`\n- Fix `neighbors` in `spring_project` {issue}`1260` {smaller}`S Rybakov`\n- Fix default size of dot in spatial plots {pr}`1255` {issue}`1253` {smaller}`giovp`\n- Bumped version requirement of `scipy` to `scipy>1.4` to support `rmatmat` argument of `LinearOperator` {issue}`1246` {smaller}`I Virshup`\n- Fix asymmetry of scores for the `'wilcoxon'` method in {func}`~scanpy.tl.rank_genes_groups` {issue}`754` {smaller}`S Rybakov`\n- Avoid trimming of gene names in {func}`~scanpy.tl.rank_genes_groups` {issue}`753` {smaller}`S Rybakov`\n\n\n(v1.9.0)=\n### 1.9.0 {small}`2022-04-01`\n\n#### Tutorials\n\n- New tutorial on the usage of Pearson Residuals: {doc}`/tutorials/experimental/pearson_residuals` {smaller}`J Lause, G Palla`\n- [Materials](https://github.com/scverse/scanpy-tutorials/tree/master/scanpy_workshop) and [recordings](https://www.youtube.com/playlist?list=PL4rcQcNPLZxWQQH7LlRBMkAo5NWuHX1e3) for Scanpy workshops by Maren Büttner\n\n#### Experimental module\n\n- Added {mod}`scanpy.experimental` module! Currently contains functionality related to pearson residuals in {mod}`scanpy.experimental.pp` {pr}`1715` {smaller}`J Lause, G Palla, I Virshup`. This includes:\n\n - {func}`~scanpy.experimental.pp.normalize_pearson_residuals` for Pearson Residuals normalization\n - {func}`~scanpy.experimental.pp.highly_variable_genes` for HVG selection with Pearson Residuals\n - {func}`~scanpy.experimental.pp.normalize_pearson_residuals_pca` for Pearson Residuals normalization and dimensionality reduction with PCA\n - {func}`~scanpy.experimental.pp.recipe_pearson_residuals` for Pearson Residuals normalization, HVG selection and dimensionality reduction with PCA\n\n#### Features\n\n- {func}`~scanpy.tl.filter_rank_genes_groups` now allows to filter with absolute values of log fold change {pr}`1649` {smaller}`S Rybakov`\n- `_choose_representation` now subsets the provided representation to n_pcs, regardless of the name of the provided representation (should affect mostly {func}`~scanpy.pp.neighbors`) {pr}`2179` {smaller}`I Virshup` {smaller}`PG Majev`\n- {func}`scanpy.pp.scrublet` (and related functions) can now be used on `AnnData` objects containing multiple batches {pr}`1965` {smaller}`J Manning`\n- Number of variables plotted with {func}`~scanpy.pl.pca_loadings` can now be controlled with `n_points` argument. Additionally, variables are no longer repeated if the anndata has less than 30 variables {pr}`2075` {smaller}`Yves33`\n- Dask arrays now work with {func}`scanpy.pp.normalize_total` {pr}`1663` {smaller}`G Buckley, I Virshup`\n- {func}`~scanpy.pl.embedding_density` now allows more than 10 groups {pr}`1936` {smaller}`A Wolf`\n- Embedding plots can now pass `colorbar_loc` to specify the location of colorbar legend, or pass `None` to not show a colorbar {pr}`1821` {smaller}`A Schaar` {smaller}`I Virshup`\n- Embedding plots now have a `dimensions` argument, which lets users select which dimensions of their embedding to plot and uses the same broadcasting rules as other arguments {pr}`1538` {smaller}`I Virshup`\n- {func}`~scanpy.logging.print_versions` now uses `session_info` {pr}`2089` {smaller}`P Angerer` {smaller}`I Virshup`\n\n#### Ecosystem\n\nMultiple packages have been added to our ecosystem page, including:\n\n- [decoupler](https://github.com/saezlab/decoupler-py) a for footprint analysis and pathway enrichement {pr}`2186` {smaller}`PB Mompel`\n- [dandelion](https://github.com/zktuong/dandelion) for B-cell receptor analysis {pr}`1953` {smaller}`Z Tuong`\n- [CIARA](https://github.com/ScialdoneLab/CIARA_python) a feature selection tools for identifying rare cell types {pr}`2175` {smaller}`M Stock`\n\n#### Bug fixes\n\n- Fixed finding variables with `use_raw=True` and `basis=None` in {func}`scanpy.pl.scatter` {pr}`2027` {smaller}`E Rice`\n- Fixed {func}`scanpy.pp.scrublet` to address {issue}`1957` {smaller}`FlMai` and ensure raw counts are used for simulation\n- Functions in {mod}`scanpy.datasets` no longer throw `OldFormatWarnings` when using `anndata` `0.8` {pr}`2096` {smaller}`I Virshup`\n- Fixed use of {func}`scanpy.pp.neighbors` with `method='rapids'`: RAPIDS cuML no longer returns a squared Euclidean distance matrix, so we should not square-root the kNN distance matrix. {pr}`1828` {smaller}`M Zaslavsky`\n- Removed `pytables` dependency by implementing `read_10x_h5` with `h5py` due to installation errors on Windows {pr}`2064`\n- Fixed bug in {func}`scanpy.external.pp.hashsolo` where default value was set improperly {pr}`2190` {smaller}`B Reiz`\n- Fixed bug in {func}`scanpy.pl.embedding` functions where an error could be raised when there were missing values and large numbers of categories {pr}`2187` {smaller}`I Virshup`\n\n\n(v1.9.5)=\n### 1.9.5 {small}`2023-09-08`\n\n#### Bug fixes\n\n- Remove use of deprecated `dtype` argument to AnnData constructor {pr}`2658` {smaller}`Isaac Virshup`\n\n\n(v1.2.1)=\n### 1.2.1 {small}`2018-06-08`\n\n#### Plotting of {ref}`pl-generic` marker genes and quality control.\n\n- {func}`~scanpy.pl.highest_expr_genes` for quality control; plot genes with highest mean fraction of cells, similar to `plotQC` of *Scater* {cite:p}`McCarthy2017` {pr}`169` {smaller}`F Ramirez`\n\n\n(v0.3.2)=\n### 0.3.2 {small}`2017-11-29`\n\n- finding marker genes via {func}`~scanpy.pl.rank_genes_groups_violin` improved,\n see {issue}`51` {smaller}`F Ramirez`\n\n\n(v1.9.6)=\n### 1.9.6 {small}`2023-10-31`\n\n#### Bug fixes\n\n- Allow {func}`scanpy.pl.scatter` to accept a {class}`str` palette name {pr}`2571` {smaller}`P Angerer`\n- Make {func}`scanpy.external.tl.palantir` compatible with palantir >=1.3 {pr}`2672` {smaller}`DJ Otto`\n- Fix {func}`scanpy.pl.pca` when `return_fig=True` and `annotate_var_explained=True` {pr}`2682` {smaller}`J Wagner`\n- Temp fix for {issue}`2680` by skipping `seaborn` version 0.13.0 {pr}`2661` {smaller}`P Angerer`\n- Fix {func}`scanpy.pp.highly_variable_genes` to not modify the used layer when `flavor=seurat` {pr}`2698` {smaller}`E Roellin`\n- Prevent pandas from causing infinite recursion when setting a slice of a categorical column {pr}`2719` {smaller}`P Angerer`\n\n\n(v1.3.1)=\n### 1.3.1 {small}`2018-09-03`\n\n#### RNA velocity in single cells {cite:p}`LaManno2018`\n\n- Scanpy and AnnData support loom’s layers so that computations for single-cell RNA velocity {cite:p}`LaManno2018` become feasible {smaller}`S Rybakov and V Bergen`\n- [scvelo] harmonizes with Scanpy and is able to process loom files with splicing information produced by Velocyto {cite:p}`LaManno2018`, it runs a lot faster than the count matrix analysis of Velocyto and provides several conceptual developments\n\n#### Plotting ({ref}`pl-generic`)\n\n- {func}`~scanpy.pl.dotplot` for visualizing genes across conditions and clusters, see [here](https://gist.github.com/fidelram/2289b7a8d6da055fb058ac9a79ed485c) {pr}`199` {smaller}`F Ramirez`\n- {func}`~scanpy.pl.heatmap` for pretty heatmaps {pr}`175` {smaller}`F Ramirez`\n- {func}`~scanpy.pl.violin` produces very compact overview figures with many panels {pr}`175` {smaller}`F Ramirez`\n\n#### There now is a section on imputation in {doc}`external <../external/index>`:\n\n- {func}`~scanpy.external.pp.magic` for imputation using data diffusion {cite:p}`vanDijk2018` {pr}`187` {smaller}`S Gigante`\n- {func}`~scanpy.external.pp.dca` for imputation and latent space construction using an autoencoder {cite:p}`Eraslan2019` {pr}`186` {smaller}`G Eraslan`\n\n[scvelo]: https://github.com/theislab/scvelo\n\n\n(v1.4.6)=\n### 1.4.6 {small}`2020-03-17`\n\n#### Functionality in `external`\n\n- {func}`~scanpy.external.tl.sam` self-assembling manifolds {cite:p}`Tarashansky2019` {pr}`903` {smaller}`A Tarashansky`\n- {func}`~scanpy.external.tl.harmony_timeseries` for trajectory inference on discrete time points {pr}`994` {smaller}`A Mousa`\n- {func}`~scanpy.external.tl.wishbone` for trajectory inference (bifurcations) {pr}`1063` {smaller}`A Mousa`\n\n#### Code design\n\n- {mod}`~scanpy.pl.violin` now reads `.uns['colors_...']` {pr}`1029` {smaller}`michalk8`\n\n#### Bug fixes\n\n- adapt {func}`~scanpy.tl.ingest` for UMAP 0.4 {pr}`1038` {pr}`1106` {smaller}`S Rybakov`\n- compat with matplotlib 3.1 and 3.2 {pr}`1090` {smaller}`I Virshup, P Angerer`\n- fix PAGA for new igraph {pr}`1037` {smaller}`P Angerer`\n- fix rapids compat of louvain {pr}`1079` {smaller}`LouisFaure`\n\n\n(v1.3.6)=\n### 1.3.6 {small}`2018-12-11`\n\n#### Major updates\n\n- a new plotting gallery for `visualizing-marker-genes` {smaller}`F Ramirez`\n- tutorials are integrated on ReadTheDocs, `pbmc3k` and `paga-paul15` {smaller}`A Wolf`\n\n#### Interactive exploration of analysis results through *manifold viewers*\n\n- CZI’s [cellxgene] directly reads `.h5ad` files {smaller}`the cellxgene developers`\n- the [UCSC Single Cell Browser] requires exporting via {func}`~scanpy.external.exporting.cellbrowser` {smaller}`M Haeussler`\n\n#### Code design\n\n- {func}`~scanpy.pp.highly_variable_genes` supersedes {func}`~scanpy.pp.filter_genes_dispersion`, it gives the same results but, by default, expects logarithmized data and doesn’t subset {smaller}`A Wolf`\n\n[cellxgene]: https://github.com/chanzuckerberg/cellxgene\n[ucsc single cell browser]: https://github.com/maximilianh/cellBrowser\n\n\n(v0.4.2)=\n### 0.4.2 {small}`2018-01-07`\n\n- amendments in [PAGA](https://github.com/theislab/paga) and its plotting functions {smaller}`A Wolf`\n\n\nUse `density_norm` instead of of `scale` (cont. from {pr}`2844`) in {func}`~scanpy.pl.violin` and {func}`~scanpy.pl.stacked_violin` {smaller}`P Angerer`\n\n\n(v1.9.7)=\n### 1.9.7 {small}`2024-01-25`\n\n#### Bug fixes\n\n- Fix handling of numpy array palettes (e.g. after write-read cycle) {pr}`2734` {smaller}`P Angerer`\n- Specify correct version of `matplotlib` dependency {pr}`2733` {smaller}`P Fisher`\n- Fix {func}`scanpy.pl.violin` usage of `seaborn.catplot` {pr}`2739` {smaller}`E Roellin`\n- Fix {func}`scanpy.pp.highly_variable_genes` to handle the combinations of `inplace` and `subset` consistently {pr}`2757` {smaller}`E Roellin`\n- Replace usage of various deprecated functionality from {mod}`anndata` and {mod}`pandas` {pr}`2678` {pr}`2779` {smaller}`P Angerer`\n- Allow to use default `n_top_genes` when using {func}`scanpy.pp.highly_variable_genes` flavor `'seurat_v3'` {pr}`2782` {smaller}`P Angerer`\n- Fix {func}`scanpy.read_10x_mtx`’s `gex_only=True` mode {pr}`2801` {smaller}`P Angerer`\n\n\n(v1.9.1)=\n### 1.9.1 {small}`2022-04-05`\n\n#### Bug fixes\n\n- {func}`~scanpy.pp.normalize_total` works when Dask is not installed {pr}`2209` {smaller}`R Cannoodt`\n- Fix embedding plots by bumping matplotlib dependency to version 3.4 {pr}`2212` {smaller}`I Virshup`\n\n\n(v1.5.1)=\n### 1.5.1 {small}`2020-05-21`\n\n#### Bug fixes\n\n- Fixed a bug in {func}`~scanpy.pp.pca`, where `random_state` did not have an effect for sparse input {pr}`1240` {smaller}`I Virshup`\n- Fixed docstring in {func}`~scanpy.pp.pca` which included an unused argument {pr}`1240` {smaller}`I Virshup`\n\n\n(v1.4.4)=\n### 1.4.4 {small}`2019-07-20`\n\n#### New functionality\n\n- {mod}`scanpy.get` adds helper functions for extracting data in convenient formats {pr}`619` {smaller}`I Virshup`\n\n#### Bug fixes\n\n- Stopped deprecations warnings from AnnData `0.6.22` {smaller}`I Virshup`\n\n#### Code design\n\n- {func}`~scanpy.pp.normalize_total` gains param `exclude_highly_expressed`, and `fraction` is renamed to `max_fraction` with better docs {smaller}`A Wolf`\n\n\n(v1.3.3)=\n### 1.3.3 {small}`2018-11-05`\n\n#### Major updates\n\n- a fully distributed preprocessing backend {smaller}`T White and the Laserson Lab`\n\n#### Code design\n\n- {func}`~scanpy.read_10x_h5` and {func}`~scanpy.read_10x_mtx` read Cell Ranger 3.0 outputs {pr}`334` {smaller}`Q Gong`\n\n```{note}\n#### Also see changes in anndata 0.6.\n\n- changed default compression to `None` in {meth}`~anndata.AnnData.write_h5ad` to speed up read and write, disk space use is usually less critical\n- performance gains in {meth}`~anndata.AnnData.write_h5ad` due to better handling of strings and categories {smaller}`S Rybakov`\n```\n\n\nAdd support for `median` as an aggregation function to the `Aggregation` class in `scanpy.get._aggregated.py`. This allows for median-based aggregation of data (e.g., pseudobulk), complementing existing methods like mean- and sum-based aggregation {smaller}`M Dehkordi (Farhad)`\n\n\nPrevent `raw` conflict with `layer` in {func}`~scanpy.tl.score_genes` {smaller}`S Dicks`\n\n\n(v0.4.3)=\n### 0.4.3 {small}`2018-02-09`\n\n- {func}`~scanpy.pl.clustermap`: heatmap from hierarchical clustering,\n based on {func}`seaborn.clustermap` {cite:p}`Waskom2016` {smaller}`A Wolf`\n- only return {class}`matplotlib.axes.Axes` in plotting functions of `sc.pl`\n when `show=False`, otherwise `None` {smaller}`A Wolf`\n\n\n(v1.1.0)=\n### 1.1.0 {small}`2018-06-01`\n\n- {func}`~scanpy.set_figure_params` by default passes `vector_friendly=True` and allows you to produce reasonablly sized pdfs by rasterizing large scatter plots {smaller}`A Wolf`\n- {func}`~scanpy.tl.draw_graph` defaults to the ForceAtlas2 layout {cite:p}`Jacomy2014,Chippada2018`, which is often more visually appealing and whose computation is much faster {smaller}`S Wollock`\n- {func}`~scanpy.pl.scatter` also plots along variables axis {smaller}`MD Luecken`\n- {func}`~scanpy.pp.pca` and {func}`~scanpy.pp.log1p` support chunk processing {smaller}`S Rybakov`\n- {func}`~scanpy.pp.regress_out` is back to multiprocessing {smaller}`F Ramirez`\n- {func}`~scanpy.read` reads compressed text files {smaller}`G Eraslan`\n- {func}`~scanpy.queries.mitochondrial_genes` for querying mito genes {smaller}`FG Brundu`\n- {func}`~scanpy.external.pp.mnn_correct` for batch correction {cite:p}`Haghverdi2018,Kang2018`\n- {func}`~scanpy.external.tl.phate` for low-dimensional embedding {cite:p}`Moon2019` {smaller}`S Gigante`\n- {func}`~scanpy.external.tl.sandbag`, {func}`~scanpy.external.tl.cyclone` for scoring genes {cite:p}`Scialdone2015,Fechtner2018`\n\n\n(v1.9.2)=\n### 1.9.2 {small}`2023-02-16`\n\n#### Bug fixes\n\n* {func}`~scanpy.pp.highly_variable_genes` `layer` argument now works in tandem with `batches` {pr}`2302` {smaller}`D Schaumont`\n* {func}`~scanpy.pp.highly_variable_genes` with `flavor='cell_ranger'` now handles the case in {issue}`2230` where the number of calculated dispersions is less than `n_top_genes` {pr}`2231` {smaller}`L Zappia`\n* Fix compatibility with matplotlib 3.7 {pr}`2414` {smaller}`I Virshup` {smaller}`P Fisher`\n* Fix scrublet numpy matrix compatibility issue {pr}`2395` {smaller}`A Gayoso`\n\n\n(v1.9.8)=\n### 1.9.8 {small}`2024-01-26`\n\n#### Bug fixes\n\n- Fix handling of numpy array palettes for old numpy versions {pr}`2832` {smaller}`P Angerer`\n\n\n(v1.2.0)=\n### 1.2.0 {small}`2018-06-08`\n\n- {func}`~scanpy.tl.paga` improved, see [PAGA](https://github.com/theislab/paga); the default model changed, restore the previous default model by passing `model='v1.0'`\n\n\n(v1.5.0)=\n### 1.5.0 {small}`2020-05-15`\n\nThe `1.5.0` release adds a lot of new functionality, much of which takes advantage of {mod}`anndata` updates `0.7.0 - 0.7.2`. Highlights of this release include support for spatial data, dedicated handling of graphs in AnnData, sparse PCA, an interface with scvi, and others.\n\n#### Spatial data support\n\n- Basic analysis {doc}`/tutorials/spatial/basic-analysis` and integration with single cell data {doc}`/tutorials/spatial/integration-scanorama` {smaller}`G Palla`\n- {func}`~scanpy.read_visium` read 10x Visium data {pr}`1034` {smaller}`G Palla, P Angerer, I Virshup`\n- {func}`~scanpy.datasets.visium_sge` load Visium data directly from 10x Genomics {pr}`1013` {smaller}`M Mirkazemi, G Palla, P Angerer`\n- {func}`~scanpy.pl.spatial` plot spatial data {pr}`1012` {smaller}`G Palla, P Angerer`\n\n#### New functionality\n\n- Many functions, like {func}`~scanpy.pp.neighbors` and {func}`~scanpy.tl.umap`, now store cell-by-cell graphs in {attr}`~anndata.AnnData.obsp` {pr}`1118` {smaller}`S Rybakov`\n- {func}`~scanpy.pp.scale` and {func}`~scanpy.pp.log1p` can be used on any element in {attr}`~anndata.AnnData.layers` or {attr}`~anndata.AnnData.obsm` {pr}`1173` {smaller}`I Virshup`\n\n#### External tools\n\n- `scanpy.external.pp.scvi` for preprocessing with scVI {pr}`1085` {smaller}`G Xing`\n- Guide for using `Scanpy in R` {pr}`1186` {smaller}`L Zappia`\n\n#### Performance\n\n- {func}`~scanpy.pp.pca` now uses efficient implicit centering for sparse matrices. This can lead to signifigantly improved performance for large datasets {pr}`1066` {smaller}`A Tarashansky`\n- {func}`~scanpy.tl.score_genes` now has an efficient implementation for sparse matrices with missing values {pr}`1196` {smaller}`redst4r`.\n\n```{warning}\nThe new {func}`~scanpy.pp.pca` implementation can result in slightly different results for sparse matrices. See the pr ({pr}`1066`) and documentation for more info.\n```\n\n#### Code design\n\n- {func}`~scanpy.pl.stacked_violin` can now be used as a subplot {pr}`1084` {smaller}`P Angerer`\n- {func}`~scanpy.tl.score_genes` has improved logging {pr}`1119` {smaller}`G Eraslan`\n- {func}`~scanpy.pp.scale` now saves mean and standard deviation in the {attr}`~anndata.AnnData.var` {pr}`1173` {smaller}`A Wolf`\n- {func}`~scanpy.external.tl.harmony_timeseries` {pr}`1091` {smaller}`A Mousa`\n\n#### Bug fixes\n\n- {func}`~scanpy.pp.combat` now works when `obs_names` aren't unique. {pr}`1215` {smaller}`I Virshup`\n- {func}`~scanpy.pp.scale` can now be used on dense arrays without centering {pr}`1160` {smaller}`simonwm`\n- {func}`~scanpy.pp.regress_out` now works when some features are constant {pr}`1194` {smaller}`simonwm`\n- {func}`~scanpy.pp.normalize_total` errored if the passed object was a view {pr}`1200` {smaller}`I Virshup`\n- {func}`~scanpy.pp.neighbors` sometimes ignored the `n_pcs` param {pr}`1124` {smaller}`V Bergen`\n- {func}`~scanpy.datasets.ebi_expression_atlas` which contained some out-of-date URLs {pr}`1102` {smaller}`I Virshup`\n- {func}`~scanpy.tl.ingest` for UMAP `0.4` {pr}`1165` {smaller}`S Rybakov`\n- {func}`~scanpy.tl.louvain` for Louvain `0.6` {pr}`1197` {smaller}`I Virshup`\n- {func}`~scanpy.pp.highly_variable_genes` which could lead to incorrect results when the `batch_key` argument was used {pr}`1180` {smaller}`G Eraslan`\n- {func}`~scanpy.tl.ingest` where an inconsistent number of neighbors was used {pr}`1111` {smaller}`S Rybakov`\n\n\n(v1.9.4)=\n### 1.9.4 {small}`2023-08-24`\n\n#### Bug fixes\n\n* Support scikit-learn 1.3 {pr}`2515` {smaller}`P Angerer`\n* Deal with `None` value vanishing from things like `.uns['log1p']` {pr}`2546` {smaller}`SP Shen`\n* Depend on `igraph` instead of `python-igraph` {pr}`2566` {smaller}`P Angerer`\n* {func}`~scanpy.tl.rank_genes_groups` now handles unsorted groups as intended {pr}`2589` {smaller}`S Dicks`\n* {func}`~scanpy.get.rank_genes_groups_df` now works for {func}`~scanpy.tl.rank_genes_groups` with `method=\"logreg\"` {pr}`2601` {smaller}`S Dicks`\n* `scanpy.tl._utils._choose_representation` now works with `n_pcs` if bigger than `settings.N_PCS` {pr}`2610` {smaller}`S Dicks`\n\n\n(v0.3.0)=\n### 0.3.0 {small}`2017-11-16`\n\n- {class}`~anndata.AnnData` gains method {meth}`~anndata.AnnData.concatenate` {smaller}`A Wolf`\n- {class}`~anndata.AnnData` is available as the separate [anndata] package {smaller}`P Angerer, A Wolf`\n- results of [PAGA](https://github.com/theislab/paga) simplified {smaller}`A Wolf`\n\n[anndata]: https://pypi.org/project/anndata/\n\n\n(v1.10.1)=\n### 1.10.1 {small}`2024-04-09`\n\n#### Documentation\n\n* Added {doc}`how-to example ` on plotting with [Marsilea](https://marsilea.readthedocs.io) {pr}`2974` {smaller}`Y Zheng`\n\n#### Bug fixes\n\n* Fix `aggregate` when aggregating by more than two groups {pr}`2965` {smaller}`I Virshup`\n\n\n#### Performance\n* {func}`~scanpy.pp.scale` now uses numba kernels for `sparse.csr_matrix` and `sparse.csc_matrix` when `zero_center==False` and `mask_obs` is provided. This greatly speed up execution {pr}`2942` {smaller}`S Dicks`\n\n\n(v1.4.3)=\n### 1.4.3 {small}`2019-05-14`\n\n#### Bug fixes\n\n- {func}`~scanpy.pp.neighbors` correctly infers `n_neighbors` again from `params`, which was temporarily broken in `v1.4.2` {smaller}`I Virshup`\n\n#### Code design\n\n- {func}`~scanpy.pp.calculate_qc_metrics` is single threaded by default for datasets under 300,000 cells -- allowing cached compilation {pr}`615` {smaller}`I Virshup`\n\n\n(v1.8.2)=\n### 1.8.2 {small}`2021-11-3`\n\n#### Documentation\n\n- Update conda installation instructions {pr}`1974` {smaller}`L Heumos`\n\n#### Bug fixes\n\n- Fix plotting after {func}`scanpy.tl.filter_rank_genes_groups` {pr}`1942` {smaller}`S Rybakov`\n- Fix `use_raw=None` using {attr}`anndata.AnnData.var_names` if {attr}`anndata.AnnData.raw`\n is present in {func}`scanpy.tl.score_genes` {pr}`1999` {smaller}`M Klein`\n- Fix compatibility with UMAP 0.5.2 {pr}`2028` {smaller}`L Mcinnes`\n- Fixed non-determinism in {func}`scanpy.pl.paga` node positions {pr}`1922` {smaller}`I Virshup`\n\n#### Ecosystem\n\n- Added PASTE (a tool to align and integrate spatial transcriptomics data) to scanpy ecosystem.\n\n\n(v1.4.5)=\n### 1.4.5 {small}`2019-12-30`\n\nPlease install `scanpy==1.4.5.post3` instead of `scanpy==1.4.5`.\n\n#### New functionality\n\n- {func}`~scanpy.tl.ingest` maps labels and embeddings of reference data to new data {doc}`/tutorials/basics/integrating-data-using-ingest` {pr}`651` {smaller}`S Rybakov, A Wolf`\n- {mod}`~scanpy.queries` recieved many updates including enrichment through [gprofiler] and more advanced biomart queries {pr}`467` {smaller}`I Virshup`\n- {func}`~scanpy.set_figure_params` allows setting `figsize` and accepts `facecolor='white'`, useful for working in dark mode {smaller}`A Wolf`\n\n#### Code design\n\n- {mod}`~scanpy.pp.downsample_counts` now always preserves the dtype of it's input, instead of converting floats to ints {pr}`865` {smaller}`I Virshup`\n- allow specifying a base for {func}`~scanpy.pp.log1p` {pr}`931` {smaller}`G Eraslan`\n- run neighbors on a GPU using rapids {pr}`830` {smaller}`T White`\n- param docs from typed params {smaller}`P Angerer`\n- {func}`~scanpy.tl.embedding_density` now only takes one positional argument; similar for {func}`~scanpy.pl.embedding_density`, which gains a param `groupby` {pr}`965` {smaller}`A Wolf`\n- webpage overhaul, ecosystem page, release notes, tutorials overhaul {pr}`960` {pr}`966` {smaller}`A Wolf`\n\n```{warning}\n- changed default `solver` in {func}`~scanpy.pp.pca` from `auto` to `arpack`\n- changed default `use_raw` in {func}`~scanpy.tl.score_genes` from `False` to `None`\n```\n\n[gprofiler]: https://biit.cs.ut.ee/gprofiler/\n\n\n(v1.10.3)=\n### 1.10.3 {small}`2024-09-17`\n\n#### Bug fixes\n\n- Prevent empty control gene set in {func}`~scanpy.tl.score_genes` {smaller}`M Müller` ({pr}`2875`)\n- Fix `subset=True` of {func}`~scanpy.pp.highly_variable_genes` when `flavor` is `seurat` or `cell_ranger`, and `batch_key!=None` {smaller}`E Roellin` ({pr}`3042`)\n- Add compatibility with {mod}`numpy` 2.0 {smaller}`P Angerer` {pr}`3065` and ({pr}`3115`)\n- Fix `legend_loc` argument in {func}`scanpy.pl.embedding` not accepting matplotlib parameters {smaller}`P Angerer` ({pr}`3163`)\n- Fix dispersion cutoff in {func}`~scanpy.pp.highly_variable_genes` in presence of `NaN`s {smaller}`P Angerer` ({pr}`3176`)\n- Fix axis labeling for swapped axes in {func}`~scanpy.pl.rank_genes_groups_stacked_violin` {smaller}`Ilan Gold` ({pr}`3196`)\n- Upper bound dask on account of {issue}`scverse/anndata#1579` {smaller}`Ilan Gold` ({pr}`3217`)\n- The [fa2-modified][] package replaces [forceatlas2][] for the latter’s lack of maintenance {smaller}`A Alam` ({pr}`3220`)\n\n [fa2-modified]: https://github.com/AminAlam/fa2_modified\n [forceatlas2]: https://github.com/bhargavchippada/forceatlas2\n\n\nAdd `layer` argument to {func}`scanpy.tl.score_genes` and {func}`scanpy.tl.score_genes_cell_cycle` {smaller}`L Zappia`\n\n\n(v1.3.8)=\n### 1.3.8 {small}`2019-02-05`\n\n- various documentation and dev process improvements\n- Added {func}`~scanpy.pp.combat` function for batch effect correction {cite:p}`Johnson2006,Leek2012,Pedersen2012` {pr}`398` {smaller}`M Lange`\n\n\n(v0.4.0)=\n### 0.4.0 {small}`2017-12-23`\n\n- export to [SPRING] {cite:p}`Weinreb2017` for interactive visualization of data:\n [spring tutorial] {smaller}`S Wollock`\n\n[spring]: https://github.com/AllonKleinLab/SPRING/\n[spring tutorial]: https://github.com/scverse/scanpy_usage/tree/master/171111_SPRING_export\n\n\nAdd `key_added` argument to {func}`~scanpy.pp.pca`, {func}`~scanpy.tl.tsne` and {func}`~scanpy.tl.umap` {smaller}`P Angerer`\n\n\n(v1.3.4)=\n### 1.3.4 {small}`2018-11-24`\n\n- {func}`~scanpy.tl.leiden` wraps the recent graph clustering package by {cite:t}`Traag2019` {smaller}`K Polanski`\n- {func}`~scanpy.external.pp.bbknn` wraps the recent batch correction package {cite:p}`Polanski2019` {smaller}`K Polanski`\n- {func}`~scanpy.pp.calculate_qc_metrics` caculates a number of quality control metrics, similar to `calculateQCMetrics` from *Scater* {cite:p}`McCarthy2017` {smaller}`I Virshup`\n\n\nSwitched all compatibility adapters for positional parameters to {exc}`FutureWarning` {smaller}`P Angerer`\n\n\n(v0.2.9)=\n### 0.2.9 {small}`2017-10-25`\n\n#### Initial release of the new trajectory inference method [PAGA](https://github.com/theislab/paga)\n\n- {func}`~scanpy.tl.paga` computes an abstracted, coarse-grained (PAGA) graph of the neighborhood graph {smaller}`A Wolf`\n- {func}`~scanpy.pl.paga_compare` plot this graph next an embedding {smaller}`A Wolf`\n- {func}`~scanpy.pl.paga_path` plots a heatmap through a node sequence in the PAGA graph {smaller}`A Wolf`\n\n\nAccept `'group'` instead of `'obs'` for `standard_scale` parameter in {func}`~scanpy.pl.stacked_violin` {smaller}`P Angerer`\n\n\n(release-notes)=\n\n# Release notes\n\n```{release-notes} .\n```\n\n\n(v1.4.2)=\n### 1.4.2 {small}`2019-05-06`\n\n#### New functionality\n\n- {func}`~scanpy.pp.combat` supports additional covariates which may include adjustment variables or biological condition {pr}`618` {smaller}`G Eraslan`\n- {func}`~scanpy.pp.highly_variable_genes` has a `batch_key` option which performs HVG selection in each batch separately to avoid selecting genes that vary strongly across batches {pr}`622` {smaller}`G Eraslan`\n\n#### Bug fixes\n\n- {func}`~scanpy.tl.rank_genes_groups` t-test implementation doesn't return NaN when variance is 0, also changed to scipy's implementation {pr}`621` {smaller}`I Virshup`\n- {func}`~scanpy.tl.umap` with `init_pos='paga'` detects correct `dtype` {smaller}`A Wolf`\n- {func}`~scanpy.tl.louvain` and {func}`~scanpy.tl.leiden` auto-generate `key_added=louvain_R` upon passing `restrict_to`, which was temporarily changed in `1.4.1` {smaller}`A Wolf`\n\n#### Code design\n\n- {func}`~scanpy.pp.neighbors` and {func}`~scanpy.tl.umap` got rid of UMAP legacy code and introduced UMAP as a dependency {pr}`576` {smaller}`S Rybakov`\n\n\n## Exporting\n\n```{eval-rst}\n.. module:: scanpy.external.exporting\n.. currentmodule:: scanpy.external\n```\n\n```{eval-rst}\n.. autosummary::\n :toctree: ../generated/\n\n exporting.spring_project\n exporting.cellbrowser\n```\n\n\n## Preprocessing: PP\n\n```{eval-rst}\n.. module:: scanpy.external.pp\n.. currentmodule:: scanpy.external\n```\n\n### Data integration\n\n```{eval-rst}\n.. autosummary::\n :toctree: ../generated/\n\n pp.bbknn\n pp.harmony_integrate\n pp.mnn_correct\n pp.scanorama_integrate\n\n```\n\n### Sample demultiplexing\n\n```{eval-rst}\n.. autosummary::\n :toctree: ../generated/\n\n pp.hashsolo\n```\n\n### Imputation\n\nNote that the fundamental limitations of imputation are still under [debate](https://github.com/scverse/scanpy/issues/189).\n\n```{eval-rst}\n.. autosummary::\n :toctree: ../generated/\n\n pp.dca\n pp.magic\n\n```\n\n\n## Tools: TL\n\n```{eval-rst}\n.. module:: scanpy.external.tl\n.. currentmodule:: scanpy.external\n```\n\n### Embeddings\n\n```{eval-rst}\n.. autosummary::\n :toctree: generated/\n\n tl.phate\n tl.palantir\n tl.trimap\n tl.sam\n```\n\n### Clustering and trajectory inference\n\n```{eval-rst}\n.. autosummary::\n :toctree: generated/\n\n tl.phenograph\n tl.harmony_timeseries\n tl.wishbone\n tl.palantir\n tl.palantir_results\n```\n\n### Gene scores, Cell cycle\n\n```{eval-rst}\n.. autosummary::\n :toctree: generated/\n\n tl.sandbag\n tl.cyclone\n\n```\n\n\n## Plotting: PL\n\n\n```{eval-rst}\n.. module:: scanpy.external.pl\n.. currentmodule:: scanpy.external\n```\n\n```{eval-rst}\n.. autosummary::\n :toctree: ../generated/\n\n pl.phate\n pl.trimap\n pl.sam\n pl.wishbone_marker_trajectory\n```\n\n\n# External API\n\n\n```{eval-rst}\n.. module:: scanpy.external\n```\n\n```{warning}\nWe are no longer accepting new tools into `scanpy.external`.\nInstead, please submit your tool to the [scverse ecosystem package listing](https://scverse.org/packages/#ecosystem).\n```\n\n```{note}\nFor tools that integrate well with scanpy and anndata, see:\n* The [scverse ecosystem](https://scverse.org/packages/#ecosystem)\n* Scanpy's ecosystem {doc}`ecosystem page <../ecosystem>`\n```\n\nImport Scanpy's wrappers to external tools as:\n\n```\nimport scanpy.external as sce\n```\n\n\n```{toctree}\n:maxdepth: 2\n\npreprocessing\ntools\nplotting\nexporting\n```\n\n\n# Tutorials\n\n:::{seealso}\nFor more tutorials featureing scanpy and other [scverse](https://scverse.org) ecosystem tools, check out the curated set of tutorials at [scverse.org/learn](https://scverse.org/learn)\n:::\n\n## Basic workflows\n\n```{toctree}\n:maxdepth: 2\n\nbasics/index\n```\n\n## Visualization\n\n```{toctree}\n:maxdepth: 2\n\nplotting/index\n```\n\n## Trajectory inference\n\n```{seealso}\nFor more powerful tools for analysing single cell dynamics, check out the Scverse ecosystem packages:\n\n* [CellRank](https://cellrank.readthedocs.io)\n* [Dynamo](https://dynamo-release.readthedocs.io/en/latest/)\n```\n\n```{toctree}\n:maxdepth: 2\n\ntrajectories/index\n```\n\n## Spatial data\n\n```{seealso}\nFor more up-to-date tutorials on working with spatial data, see:\n\n* [SquidPy tutorials](https://squidpy.readthedocs.io/en/stable/notebooks/tutorials/index.html)\n* [SpatialData tutorials](https://spatialdata.scverse.org/en/latest/tutorials/notebooks/notebooks.html)\n* [Scverse ecosystem spatial tutorials](https://scverse.org/learn/)\n```\n\n```{toctree}\n:maxdepth: 2\n\nspatial/index\n```\n\n## Experimental\n\n```{toctree}\n:maxdepth: 2\n\nexperimental/index\n```\n\n## Older tutorials\n\nA number of older tutorials can be found at:\n\n* The [`scanpy_usage`](https://github.com/scverse/scanpy_usage) repository\n\n\n# Plotting\n\n```{toctree}\n:maxdepth: 1\n\ncore\nadvanced\n```\n\n\n## Spatial\n\n```{toctree}\n:maxdepth: 1\n\nbasic-analysis\nintegration-scanorama\n```\n\n\n## Experimental\n\n```{toctree}\n:maxdepth: 1\n\npearson_residuals\ndask\n```\n\n\n# Basics\n\n```{toctree}\n:maxdepth: 1\n\nclustering\nclustering-2017\nintegrating-data-using-ingest\n```\n\n\n## Trajectories\n\n```{toctree}\n:maxdepth: 1\n\npaga-paul15\n```\n\n\nfrom __future__ import annotations\n\nfrom inspect import get_annotations\nfrom typing import TYPE_CHECKING\n\nfrom jinja2.defaults import DEFAULT_NAMESPACE\nfrom jinja2.utils import import_string\n\nif TYPE_CHECKING:\n from sphinx.application import Sphinx\n\n\ndef has_member(obj_path: str, attr: str) -> bool:\n # https://jinja.palletsprojects.com/en/3.0.x/api/#custom-tests\n obj = import_string(obj_path)\n return hasattr(obj, attr) or attr in get_annotations(obj)\n\n\ndef setup(app: Sphinx):\n DEFAULT_NAMESPACE[\"has_member\"] = has_member\n\n\n\"\"\"Extension to patch https://github.com/executablebooks/MyST-NB/pull/599.\"\"\"\n\n# TODO once MyST-NB 1.1.1/1.2.0 is out, this can be removed.\n\nfrom __future__ import annotations\n\nfrom copy import copy\nfrom typing import TYPE_CHECKING\n\nfrom myst_nb.core.render import MditRenderMixin\n\nif TYPE_CHECKING:\n from sphinx.application import Sphinx\n\n\nget_orig = MditRenderMixin.get_cell_level_config\n\n\ndef get_cell_level_config(\n self: MditRenderMixin,\n field: str,\n cell_metadata: dict[str, object],\n line: int | None = None,\n):\n rv = get_orig(self, field, cell_metadata, line)\n return copy(rv)\n\n\ndef setup(app: Sphinx):\n MditRenderMixin.get_cell_level_config = get_cell_level_config\n\n\n# Just do the following to see the rst of a function:\n# rm ./_build/doctrees/api/generated/scanpy..doctree; DEBUG=1 make html\nfrom __future__ import annotations\n\nimport os\nfrom typing import TYPE_CHECKING\n\nimport sphinx.ext.napoleon\n\nif TYPE_CHECKING:\n from sphinx.application import Sphinx\n\n_pd_orig = sphinx.ext.napoleon._process_docstring\n\n\ndef pd_new(app, what, name, obj, options, lines): # noqa: PLR0917\n _pd_orig(app, what, name, obj, options, lines)\n print(*lines, sep=\"\\n\")\n\n\ndef setup(app: Sphinx):\n if os.environ.get(\"DEBUG\") is not None:\n sphinx.ext.napoleon._process_docstring = pd_new\n\n\n\"\"\"Extension to inject ``html_theme_options[\"repository_branch\"]``.\"\"\"\n\nfrom __future__ import annotations\n\nimport re\nimport subprocess\nfrom functools import lru_cache\nfrom typing import TYPE_CHECKING\n\nif TYPE_CHECKING:\n from sphinx.application import Sphinx\n from sphinx.config import Config\n\n\ndef git(*args: str) -> str:\n return subprocess.check_output([\"git\", *args]).strip().decode()\n\n\n# https://github.com/DisnakeDev/disnake/blob/7853da70b13fcd2978c39c0b7efa59b34d298186/docs/conf.py#L192\n@lru_cache\ndef get() -> str | None:\n \"\"\"Current git reference. Uses branch/tag name if found, otherwise uses commit hash\"\"\"\n git_ref = None\n try:\n git_ref = git(\"name-rev\", \"--name-only\", \"--no-undefined\", \"HEAD\")\n git_ref = re.sub(r\"^(remotes/[^/]+|tags)/\", \"\", git_ref)\n except Exception:\n pass\n\n # (if no name found or relative ref, use commit hash instead)\n if not git_ref or re.search(r\"[\\^~]\", git_ref):\n try:\n git_ref = git(\"rev-parse\", \"HEAD\")\n except Exception:\n git_ref = \"main\"\n return git_ref\n\n\ndef set_ref(app: Sphinx, config: Config):\n app.config[\"html_theme_options\"][\"repository_branch\"] = get() or \"main\"\n\n\ndef setup(app: Sphinx) -> None:\n app.connect(\"config-inited\", set_ref)\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nfrom sphinx.util.docutils import SphinxDirective\n\nif TYPE_CHECKING:\n from typing import ClassVar\n\n from docutils import nodes\n from sphinx.application import Sphinx\n\n\nclass CanonicalTutorial(SphinxDirective):\n \"\"\"In the scanpy-tutorials repo, this links to the canonical location (here!).\"\"\"\n\n required_arguments: ClassVar = 1\n\n def run(self) -> list[nodes.Node]:\n return []\n\n\ndef setup(app: Sphinx) -> None:\n app.add_directive(\"canonical-tutorial\", CanonicalTutorial)\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom typing import TYPE_CHECKING\n\nfrom sphinx.ext.napoleon import NumpyDocstring\n\nif TYPE_CHECKING:\n from sphinx.application import Sphinx\n\n_format_docutils_params_orig = NumpyDocstring._format_docutils_params\nparam_warnings = {}\n\n\ndef scanpy_log_param_types(self, fields, field_role=\"param\", type_role=\"type\"):\n for _name, _type, _desc in fields:\n if not _type or not self._obj.__module__.startswith(\"scanpy\"):\n continue\n w_list = param_warnings.setdefault((self._name, self._obj), [])\n if (_name, _type) not in w_list:\n w_list.append((_name, _type))\n return _format_docutils_params_orig(self, fields, field_role, type_role)\n\n\ndef show_param_warnings(app, exception):\n import inspect\n\n for (fname, fun), params in param_warnings.items():\n _, line = inspect.getsourcelines(fun)\n file_name = inspect.getsourcefile(fun)\n params_str = \"\\n\".join(f\"\\t{n}: {t}\" for n, t in params)\n warnings.warn_explicit(\n f\"\\nParameters in `{fname}` have types in docstring.\\n\"\n f\"Replace them with type annotations.\\n{params_str}\",\n UserWarning,\n file_name,\n line,\n )\n if param_warnings:\n raise RuntimeError(\"Encountered text parameter type. Use annotations.\")\n\n\ndef setup(app: Sphinx):\n NumpyDocstring._format_docutils_params = scanpy_log_param_types\n app.connect(\"build-finished\", show_param_warnings)\n\n\n\"\"\"Images for plot functions\"\"\"\n\nfrom __future__ import annotations\n\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nif TYPE_CHECKING:\n from typing import Any\n\n from sphinx.application import Sphinx\n from sphinx.ext.autodoc import Options\n\n\ndef insert_function_images( # noqa: PLR0917\n app: Sphinx, what: str, name: str, obj: Any, options: Options, lines: list[str]\n):\n path = app.config.api_dir / f\"{name}.png\"\n if what != \"function\" or not path.is_file():\n return\n lines[0:0] = [\n f\".. image:: {path.name}\",\n \" :width: 200\",\n \" :align: right\",\n \"\",\n ]\n\n\ndef setup(app: Sphinx):\n app.add_config_value(\"api_dir\", Path(), \"env\")\n app.connect(\"autodoc-process-docstring\", insert_function_images)\n\n\n{\n // The version of the config file format. Do not change, unless\n // you know what you are doing.\n \"version\": 1,\n\n // The name of the project being benchmarked\n \"project\": \"scanpy\",\n\n // The project's homepage\n \"project_url\": \"https://scanpy.readthedocs.io/\",\n\n // The URL or local path of the source code repository for the\n // project being benchmarked\n \"repo\": \"..\",\n\n // The Python project's subdirectory in your repo. If missing or\n // the empty string, the project is assumed to be located at the root\n // of the repository.\n // \"repo_subdir\": \"\",\n\n // Customizable commands for building, installing, and\n // uninstalling the project. See asv.conf.json documentation.\n //\n // \"install_command\": [\"python -mpip install {wheel_file}\"],\n // \"uninstall_command\": [\"return-code=any python -mpip uninstall -y {project}\"],\n \"build_command\": [\n \"python -m pip install build\",\n \"python -m build --wheel -o {build_cache_dir} {build_dir}\",\n ],\n\n // List of branches to benchmark. If not provided, defaults to \"master\"\n // (for git) or \"default\" (for mercurial).\n \"branches\": [\"main\"], // for git\n\n // The DVCS being used. If not set, it will be automatically\n // determined from \"repo\" by looking at the protocol in the URL\n // (if remote), or by looking for special directories, such as\n // \".git\" (if local).\n \"dvcs\": \"git\",\n\n // The tool to use to create environments. May be \"conda\",\n // \"virtualenv\" or other value depending on the plugins in use.\n // If missing or the empty string, the tool will be automatically\n // determined by looking for tools on the PATH environment\n // variable.\n \"environment_type\": \"conda\",\n\n // timeout in seconds for installing any dependencies in environment\n // defaults to 10 min\n //\"install_timeout\": 600,\n\n // the base URL to show a commit for the project.\n \"show_commit_url\": \"https://github.com/scverse/scanpy/commit/\",\n\n // The Pythons you'd like to test against. If not provided, defaults\n // to the current version of Python used to run `asv`.\n // \"pythons\": [\"3.9\", \"3.12\"],\n\n // The list of conda channel names to be searched for benchmark\n // dependency packages in the specified order\n \"conda_channels\": [\"conda-forge\", \"defaults\"],\n\n // The matrix of dependencies to test. Each key is the name of a\n // package (in PyPI) and the values are version numbers. An empty\n // list or empty string indicates to just test against the default\n // (latest) version. null indicates that the package is to not be\n // installed. If the package to be tested is only available from\n // PyPi, and the 'environment_type' is conda, then you can preface\n // the package name by 'pip+', and the package will be installed via\n // pip (with all the conda available packages installed first,\n // followed by the pip installed packages).\n //\n \"matrix\": {\n \"numpy\": [\"\"],\n // \"scipy\": [\"1.2\", \"\"],\n \"scipy\": [\"\"],\n \"h5py\": [\"\"],\n \"natsort\": [\"\"],\n \"pandas\": [\"\"],\n \"memory_profiler\": [\"\"],\n \"zarr\": [\"\"],\n \"pytest\": [\"\"],\n \"scanpy\": [\"\"],\n \"python-igraph\": [\"\"],\n // \"psutil\": [\"\"]\n \"pooch\": [\"\"],\n \"scikit-image\": [\"\"],\n // \"scikit-misc\": [\"\"],\n },\n\n // Combinations of libraries/python versions can be excluded/included\n // from the set to test. Each entry is a dictionary containing additional\n // key-value pairs to include/exclude.\n //\n // An exclude entry excludes entries where all values match. The\n // values are regexps that should match the whole string.\n //\n // An include entry adds an environment. Only the packages listed\n // are installed. The 'python' key is required. The exclude rules\n // do not apply to includes.\n //\n // In addition to package names, the following keys are available:\n //\n // - python\n // Python version, as in the *pythons* variable above.\n // - environment_type\n // Environment type, as above.\n // - sys_platform\n // Platform, as in sys.platform. Possible values for the common\n // cases: 'linux2', 'win32', 'cygwin', 'darwin'.\n //\n // \"exclude\": [\n // {\"python\": \"3.2\", \"sys_platform\": \"win32\"}, // skip py3.2 on windows\n // {\"environment_type\": \"conda\", \"six\": null}, // don't run without six on conda\n // ],\n //\n // \"include\": [\n // // additional env for python2.7\n // {\"python\": \"2.7\", \"numpy\": \"1.8\"},\n // // additional env if run on windows+conda\n // {\"platform\": \"win32\", \"environment_type\": \"conda\", \"python\": \"2.7\", \"libpython\": \"\"},\n // ],\n\n // The directory (relative to the current directory) that benchmarks are\n // stored in. If not provided, defaults to \"benchmarks\"\n // \"benchmark_dir\": \"benchmarks\",\n\n // The directory (relative to the current directory) to cache the Python\n // environments in. If not provided, defaults to \"env\"\n \"env_dir\": \".asv/env\",\n\n // The directory (relative to the current directory) that raw benchmark\n // results are stored in. If not provided, defaults to \"results\".\n \"results_dir\": \".asv/results\",\n\n // The directory (relative to the current directory) that the html tree\n // should be written to. If not provided, defaults to \"html\".\n \"html_dir\": \".asv/html\",\n\n // The number of characters to retain in the commit hashes.\n // \"hash_length\": 8,\n\n // `asv` will cache results of the recent builds in each\n // environment, making them faster to install next time. This is\n // the number of builds to keep, per environment.\n // \"build_cache_size\": 2,\n\n // The commits after which the regression search in `asv publish`\n // should start looking for regressions. Dictionary whose keys are\n // regexps matching to benchmark names, and values corresponding to\n // the commit (exclusive) after which to start looking for\n // regressions. The default is to start from the first commit\n // with results. If the commit is `null`, regression detection is\n // skipped for the matching benchmark.\n //\n // \"regressions_first_commits\": {\n // \"some_benchmark\": \"352cdf\", // Consider regressions only after this commit\n // \"another_benchmark\": null, // Skip regression detection altogether\n // },\n\n // The thresholds for relative change in results, after which `asv\n // publish` starts reporting regressions. Dictionary of the same\n // form as in ``regressions_first_commits``, with values\n // indicating the thresholds. If multiple entries match, the\n // maximum is taken. If no entry matches, the default is 5%.\n //\n // \"regressions_thresholds\": {\n // \"some_benchmark\": 0.01, // Threshold of 1%\n // \"another_benchmark\": 0.5, // Threshold of 50%\n // },\n}\n\n\n# Scanpy Benchmarks\n\nThis directory contains code for benchmarking Scanpy using [asv][].\n\nThe functionality is checked using the [`benchmark.yml`][] workflow.\nBenchmarks are run using the [benchmark bot][].\n\n[asv]: https://asv.readthedocs.io/\n[`benchmark.yml`]: ../.github/workflows/benchmark.yml\n[benchmark bot]: https://github.com/apps/scverse-benchmark\n\n## Data processing in benchmarks\n\nEach dataset is processed so it has\n\n- `.layers['counts']` (containing data in C/row-major format) and `.layers['counts-off-axis']` (containing data in FORTRAN/column-major format)\n- `.X` and `.layers['off-axis']` with log-transformed data (formats like above)\n- a `.var['mt']` boolean column indicating mitochondrial genes\n\nThe benchmarks are set up so the `layer` parameter indicates the layer that will be moved into `.X` before the benchmark.\nThat way, we don’t need to add `layer=layer` everywhere.\n\n\n\"\"\"\nThis module will benchmark preprocessing operations in Scanpy that run on log-transformed data\nAPI documentation: https://scanpy.readthedocs.io/en/stable/api/preprocessing.html\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport scanpy as sc\nfrom scanpy.preprocessing._utils import _get_mean_var\n\nfrom ._utils import get_dataset, param_skipper\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n from ._utils import Dataset, KeyX\n\n# setup variables\n\n\nadata: AnnData\nbatch_key: str | None\n\n\ndef setup(dataset: Dataset, layer: KeyX, *_):\n \"\"\"Setup global variables before each benchmark.\"\"\"\n global adata, batch_key\n adata, batch_key = get_dataset(dataset, layer=layer)\n\n\n# ASV suite\n\nparams: tuple[list[Dataset], list[KeyX]] = (\n [\"pbmc68k_reduced\", \"pbmc3k\"],\n [None, \"off-axis\"],\n)\nparam_names = [\"dataset\", \"layer\"]\n\nskip_when = param_skipper(param_names, params)\n\n\ndef time_pca(*_):\n sc.pp.pca(adata, svd_solver=\"arpack\")\n\n\ndef peakmem_pca(*_):\n sc.pp.pca(adata, svd_solver=\"arpack\")\n\n\ndef time_highly_variable_genes(*_):\n # the default flavor runs on log-transformed data\n sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)\n\n\ndef peakmem_highly_variable_genes(*_):\n sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)\n\n\n# regress_out is very slow for this dataset\n@skip_when(dataset={\"pbmc3k\"})\ndef time_regress_out(*_):\n sc.pp.regress_out(adata, [\"total_counts\", \"pct_counts_mt\"])\n\n\n@skip_when(dataset={\"pbmc3k\"})\ndef peakmem_regress_out(*_):\n sc.pp.regress_out(adata, [\"total_counts\", \"pct_counts_mt\"])\n\n\ndef time_scale(*_):\n sc.pp.scale(adata, max_value=10)\n\n\ndef peakmem_scale(*_):\n sc.pp.scale(adata, max_value=10)\n\n\nclass FastSuite:\n \"\"\"Suite for fast preprocessing operations.\"\"\"\n\n params: tuple[list[Dataset], list[KeyX]] = (\n [\"pbmc3k\", \"pbmc68k_reduced\", \"bmmc\", \"lung93k\"],\n [None, \"off-axis\"],\n )\n param_names = [\"dataset\", \"layer\"]\n\n def time_mean_var(self, *_):\n _get_mean_var(adata.X)\n\n def peakmem_mean_var(self, *_):\n _get_mean_var(adata.X)\n\n\n\"\"\"\nThis module will benchmark preprocessing operations in Scanpy that run on counts\nAPI documentation: https://scanpy.readthedocs.io/en/stable/api/preprocessing.html\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport scanpy as sc\n\nfrom ._utils import get_count_dataset\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n from ._utils import Dataset, KeyCount\n\n# setup variables\n\nadata: AnnData\nbatch_key: str | None\n\n\ndef setup(dataset: Dataset, layer: KeyCount, *_):\n \"\"\"Setup global variables before each benchmark.\"\"\"\n global adata, batch_key\n adata, batch_key = get_count_dataset(dataset, layer=layer)\n assert \"log1p\" not in adata.uns\n\n\n# ASV suite\n\nparams: tuple[list[Dataset], list[KeyCount]] = (\n [\"pbmc68k_reduced\", \"pbmc3k\"],\n [\"counts\", \"counts-off-axis\"],\n)\nparam_names = [\"dataset\", \"layer\"]\n\n\ndef time_filter_cells(*_):\n sc.pp.filter_cells(adata, min_genes=100)\n\n\ndef peakmem_filter_cells(*_):\n sc.pp.filter_cells(adata, min_genes=100)\n\n\ndef time_filter_genes(*_):\n sc.pp.filter_genes(adata, min_cells=3)\n\n\ndef peakmem_filter_genes(*_):\n sc.pp.filter_genes(adata, min_cells=3)\n\n\ndef time_scrublet(*_):\n sc.pp.scrublet(adata, batch_key=batch_key)\n\n\ndef peakmem_scrublet(*_):\n sc.pp.scrublet(adata, batch_key=batch_key)\n\n\n# Can’t do seurat v3 yet: https://github.com/conda-forge/scikit-misc-feedstock/issues/17\n\"\"\"\ndef time_hvg_seurat_v3(*_):\n # seurat v3 runs on counts\n sc.pp.highly_variable_genes(adata, flavor=\"seurat_v3_paper\")\n\n\ndef peakmem_hvg_seurat_v3(*_):\n sc.pp.highly_variable_genes(adata, flavor=\"seurat_v3_paper\")\n\"\"\"\n\n\nclass FastSuite:\n \"\"\"Suite for fast preprocessing operations.\"\"\"\n\n params: tuple[list[Dataset], list[KeyCount]] = (\n [\"pbmc3k\", \"pbmc68k_reduced\", \"bmmc\", \"lung93k\"],\n [\"counts\", \"counts-off-axis\"],\n )\n param_names = [\"dataset\", \"layer\"]\n\n def time_calculate_qc_metrics(self, *_):\n sc.pp.calculate_qc_metrics(\n adata, qc_vars=[\"mt\"], percent_top=None, log1p=False, inplace=True\n )\n\n def peakmem_calculate_qc_metrics(self, *_):\n sc.pp.calculate_qc_metrics(\n adata, qc_vars=[\"mt\"], percent_top=None, log1p=False, inplace=True\n )\n\n def time_normalize_total(self, *_):\n sc.pp.normalize_total(adata, target_sum=1e4)\n\n def peakmem_normalize_total(self, *_):\n sc.pp.normalize_total(adata, target_sum=1e4)\n\n def time_log1p(self, *_):\n # TODO: This would fail: assert \"log1p\" not in adata.uns, \"ASV bug?\"\n # https://github.com/scverse/scanpy/issues/3052\n adata.uns.pop(\"log1p\", None)\n sc.pp.log1p(adata)\n\n def peakmem_log1p(self, *_):\n adata.uns.pop(\"log1p\", None)\n sc.pp.log1p(adata)\n\n\nfrom __future__ import annotations\n\nimport itertools\nimport warnings\nfrom functools import cache\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pooch\nfrom anndata import concat\nfrom asv_runner.benchmarks.mark import skip_for_params\nfrom scipy import sparse\n\nimport scanpy as sc\n\nif TYPE_CHECKING:\n from collections.abc import Callable, Sequence, Set\n from typing import Literal, Protocol, TypeVar\n\n from anndata import AnnData\n\n C = TypeVar(\"C\", bound=Callable)\n\n class ParamSkipper(Protocol):\n def __call__(self, **skipped: Set) -> Callable[[C], C]: ...\n\n Dataset = Literal[\"pbmc68k_reduced\", \"pbmc3k\", \"bmmc\", \"lung93k\"]\n KeyX = Literal[None, \"off-axis\"]\n KeyCount = Literal[\"counts\", \"counts-off-axis\"]\n\n\n@cache\ndef _pbmc68k_reduced() -> AnnData:\n \"\"\"A small datasets with a dense `.X`\"\"\"\n adata = sc.datasets.pbmc68k_reduced()\n assert isinstance(adata.X, np.ndarray)\n assert not np.isfortran(adata.X)\n\n # raw has the same number of genes, so we can use it for counts\n # it doesn’t actually contain counts for some reason, but close enough\n assert isinstance(adata.raw.X, sparse.csr_matrix)\n adata.layers[\"counts\"] = adata.raw.X.toarray(order=\"C\")\n mapper = dict(\n percent_mito=\"pct_counts_mt\",\n n_counts=\"total_counts\",\n )\n adata.obs.rename(columns=mapper, inplace=True)\n return adata\n\n\ndef pbmc68k_reduced() -> AnnData:\n return _pbmc68k_reduced().copy()\n\n\n@cache\ndef _pbmc3k() -> AnnData:\n adata = sc.datasets.pbmc3k()\n assert isinstance(adata.X, sparse.csr_matrix)\n adata.layers[\"counts\"] = adata.X.astype(np.int32, copy=True)\n sc.pp.log1p(adata)\n return adata\n\n\ndef pbmc3k() -> AnnData:\n return _pbmc3k().copy()\n\n\n@cache\ndef _bmmc(n_obs: int = 4000) -> AnnData:\n registry = pooch.create(\n path=pooch.os_cache(\"pooch\"),\n base_url=\"doi:10.6084/m9.figshare.22716739.v1/\",\n )\n registry.load_registry_from_doi()\n samples = {smp: f\"{smp}_filtered_feature_bc_matrix.h5\" for smp in (\"s1d1\", \"s1d3\")}\n adatas = {}\n\n for sample_id, filename in samples.items():\n path = registry.fetch(filename)\n with warnings.catch_warnings():\n warnings.filterwarnings(\"ignore\", r\"Variable names are not unique\")\n sample_adata = sc.read_10x_h5(path)\n sample_adata.var_names_make_unique()\n sc.pp.subsample(sample_adata, n_obs=n_obs // len(samples))\n adatas[sample_id] = sample_adata\n\n with warnings.catch_warnings():\n warnings.filterwarnings(\"ignore\", r\"Observation names are not unique\")\n adata = concat(adatas, label=\"sample\")\n adata.obs_names_make_unique()\n\n assert isinstance(adata.X, sparse.csr_matrix)\n adata.layers[\"counts\"] = adata.X.astype(np.int32, copy=True)\n sc.pp.log1p(adata)\n adata.obs[\"n_counts\"] = adata.layers[\"counts\"].sum(axis=1).A1\n return adata\n\n\ndef bmmc(n_obs: int = 400) -> AnnData:\n return _bmmc(n_obs).copy()\n\n\n@cache\ndef _lung93k() -> AnnData:\n path = pooch.retrieve(\n url=\"https://figshare.com/ndownloader/files/45788454\",\n known_hash=\"md5:4f28af5ff226052443e7e0b39f3f9212\",\n )\n adata = sc.read_h5ad(path)\n assert isinstance(adata.X, sparse.csr_matrix)\n adata.layers[\"counts\"] = adata.X.astype(np.int32, copy=True)\n sc.pp.log1p(adata)\n return adata\n\n\ndef lung93k() -> AnnData:\n return _lung93k().copy()\n\n\ndef to_off_axis(x: np.ndarray | sparse.csr_matrix) -> np.ndarray | sparse.csc_matrix:\n if isinstance(x, sparse.csr_matrix):\n return x.tocsc()\n if isinstance(x, np.ndarray):\n assert not np.isfortran(x)\n return x.copy(order=\"F\")\n msg = f\"Unexpected type {type(x)}\"\n raise TypeError(msg)\n\n\ndef _get_dataset_raw(dataset: Dataset) -> tuple[AnnData, str | None]:\n match dataset:\n case \"pbmc68k_reduced\":\n adata, batch_key = pbmc68k_reduced(), None\n case \"pbmc3k\":\n adata, batch_key = pbmc3k(), None # can’t use this with batches\n case \"bmmc\":\n # TODO: allow specifying bigger variant\n adata, batch_key = bmmc(400), \"sample\"\n case \"lung93k\":\n adata, batch_key = lung93k(), \"PatientNumber\"\n case _:\n msg = f\"Unknown dataset {dataset}\"\n raise AssertionError(msg)\n\n # add off-axis layers\n adata.layers[\"off-axis\"] = to_off_axis(adata.X)\n adata.layers[\"counts-off-axis\"] = to_off_axis(adata.layers[\"counts\"])\n\n # add mitochondrial gene and pre-compute qc metrics\n adata.var[\"mt\"] = adata.var_names.str.startswith(\"MT-\")\n assert adata.var[\"mt\"].sum() > 0, \"no MT genes in dataset\"\n sc.pp.calculate_qc_metrics(\n adata, qc_vars=[\"mt\"], percent_top=None, log1p=False, inplace=True\n )\n\n return adata, batch_key\n\n\ndef get_dataset(dataset: Dataset, *, layer: KeyX = None) -> tuple[AnnData, str | None]:\n adata, batch_key = _get_dataset_raw(dataset)\n if layer is not None:\n adata.X = adata.layers.pop(layer)\n return adata, batch_key\n\n\ndef get_count_dataset(\n dataset: Dataset, *, layer: KeyCount = \"counts\"\n) -> tuple[AnnData, str | None]:\n adata, batch_key = _get_dataset_raw(dataset)\n\n adata.X = adata.layers.pop(layer)\n # remove indicators that X was transformed\n adata.uns.pop(\"log1p\", None)\n\n return adata, batch_key\n\n\ndef param_skipper(\n param_names: Sequence[str], params: tuple[Sequence[object], ...]\n) -> ParamSkipper:\n \"\"\"Creates a decorator that will skip all combinations that contain any of the given parameters.\n\n Examples\n --------\n\n >>> param_names = [\"letters\", \"numbers\"]\n >>> params = [[\"a\", \"b\"], [3, 4, 5]]\n >>> skip_when = param_skipper(param_names, params)\n\n >>> @skip_when(letters={\"a\"}, numbers={3})\n ... def func(a, b):\n ... print(a, b)\n >>> run_as_asv_benchmark(func)\n b 4\n b 5\n \"\"\"\n\n def skip(**skipped: Set) -> Callable[[C], C]:\n skipped_combs = [\n tuple(record.values())\n for record in (\n dict(zip(param_names, vals)) for vals in itertools.product(*params)\n )\n if any(v in skipped.get(n, set()) for n, v in record.items())\n ]\n # print(skipped_combs, file=sys.stderr)\n return skip_for_params(skipped_combs)\n\n return skip\n\n\n\"\"\"\nThis module will benchmark tool operations in Scanpy\nAPI documentation: https://scanpy.readthedocs.io/en/stable/api/tools.html\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport scanpy as sc\n\nfrom ._utils import pbmc68k_reduced\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n# setup variables\n\nadata: AnnData\n\n\ndef setup():\n global adata\n adata = pbmc68k_reduced()\n assert \"X_pca\" in adata.obsm\n\n\ndef time_umap():\n sc.tl.umap(adata)\n\n\ndef peakmem_umap():\n sc.tl.umap(adata)\n\n\ndef time_diffmap():\n sc.tl.diffmap(adata)\n\n\ndef peakmem_diffmap():\n sc.tl.diffmap(adata)\n\n\ndef time_leiden():\n sc.tl.leiden(adata, flavor=\"igraph\")\n\n\ndef peakmem_leiden():\n sc.tl.leiden(adata, flavor=\"igraph\")\n\n\n\n\n\"\"\"Reading and Writing\"\"\"\n\nfrom __future__ import annotations\n\nimport json\nfrom pathlib import Path, PurePath\nfrom typing import TYPE_CHECKING\n\nimport anndata.utils\nimport h5py\nimport numpy as np\nimport pandas as pd\nfrom anndata import (\n AnnData,\n read_csv,\n read_excel,\n read_h5ad,\n read_hdf,\n read_loom,\n read_mtx,\n read_text,\n)\nfrom matplotlib.image import imread\n\nfrom . import logging as logg\nfrom ._compat import old_positionals\nfrom ._settings import settings\nfrom ._utils import _empty\n\nif TYPE_CHECKING:\n from typing import BinaryIO, Literal\n\n from ._utils import Empty\n\n# .gz and .bz2 suffixes are also allowed for text formats\ntext_exts = {\n \"csv\",\n \"tsv\",\n \"tab\",\n \"data\",\n \"txt\", # these four are all equivalent\n}\navail_exts = {\n \"anndata\",\n \"xlsx\",\n \"h5\",\n \"h5ad\",\n \"mtx\",\n \"mtx.gz\",\n \"soft.gz\",\n \"loom\",\n} | text_exts\n\"\"\"Available file formats for reading data. \"\"\"\n\n\n# --------------------------------------------------------------------------------\n# Reading and Writing data files and AnnData objects\n# --------------------------------------------------------------------------------\n\n\n@old_positionals(\n \"sheet\",\n \"ext\",\n \"delimiter\",\n \"first_column_names\",\n \"backup_url\",\n \"cache\",\n \"cache_compression\",\n)\ndef read(\n filename: Path | str,\n backed: Literal[\"r\", \"r+\"] | None = None,\n *,\n sheet: str | None = None,\n ext: str | None = None,\n delimiter: str | None = None,\n first_column_names: bool = False,\n backup_url: str | None = None,\n cache: bool = False,\n cache_compression: Literal[\"gzip\", \"lzf\"] | None | Empty = _empty,\n **kwargs,\n) -> AnnData:\n \"\"\"\\\n Read file and return :class:`~anndata.AnnData` object.\n\n To speed up reading, consider passing ``cache=True``, which creates an hdf5\n cache file.\n\n Parameters\n ----------\n filename\n If the filename has no file extension, it is interpreted as a key for\n generating a filename via ``sc.settings.writedir / (filename +\n sc.settings.file_format_data)``. This is the same behavior as in\n ``sc.read(filename, ...)``.\n backed\n If ``'r'``, load :class:`~anndata.AnnData` in ``backed`` mode instead\n of fully loading it into memory (`memory` mode). If you want to modify\n backed attributes of the AnnData object, you need to choose ``'r+'``.\n sheet\n Name of sheet/table in hdf5 or Excel file.\n ext\n Extension that indicates the file type. If ``None``, uses extension of\n filename.\n delimiter\n Delimiter that separates data within text file. If ``None``, will split at\n arbitrary number of white spaces, which is different from enforcing\n splitting at any single white space ``' '``.\n first_column_names\n Assume the first column stores row names. This is only necessary if\n these are not strings: strings in the first column are automatically\n assumed to be row names.\n backup_url\n Retrieve the file from an URL if not present on disk.\n cache\n If `False`, read from source, if `True`, read from fast 'h5ad' cache.\n cache_compression\n See the h5py :ref:`dataset_compression`.\n (Default: `settings.cache_compression`)\n kwargs\n Parameters passed to :func:`~anndata.read_loom`.\n\n Returns\n -------\n An :class:`~anndata.AnnData` object\n \"\"\"\n filename = Path(filename) # allow passing strings\n if is_valid_filename(filename):\n return _read(\n filename,\n backed=backed,\n sheet=sheet,\n ext=ext,\n delimiter=delimiter,\n first_column_names=first_column_names,\n backup_url=backup_url,\n cache=cache,\n cache_compression=cache_compression,\n **kwargs,\n )\n # generate filename and read to dict\n filekey = str(filename)\n filename = settings.writedir / (filekey + \".\" + settings.file_format_data)\n if not filename.exists():\n raise ValueError(\n f\"Reading with filekey {filekey!r} failed, \"\n f\"the inferred filename {filename!r} does not exist. \"\n \"If you intended to provide a filename, either use a filename \"\n f\"ending on one of the available extensions {avail_exts} \"\n \"or pass the parameter `ext`.\"\n )\n return read_h5ad(filename, backed=backed)\n\n\n@old_positionals(\"genome\", \"gex_only\", \"backup_url\")\ndef read_10x_h5(\n filename: Path | str,\n *,\n genome: str | None = None,\n gex_only: bool = True,\n backup_url: str | None = None,\n) -> AnnData:\n \"\"\"\\\n Read 10x-Genomics-formatted hdf5 file.\n\n Parameters\n ----------\n filename\n Path to a 10x hdf5 file.\n genome\n Filter expression to genes within this genome. For legacy 10x h5\n files, this must be provided if the data contains more than one genome.\n gex_only\n Only keep 'Gene Expression' data and ignore other feature types,\n e.g. 'Antibody Capture', 'CRISPR Guide Capture', or 'Custom'\n backup_url\n Retrieve the file from an URL if not present on disk.\n\n Returns\n -------\n Annotated data matrix, where observations/cells are named by their\n barcode and variables/genes by gene name. Stores the following information:\n\n :attr:`~anndata.AnnData.X`\n The data matrix is stored\n :attr:`~anndata.AnnData.obs_names`\n Cell names\n :attr:`~anndata.AnnData.var_names`\n Gene names for a feature barcode matrix, probe names for a probe bc matrix\n :attr:`~anndata.AnnData.var`\\\\ `['gene_ids']`\n Gene IDs\n :attr:`~anndata.AnnData.var`\\\\ `['feature_types']`\n Feature types\n :attr:`~anndata.AnnData.obs`\\\\ `[filtered_barcodes]`\n filtered barcodes if present in the matrix\n :attr:`~anndata.AnnData.var`\n Any additional metadata present in /matrix/features is read in.\n \"\"\"\n start = logg.info(f\"reading {filename}\")\n is_present = _check_datafile_present_and_download(filename, backup_url=backup_url)\n if not is_present:\n logg.debug(f\"... did not find original file {filename}\")\n with h5py.File(str(filename), \"r\") as f:\n v3 = \"/matrix\" in f\n if v3:\n adata = _read_v3_10x_h5(filename, start=start)\n if genome:\n if genome not in adata.var[\"genome\"].values:\n raise ValueError(\n f\"Could not find data corresponding to genome '{genome}' in '{filename}'. \"\n f'Available genomes are: {list(adata.var[\"genome\"].unique())}.'\n )\n adata = adata[:, adata.var[\"genome\"] == genome]\n if gex_only:\n adata = adata[:, adata.var[\"feature_types\"] == \"Gene Expression\"]\n if adata.is_view:\n adata = adata.copy()\n else:\n adata = _read_legacy_10x_h5(filename, genome=genome, start=start)\n return adata\n\n\ndef _read_legacy_10x_h5(filename, *, genome=None, start=None):\n \"\"\"\n Read hdf5 file from Cell Ranger v2 or earlier versions.\n \"\"\"\n with h5py.File(str(filename), \"r\") as f:\n try:\n children = list(f.keys())\n if not genome:\n if len(children) > 1:\n raise ValueError(\n f\"'{filename}' contains more than one genome. For legacy 10x h5 \"\n \"files you must specify the genome if more than one is present. \"\n f\"Available genomes are: {children}\"\n )\n genome = children[0]\n elif genome not in children:\n raise ValueError(\n f\"Could not find genome '{genome}' in '{filename}'. \"\n f\"Available genomes are: {children}\"\n )\n\n dsets = {}\n _collect_datasets(dsets, f[genome])\n\n # AnnData works with csr matrices\n # 10x stores the transposed data, so we do the transposition right away\n from scipy.sparse import csr_matrix\n\n M, N = dsets[\"shape\"]\n data = dsets[\"data\"]\n if dsets[\"data\"].dtype == np.dtype(\"int32\"):\n data = dsets[\"data\"].view(\"float32\")\n data[:] = dsets[\"data\"]\n matrix = csr_matrix(\n (data, dsets[\"indices\"], dsets[\"indptr\"]),\n shape=(N, M),\n )\n # the csc matrix is automatically the transposed csr matrix\n # as scanpy expects it, so, no need for a further transpostion\n adata = AnnData(\n matrix,\n obs=dict(obs_names=dsets[\"barcodes\"].astype(str)),\n var=dict(\n var_names=dsets[\"gene_names\"].astype(str),\n gene_ids=dsets[\"genes\"].astype(str),\n ),\n )\n logg.info(\"\", time=start)\n return adata\n except KeyError:\n raise Exception(\"File is missing one or more required datasets.\")\n\n\ndef _collect_datasets(dsets: dict, group: h5py.Group):\n for k, v in group.items():\n if isinstance(v, h5py.Dataset):\n dsets[k] = v[()]\n else:\n _collect_datasets(dsets, v)\n\n\ndef _read_v3_10x_h5(filename, *, start=None):\n \"\"\"\n Read hdf5 file from Cell Ranger v3 or later versions.\n \"\"\"\n with h5py.File(str(filename), \"r\") as f:\n try:\n dsets = {}\n _collect_datasets(dsets, f[\"matrix\"])\n\n from scipy.sparse import csr_matrix\n\n M, N = dsets[\"shape\"]\n data = dsets[\"data\"]\n if dsets[\"data\"].dtype == np.dtype(\"int32\"):\n data = dsets[\"data\"].view(\"float32\")\n data[:] = dsets[\"data\"]\n matrix = csr_matrix(\n (data, dsets[\"indices\"], dsets[\"indptr\"]),\n shape=(N, M),\n )\n obs_dict = {\"obs_names\": dsets[\"barcodes\"].astype(str)}\n var_dict = {\"var_names\": dsets[\"name\"].astype(str)}\n\n if \"gene_id\" not in dsets:\n # Read metadata specific to a feature-barcode matrix\n var_dict[\"gene_ids\"] = dsets[\"id\"].astype(str)\n else:\n # Read metadata specific to a probe-barcode matrix\n var_dict.update(\n {\n \"gene_ids\": dsets[\"gene_id\"].astype(str),\n \"probe_ids\": dsets[\"id\"].astype(str),\n }\n )\n var_dict[\"feature_types\"] = dsets[\"feature_type\"].astype(str)\n if \"filtered_barcodes\" in f[\"matrix\"]:\n obs_dict[\"filtered_barcodes\"] = dsets[\"filtered_barcodes\"].astype(bool)\n\n if \"features\" in f[\"matrix\"]:\n var_dict.update(\n (\n feature_metadata_name,\n dsets[feature_metadata_name].astype(\n bool if feature_metadata_item.dtype.kind == \"b\" else str\n ),\n )\n for feature_metadata_name, feature_metadata_item in f[\"matrix\"][\n \"features\"\n ].items()\n if isinstance(feature_metadata_item, h5py.Dataset)\n and feature_metadata_name\n not in [\n \"name\",\n \"feature_type\",\n \"id\",\n \"gene_id\",\n \"_all_tag_keys\",\n ]\n )\n else:\n raise ValueError(\"10x h5 has no features group\")\n adata = AnnData(\n matrix,\n obs=obs_dict,\n var=var_dict,\n )\n logg.info(\"\", time=start)\n return adata\n except KeyError:\n raise Exception(\"File is missing one or more required datasets.\")\n\n\ndef read_visium(\n path: Path | str,\n genome: str | None = None,\n *,\n count_file: str = \"filtered_feature_bc_matrix.h5\",\n library_id: str | None = None,\n load_images: bool | None = True,\n source_image_path: Path | str | None = None,\n) -> AnnData:\n \"\"\"\\\n Read 10x-Genomics-formatted visum dataset.\n\n In addition to reading regular 10x output,\n this looks for the `spatial` folder and loads images,\n coordinates and scale factors.\n Based on the `Space Ranger output docs`_.\n\n See :func:`~scanpy.pl.spatial` for a compatible plotting function.\n\n .. _Space Ranger output docs: https://support.10xgenomics.com/spatial-gene-expression/software/pipelines/latest/output/overview\n\n Parameters\n ----------\n path\n Path to directory for visium datafiles.\n genome\n Filter expression to genes within this genome.\n count_file\n Which file in the passed directory to use as the count file. Typically would be one of:\n 'filtered_feature_bc_matrix.h5' or 'raw_feature_bc_matrix.h5'.\n library_id\n Identifier for the visium library. Can be modified when concatenating multiple adata objects.\n source_image_path\n Path to the high-resolution tissue image. Path will be included in\n `.uns[\"spatial\"][library_id][\"metadata\"][\"source_image_path\"]`.\n\n Returns\n -------\n Annotated data matrix, where observations/cells are named by their\n barcode and variables/genes by gene name. Stores the following information:\n\n :attr:`~anndata.AnnData.X`\n The data matrix is stored\n :attr:`~anndata.AnnData.obs_names`\n Cell names\n :attr:`~anndata.AnnData.var_names`\n Gene names for a feature barcode matrix, probe names for a probe bc matrix\n :attr:`~anndata.AnnData.var`\\\\ `['gene_ids']`\n Gene IDs\n :attr:`~anndata.AnnData.var`\\\\ `['feature_types']`\n Feature types\n :attr:`~anndata.AnnData.obs`\\\\ `[filtered_barcodes]`\n filtered barcodes if present in the matrix\n :attr:`~anndata.AnnData.var`\n Any additional metadata present in /matrix/features is read in.\n :attr:`~anndata.AnnData.uns`\\\\ `['spatial']`\n Dict of spaceranger output files with 'library_id' as key\n :attr:`~anndata.AnnData.uns`\\\\ `['spatial'][library_id]['images']`\n Dict of images (`'hires'` and `'lowres'`)\n :attr:`~anndata.AnnData.uns`\\\\ `['spatial'][library_id]['scalefactors']`\n Scale factors for the spots\n :attr:`~anndata.AnnData.uns`\\\\ `['spatial'][library_id]['metadata']`\n Files metadata: 'chemistry_description', 'software_version', 'source_image_path'\n :attr:`~anndata.AnnData.obsm`\\\\ `['spatial']`\n Spatial spot coordinates, usable as `basis` by :func:`~scanpy.pl.embedding`.\n \"\"\"\n path = Path(path)\n adata = read_10x_h5(path / count_file, genome=genome)\n\n adata.uns[\"spatial\"] = dict()\n\n from h5py import File\n\n with File(path / count_file, mode=\"r\") as f:\n attrs = dict(f.attrs)\n if library_id is None:\n library_id = str(attrs.pop(\"library_ids\")[0], \"utf-8\")\n\n adata.uns[\"spatial\"][library_id] = dict()\n\n if load_images:\n tissue_positions_file = (\n path / \"spatial/tissue_positions.csv\"\n if (path / \"spatial/tissue_positions.csv\").exists()\n else path / \"spatial/tissue_positions_list.csv\"\n )\n files = dict(\n tissue_positions_file=tissue_positions_file,\n scalefactors_json_file=path / \"spatial/scalefactors_json.json\",\n hires_image=path / \"spatial/tissue_hires_image.png\",\n lowres_image=path / \"spatial/tissue_lowres_image.png\",\n )\n\n # check if files exists, continue if images are missing\n for f in files.values():\n if not f.exists():\n if any(x in str(f) for x in [\"hires_image\", \"lowres_image\"]):\n logg.warning(\n f\"You seem to be missing an image file.\\n\"\n f\"Could not find '{f}'.\"\n )\n else:\n raise OSError(f\"Could not find '{f}'\")\n\n adata.uns[\"spatial\"][library_id][\"images\"] = dict()\n for res in [\"hires\", \"lowres\"]:\n try:\n adata.uns[\"spatial\"][library_id][\"images\"][res] = imread(\n str(files[f\"{res}_image\"])\n )\n except Exception:\n raise OSError(f\"Could not find '{res}_image'\")\n\n # read json scalefactors\n adata.uns[\"spatial\"][library_id][\"scalefactors\"] = json.loads(\n files[\"scalefactors_json_file\"].read_bytes()\n )\n\n adata.uns[\"spatial\"][library_id][\"metadata\"] = {\n k: (str(attrs[k], \"utf-8\") if isinstance(attrs[k], bytes) else attrs[k])\n for k in (\"chemistry_description\", \"software_version\")\n if k in attrs\n }\n\n # read coordinates\n positions = pd.read_csv(\n files[\"tissue_positions_file\"],\n header=0 if tissue_positions_file.name == \"tissue_positions.csv\" else None,\n index_col=0,\n )\n positions.columns = [\n \"in_tissue\",\n \"array_row\",\n \"array_col\",\n \"pxl_col_in_fullres\",\n \"pxl_row_in_fullres\",\n ]\n\n adata.obs = adata.obs.join(positions, how=\"left\")\n\n adata.obsm[\"spatial\"] = adata.obs[\n [\"pxl_row_in_fullres\", \"pxl_col_in_fullres\"]\n ].to_numpy()\n adata.obs.drop(\n columns=[\"pxl_row_in_fullres\", \"pxl_col_in_fullres\"],\n inplace=True,\n )\n\n # put image path in uns\n if source_image_path is not None:\n # get an absolute path\n source_image_path = str(Path(source_image_path).resolve())\n adata.uns[\"spatial\"][library_id][\"metadata\"][\"source_image_path\"] = str(\n source_image_path\n )\n\n return adata\n\n\n@old_positionals(\"var_names\", \"make_unique\", \"cache\", \"cache_compression\", \"gex_only\")\ndef read_10x_mtx(\n path: Path | str,\n *,\n var_names: Literal[\"gene_symbols\", \"gene_ids\"] = \"gene_symbols\",\n make_unique: bool = True,\n cache: bool = False,\n cache_compression: Literal[\"gzip\", \"lzf\"] | None | Empty = _empty,\n gex_only: bool = True,\n prefix: str | None = None,\n) -> AnnData:\n \"\"\"\\\n Read 10x-Genomics-formatted mtx directory.\n\n Parameters\n ----------\n path\n Path to directory for `.mtx` and `.tsv` files,\n e.g. './filtered_gene_bc_matrices/hg19/'.\n var_names\n The variables index.\n make_unique\n Whether to make the variables index unique by appending '-1',\n '-2' etc. or not.\n cache\n If `False`, read from source, if `True`, read from fast 'h5ad' cache.\n cache_compression\n See the h5py :ref:`dataset_compression`.\n (Default: `settings.cache_compression`)\n gex_only\n Only keep 'Gene Expression' data and ignore other feature types,\n e.g. 'Antibody Capture', 'CRISPR Guide Capture', or 'Custom'\n prefix\n Any prefix before `matrix.mtx`, `genes.tsv` and `barcodes.tsv`. For instance,\n if the files are named `patientA_matrix.mtx`, `patientA_genes.tsv` and\n `patientA_barcodes.tsv` the prefix is `patientA_`.\n (Default: no prefix)\n\n Returns\n -------\n An :class:`~anndata.AnnData` object\n \"\"\"\n path = Path(path)\n prefix = \"\" if prefix is None else prefix\n is_legacy = (path / f\"{prefix}genes.tsv\").is_file()\n adata = _read_10x_mtx(\n path,\n var_names=var_names,\n make_unique=make_unique,\n cache=cache,\n cache_compression=cache_compression,\n prefix=prefix,\n is_legacy=is_legacy,\n )\n if is_legacy or not gex_only:\n return adata\n gex_rows = adata.var[\"feature_types\"] == \"Gene Expression\"\n return adata[:, gex_rows].copy()\n\n\ndef _read_10x_mtx(\n path: Path,\n *,\n var_names: Literal[\"gene_symbols\", \"gene_ids\"] = \"gene_symbols\",\n make_unique: bool = True,\n cache: bool = False,\n cache_compression: Literal[\"gzip\", \"lzf\"] | None | Empty = _empty,\n prefix: str = \"\",\n is_legacy: bool,\n) -> AnnData:\n \"\"\"\n Read mex from output from Cell Ranger v2- or v3+\n \"\"\"\n suffix = \"\" if is_legacy else \".gz\"\n adata = read(\n path / f\"{prefix}matrix.mtx{suffix}\",\n cache=cache,\n cache_compression=cache_compression,\n ).T # transpose the data\n genes = pd.read_csv(\n path / f\"{prefix}{'genes' if is_legacy else 'features'}.tsv{suffix}\",\n header=None,\n sep=\"\\t\",\n )\n if var_names == \"gene_symbols\":\n var_names_idx = pd.Index(genes[1].values)\n if make_unique:\n var_names_idx = anndata.utils.make_index_unique(var_names_idx)\n adata.var_names = var_names_idx\n adata.var[\"gene_ids\"] = genes[0].values\n elif var_names == \"gene_ids\":\n adata.var_names = genes[0].values\n adata.var[\"gene_symbols\"] = genes[1].values\n else:\n raise ValueError(\"`var_names` needs to be 'gene_symbols' or 'gene_ids'\")\n if not is_legacy:\n adata.var[\"feature_types\"] = genes[2].values\n barcodes = pd.read_csv(path / f\"{prefix}barcodes.tsv{suffix}\", header=None)\n adata.obs_names = barcodes[0].values\n return adata\n\n\n@old_positionals(\"ext\", \"compression\", \"compression_opts\")\ndef write(\n filename: Path | str,\n adata: AnnData,\n *,\n ext: Literal[\"h5\", \"csv\", \"txt\", \"npz\"] | None = None,\n compression: Literal[\"gzip\", \"lzf\"] | None = \"gzip\",\n compression_opts: int | None = None,\n):\n \"\"\"\\\n Write :class:`~anndata.AnnData` objects to file.\n\n Parameters\n ----------\n filename\n If the filename has no file extension, it is interpreted as a key for\n generating a filename via `sc.settings.writedir / (filename +\n sc.settings.file_format_data)`. This is the same behavior as in\n :func:`~scanpy.read`.\n adata\n Annotated data matrix.\n ext\n File extension from wich to infer file format. If `None`, defaults to\n `sc.settings.file_format_data`.\n compression\n See https://docs.h5py.org/en/latest/high/dataset.html.\n compression_opts\n See https://docs.h5py.org/en/latest/high/dataset.html.\n \"\"\"\n filename = Path(filename) # allow passing strings\n if is_valid_filename(filename):\n filename = filename\n ext_ = is_valid_filename(filename, return_ext=True)\n if ext is None:\n ext = ext_\n elif ext != ext_:\n raise ValueError(\n \"It suffices to provide the file type by \"\n \"providing a proper extension to the filename.\"\n 'One of \"txt\", \"csv\", \"h5\" or \"npz\".'\n )\n else:\n key = filename\n ext = settings.file_format_data if ext is None else ext\n filename = _get_filename_from_key(key, ext)\n if ext == \"csv\":\n adata.write_csvs(filename)\n else:\n adata.write(\n filename, compression=compression, compression_opts=compression_opts\n )\n\n\n# -------------------------------------------------------------------------------\n# Reading and writing parameter files\n# -------------------------------------------------------------------------------\n\n\n@old_positionals(\"as_header\")\ndef read_params(\n filename: Path | str, *, as_header: bool = False\n) -> dict[str, int | float | bool | str | None]:\n \"\"\"\\\n Read parameter dictionary from text file.\n\n Assumes that parameters are specified in the format::\n\n par1 = value1\n par2 = value2\n\n Comments that start with '#' are allowed.\n\n Parameters\n ----------\n filename\n Filename of data file.\n asheader\n Read the dictionary from the header (comment section) of a file.\n\n Returns\n -------\n Dictionary that stores parameters.\n \"\"\"\n filename = Path(filename) # allow passing str objects\n from collections import OrderedDict\n\n params = OrderedDict([])\n for line in filename.open():\n if \"=\" in line and (not as_header or line.startswith(\"#\")):\n line = line[1:] if line.startswith(\"#\") else line\n key, val = line.split(\"=\")\n key = key.strip()\n val = val.strip()\n params[key] = convert_string(val)\n return params\n\n\ndef write_params(path: Path | str, *args, **maps):\n \"\"\"\\\n Write parameters to file, so that it's readable by read_params.\n\n Uses INI file format.\n \"\"\"\n path = Path(path)\n if not path.parent.is_dir():\n path.parent.mkdir(parents=True)\n if len(args) == 1:\n maps[None] = args[0]\n with path.open(\"w\") as f:\n for header, map in maps.items():\n if header is not None:\n f.write(f\"[{header}]\\n\")\n for key, val in map.items():\n f.write(f\"{key} = {val}\\n\")\n\n\n# -------------------------------------------------------------------------------\n# Reading and Writing data files\n# -------------------------------------------------------------------------------\n\n\ndef _read(\n filename: Path,\n *,\n backed=None,\n sheet=None,\n ext=None,\n delimiter=None,\n first_column_names=None,\n backup_url=None,\n cache=False,\n cache_compression=None,\n suppress_cache_warning=False,\n **kwargs,\n):\n if ext is not None and ext not in avail_exts:\n raise ValueError(\n \"Please provide one of the available extensions.\\n\" f\"{avail_exts}\"\n )\n else:\n ext = is_valid_filename(filename, return_ext=True)\n is_present = _check_datafile_present_and_download(filename, backup_url=backup_url)\n if not is_present:\n logg.debug(f\"... did not find original file {filename}\")\n # read hdf5 files\n if ext in {\"h5\", \"h5ad\"}:\n if sheet is None:\n return read_h5ad(filename, backed=backed)\n else:\n logg.debug(f\"reading sheet {sheet} from file {filename}\")\n return read_hdf(filename, sheet)\n # read other file types\n path_cache: Path = settings.cachedir / _slugify(filename).replace(\n f\".{ext}\", \".h5ad\"\n )\n if path_cache.suffix in {\".gz\", \".bz2\"}:\n path_cache = path_cache.with_suffix(\"\")\n if cache and path_cache.is_file():\n logg.info(f\"... reading from cache file {path_cache}\")\n return read_h5ad(path_cache)\n\n if not is_present:\n raise FileNotFoundError(f\"Did not find file {filename}.\")\n logg.debug(f\"reading {filename}\")\n if not cache and not suppress_cache_warning:\n logg.hint(\n \"This might be very slow. Consider passing `cache=True`, \"\n \"which enables much faster reading from a cache file.\"\n )\n # do the actual reading\n if ext == \"xlsx\" or ext == \"xls\":\n if sheet is None:\n raise ValueError(\"Provide `sheet` parameter when reading '.xlsx' files.\")\n else:\n adata = read_excel(filename, sheet)\n elif ext in {\"mtx\", \"mtx.gz\"}:\n adata = read_mtx(filename)\n elif ext == \"csv\":\n if delimiter is None:\n delimiter = \",\"\n adata = read_csv(\n filename, first_column_names=first_column_names, delimiter=delimiter\n )\n elif ext in {\"txt\", \"tab\", \"data\", \"tsv\"}:\n if ext == \"data\":\n logg.hint(\n \"... assuming '.data' means tab or white-space \" \"separated text file\",\n )\n logg.hint(\"change this by passing `ext` to sc.read\")\n adata = read_text(filename, delimiter, first_column_names)\n elif ext == \"soft.gz\":\n adata = _read_softgz(filename)\n elif ext == \"loom\":\n adata = read_loom(filename=filename, **kwargs)\n else:\n raise ValueError(f\"Unknown extension {ext}.\")\n if cache:\n logg.info(\n f\"... writing an {settings.file_format_data} \"\n \"cache file to speedup reading next time\"\n )\n if cache_compression is _empty:\n cache_compression = settings.cache_compression\n if not path_cache.parent.is_dir():\n path_cache.parent.mkdir(parents=True)\n # write for faster reading when calling the next time\n adata.write(path_cache, compression=cache_compression)\n return adata\n\n\ndef _slugify(path: str | PurePath) -> str:\n \"\"\"Make a path into a filename.\"\"\"\n if not isinstance(path, PurePath):\n path = PurePath(path)\n parts = list(path.parts)\n if parts[0] == \"/\":\n parts.pop(0)\n elif len(parts[0]) == 3 and parts[0][1:] == \":\\\\\":\n parts[0] = parts[0][0] # C:\\ → C\n filename = \"-\".join(parts)\n assert \"/\" not in filename, filename\n assert not filename[1:].startswith(\":\"), filename\n return filename\n\n\ndef _read_softgz(filename: str | bytes | Path | BinaryIO) -> AnnData:\n \"\"\"\\\n Read a SOFT format data file.\n\n The SOFT format is documented here\n https://www.ncbi.nlm.nih.gov/geo/info/soft.html.\n\n Notes\n -----\n The function is based on a script by Kerby Shedden.\n https://dept.stat.lsa.umich.edu/~kshedden/Python-Workshop/gene_expression_comparison.html\n \"\"\"\n import gzip\n\n with gzip.open(filename, mode=\"rt\") as file:\n # The header part of the file contains information about the\n # samples. Read that information first.\n samples_info = {}\n for line in file:\n if line.startswith(\"!dataset_table_begin\"):\n break\n elif line.startswith(\"!subset_description\"):\n subset_description = line.split(\"=\")[1].strip()\n elif line.startswith(\"!subset_sample_id\"):\n subset_ids = line.split(\"=\")[1].split(\",\")\n subset_ids = [x.strip() for x in subset_ids]\n for k in subset_ids:\n samples_info[k] = subset_description\n # Next line is the column headers (sample id's)\n sample_names = file.readline().strip().split(\"\\t\")\n # The column indices that contain gene expression data\n indices = [i for i, x in enumerate(sample_names) if x.startswith(\"GSM\")]\n # Restrict the column headers to those that we keep\n sample_names = [sample_names[i] for i in indices]\n # Get a list of sample labels\n groups = [samples_info[k] for k in sample_names]\n # Read the gene expression data as a list of lists, also get the gene\n # identifiers\n gene_names, X = [], []\n for line in file:\n # This is what signals the end of the gene expression data\n # section in the file\n if line.startswith(\"!dataset_table_end\"):\n break\n V = line.split(\"\\t\")\n # Extract the values that correspond to gene expression measures\n # and convert the strings to numbers\n x = [float(V[i]) for i in indices]\n X.append(x)\n gene_names.append(V[1])\n # Convert the Python list of lists to a Numpy array and transpose to match\n # the Scanpy convention of storing samples in rows and variables in colums.\n X = np.array(X).T\n obs = pd.DataFrame({\"groups\": groups}, index=sample_names)\n var = pd.DataFrame(index=gene_names)\n return AnnData(X=X, obs=obs, var=var)\n\n\n# -------------------------------------------------------------------------------\n# Type conversion\n# -------------------------------------------------------------------------------\n\n\ndef is_float(string: str) -> float:\n \"\"\"Check whether string is float.\n\n See also\n --------\n https://stackoverflow.com/questions/736043/checking-if-a-string-can-be-converted-to-float-in-python\n \"\"\"\n try:\n float(string)\n return True\n except ValueError:\n return False\n\n\ndef is_int(string: str) -> bool:\n \"\"\"Check whether string is integer.\"\"\"\n try:\n int(string)\n return True\n except ValueError:\n return False\n\n\ndef convert_bool(string: str) -> tuple[bool, bool]:\n \"\"\"Check whether string is boolean.\"\"\"\n if string == \"True\":\n return True, True\n elif string == \"False\":\n return True, False\n else:\n return False, False\n\n\ndef convert_string(string: str) -> int | float | bool | str | None:\n \"\"\"Convert string to int, float or bool.\"\"\"\n if is_int(string):\n return int(string)\n elif is_float(string):\n return float(string)\n elif convert_bool(string)[0]:\n return convert_bool(string)[1]\n elif string == \"None\":\n return None\n else:\n return string\n\n\n# -------------------------------------------------------------------------------\n# Helper functions for reading and writing\n# -------------------------------------------------------------------------------\n\n\ndef get_used_files():\n \"\"\"Get files used by processes with name scanpy.\"\"\"\n import psutil\n\n loop_over_scanpy_processes = (\n proc for proc in psutil.process_iter() if proc.name() == \"scanpy\"\n )\n filenames = []\n for proc in loop_over_scanpy_processes:\n try:\n flist = proc.open_files()\n for nt in flist:\n filenames.append(nt.path)\n # This catches a race condition where a process ends\n # before we can examine its files\n except psutil.NoSuchProcess:\n pass\n return set(filenames)\n\n\ndef _get_filename_from_key(key, ext=None) -> Path:\n ext = settings.file_format_data if ext is None else ext\n return settings.writedir / f\"{key}.{ext}\"\n\n\ndef _download(url: str, path: Path):\n try:\n import ipywidgets # noqa: F401\n from tqdm.auto import tqdm\n except ImportError:\n from tqdm import tqdm\n\n from urllib.error import URLError\n from urllib.request import Request, urlopen\n\n blocksize = 1024 * 8\n blocknum = 0\n\n try:\n req = Request(url, headers={\"User-agent\": \"scanpy-user\"})\n\n try:\n open_url = urlopen(req)\n except URLError:\n logg.warning(\n \"Failed to open the url with default certificates, trying with certifi.\"\n )\n\n from ssl import create_default_context\n\n from certifi import where\n\n open_url = urlopen(req, context=create_default_context(cafile=where()))\n\n with open_url as resp:\n total = resp.info().get(\"content-length\", None)\n with (\n tqdm(\n unit=\"B\",\n unit_scale=True,\n miniters=1,\n unit_divisor=1024,\n total=total if total is None else int(total),\n ) as t,\n path.open(\"wb\") as f,\n ):\n block = resp.read(blocksize)\n while block:\n f.write(block)\n blocknum += 1\n t.update(len(block))\n block = resp.read(blocksize)\n\n except (KeyboardInterrupt, Exception):\n # Make sure file doesn’t exist half-downloaded\n if path.is_file():\n path.unlink()\n raise\n\n\ndef _check_datafile_present_and_download(path, backup_url=None):\n \"\"\"Check whether the file is present, otherwise download.\"\"\"\n path = Path(path)\n if path.is_file():\n return True\n if backup_url is None:\n return False\n logg.info(\n f\"try downloading from url\\n{backup_url}\\n\"\n \"... this may take a while but only happens once\"\n )\n if not path.parent.is_dir():\n logg.info(f\"creating directory {path.parent}/ for saving data\")\n path.parent.mkdir(parents=True)\n\n _download(backup_url, path)\n return True\n\n\ndef is_valid_filename(filename: Path, *, return_ext: bool = False):\n \"\"\"Check whether the argument is a filename.\"\"\"\n ext = filename.suffixes\n\n if len(ext) > 2:\n logg.warning(\n f\"Your filename has more than two extensions: {ext}.\\n\"\n f\"Only considering the two last: {ext[-2:]}.\"\n )\n ext = ext[-2:]\n\n # cases for gzipped/bzipped text files\n if len(ext) == 2 and ext[0][1:] in text_exts and ext[1][1:] in (\"gz\", \"bz2\"):\n return ext[0][1:] if return_ext else True\n elif ext and ext[-1][1:] in avail_exts:\n return ext[-1][1:] if return_ext else True\n elif \"\".join(ext) == \".soft.gz\":\n return \"soft.gz\" if return_ext else True\n elif \"\".join(ext) == \".mtx.gz\":\n return \"mtx.gz\" if return_ext else True\n elif not return_ext:\n return False\n raise ValueError(\n f\"\"\"\\\n{filename!r} does not end on a valid extension.\nPlease, provide one of the available extensions.\n{avail_exts}\nText files with .gz and .bz2 extensions are also supported.\\\n\"\"\"\n )\n\n\nfrom __future__ import annotations\n\nimport inspect\nimport sys\nfrom contextlib import contextmanager\nfrom enum import IntEnum\nfrom logging import getLevelName\nfrom pathlib import Path\nfrom time import time\nfrom typing import TYPE_CHECKING\n\nfrom . import logging\nfrom ._compat import old_positionals\nfrom .logging import _RootLogger, _set_log_file, _set_log_level\n\nif TYPE_CHECKING:\n from collections.abc import Generator, Iterable\n from typing import Any, Literal, TextIO, Union\n\n # Collected from the print_* functions in matplotlib.backends\n _Format = Union[\n Literal[\"png\", \"jpg\", \"tif\", \"tiff\"],\n Literal[\"pdf\", \"ps\", \"eps\", \"svg\", \"svgz\", \"pgf\"],\n Literal[\"raw\", \"rgba\"],\n ]\n\n_VERBOSITY_TO_LOGLEVEL = {\n \"error\": \"ERROR\",\n \"warning\": \"WARNING\",\n \"info\": \"INFO\",\n \"hint\": \"HINT\",\n \"debug\": \"DEBUG\",\n}\n# Python 3.7+ ensures iteration order\nfor v, level in enumerate(list(_VERBOSITY_TO_LOGLEVEL.values())):\n _VERBOSITY_TO_LOGLEVEL[v] = level\n\n\nclass Verbosity(IntEnum):\n \"\"\"Logging verbosity levels.\"\"\"\n\n error = 0\n warning = 1\n info = 2\n hint = 3\n debug = 4\n\n def __eq__(self, other: Verbosity | int | str) -> bool:\n if isinstance(other, Verbosity):\n return self is other\n if isinstance(other, int):\n return self.value == other\n if isinstance(other, str):\n return self.name == other\n return NotImplemented\n\n @property\n def level(self) -> int:\n # getLevelName(str) returns the int level…\n return getLevelName(_VERBOSITY_TO_LOGLEVEL[self.name])\n\n @contextmanager\n def override(\n self, verbosity: Verbosity | str | int\n ) -> Generator[Verbosity, None, None]:\n \"\"\"\\\n Temporarily override verbosity\n \"\"\"\n settings.verbosity = verbosity\n yield self\n settings.verbosity = self\n\n\n# backwards compat\nVerbosity.warn = Verbosity.warning\n\n\ndef _type_check(var: Any, varname: str, types: type | tuple[type, ...]):\n if isinstance(var, types):\n return\n if isinstance(types, type):\n possible_types_str = types.__name__\n else:\n type_names = [t.__name__ for t in types]\n possible_types_str = \"{} or {}\".format(\n \", \".join(type_names[:-1]), type_names[-1]\n )\n raise TypeError(f\"{varname} must be of type {possible_types_str}\")\n\n\nclass ScanpyConfig:\n \"\"\"\\\n Config manager for scanpy.\n \"\"\"\n\n N_PCS: int\n \"\"\"Default number of principal components to use.\"\"\"\n\n def __init__(\n self,\n *,\n verbosity: Verbosity | int | str = Verbosity.warning,\n plot_suffix: str = \"\",\n file_format_data: str = \"h5ad\",\n file_format_figs: str = \"pdf\",\n autosave: bool = False,\n autoshow: bool = True,\n writedir: Path | str = \"./write/\",\n cachedir: Path | str = \"./cache/\",\n datasetdir: Path | str = \"./data/\",\n figdir: Path | str = \"./figures/\",\n cache_compression: str | None = \"lzf\",\n max_memory=15,\n n_jobs=1,\n logfile: Path | str | None = None,\n categories_to_ignore: Iterable[str] = (\"N/A\", \"dontknow\", \"no_gate\", \"?\"),\n _frameon: bool = True,\n _vector_friendly: bool = False,\n _low_resolution_warning: bool = True,\n n_pcs=50,\n ):\n # logging\n self._root_logger = _RootLogger(logging.INFO) # level will be replaced\n self.logfile = logfile\n self.verbosity = verbosity\n # rest\n self.plot_suffix = plot_suffix\n self.file_format_data = file_format_data\n self.file_format_figs = file_format_figs\n self.autosave = autosave\n self.autoshow = autoshow\n self.writedir = writedir\n self.cachedir = cachedir\n self.datasetdir = datasetdir\n self.figdir = figdir\n self.cache_compression = cache_compression\n self.max_memory = max_memory\n self.n_jobs = n_jobs\n self.categories_to_ignore = categories_to_ignore\n self._frameon = _frameon\n \"\"\"bool: See set_figure_params.\"\"\"\n\n self._vector_friendly = _vector_friendly\n \"\"\"Set to true if you want to include pngs in svgs and pdfs.\"\"\"\n\n self._low_resolution_warning = _low_resolution_warning\n \"\"\"Print warning when saving a figure with low resolution.\"\"\"\n\n self._start = time()\n \"\"\"Time when the settings module is first imported.\"\"\"\n\n self._previous_time = self._start\n \"\"\"Variable for timing program parts.\"\"\"\n\n self._previous_memory_usage = -1\n \"\"\"Stores the previous memory usage.\"\"\"\n\n self.N_PCS = n_pcs\n\n @property\n def verbosity(self) -> Verbosity:\n \"\"\"\n Verbosity level (default `warning`)\n\n Level 0: only show 'error' messages.\n Level 1: also show 'warning' messages.\n Level 2: also show 'info' messages.\n Level 3: also show 'hint' messages.\n Level 4: also show very detailed progress for 'debug'ging.\n \"\"\"\n return self._verbosity\n\n @verbosity.setter\n def verbosity(self, verbosity: Verbosity | int | str):\n verbosity_str_options = [\n v for v in _VERBOSITY_TO_LOGLEVEL if isinstance(v, str)\n ]\n if isinstance(verbosity, Verbosity):\n self._verbosity = verbosity\n elif isinstance(verbosity, int):\n self._verbosity = Verbosity(verbosity)\n elif isinstance(verbosity, str):\n verbosity = verbosity.lower()\n if verbosity not in verbosity_str_options:\n raise ValueError(\n f\"Cannot set verbosity to {verbosity}. \"\n f\"Accepted string values are: {verbosity_str_options}\"\n )\n else:\n self._verbosity = Verbosity(verbosity_str_options.index(verbosity))\n else:\n _type_check(verbosity, \"verbosity\", (str, int))\n _set_log_level(self, _VERBOSITY_TO_LOGLEVEL[self._verbosity.name])\n\n @property\n def plot_suffix(self) -> str:\n \"\"\"Global suffix that is appended to figure filenames.\"\"\"\n return self._plot_suffix\n\n @plot_suffix.setter\n def plot_suffix(self, plot_suffix: str):\n _type_check(plot_suffix, \"plot_suffix\", str)\n self._plot_suffix = plot_suffix\n\n @property\n def file_format_data(self) -> str:\n \"\"\"File format for saving AnnData objects.\n\n Allowed are 'txt', 'csv' (comma separated value file) for exporting and 'h5ad'\n (hdf5) for lossless saving.\n \"\"\"\n return self._file_format_data\n\n @file_format_data.setter\n def file_format_data(self, file_format: str):\n _type_check(file_format, \"file_format_data\", str)\n file_format_options = {\"txt\", \"csv\", \"h5ad\"}\n if file_format not in file_format_options:\n raise ValueError(\n f\"Cannot set file_format_data to {file_format}. \"\n f\"Must be one of {file_format_options}\"\n )\n self._file_format_data = file_format\n\n @property\n def file_format_figs(self) -> str:\n \"\"\"File format for saving figures.\n\n For example 'png', 'pdf' or 'svg'. Many other formats work as well (see\n `matplotlib.pyplot.savefig`).\n \"\"\"\n return self._file_format_figs\n\n @file_format_figs.setter\n def file_format_figs(self, figure_format: str):\n _type_check(figure_format, \"figure_format_data\", str)\n self._file_format_figs = figure_format\n\n @property\n def autosave(self) -> bool:\n \"\"\"\\\n Automatically save figures in :attr:`~scanpy._settings.ScanpyConfig.figdir` (default `False`).\n\n Do not show plots/figures interactively.\n \"\"\"\n return self._autosave\n\n @autosave.setter\n def autosave(self, autosave: bool):\n _type_check(autosave, \"autosave\", bool)\n self._autosave = autosave\n\n @property\n def autoshow(self) -> bool:\n \"\"\"\\\n Automatically show figures if `autosave == False` (default `True`).\n\n There is no need to call the matplotlib pl.show() in this case.\n \"\"\"\n return self._autoshow\n\n @autoshow.setter\n def autoshow(self, autoshow: bool):\n _type_check(autoshow, \"autoshow\", bool)\n self._autoshow = autoshow\n\n @property\n def writedir(self) -> Path:\n \"\"\"\\\n Directory where the function scanpy.write writes to by default.\n \"\"\"\n return self._writedir\n\n @writedir.setter\n def writedir(self, writedir: Path | str):\n _type_check(writedir, \"writedir\", (str, Path))\n self._writedir = Path(writedir)\n\n @property\n def cachedir(self) -> Path:\n \"\"\"\\\n Directory for cache files (default `'./cache/'`).\n \"\"\"\n return self._cachedir\n\n @cachedir.setter\n def cachedir(self, cachedir: Path | str):\n _type_check(cachedir, \"cachedir\", (str, Path))\n self._cachedir = Path(cachedir)\n\n @property\n def datasetdir(self) -> Path:\n \"\"\"\\\n Directory for example :mod:`~scanpy.datasets` (default `'./data/'`).\n \"\"\"\n return self._datasetdir\n\n @datasetdir.setter\n def datasetdir(self, datasetdir: Path | str):\n _type_check(datasetdir, \"datasetdir\", (str, Path))\n self._datasetdir = Path(datasetdir).resolve()\n\n @property\n def figdir(self) -> Path:\n \"\"\"\\\n Directory for saving figures (default `'./figures/'`).\n \"\"\"\n return self._figdir\n\n @figdir.setter\n def figdir(self, figdir: Path | str):\n _type_check(figdir, \"figdir\", (str, Path))\n self._figdir = Path(figdir)\n\n @property\n def cache_compression(self) -> str | None:\n \"\"\"\\\n Compression for `sc.read(..., cache=True)` (default `'lzf'`).\n\n May be `'lzf'`, `'gzip'`, or `None`.\n \"\"\"\n return self._cache_compression\n\n @cache_compression.setter\n def cache_compression(self, cache_compression: str | None):\n if cache_compression not in {\"lzf\", \"gzip\", None}:\n raise ValueError(\n f\"`cache_compression` ({cache_compression}) \"\n \"must be in {'lzf', 'gzip', None}\"\n )\n self._cache_compression = cache_compression\n\n @property\n def max_memory(self) -> int | float:\n \"\"\"\\\n Maximum memory usage in Gigabyte.\n\n Is currently not well respected…\n \"\"\"\n return self._max_memory\n\n @max_memory.setter\n def max_memory(self, max_memory: int | float):\n _type_check(max_memory, \"max_memory\", (int, float))\n self._max_memory = max_memory\n\n @property\n def n_jobs(self) -> int:\n \"\"\"\\\n Default number of jobs/ CPUs to use for parallel computing.\n\n Set to `-1` in order to use all available cores.\n Not all algorithms support special behavior for numbers < `-1`,\n so make sure to leave this setting as >= `-1`.\n \"\"\"\n return self._n_jobs\n\n @n_jobs.setter\n def n_jobs(self, n_jobs: int):\n _type_check(n_jobs, \"n_jobs\", int)\n self._n_jobs = n_jobs\n\n @property\n def logpath(self) -> Path | None:\n \"\"\"\\\n The file path `logfile` was set to.\n \"\"\"\n return self._logpath\n\n @logpath.setter\n def logpath(self, logpath: Path | str | None):\n _type_check(logpath, \"logfile\", (str, Path))\n # set via “file object” branch of logfile.setter\n self.logfile = Path(logpath).open(\"a\") # noqa: SIM115\n self._logpath = Path(logpath)\n\n @property\n def logfile(self) -> TextIO:\n \"\"\"\\\n The open file to write logs to.\n\n Set it to a :class:`~pathlib.Path` or :class:`str` to open a new one.\n The default `None` corresponds to :obj:`sys.stdout` in jupyter notebooks\n and to :obj:`sys.stderr` otherwise.\n\n For backwards compatibility, setting it to `''` behaves like setting it to `None`.\n \"\"\"\n return self._logfile\n\n @logfile.setter\n def logfile(self, logfile: Path | str | TextIO | None):\n if not hasattr(logfile, \"write\") and logfile:\n self.logpath = logfile\n else: # file object\n if not logfile: # None or ''\n logfile = sys.stdout if self._is_run_from_ipython() else sys.stderr\n self._logfile = logfile\n self._logpath = None\n _set_log_file(self)\n\n @property\n def categories_to_ignore(self) -> list[str]:\n \"\"\"\\\n Categories that are omitted in plotting etc.\n \"\"\"\n return self._categories_to_ignore\n\n @categories_to_ignore.setter\n def categories_to_ignore(self, categories_to_ignore: Iterable[str]):\n categories_to_ignore = list(categories_to_ignore)\n for i, cat in enumerate(categories_to_ignore):\n _type_check(cat, f\"categories_to_ignore[{i}]\", str)\n self._categories_to_ignore = categories_to_ignore\n\n # --------------------------------------------------------------------------------\n # Functions\n # --------------------------------------------------------------------------------\n\n @old_positionals(\n \"scanpy\",\n \"dpi\",\n \"dpi_save\",\n \"frameon\",\n \"vector_friendly\",\n \"fontsize\",\n \"figsize\",\n \"color_map\",\n \"format\",\n \"facecolor\",\n \"transparent\",\n \"ipython_format\",\n )\n def set_figure_params(\n self,\n *,\n scanpy: bool = True,\n dpi: int = 80,\n dpi_save: int = 150,\n frameon: bool = True,\n vector_friendly: bool = True,\n fontsize: int = 14,\n figsize: int | None = None,\n color_map: str | None = None,\n format: _Format = \"pdf\",\n facecolor: str | None = None,\n transparent: bool = False,\n ipython_format: str = \"png2x\",\n ) -> None:\n \"\"\"\\\n Set resolution/size, styling and format of figures.\n\n Parameters\n ----------\n scanpy\n Init default values for :obj:`matplotlib.rcParams` suited for Scanpy.\n dpi\n Resolution of rendered figures – this influences the size of figures in notebooks.\n dpi_save\n Resolution of saved figures. This should typically be higher to achieve\n publication quality.\n frameon\n Add frames and axes labels to scatter plots.\n vector_friendly\n Plot scatter plots using `png` backend even when exporting as `pdf` or `svg`.\n fontsize\n Set the fontsize for several `rcParams` entries. Ignored if `scanpy=False`.\n figsize\n Set plt.rcParams['figure.figsize'].\n color_map\n Convenience method for setting the default color map. Ignored if `scanpy=False`.\n format\n This sets the default format for saving figures: `file_format_figs`.\n facecolor\n Sets backgrounds via `rcParams['figure.facecolor'] = facecolor` and\n `rcParams['axes.facecolor'] = facecolor`.\n transparent\n Save figures with transparent back ground. Sets\n `rcParams['savefig.transparent']`.\n ipython_format\n Only concerns the notebook/IPython environment; see\n :func:`~IPython.display.set_matplotlib_formats` for details.\n \"\"\"\n if self._is_run_from_ipython():\n import IPython\n\n if isinstance(ipython_format, str):\n ipython_format = [ipython_format]\n IPython.display.set_matplotlib_formats(*ipython_format)\n\n from matplotlib import rcParams\n\n self._vector_friendly = vector_friendly\n self.file_format_figs = format\n if dpi is not None:\n rcParams[\"figure.dpi\"] = dpi\n if dpi_save is not None:\n rcParams[\"savefig.dpi\"] = dpi_save\n if transparent is not None:\n rcParams[\"savefig.transparent\"] = transparent\n if facecolor is not None:\n rcParams[\"figure.facecolor\"] = facecolor\n rcParams[\"axes.facecolor\"] = facecolor\n if scanpy:\n from .plotting._rcmod import set_rcParams_scanpy\n\n set_rcParams_scanpy(fontsize=fontsize, color_map=color_map)\n if figsize is not None:\n rcParams[\"figure.figsize\"] = figsize\n self._frameon = frameon\n\n @staticmethod\n def _is_run_from_ipython():\n \"\"\"Determines whether we're currently in IPython.\"\"\"\n import builtins\n\n return getattr(builtins, \"__IPYTHON__\", False)\n\n def __str__(self) -> str:\n return \"\\n\".join(\n f\"{k} = {v!r}\"\n for k, v in inspect.getmembers(self)\n if not k.startswith(\"_\") and k != \"getdoc\"\n )\n\n\nsettings = ScanpyConfig()\n\n\nfrom __future__ import annotations\n\nfrom .cli import console_main\n\nconsole_main()\n\n\n\"\"\"Logging and Profiling\"\"\"\n\nfrom __future__ import annotations\n\nimport logging\nimport sys\nimport warnings\nfrom datetime import datetime, timedelta, timezone\nfrom functools import partial, update_wrapper\nfrom logging import CRITICAL, DEBUG, ERROR, INFO, WARNING\nfrom typing import TYPE_CHECKING\n\nimport anndata.logging\n\nif TYPE_CHECKING:\n from typing import IO\n\n from ._settings import ScanpyConfig\n\n\n# This is currently the only documented API\n__all__ = [\"print_versions\"]\n\nHINT = (INFO + DEBUG) // 2\nlogging.addLevelName(HINT, \"HINT\")\n\n\nclass _RootLogger(logging.RootLogger):\n def __init__(self, level):\n super().__init__(level)\n self.propagate = False\n _RootLogger.manager = logging.Manager(self)\n\n def log(\n self,\n level: int,\n msg: str,\n *,\n extra: dict | None = None,\n time: datetime | None = None,\n deep: str | None = None,\n ) -> datetime:\n from ._settings import settings\n\n now = datetime.now(timezone.utc)\n time_passed: timedelta = None if time is None else now - time\n extra = {\n **(extra or {}),\n \"deep\": deep if settings.verbosity.level < level else None,\n \"time_passed\": time_passed,\n }\n super().log(level, msg, extra=extra)\n return now\n\n def critical(self, msg, *, time=None, deep=None, extra=None) -> datetime:\n return self.log(CRITICAL, msg, time=time, deep=deep, extra=extra)\n\n def error(self, msg, *, time=None, deep=None, extra=None) -> datetime:\n return self.log(ERROR, msg, time=time, deep=deep, extra=extra)\n\n def warning(self, msg, *, time=None, deep=None, extra=None) -> datetime:\n return self.log(WARNING, msg, time=time, deep=deep, extra=extra)\n\n def info(self, msg, *, time=None, deep=None, extra=None) -> datetime:\n return self.log(INFO, msg, time=time, deep=deep, extra=extra)\n\n def hint(self, msg, *, time=None, deep=None, extra=None) -> datetime:\n return self.log(HINT, msg, time=time, deep=deep, extra=extra)\n\n def debug(self, msg, *, time=None, deep=None, extra=None) -> datetime:\n return self.log(DEBUG, msg, time=time, deep=deep, extra=extra)\n\n\ndef _set_log_file(settings: ScanpyConfig):\n file = settings.logfile\n name = settings.logpath\n root = settings._root_logger\n h = logging.StreamHandler(file) if name is None else logging.FileHandler(name)\n h.setFormatter(_LogFormatter())\n h.setLevel(root.level)\n for handler in list(root.handlers):\n root.removeHandler(handler)\n root.addHandler(h)\n\n\ndef _set_log_level(settings: ScanpyConfig, level: int):\n root = settings._root_logger\n root.setLevel(level)\n for h in list(root.handlers):\n h.setLevel(level)\n\n\nclass _LogFormatter(logging.Formatter):\n def __init__(\n self, fmt=\"{levelname}: {message}\", datefmt=\"%Y-%m-%d %H:%M\", style=\"{\"\n ):\n super().__init__(fmt, datefmt, style)\n\n def format(self, record: logging.LogRecord):\n format_orig = self._style._fmt\n if record.levelno == INFO:\n self._style._fmt = \"{message}\"\n elif record.levelno == HINT:\n self._style._fmt = \"--> {message}\"\n elif record.levelno == DEBUG:\n self._style._fmt = \" {message}\"\n if record.time_passed:\n # strip microseconds\n if record.time_passed.microseconds:\n record.time_passed = timedelta(\n seconds=int(record.time_passed.total_seconds())\n )\n if \"{time_passed}\" in record.msg:\n record.msg = record.msg.replace(\n \"{time_passed}\", str(record.time_passed)\n )\n else:\n self._style._fmt += \" ({time_passed})\"\n if record.deep:\n record.msg = f\"{record.msg}: {record.deep}\"\n result = logging.Formatter.format(self, record)\n self._style._fmt = format_orig\n return result\n\n\nprint_memory_usage = anndata.logging.print_memory_usage\nget_memory_usage = anndata.logging.get_memory_usage\n\n\n_DEPENDENCIES_NUMERICS = [\n \"anndata\", # anndata actually shouldn't, but as long as it's in development\n \"umap\",\n \"numpy\",\n \"scipy\",\n \"pandas\",\n (\"sklearn\", \"scikit-learn\"),\n \"statsmodels\",\n \"igraph\",\n \"louvain\",\n \"leidenalg\",\n \"pynndescent\",\n]\n\n\ndef _versions_dependencies(dependencies):\n # this is not the same as the requirements!\n for mod in dependencies:\n mod_name, dist_name = mod if isinstance(mod, tuple) else (mod, mod)\n try:\n imp = __import__(mod_name)\n yield dist_name, imp.__version__\n except (ImportError, AttributeError):\n pass\n\n\ndef print_header(*, file=None):\n \"\"\"\\\n Versions that might influence the numerical results.\n Matplotlib and Seaborn are excluded from this.\n\n Parameters\n ----------\n file\n Optional path for dependency output.\n \"\"\"\n\n modules = [\"scanpy\"] + _DEPENDENCIES_NUMERICS\n print(\n \" \".join(f\"{mod}=={ver}\" for mod, ver in _versions_dependencies(modules)),\n file=file or sys.stdout,\n )\n\n\ndef print_versions(*, file: IO[str] | None = None):\n \"\"\"\\\n Print versions of imported packages, OS, and jupyter environment.\n\n For more options (including rich output) use `session_info.show` directly.\n\n Parameters\n ----------\n file\n Optional path for output.\n \"\"\"\n import session_info\n\n if file is not None:\n from contextlib import redirect_stdout\n\n warnings.warn(\n \"Passing argument 'file' to print_versions is deprecated, and will be \"\n \"removed in a future version.\",\n FutureWarning,\n )\n with redirect_stdout(file):\n print_versions()\n else:\n session_info.show(\n dependencies=True,\n html=False,\n excludes=[\n \"builtins\",\n \"stdlib_list\",\n \"importlib_metadata\",\n # Special module present if test coverage being calculated\n # https://gitlab.com/joelostblom/session_info/-/issues/10\n \"$coverage\",\n ],\n )\n\n\ndef print_version_and_date(*, file=None):\n \"\"\"\\\n Useful for starting a notebook so you see when you started working.\n\n Parameters\n ----------\n file\n Optional path for output.\n \"\"\"\n from . import __version__\n\n if file is None:\n file = sys.stdout\n print(\n f\"Running Scanpy {__version__}, \" f\"on {datetime.now():%Y-%m-%d %H:%M}.\",\n file=file,\n )\n\n\ndef _copy_docs_and_signature(fn):\n return partial(update_wrapper, wrapped=fn, assigned=[\"__doc__\", \"__annotations__\"])\n\n\ndef error(\n msg: str,\n *,\n time: datetime = None,\n deep: str | None = None,\n extra: dict | None = None,\n) -> datetime:\n \"\"\"\\\n Log message with specific level and return current time.\n\n Parameters\n ----------\n msg\n Message to display.\n time\n A time in the past. If this is passed, the time difference from then\n to now is appended to `msg` as ` (HH:MM:SS)`.\n If `msg` contains `{time_passed}`, the time difference is instead\n inserted at that position.\n deep\n If the current verbosity is higher than the log function’s level,\n this gets displayed as well\n extra\n Additional values you can specify in `msg` like `{time_passed}`.\n \"\"\"\n from ._settings import settings\n\n return settings._root_logger.error(msg, time=time, deep=deep, extra=extra)\n\n\n@_copy_docs_and_signature(error)\ndef warning(msg, *, time=None, deep=None, extra=None) -> datetime:\n from ._settings import settings\n\n return settings._root_logger.warning(msg, time=time, deep=deep, extra=extra)\n\n\n@_copy_docs_and_signature(error)\ndef info(msg, *, time=None, deep=None, extra=None) -> datetime:\n from ._settings import settings\n\n return settings._root_logger.info(msg, time=time, deep=deep, extra=extra)\n\n\n@_copy_docs_and_signature(error)\ndef hint(msg, *, time=None, deep=None, extra=None) -> datetime:\n from ._settings import settings\n\n return settings._root_logger.hint(msg, time=time, deep=deep, extra=extra)\n\n\n@_copy_docs_and_signature(error)\ndef debug(msg, *, time=None, deep=None, extra=None) -> datetime:\n from ._settings import settings\n\n return settings._root_logger.debug(msg, time=time, deep=deep, extra=extra)\n\n\nfrom __future__ import annotations\n\nimport os\nimport sys\nfrom argparse import ArgumentParser, Namespace, _SubParsersAction\nfrom collections.abc import MutableMapping\nfrom functools import lru_cache, partial\nfrom pathlib import Path\nfrom shutil import which\nfrom subprocess import run\nfrom typing import TYPE_CHECKING\n\nif TYPE_CHECKING:\n from collections.abc import Generator, Mapping, Sequence\n from subprocess import CompletedProcess\n from typing import Any\n\n\nclass _DelegatingSubparsersAction(_SubParsersAction):\n \"\"\"Like a normal subcommand action, but uses a delegator for more choices\"\"\"\n\n def __init__(self, *args, _command: str, _runargs: dict[str, Any], **kwargs):\n super().__init__(*args, **kwargs)\n self.command = _command\n self._name_parser_map = self.choices = _CommandDelegator(\n _command, self, **_runargs\n )\n\n\nclass _CommandDelegator(MutableMapping):\n \"\"\"\\\n Provide the ability to delegate,\n but don’t calculate the whole list until necessary\n \"\"\"\n\n def __init__(self, command: str, action: _DelegatingSubparsersAction, **runargs):\n self.command = command\n self.action = action\n self.parser_map = {}\n self.runargs = runargs\n\n def __contains__(self, k: str) -> bool:\n if k in self.parser_map:\n return True\n try:\n self[k]\n except KeyError:\n return False\n return True\n\n def __getitem__(self, k: str) -> ArgumentParser:\n try:\n return self.parser_map[k]\n except KeyError:\n if which(f\"{self.command}-{k}\"):\n return _DelegatingParser(self, k)\n raise\n\n def __setitem__(self, k: str, v: ArgumentParser) -> None:\n self.parser_map[k] = v\n\n def __delitem__(self, k: str) -> None:\n del self.parser_map[k]\n\n # These methods retrieve the command list or help with doing it\n\n def __iter__(self) -> Generator[str, None, None]:\n yield from self.parser_map\n yield from self.commands\n\n def __len__(self) -> int:\n return len(self.parser_map) + len(self.commands)\n\n def __hash__(self) -> int:\n return hash(self.command)\n\n def __eq__(self, other: Mapping[str, ArgumentParser]):\n if isinstance(other, _CommandDelegator):\n return all(\n getattr(self, attr) == getattr(other, attr)\n for attr in [\"command\", \"action\", \"parser_map\", \"runargs\"]\n )\n return self.parser_map == other\n\n @property\n @lru_cache\n def commands(self) -> frozenset[str]:\n return frozenset(\n binary.name[len(self.command) + 1 :]\n for bin_dir in os.environ[\"PATH\"].split(os.pathsep)\n for binary in Path(bin_dir).glob(f\"{self.command}-*\")\n if os.access(binary, os.X_OK)\n )\n\n\nclass _DelegatingParser(ArgumentParser):\n \"\"\"Just sets parse_args().func to run the subcommand\"\"\"\n\n def __init__(self, cd: _CommandDelegator, subcmd: str):\n super().__init__(f\"{cd.command}-{subcmd}\", add_help=False)\n self.cd = cd\n self.subcmd = subcmd\n\n def parse_known_args(\n self,\n args: Sequence[str] | None = None,\n namespace: Namespace | None = None,\n ) -> tuple[Namespace, list[str]]:\n msg = \"Only use DelegatingParser as subparser\"\n assert args is not None, msg\n assert namespace is None, msg\n return Namespace(func=partial(run, [self.prog, *args], **self.cd.runargs)), []\n\n\ndef _cmd_settings() -> None:\n from ._settings import settings\n\n print(settings)\n\n\ndef main(\n argv: Sequence[str] | None = None, *, check: bool = True, **runargs\n) -> CompletedProcess | None:\n \"\"\"\\\n Run a builtin scanpy command or a scanpy-* subcommand.\n\n Uses :func:`subcommand.run` for the latter:\n `~run(['scanpy', *argv], **runargs)`\n \"\"\"\n parser = ArgumentParser(\n description=(\n \"There are a few packages providing commands. \"\n \"Try e.g. `pip install scanpy-scripts`!\"\n )\n )\n parser.set_defaults(func=parser.print_help)\n\n subparsers: _DelegatingSubparsersAction = parser.add_subparsers(\n action=_DelegatingSubparsersAction,\n _command=\"scanpy\",\n _runargs={**runargs, \"check\": check},\n )\n\n parser_settings = subparsers.add_parser(\"settings\")\n parser_settings.set_defaults(func=_cmd_settings)\n\n args = parser.parse_args(argv)\n return args.func()\n\n\ndef console_main():\n \"\"\"\\\n This serves as CLI entry point and will not show a Python traceback\n if a called command fails\n \"\"\"\n cmd = main(check=False)\n if cmd is not None:\n sys.exit(cmd.returncode)\n\n\n\"\"\"Single-Cell Analysis in Python.\"\"\"\n\nfrom __future__ import annotations\n\nimport sys\n\ntry: # See https://github.com/maresb/hatch-vcs-footgun-example\n from setuptools_scm import get_version\n\n __version__ = get_version(root=\"../..\", relative_to=__file__)\n del get_version\nexcept (ImportError, LookupError):\n try:\n from ._version import __version__\n except ModuleNotFoundError:\n raise RuntimeError(\n \"scanpy is not correctly installed. Please install it, e.g. with pip.\"\n )\n\nfrom ._utils import check_versions\n\ncheck_versions()\ndel check_versions\n\n# the actual API\n# (start with settings as several tools are using it)\n\nfrom ._settings import Verbosity, settings\n\nset_figure_params = settings.set_figure_params\n\nfrom anndata import (\n AnnData,\n concat,\n read_csv,\n read_excel,\n read_h5ad,\n read_hdf,\n read_loom,\n read_mtx,\n read_text,\n read_umi_tools,\n)\n\nfrom . import datasets, experimental, external, get, logging, metrics, queries\nfrom . import plotting as pl\nfrom . import preprocessing as pp\nfrom . import tools as tl\nfrom .neighbors import Neighbors\nfrom .readwrite import read, read_10x_h5, read_10x_mtx, read_visium, write\n\n# has to be done at the end, after everything has been imported\nsys.modules.update({f\"{__name__}.{m}\": globals()[m] for m in [\"tl\", \"pp\", \"pl\"]})\nfrom ._utils import annotate_doc_types\n\nannotate_doc_types(sys.modules[__name__], \"scanpy\")\ndel sys, annotate_doc_types\n\n__all__ = [\n \"__version__\",\n \"AnnData\",\n \"concat\",\n \"read_csv\",\n \"read_excel\",\n \"read_h5ad\",\n \"read_hdf\",\n \"read_loom\",\n \"read_mtx\",\n \"read_text\",\n \"read_umi_tools\",\n \"read\",\n \"read_10x_h5\",\n \"read_10x_mtx\",\n \"read_visium\",\n \"write\",\n \"datasets\",\n \"experimental\",\n \"external\",\n \"get\",\n \"logging\",\n \"metrics\",\n \"queries\",\n \"pl\",\n \"pp\",\n \"tl\",\n \"Verbosity\",\n \"settings\",\n \"Neighbors\",\n \"set_figure_params\",\n]\n\n\nfrom __future__ import annotations\n\nimport sys\nfrom dataclasses import dataclass, field\nfrom functools import cache, partial\nfrom importlib.util import find_spec\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nfrom packaging.version import Version\n\nif TYPE_CHECKING:\n from importlib.metadata import PackageMetadata\n\n\nif TYPE_CHECKING:\n # type checkers are confused and can only see …core.Array\n from dask.array.core import Array as DaskArray\nelif find_spec(\"dask\"):\n from dask.array import Array as DaskArray\nelse:\n\n class DaskArray:\n pass\n\n\nif find_spec(\"zappy\") or TYPE_CHECKING:\n from zappy.base import ZappyArray\nelse:\n\n class ZappyArray:\n pass\n\n\n__all__ = [\n \"DaskArray\",\n \"ZappyArray\",\n \"fullname\",\n \"pkg_metadata\",\n \"pkg_version\",\n]\n\n\ndef fullname(typ: type) -> str:\n module = typ.__module__\n name = typ.__qualname__\n if module == \"builtins\" or module is None:\n return name\n return f\"{module}.{name}\"\n\n\nif sys.version_info >= (3, 11):\n from contextlib import chdir\nelse:\n import os\n from contextlib import AbstractContextManager\n\n @dataclass\n class chdir(AbstractContextManager):\n path: Path\n _old_cwd: list[Path] = field(default_factory=list)\n\n def __enter__(self) -> None:\n self._old_cwd.append(Path.cwd())\n os.chdir(self.path)\n\n def __exit__(self, *_excinfo) -> None:\n os.chdir(self._old_cwd.pop())\n\n\ndef pkg_metadata(package: str) -> PackageMetadata:\n from importlib.metadata import metadata\n\n return metadata(package)\n\n\n@cache\ndef pkg_version(package: str) -> Version:\n from importlib.metadata import version\n\n return Version(version(package))\n\n\nif find_spec(\"legacy_api_wrap\") or TYPE_CHECKING:\n from legacy_api_wrap import legacy_api # noqa: TID251\n\n old_positionals = partial(legacy_api, category=FutureWarning)\nelse:\n # legacy_api_wrap is currently a hard dependency,\n # but this code makes it possible to run scanpy without it.\n def old_positionals(*old_positionals: str):\n return lambda func: func\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom typing import TypeVar\n\nimport numpy as np\nfrom numpy.typing import NDArray\nfrom scipy.sparse import coo_matrix, csr_matrix, issparse\n\nfrom ._common import (\n _get_indices_distances_from_dense_matrix,\n _get_indices_distances_from_sparse_matrix,\n)\n\nD = TypeVar(\"D\", NDArray[np.float32], csr_matrix)\n\n\ndef gauss(distances: D, n_neighbors: int, *, knn: bool) -> D:\n \"\"\"\n Derive gaussian connectivities between data points from their distances.\n\n Parameters\n ----------\n distances\n The input matrix of distances between data points.\n n_neighbors\n The number of nearest neighbors to consider.\n knn\n Specify if the distances have been restricted to k nearest neighbors.\n \"\"\"\n # init distances\n if isinstance(distances, csr_matrix):\n Dsq = distances.power(2)\n indices, distances_sq = _get_indices_distances_from_sparse_matrix(\n Dsq, n_neighbors\n )\n else:\n assert isinstance(distances, np.ndarray)\n Dsq = np.power(distances, 2)\n indices, distances_sq = _get_indices_distances_from_dense_matrix(\n Dsq, n_neighbors\n )\n\n # exclude the first point, the 0th neighbor\n indices = indices[:, 1:]\n distances_sq = distances_sq[:, 1:]\n\n # choose sigma, the heuristic here doesn't seem to make much of a difference,\n # but is used to reproduce the figures of Haghverdi et al. (2016)\n if issparse(distances):\n # as the distances are not sorted\n # we have decay within the n_neighbors first neighbors\n sigmas_sq = np.median(distances_sq, axis=1)\n else:\n # the last item is already in its sorted position through argpartition\n # we have decay beyond the n_neighbors neighbors\n sigmas_sq = distances_sq[:, -1] / 4\n sigmas = np.sqrt(sigmas_sq)\n\n # compute the symmetric weight matrix\n if not issparse(distances):\n Num = 2 * np.multiply.outer(sigmas, sigmas)\n Den = np.add.outer(sigmas_sq, sigmas_sq)\n W = np.sqrt(Num / Den) * np.exp(-Dsq / Den)\n # make the weight matrix sparse\n if not knn:\n mask = W > 1e-14\n W[~mask] = 0\n else:\n # restrict number of neighbors to ~k\n # build a symmetric mask\n mask = np.zeros(Dsq.shape, dtype=bool)\n for i, row in enumerate(indices):\n mask[i, row] = True\n for j in row:\n if i not in set(indices[j]):\n W[j, i] = W[i, j]\n mask[j, i] = True\n # set all entries that are not nearest neighbors to zero\n W[~mask] = 0\n else:\n assert isinstance(Dsq, csr_matrix)\n W = Dsq.copy() # need to copy the distance matrix here; what follows is inplace\n for i in range(len(Dsq.indptr[:-1])):\n row = Dsq.indices[Dsq.indptr[i] : Dsq.indptr[i + 1]]\n num = 2 * sigmas[i] * sigmas[row]\n den = sigmas_sq[i] + sigmas_sq[row]\n W.data[Dsq.indptr[i] : Dsq.indptr[i + 1]] = np.sqrt(num / den) * np.exp(\n -Dsq.data[Dsq.indptr[i] : Dsq.indptr[i + 1]] / den\n )\n W = W.tolil()\n for i, row in enumerate(indices):\n for j in row:\n if i not in set(indices[j]):\n W[j, i] = W[i, j]\n W = W.tocsr()\n\n return W\n\n\ndef umap(\n knn_indices: NDArray[np.int32 | np.int64],\n knn_dists: NDArray[np.float32 | np.float64],\n *,\n n_obs: int,\n n_neighbors: int,\n set_op_mix_ratio: float = 1.0,\n local_connectivity: float = 1.0,\n) -> csr_matrix:\n \"\"\"\\\n This is from umap.fuzzy_simplicial_set :cite:p:`McInnes2018`.\n\n Given a set of data X, a neighborhood size, and a measure of distance\n compute the fuzzy simplicial set (here represented as a fuzzy graph in\n the form of a sparse matrix) associated to the data. This is done by\n locally approximating geodesic distance at each point, creating a fuzzy\n simplicial set for each such point, and then combining all the local\n fuzzy simplicial sets into a global one via a fuzzy union.\n \"\"\"\n with warnings.catch_warnings():\n # umap 0.5.0\n warnings.filterwarnings(\"ignore\", message=r\"Tensorflow not installed\")\n from umap.umap_ import fuzzy_simplicial_set\n\n X = coo_matrix(([], ([], [])), shape=(n_obs, 1))\n connectivities, _sigmas, _rhos = fuzzy_simplicial_set(\n X,\n n_neighbors,\n None,\n None,\n knn_indices=knn_indices,\n knn_dists=knn_dists,\n set_op_mix_ratio=set_op_mix_ratio,\n local_connectivity=local_connectivity,\n )\n\n return connectivities.tocsr()\n\n\nfrom __future__ import annotations\n\nfrom collections.abc import Callable\nfrom typing import TYPE_CHECKING, Literal, Protocol, Union\n\nimport numpy as np\n\nif TYPE_CHECKING:\n from typing import Any, Self\n\n from scipy.sparse import spmatrix\n\n\n# These two are used with get_args elsewhere\n_Method = Literal[\"umap\", \"gauss\"]\n_KnownTransformer = Literal[\"pynndescent\", \"sklearn\", \"rapids\"]\n\n# sphinx-autodoc-typehints can’t transitively import types from if TYPE_CHECKING blocks,\n# so these four needs to be importable\n\n_MetricFn = Callable[[np.ndarray, np.ndarray], float]\n# from sklearn.metrics.pairwise_distances.__doc__:\n_MetricSparseCapable = Literal[\n \"cityblock\", \"cosine\", \"euclidean\", \"l1\", \"l2\", \"manhattan\"\n]\n_MetricScipySpatial = Literal[\n \"braycurtis\",\n \"canberra\",\n \"chebyshev\",\n \"correlation\",\n \"dice\",\n \"hamming\",\n \"jaccard\",\n \"kulsinski\",\n \"mahalanobis\",\n \"minkowski\",\n \"rogerstanimoto\",\n \"russellrao\",\n \"seuclidean\",\n \"sokalmichener\",\n \"sokalsneath\",\n \"sqeuclidean\",\n \"yule\",\n]\n_Metric = Union[_MetricSparseCapable, _MetricScipySpatial]\n\n\nclass KnnTransformerLike(Protocol):\n \"\"\"See :class:`~sklearn.neighbors.KNeighborsTransformer`.\"\"\"\n\n def fit(self, X, y: None = None): ...\n def transform(self, X) -> spmatrix: ...\n\n # from TransformerMixin\n def fit_transform(self, X, y: None = None) -> spmatrix: ...\n\n # from BaseEstimator\n def get_params(self, *, deep: bool = True) -> dict[str, Any]: ...\n def set_params(self, **params: Any) -> Self: ...\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\nfrom warnings import warn\n\nimport numpy as np\nfrom scipy.sparse import csr_matrix\n\nfrom scanpy._utils.compute.is_constant import is_constant\n\nif TYPE_CHECKING:\n from numpy.typing import NDArray\n\n\ndef _has_self_column(\n indices: NDArray[np.int32 | np.int64],\n distances: NDArray[np.float32 | np.float64],\n) -> bool:\n # some algorithms have some messed up reordering.\n return (indices[:, 0] == np.arange(indices.shape[0])).any()\n\n\ndef _remove_self_column(\n indices: NDArray[np.int32 | np.int64],\n distances: NDArray[np.float32 | np.float64],\n) -> tuple[NDArray[np.int32 | np.int64], NDArray[np.float32 | np.float64]]:\n if not _has_self_column(indices, distances):\n msg = \"The first neighbor should be the cell itself.\"\n raise AssertionError(msg)\n return indices[:, 1:], distances[:, 1:]\n\n\ndef _get_sparse_matrix_from_indices_distances(\n indices: NDArray[np.int32 | np.int64],\n distances: NDArray[np.float32 | np.float64],\n *,\n keep_self: bool,\n) -> csr_matrix:\n \"\"\"\\\n Create a sparse matrix from a pair of indices and distances.\n\n If keep_self=False, it verifies that the first column is the cell itself,\n then removes it from the explicitly stored zeroes.\n\n Duplicates in the data are kept as explicitly stored zeroes.\n \"\"\"\n # instead of calling .eliminate_zeros() on our sparse matrix,\n # we manually handle the nearest neighbor being the cell itself.\n # This allows us to use _ind_dist_shortcut even when the data has duplicates.\n if not keep_self:\n indices, distances = _remove_self_column(indices, distances)\n indptr = np.arange(0, np.prod(indices.shape) + 1, indices.shape[1])\n return csr_matrix(\n (\n distances.copy().ravel(), # copy the data, otherwise strange behavior here\n indices.copy().ravel(),\n indptr,\n ),\n shape=(indices.shape[0],) * 2,\n )\n\n\ndef _get_indices_distances_from_dense_matrix(\n D: NDArray[np.float32 | np.float64], n_neighbors: int\n):\n sample_range = np.arange(D.shape[0])[:, None]\n indices = np.argpartition(D, n_neighbors - 1, axis=1)[:, :n_neighbors]\n indices = indices[sample_range, np.argsort(D[sample_range, indices])]\n distances = D[sample_range, indices]\n return indices, distances\n\n\ndef _get_indices_distances_from_sparse_matrix(\n D: csr_matrix, n_neighbors: int\n) -> tuple[NDArray[np.int32 | np.int64], NDArray[np.float32 | np.float64]]:\n \"\"\"\\\n Get indices and distances from a sparse matrix.\n\n Makes sure that for both of the returned matrices:\n 1. the first column corresponds to the cell itself as nearest neighbor.\n 2. the number of neighbors (`.shape[1]`) is restricted to `n_neighbors`.\n \"\"\"\n if (shortcut := _ind_dist_shortcut(D)) is not None:\n indices, distances = shortcut\n else:\n indices, distances = _ind_dist_slow(D, n_neighbors)\n\n # handle RAPIDS style indices_distances lacking the self-column\n if not _has_self_column(indices, distances):\n indices = np.hstack([np.arange(indices.shape[0])[:, None], indices])\n distances = np.hstack([np.zeros(distances.shape[0])[:, None], distances])\n\n # If using the shortcut or adding the self column resulted in too many neighbors,\n # restrict the output matrices to the correct size\n if indices.shape[1] > n_neighbors:\n indices, distances = indices[:, :n_neighbors], distances[:, :n_neighbors]\n\n return indices, distances\n\n\ndef _ind_dist_slow(\n D: csr_matrix, n_neighbors: int\n) -> tuple[NDArray[np.int32 | np.int64], NDArray[np.float32 | np.float64]]:\n indices = np.zeros((D.shape[0], n_neighbors), dtype=int)\n distances = np.zeros((D.shape[0], n_neighbors), dtype=D.dtype)\n n_neighbors_m1 = n_neighbors - 1\n for i in range(indices.shape[0]):\n neighbors = D[i].nonzero() # 'true' and 'spurious' zeros\n indices[i, 0] = i\n distances[i, 0] = 0\n # account for the fact that there might be more than n_neighbors\n # due to an approximate search\n # [the point itself was not detected as its own neighbor during the search]\n if len(neighbors[1]) > n_neighbors_m1:\n sorted_indices = np.argsort(D[i][neighbors].A1)[:n_neighbors_m1]\n indices[i, 1:] = neighbors[1][sorted_indices]\n distances[i, 1:] = D[i][\n neighbors[0][sorted_indices], neighbors[1][sorted_indices]\n ]\n else:\n indices[i, 1:] = neighbors[1]\n distances[i, 1:] = D[i][neighbors]\n return indices, distances\n\n\ndef _ind_dist_shortcut(\n D: csr_matrix,\n) -> tuple[NDArray[np.int32 | np.int64], NDArray[np.float32 | np.float64]] | None:\n \"\"\"Shortcut for scipy or RAPIDS style distance matrices.\"\"\"\n # Check if each row has the correct number of entries\n nnzs = D.getnnz(axis=1)\n if not is_constant(nnzs):\n msg = (\n \"Sparse matrix has no constant number of neighbors per row. \"\n \"Cannot efficiently get indices and distances.\"\n )\n warn(msg, category=RuntimeWarning)\n return None\n n_obs, n_neighbors = D.shape[0], int(nnzs[0])\n return (\n D.indices.reshape(n_obs, n_neighbors),\n D.data.reshape(n_obs, n_neighbors),\n )\n\n\nfrom __future__ import annotations\n\ndoc_use_rep = \"\"\"\\\nuse_rep\n Use the indicated representation. `'X'` or any key for `.obsm` is valid.\n If `None`, the representation is chosen automatically:\n For `.n_vars` < :attr:`~scanpy._settings.ScanpyConfig.N_PCS` (default: 50), `.X` is used, otherwise 'X_pca' is used.\n If 'X_pca' is not present, it’s computed with default parameters or `n_pcs` if present.\\\n\"\"\"\n\ndoc_n_pcs = \"\"\"\\\nn_pcs\n Use this many PCs. If `n_pcs==0` use `.X` if `use_rep is None`.\\\n\"\"\"\n\n\nfrom __future__ import annotations\n\nimport contextlib\nfrom collections.abc import Mapping\nfrom textwrap import indent\nfrom types import MappingProxyType\nfrom typing import TYPE_CHECKING, NamedTuple, TypedDict, get_args\nfrom warnings import warn\n\nimport numpy as np\nimport scipy\nfrom scipy.sparse import issparse\nfrom sklearn.utils import check_random_state\n\nfrom .. import _utils\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._settings import settings\nfrom .._utils import NeighborsView, _doc_params\nfrom . import _connectivity\nfrom ._common import (\n _get_indices_distances_from_sparse_matrix,\n _get_sparse_matrix_from_indices_distances,\n)\nfrom ._doc import doc_n_pcs, doc_use_rep\nfrom ._types import _KnownTransformer, _Method\n\nif TYPE_CHECKING:\n from collections.abc import Callable, MutableMapping\n from typing import Any, Literal, NotRequired\n\n from anndata import AnnData\n from igraph import Graph\n from scipy.sparse import csr_matrix\n\n from .._utils import AnyRandom\n from ._types import KnnTransformerLike, _Metric, _MetricFn\n\n\nRPForestDict = Mapping[str, Mapping[str, np.ndarray]]\n\nN_DCS = 15 # default number of diffusion components\n# Backwards compat, constants should be defined in only one place.\nN_PCS = settings.N_PCS\n\n\nclass KwdsForTransformer(TypedDict):\n \"\"\"Keyword arguments passed to a _KnownTransformer.\n\n IMPORTANT: when changing the parameters set here,\n update the “*ignored*” part in the parameter docs!\n \"\"\"\n\n n_neighbors: int\n metric: _Metric | _MetricFn\n metric_params: Mapping[str, Any]\n random_state: AnyRandom\n\n\nclass NeighborsParams(TypedDict):\n n_neighbors: int\n method: _Method\n random_state: AnyRandom\n metric: _Metric | _MetricFn\n metric_kwds: NotRequired[Mapping[str, Any]]\n use_rep: NotRequired[str]\n n_pcs: NotRequired[int]\n\n\n@_doc_params(n_pcs=doc_n_pcs, use_rep=doc_use_rep)\ndef neighbors(\n adata: AnnData,\n n_neighbors: int = 15,\n n_pcs: int | None = None,\n *,\n use_rep: str | None = None,\n knn: bool = True,\n method: _Method = \"umap\",\n transformer: KnnTransformerLike | _KnownTransformer | None = None,\n metric: _Metric | _MetricFn = \"euclidean\",\n metric_kwds: Mapping[str, Any] = MappingProxyType({}),\n random_state: AnyRandom = 0,\n key_added: str | None = None,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Computes the nearest neighbors distance matrix and a neighborhood graph of observations :cite:p:`McInnes2018`.\n\n The neighbor search efficiency of this heavily relies on UMAP :cite:p:`McInnes2018`,\n which also provides a method for estimating connectivities of data points -\n the connectivity of the manifold (`method=='umap'`). If `method=='gauss'`,\n connectivities are computed according to :cite:t:`Coifman2005`, in the adaption of\n :cite:t:`Haghverdi2016`.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n n_neighbors\n The size of local neighborhood (in terms of number of neighboring data\n points) used for manifold approximation. Larger values result in more\n global views of the manifold, while smaller values result in more local\n data being preserved. In general values should be in the range 2 to 100.\n If `knn` is `True`, number of nearest neighbors to be searched. If `knn`\n is `False`, a Gaussian kernel width is set to the distance of the\n `n_neighbors` neighbor.\n\n *ignored if ``transformer`` is an instance.*\n {n_pcs}\n {use_rep}\n knn\n If `True`, use a hard threshold to restrict the number of neighbors to\n `n_neighbors`, that is, consider a knn graph. Otherwise, use a Gaussian\n Kernel to assign low weights to neighbors more distant than the\n `n_neighbors` nearest neighbor.\n method\n Use 'umap' :cite:p:`McInnes2018` or 'gauss' (Gauss kernel following :cite:t:`Coifman2005`\n with adaptive width :cite:t:`Haghverdi2016`) for computing connectivities.\n transformer\n Approximate kNN search implementation following the API of\n :class:`~sklearn.neighbors.KNeighborsTransformer`.\n See :doc:`/how-to/knn-transformers` for more details.\n Also accepts the following known options:\n\n `None` (the default)\n Behavior depends on data size.\n For small data, we will calculate exact kNN, otherwise we use\n :class:`~pynndescent.pynndescent_.PyNNDescentTransformer`\n `'pynndescent'`\n :class:`~pynndescent.pynndescent_.PyNNDescentTransformer`\n `'rapids'`\n A transformer based on :class:`cuml.neighbors.NearestNeighbors`.\n\n .. deprecated:: 1.10.0\n Use :func:`rapids_singlecell.pp.neighbors` instead.\n metric\n A known metric’s name or a callable that returns a distance.\n\n *ignored if ``transformer`` is an instance.*\n metric_kwds\n Options for the metric.\n\n *ignored if ``transformer`` is an instance.*\n random_state\n A numpy random seed.\n\n *ignored if ``transformer`` is an instance.*\n key_added\n If not specified, the neighbors data is stored in `.uns['neighbors']`,\n distances and connectivities are stored in `.obsp['distances']` and\n `.obsp['connectivities']` respectively.\n If specified, the neighbors data is added to .uns[key_added],\n distances are stored in `.obsp[key_added+'_distances']` and\n connectivities in `.obsp[key_added+'_connectivities']`.\n copy\n Return a copy instead of writing to adata.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.obsp['distances' | key_added+'_distances']` : :class:`scipy.sparse.csr_matrix` (dtype `float`)\n Distance matrix of the nearest neighbors search. Each row (cell) has `n_neighbors`-1 non-zero entries. These are the distances to their `n_neighbors`-1 nearest neighbors (excluding the cell itself).\n `adata.obsp['connectivities' | key_added+'_connectivities']` : :class:`scipy.sparse._csr.csr_matrix` (dtype `float`)\n Weighted adjacency matrix of the neighborhood graph of data\n points. Weights should be interpreted as connectivities.\n `adata.uns['neighbors' | key_added]` : :class:`dict`\n neighbors parameters.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> # Basic usage\n >>> sc.pp.neighbors(adata, 20, metric='cosine')\n >>> # Provide your own transformer for more control and flexibility\n >>> from sklearn.neighbors import KNeighborsTransformer\n >>> transformer = KNeighborsTransformer(n_neighbors=10, metric='manhattan', algorithm='kd_tree')\n >>> sc.pp.neighbors(adata, transformer=transformer)\n >>> # now you can e.g. access the index: `transformer._tree`\n\n See also\n --------\n :doc:`/how-to/knn-transformers`\n \"\"\"\n start = logg.info(\"computing neighbors\")\n adata = adata.copy() if copy else adata\n if adata.is_view: # we shouldn't need this here...\n adata._init_as_actual(adata.copy())\n neighbors = Neighbors(adata)\n neighbors.compute_neighbors(\n n_neighbors,\n n_pcs=n_pcs,\n use_rep=use_rep,\n knn=knn,\n method=method,\n transformer=transformer,\n metric=metric,\n metric_kwds=metric_kwds,\n random_state=random_state,\n )\n\n if key_added is None:\n key_added = \"neighbors\"\n conns_key = \"connectivities\"\n dists_key = \"distances\"\n else:\n conns_key = key_added + \"_connectivities\"\n dists_key = key_added + \"_distances\"\n\n adata.uns[key_added] = {}\n\n neighbors_dict = adata.uns[key_added]\n\n neighbors_dict[\"connectivities_key\"] = conns_key\n neighbors_dict[\"distances_key\"] = dists_key\n\n neighbors_dict[\"params\"] = NeighborsParams(\n n_neighbors=neighbors.n_neighbors,\n method=method,\n random_state=random_state,\n metric=metric,\n )\n if metric_kwds:\n neighbors_dict[\"params\"][\"metric_kwds\"] = metric_kwds\n if use_rep is not None:\n neighbors_dict[\"params\"][\"use_rep\"] = use_rep\n if n_pcs is not None:\n neighbors_dict[\"params\"][\"n_pcs\"] = n_pcs\n\n adata.obsp[dists_key] = neighbors.distances\n adata.obsp[conns_key] = neighbors.connectivities\n\n if neighbors.rp_forest is not None:\n neighbors_dict[\"rp_forest\"] = neighbors.rp_forest\n logg.info(\n \" finished\",\n time=start,\n deep=(\n f\"added to `.uns[{key_added!r}]`\\n\"\n f\" `.obsp[{dists_key!r}]`, distances for each pair of neighbors\\n\"\n f\" `.obsp[{conns_key!r}]`, weighted adjacency matrix\"\n ),\n )\n return adata if copy else None\n\n\nclass FlatTree(NamedTuple):\n hyperplanes: None\n offsets: None\n children: None\n indices: None\n\n\ndef _backwards_compat_get_full_X_diffmap(adata: AnnData) -> np.ndarray:\n if \"X_diffmap0\" in adata.obs:\n return np.c_[adata.obs[\"X_diffmap0\"].values[:, None], adata.obsm[\"X_diffmap\"]]\n else:\n return adata.obsm[\"X_diffmap\"]\n\n\ndef _backwards_compat_get_full_eval(adata: AnnData):\n if \"X_diffmap0\" in adata.obs:\n return np.r_[1, adata.uns[\"diffmap_evals\"]]\n else:\n return adata.uns[\"diffmap_evals\"]\n\n\ndef _make_forest_dict(forest):\n d = {}\n props = (\"hyperplanes\", \"offsets\", \"children\", \"indices\")\n for prop in props:\n d[prop] = {}\n sizes = np.fromiter(\n (getattr(tree, prop).shape[0] for tree in forest), dtype=int\n )\n d[prop][\"start\"] = np.zeros_like(sizes)\n if prop == \"offsets\":\n dims = sizes.sum()\n else:\n dims = (sizes.sum(), getattr(forest[0], prop).shape[1])\n dtype = getattr(forest[0], prop).dtype\n dat = np.empty(dims, dtype=dtype)\n start = 0\n for i, size in enumerate(sizes):\n d[prop][\"start\"][i] = start\n end = start + size\n dat[start:end] = getattr(forest[i], prop)\n start = end\n d[prop][\"data\"] = dat\n return d\n\n\nclass OnFlySymMatrix:\n \"\"\"Emulate a matrix where elements are calculated on the fly.\"\"\"\n\n def __init__(\n self,\n get_row: Callable[[Any], np.ndarray],\n shape: tuple[int, int],\n *,\n DC_start: int = 0,\n DC_end: int = -1,\n rows: MutableMapping[Any, np.ndarray] | None = None,\n restrict_array: np.ndarray | None = None,\n ):\n self.get_row = get_row\n self.shape = shape\n self.DC_start = DC_start\n self.DC_end = DC_end\n self.rows = {} if rows is None else rows\n self.restrict_array = restrict_array # restrict the array to a subset\n\n def __getitem__(self, index):\n if isinstance(index, (int, np.integer)):\n if self.restrict_array is None:\n glob_index = index\n else:\n # map the index back to the global index\n glob_index = self.restrict_array[index]\n if glob_index not in self.rows:\n self.rows[glob_index] = self.get_row(glob_index)\n row = self.rows[glob_index]\n if self.restrict_array is None:\n return row\n else:\n return row[self.restrict_array]\n else:\n if self.restrict_array is None:\n glob_index_0, glob_index_1 = index\n else:\n glob_index_0 = self.restrict_array[index[0]]\n glob_index_1 = self.restrict_array[index[1]]\n if glob_index_0 not in self.rows:\n self.rows[glob_index_0] = self.get_row(glob_index_0)\n return self.rows[glob_index_0][glob_index_1]\n\n def restrict(self, index_array):\n \"\"\"Generate a view restricted to a subset of indices.\"\"\"\n new_shape = index_array.shape[0], index_array.shape[0]\n return OnFlySymMatrix(\n self.get_row,\n new_shape,\n DC_start=self.DC_start,\n DC_end=self.DC_end,\n rows=self.rows,\n restrict_array=index_array,\n )\n\n\nclass Neighbors:\n \"\"\"\\\n Data represented as graph of nearest neighbors.\n\n Represent a data matrix as a graph of nearest neighbor relations (edges)\n among data points (nodes).\n\n Parameters\n ----------\n adata\n Annotated data object.\n n_dcs\n Number of diffusion components to use.\n neighbors_key\n Where to look in `.uns` and `.obsp` for neighbors data\n \"\"\"\n\n @old_positionals(\"n_dcs\", \"neighbors_key\")\n def __init__(\n self,\n adata: AnnData,\n *,\n n_dcs: int | None = None,\n neighbors_key: str | None = None,\n ):\n self._adata = adata\n self._init_iroot()\n # use the graph in adata\n info_str = \"\"\n self.knn: bool | None = None\n self._distances: np.ndarray | csr_matrix | None = None\n self._connectivities: np.ndarray | csr_matrix | None = None\n self._transitions_sym: np.ndarray | csr_matrix | None = None\n self._number_connected_components: int | None = None\n self._rp_forest: RPForestDict | None = None\n if neighbors_key is None:\n neighbors_key = \"neighbors\"\n if neighbors_key in adata.uns:\n neighbors = NeighborsView(adata, neighbors_key)\n if \"distances\" in neighbors:\n self.knn = issparse(neighbors[\"distances\"])\n self._distances = neighbors[\"distances\"]\n if \"connectivities\" in neighbors:\n self.knn = issparse(neighbors[\"connectivities\"])\n self._connectivities = neighbors[\"connectivities\"]\n if \"rp_forest\" in neighbors:\n self._rp_forest = neighbors[\"rp_forest\"]\n if \"params\" in neighbors:\n self.n_neighbors = neighbors[\"params\"][\"n_neighbors\"]\n else:\n\n def count_nonzero(a: np.ndarray | csr_matrix) -> int:\n return a.count_nonzero() if issparse(a) else np.count_nonzero(a)\n\n # estimating n_neighbors\n if self._connectivities is None:\n self.n_neighbors = int(\n count_nonzero(self._distances) / self._distances.shape[0]\n )\n else:\n self.n_neighbors = int(\n count_nonzero(self._connectivities)\n / self._connectivities.shape[0]\n / 2\n )\n info_str += \"`.distances` `.connectivities` \"\n self._number_connected_components = 1\n if issparse(self._connectivities):\n from scipy.sparse.csgraph import connected_components\n\n self._connected_components = connected_components(self._connectivities)\n self._number_connected_components = self._connected_components[0]\n if \"X_diffmap\" in adata.obsm_keys():\n self._eigen_values = _backwards_compat_get_full_eval(adata)\n self._eigen_basis = _backwards_compat_get_full_X_diffmap(adata)\n if n_dcs is not None:\n if n_dcs > len(self._eigen_values):\n raise ValueError(\n f\"Cannot instantiate using `n_dcs`={n_dcs}. \"\n \"Compute diffmap/spectrum with more components first.\"\n )\n self._eigen_values = self._eigen_values[:n_dcs]\n self._eigen_basis = self._eigen_basis[:, :n_dcs]\n self.n_dcs = len(self._eigen_values)\n info_str += \"`.eigen_values` `.eigen_basis` `.distances_dpt`\"\n else:\n self._eigen_values = None\n self._eigen_basis = None\n self.n_dcs = None\n if info_str != \"\":\n logg.debug(f\" initialized {info_str}\")\n\n @property\n def rp_forest(self) -> RPForestDict | None:\n return self._rp_forest\n\n @property\n def distances(self) -> np.ndarray | csr_matrix | None:\n \"\"\"Distances between data points (sparse matrix).\"\"\"\n return self._distances\n\n @property\n def connectivities(self) -> np.ndarray | csr_matrix | None:\n \"\"\"Connectivities between data points (sparse matrix).\"\"\"\n return self._connectivities\n\n @property\n def transitions(self) -> np.ndarray | csr_matrix:\n \"\"\"Transition matrix (sparse matrix).\n\n Is conjugate to the symmetrized transition matrix via::\n\n self.transitions = self.Z * self.transitions_sym / self.Z\n\n where ``self.Z`` is the diagonal matrix storing the normalization of the\n underlying kernel matrix.\n\n Notes\n -----\n This has not been tested, in contrast to `transitions_sym`.\n \"\"\"\n Zinv = self.Z.power(-1) if issparse(self.Z) else np.diag(1.0 / np.diag(self.Z))\n return self.Z @ self.transitions_sym @ Zinv\n\n @property\n def transitions_sym(self) -> np.ndarray | csr_matrix | None:\n \"\"\"Symmetrized transition matrix (sparse matrix).\n\n Is conjugate to the transition matrix via::\n\n self.transitions_sym = self.Z / self.transitions * self.Z\n\n where ``self.Z`` is the diagonal matrix storing the normalization of the\n underlying kernel matrix.\n \"\"\"\n return self._transitions_sym\n\n @property\n def eigen_values(self) -> np.ndarray:\n \"\"\"Eigen values of transition matrix.\"\"\"\n return self._eigen_values\n\n @property\n def eigen_basis(self) -> np.ndarray:\n \"\"\"Eigen basis of transition matrix.\"\"\"\n return self._eigen_basis\n\n @property\n def distances_dpt(self) -> OnFlySymMatrix:\n \"\"\"DPT distances.\n\n This is yields :cite:p:`Haghverdi2016`, Eq. 15 from the supplement with the\n extensions of :cite:p:`Wolf2019`, supplement on random-walk based distance\n measures.\n \"\"\"\n return OnFlySymMatrix(self._get_dpt_row, shape=self._adata.shape)\n\n def to_igraph(self) -> Graph:\n \"\"\"Generate igraph from connectiviies.\"\"\"\n return _utils.get_igraph_from_adjacency(self.connectivities)\n\n @_doc_params(n_pcs=doc_n_pcs, use_rep=doc_use_rep)\n def compute_neighbors(\n self,\n n_neighbors: int = 30,\n n_pcs: int | None = None,\n *,\n use_rep: str | None = None,\n knn: bool = True,\n method: _Method | None = \"umap\",\n transformer: KnnTransformerLike | _KnownTransformer | None = None,\n metric: _Metric | _MetricFn = \"euclidean\",\n metric_kwds: Mapping[str, Any] = MappingProxyType({}),\n random_state: AnyRandom = 0,\n ) -> None:\n \"\"\"\\\n Compute distances and connectivities of neighbors.\n\n Parameters\n ----------\n n_neighbors\n Use this number of nearest neighbors.\n {n_pcs}\n {use_rep}\n knn\n Restrict result to `n_neighbors` nearest neighbors.\n method\n See :func:`scanpy.pp.neighbors`.\n If `None`, skip calculating connectivities.\n\n Returns\n -------\n Writes sparse graph attributes `.distances` and,\n if `method` is not `None`, `.connectivities`.\n \"\"\"\n from ..tools._utils import _choose_representation\n\n start_neighbors = logg.debug(\"computing neighbors\")\n if transformer is not None and not isinstance(transformer, str):\n n_neighbors = transformer.get_params()[\"n_neighbors\"]\n elif n_neighbors > self._adata.shape[0]: # very small datasets\n n_neighbors = 1 + int(0.5 * self._adata.shape[0])\n logg.warning(f\"n_obs too small: adjusting to `n_neighbors = {n_neighbors}`\")\n\n # default keyword arguments when `transformer` is not an instance\n transformer_kwds_default = KwdsForTransformer(\n n_neighbors=n_neighbors,\n metric=metric,\n metric_params=metric_kwds, # most use _params, not _kwds\n random_state=random_state,\n )\n method, transformer, shortcut = self._handle_transformer(\n method, transformer, knn=knn, kwds=transformer_kwds_default\n )\n\n if self._adata.shape[0] >= 10000 and not knn:\n logg.warning(\"Using high n_obs without `knn=True` takes a lot of memory...\")\n # do not use the cached rp_forest\n self._rp_forest = None\n self.n_neighbors = n_neighbors\n self.knn = knn\n X = _choose_representation(self._adata, use_rep=use_rep, n_pcs=n_pcs)\n self._distances = transformer.fit_transform(X)\n knn_indices, knn_distances = _get_indices_distances_from_sparse_matrix(\n self._distances, n_neighbors\n )\n if shortcut:\n # self._distances is a sparse matrix with a diag of 1, fix that\n self._distances[np.diag_indices_from(self.distances)] = 0\n if knn: # remove too far away entries in self._distances\n self._distances = _get_sparse_matrix_from_indices_distances(\n knn_indices, knn_distances, keep_self=False\n )\n else: # convert to dense\n self._distances = self._distances.toarray()\n if index := getattr(transformer, \"index_\", None):\n from pynndescent import NNDescent\n\n if isinstance(index, NNDescent):\n # very cautious here\n # TODO catch the correct exception\n with contextlib.suppress(Exception):\n self._rp_forest = _make_forest_dict(index)\n start_connect = logg.debug(\"computed neighbors\", time=start_neighbors)\n\n if method == \"umap\":\n self._connectivities = _connectivity.umap(\n knn_indices,\n knn_distances,\n n_obs=self._adata.shape[0],\n n_neighbors=self.n_neighbors,\n )\n elif method == \"gauss\":\n self._connectivities = _connectivity.gauss(\n self._distances, self.n_neighbors, knn=self.knn\n )\n elif method is not None:\n msg = f\"{method!r} should have been coerced in _handle_transform_args\"\n raise AssertionError(msg)\n self._number_connected_components = 1\n if issparse(self._connectivities):\n from scipy.sparse.csgraph import connected_components\n\n self._connected_components = connected_components(self._connectivities)\n self._number_connected_components = self._connected_components[0]\n if method is not None:\n logg.debug(\"computed connectivities\", time=start_connect)\n\n def _handle_transformer(\n self,\n method: _Method | Literal[\"gauss\"] | None,\n transformer: KnnTransformerLike | _KnownTransformer | None,\n *,\n knn: bool,\n kwds: KwdsForTransformer,\n ) -> tuple[_Method | None, KnnTransformerLike, bool]:\n \"\"\"Return effective `method` and transformer.\n\n `method` will be coerced to `'gauss'` or `'umap'`.\n `transformer` is coerced from a str or instance to an instance class.\n\n If `transformer` is `None` and there are few data points,\n `transformer` will be set to a brute force\n :class:`~sklearn.neighbors.KNeighborsTransformer`.\n\n If `transformer` is `None` and there are many data points,\n `transformer` will be set like `umap` does (i.e. to a\n ~`pynndescent.PyNNDescentTransformer` with custom `n_trees` and `n_iter`).\n \"\"\"\n # legacy logic\n use_dense_distances = (\n kwds[\"metric\"] == \"euclidean\" and self._adata.n_obs < 8192\n ) or not knn\n shortcut = transformer == \"sklearn\" or (\n transformer is None and (use_dense_distances or self._adata.n_obs < 4096)\n )\n\n # Coerce `method` to 'gauss' or 'umap'\n if method == \"rapids\":\n if transformer is not None:\n msg = \"Can’t specify both `method = 'rapids'` and `transformer`.\"\n raise ValueError(msg)\n method = \"umap\"\n transformer = \"rapids\"\n elif method not in (methods := set(get_args(_Method))) and method is not None:\n msg = f\"`method` needs to be one of {methods}.\"\n raise ValueError(msg)\n\n # Validate `knn`\n conn_method = method if method in {\"gauss\", None} else \"umap\"\n if not knn and not (conn_method == \"gauss\" and transformer is None):\n # “knn=False” seems to be only intended for method “gauss”\n msg = f\"`method = {method!r} only with `knn = True`.\"\n raise ValueError(msg)\n\n # Coerce `transformer` to an instance\n if shortcut:\n from sklearn.neighbors import KNeighborsTransformer\n\n assert transformer in {None, \"sklearn\"}\n n_neighbors = self._adata.n_obs - 1\n if knn: # only obey n_neighbors arg if knn set\n n_neighbors = min(n_neighbors, kwds[\"n_neighbors\"])\n transformer = KNeighborsTransformer(\n algorithm=\"brute\",\n n_jobs=settings.n_jobs,\n n_neighbors=n_neighbors,\n metric=kwds[\"metric\"],\n metric_params=dict(kwds[\"metric_params\"]), # needs dict\n # no random_state\n )\n elif transformer is None or transformer == \"pynndescent\":\n from pynndescent import PyNNDescentTransformer\n\n kwds = kwds.copy()\n kwds[\"metric_kwds\"] = kwds.pop(\"metric_params\")\n if transformer is None:\n # Use defaults from UMAP’s `nearest_neighbors` function\n kwds.update(\n n_jobs=settings.n_jobs,\n n_trees=min(64, 5 + int(round((self._adata.n_obs) ** 0.5 / 20.0))),\n n_iters=max(5, int(round(np.log2(self._adata.n_obs)))),\n )\n transformer = PyNNDescentTransformer(**kwds)\n elif transformer == \"rapids\":\n msg = (\n \"`transformer='rapids'` is deprecated. \"\n \"Use `rapids_singlecell.tl.neighbors` instead.\"\n )\n warn(msg, FutureWarning)\n from scanpy.neighbors._backends.rapids import RapidsKNNTransformer\n\n transformer = RapidsKNNTransformer(**kwds)\n elif isinstance(transformer, str):\n msg = (\n f\"Unknown transformer: {transformer}. \"\n f\"Try passing a class or one of {set(get_args(_KnownTransformer))}\"\n )\n raise ValueError(msg)\n # else `transformer` is probably an instance\n return conn_method, transformer, shortcut\n\n @old_positionals(\"density_normalize\")\n def compute_transitions(self, *, density_normalize: bool = True):\n \"\"\"\\\n Compute transition matrix.\n\n Parameters\n ----------\n density_normalize\n The density rescaling of Coifman and Lafon (2006): Then only the\n geometry of the data matters, not the sampled density.\n\n Returns\n -------\n Makes attributes `.transitions_sym` and `.transitions` available.\n \"\"\"\n start = logg.info(\"computing transitions\")\n W = self._connectivities\n # density normalization as of Coifman et al. (2005)\n # ensures that kernel matrix is independent of sampling density\n if density_normalize:\n # q[i] is an estimate for the sampling density at point i\n # it's also the degree of the underlying graph\n q = np.asarray(W.sum(axis=0))\n if not issparse(W):\n Q = np.diag(1.0 / q)\n else:\n Q = scipy.sparse.spdiags(1.0 / q, 0, W.shape[0], W.shape[0])\n K = Q @ W @ Q\n else:\n K = W\n\n # z[i] is the square root of the row sum of K\n z = np.sqrt(np.asarray(K.sum(axis=0)))\n if not issparse(K):\n self.Z = np.diag(1.0 / z)\n else:\n self.Z = scipy.sparse.spdiags(1.0 / z, 0, K.shape[0], K.shape[0])\n self._transitions_sym = self.Z @ K @ self.Z\n logg.info(\" finished\", time=start)\n\n def compute_eigen(\n self,\n n_comps: int = 15,\n sym: bool | None = None,\n sort: Literal[\"decrease\", \"increase\"] = \"decrease\",\n random_state: AnyRandom = 0,\n ):\n \"\"\"\\\n Compute eigen decomposition of transition matrix.\n\n Parameters\n ----------\n n_comps\n Number of eigenvalues/vectors to be computed, set `n_comps = 0` if\n you need all eigenvectors.\n sym\n Instead of computing the eigendecomposition of the assymetric\n transition matrix, computed the eigendecomposition of the symmetric\n Ktilde matrix.\n random_state\n A numpy random seed\n\n Returns\n -------\n Writes the following attributes.\n\n eigen_values : :class:`~numpy.ndarray`\n Eigenvalues of transition matrix.\n eigen_basis : :class:`~numpy.ndarray`\n Matrix of eigenvectors (stored in columns). `.eigen_basis` is\n projection of data matrix on right eigenvectors, that is, the\n projection on the diffusion components. these are simply the\n components of the right eigenvectors and can directly be used for\n plotting.\n \"\"\"\n np.set_printoptions(precision=10)\n if self._transitions_sym is None:\n raise ValueError(\"Run `.compute_transitions` first.\")\n matrix = self._transitions_sym\n # compute the spectrum\n if n_comps == 0:\n evals, evecs = scipy.linalg.eigh(matrix)\n else:\n n_comps = min(matrix.shape[0] - 1, n_comps)\n # ncv = max(2 * n_comps + 1, int(np.sqrt(matrix.shape[0])))\n ncv = None\n which = \"LM\" if sort == \"decrease\" else \"SM\"\n # it pays off to increase the stability with a bit more precision\n matrix = matrix.astype(np.float64)\n\n # Setting the random initial vector\n random_state = check_random_state(random_state)\n v0 = random_state.standard_normal(matrix.shape[0])\n evals, evecs = scipy.sparse.linalg.eigsh(\n matrix, k=n_comps, which=which, ncv=ncv, v0=v0\n )\n evals, evecs = evals.astype(np.float32), evecs.astype(np.float32)\n if sort == \"decrease\":\n evals = evals[::-1]\n evecs = evecs[:, ::-1]\n logg.info(\n f\" eigenvalues of transition matrix\\n\" f\"{indent(str(evals), ' ')}\"\n )\n if self._number_connected_components > len(evals) / 2:\n logg.warning(\"Transition matrix has many disconnected components!\")\n self._eigen_values = evals\n self._eigen_basis = evecs\n\n def _init_iroot(self):\n self.iroot = None\n # set iroot directly\n if \"iroot\" in self._adata.uns:\n if self._adata.uns[\"iroot\"] >= self._adata.n_obs:\n logg.warning(\n f'Root cell index {self._adata.uns[\"iroot\"]} does not '\n f\"exist for {self._adata.n_obs} samples. It’s ignored.\"\n )\n else:\n self.iroot = self._adata.uns[\"iroot\"]\n return\n # set iroot via xroot\n xroot = None\n if \"xroot\" in self._adata.uns:\n xroot = self._adata.uns[\"xroot\"]\n elif \"xroot\" in self._adata.var:\n xroot = self._adata.var[\"xroot\"]\n # see whether we can set self.iroot using the full data matrix\n if xroot is not None and xroot.size == self._adata.shape[1]:\n self._set_iroot_via_xroot(xroot)\n\n def _get_dpt_row(self, i: int) -> np.ndarray:\n mask = None\n if self._number_connected_components > 1:\n label = self._connected_components[1][i]\n mask = self._connected_components[1] == label\n row = sum(\n (\n self.eigen_values[j]\n / (1 - self.eigen_values[j])\n * (self.eigen_basis[i, j] - self.eigen_basis[:, j])\n )\n ** 2\n # account for float32 precision\n for j in range(0, self.eigen_values.size)\n if self.eigen_values[j] < 0.9994\n )\n # thanks to Marius Lange for pointing Alex to this:\n # we will likely remove the contributions from the stationary state below when making\n # backwards compat breaking changes, they originate from an early implementation in 2015\n # they never seem to have deteriorated results, but also other distance measures (see e.g.\n # PAGA paper) don't have it, which makes sense\n row += sum(\n (self.eigen_basis[i, k] - self.eigen_basis[:, k]) ** 2\n for k in range(0, self.eigen_values.size)\n if self.eigen_values[k] >= 0.9994\n )\n if mask is not None:\n row[~mask] = np.inf\n return np.sqrt(row)\n\n def _set_pseudotime(self):\n \"\"\"Return pseudotime with respect to root point.\"\"\"\n self.pseudotime = self.distances_dpt[self.iroot].copy()\n self.pseudotime /= np.max(self.pseudotime[self.pseudotime < np.inf])\n\n def _set_iroot_via_xroot(self, xroot: np.ndarray):\n \"\"\"Determine the index of the root cell.\n\n Given an expression vector, find the observation index that is closest\n to this vector.\n\n Parameters\n ----------\n xroot\n Vector that marks the root cell, the vector storing the initial\n condition, only relevant for computing pseudotime.\n \"\"\"\n if self._adata.shape[1] != xroot.size:\n raise ValueError(\n \"The root vector you provided does not have the \" \"correct dimension.\"\n )\n # this is the squared distance\n dsqroot = 1e10\n iroot = 0\n for i in range(self._adata.shape[0]):\n diff = self._adata.X[i, :] - xroot\n dsq = diff @ diff\n if dsq < dsqroot:\n dsqroot = dsq\n iroot = i\n if np.sqrt(dsqroot) < 1e-10:\n break\n logg.debug(f\"setting root index to {iroot}\")\n if self.iroot is not None and iroot != self.iroot:\n logg.warning(f\"Changing index of iroot from {self.iroot} to {iroot}.\")\n self.iroot = iroot\n\n\nfrom __future__ import annotations\n\n\nclass TransformerChecksMixin:\n def _transform_checks(self, X, *fitted_props, **check_params):\n from sklearn.utils.validation import check_is_fitted\n\n if X is not None:\n X = self._validate_data(X, reset=False, **check_params)\n check_is_fitted(self, *fitted_props)\n return X\n\n\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nfrom sklearn.base import BaseEstimator, TransformerMixin\nfrom sklearn.exceptions import NotFittedError\nfrom sklearn.utils.validation import check_is_fitted\n\nfrom ..._settings import settings\nfrom ._common import TransformerChecksMixin\n\nif TYPE_CHECKING:\n from collections.abc import Mapping\n from typing import Any, Literal\n\n from numpy.typing import ArrayLike\n from scipy.sparse import csr_matrix\n\n _Algorithm = Literal[\"rbc\", \"brute\", \"ivfflat\", \"ivfpq\"]\n _Metric = Literal[\n \"l1\",\n \"cityblock\",\n \"taxicab\",\n \"manhattan\",\n \"euclidean\",\n \"l2\",\n \"braycurtis\",\n \"canberra\",\n \"minkowski\",\n \"chebyshev\",\n \"jensenshannon\",\n \"cosine\",\n \"correlation\",\n ]\n\n\nclass RapidsKNNTransformer(TransformerChecksMixin, TransformerMixin, BaseEstimator):\n \"\"\"Compute nearest neighbors using RAPIDS cuml.\n\n See :class:`cuml.neighbors.NearestNeighbors`.\n \"\"\"\n\n def __init__(\n self,\n *,\n handle=None,\n algorithm: _Algorithm | Literal[\"auto\"] = \"auto\",\n n_neighbors: int,\n metric: _Metric = \"euclidean\",\n p: int = 2,\n algo_params: Mapping[str, Any] | None = None,\n metric_params: Mapping[str, Any] | None = None,\n random_state=None,\n ) -> None:\n from cuml.neighbors import NearestNeighbors\n\n self.n_neighbors = n_neighbors\n self.metric = metric\n self.p = p\n self.nn = NearestNeighbors(\n n_neighbors=n_neighbors,\n # https://docs.rapids.ai/api/cuml/nightly/api/#verbosity-levels\n verbose=settings.verbosity + 2,\n handle=handle,\n algorithm=algorithm,\n metric=metric,\n p=p,\n algo_params=algo_params,\n metric_params=metric_params,\n output_type=\"input\", # could also be None to respect global setting\n )\n\n def __sklearn_is_fitted__(self) -> bool:\n try:\n check_is_fitted(self.nn)\n except NotFittedError:\n return False\n else:\n return True\n\n def fit(self, X: ArrayLike, y: Any = None) -> RapidsKNNTransformer:\n \"\"\"Index data for knn search.\"\"\"\n X_contiguous = np.ascontiguousarray(X, dtype=np.float32)\n self.nn.fit(X_contiguous)\n return self\n\n def transform(self, X: ArrayLike) -> csr_matrix:\n \"\"\"Perform knn search on the index.\"\"\"\n self._transform_checks(X)\n X_contiguous = np.ascontiguousarray(X, dtype=np.float32)\n return self.nn.kneighbors_graph(X_contiguous, mode=\"distance\")\n\n def _more_tags(self) -> dict[str, Any]:\n \"\"\"See :label:`sklearn:estimator_tags`\"\"\"\n return {\n \"requires_y\": False,\n \"preserves_dtype\": [np.float32],\n \"non_deterministic\": True,\n }\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nif TYPE_CHECKING:\n from collections.abc import Iterable\n\n import numpy as np\n import pandas as pd\n from anndata import AnnData\n from numpy.typing import NDArray\n from scipy.sparse import spmatrix\n\n\ndef rename_groups(\n adata: AnnData,\n restrict_key: str,\n *,\n key_added: str | None,\n restrict_categories: Iterable[str],\n restrict_indices: NDArray[np.bool_],\n groups: NDArray,\n) -> pd.Series[str]:\n key_added = f\"{restrict_key}_R\" if key_added is None else key_added\n all_groups = adata.obs[restrict_key].astype(\"U\")\n prefix = \"-\".join(restrict_categories) + \",\"\n new_groups = [prefix + g for g in groups.astype(\"U\")]\n all_groups.iloc[restrict_indices] = new_groups\n return all_groups\n\n\ndef restrict_adjacency(\n adata: AnnData,\n restrict_key: str,\n *,\n restrict_categories: Iterable[str],\n adjacency: spmatrix,\n) -> tuple[spmatrix, NDArray[np.bool_]]:\n if not isinstance(restrict_categories[0], str):\n raise ValueError(\n \"You need to use strings to label categories, \" \"e.g. '1' instead of 1.\"\n )\n for c in restrict_categories:\n if c not in adata.obs[restrict_key].cat.categories:\n raise ValueError(f\"'{c}' is not a valid category for '{restrict_key}'\")\n restrict_indices = adata.obs[restrict_key].isin(restrict_categories).values\n adjacency = adjacency[restrict_indices, :]\n adjacency = adjacency[:, restrict_indices]\n return adjacency, restrict_indices\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nimport scipy as sp\nfrom natsort import natsorted\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom ..neighbors import Neighbors, OnFlySymMatrix\n\nif TYPE_CHECKING:\n from collections.abc import Sequence\n\n from anndata import AnnData\n\n\ndef _diffmap(adata, n_comps=15, neighbors_key=None, random_state=0):\n start = logg.info(f\"computing Diffusion Maps using n_comps={n_comps}(=n_dcs)\")\n dpt = DPT(adata, neighbors_key=neighbors_key)\n dpt.compute_transitions()\n dpt.compute_eigen(n_comps=n_comps, random_state=random_state)\n adata.obsm[\"X_diffmap\"] = dpt.eigen_basis\n adata.uns[\"diffmap_evals\"] = dpt.eigen_values\n logg.info(\n \" finished\",\n time=start,\n deep=(\n \"added\\n\"\n \" 'X_diffmap', diffmap coordinates (adata.obsm)\\n\"\n \" 'diffmap_evals', eigenvalues of transition matrix (adata.uns)\"\n ),\n )\n\n\n@old_positionals(\n \"n_branchings\", \"min_group_size\", \"allow_kendall_tau_shift\", \"neighbors_key\", \"copy\"\n)\ndef dpt(\n adata: AnnData,\n n_dcs: int = 10,\n *,\n n_branchings: int = 0,\n min_group_size: float = 0.01,\n allow_kendall_tau_shift: bool = True,\n neighbors_key: str | None = None,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Infer progression of cells through geodesic distance along the graph\n :cite:p:`Haghverdi2016,Wolf2019`.\n\n Reconstruct the progression of a biological process from snapshot\n data. `Diffusion Pseudotime` has been introduced by :cite:t:`Haghverdi2016` and\n implemented within Scanpy :cite:p:`Wolf2018`. Here, we use a further developed\n version, which is able to deal with disconnected graphs :cite:p:`Wolf2019` and can\n be run in a `hierarchical` mode by setting the parameter\n `n_branchings>1`. We recommend, however, to only use\n :func:`~scanpy.tl.dpt` for computing pseudotime (`n_branchings=0`) and\n to detect branchings via :func:`~scanpy.tl.paga`. For pseudotime, you need\n to annotate your data with a root cell. For instance::\n\n adata.uns['iroot'] = np.flatnonzero(adata.obs['cell_types'] == 'Stem')[0]\n\n This requires to run :func:`~scanpy.pp.neighbors`, first. In order to\n reproduce the original implementation of DPT, use `method=='gauss'` in\n this. Using the default `method=='umap'` only leads to minor quantitative\n differences, though.\n\n .. versionadded:: 1.1\n\n :func:`~scanpy.tl.dpt` also requires to run\n :func:`~scanpy.tl.diffmap` first. As previously,\n :func:`~scanpy.tl.dpt` came with a default parameter of ``n_dcs=10`` but\n :func:`~scanpy.tl.diffmap` has a default parameter of ``n_comps=15``,\n you need to pass ``n_comps=10`` in :func:`~scanpy.tl.diffmap` in order\n to exactly reproduce previous :func:`~scanpy.tl.dpt` results.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n n_dcs\n The number of diffusion components to use.\n n_branchings\n Number of branchings to detect.\n min_group_size\n During recursive splitting of branches ('dpt groups') for `n_branchings`\n > 1, do not consider groups that contain less than `min_group_size` data\n points. If a float, `min_group_size` refers to a fraction of the total\n number of data points.\n allow_kendall_tau_shift\n If a very small branch is detected upon splitting, shift away from\n maximum correlation in Kendall tau criterion of :cite:t:`Haghverdi2016` to\n stabilize the splitting.\n neighbors_key\n If not specified, dpt looks .uns['neighbors'] for neighbors settings\n and .obsp['connectivities'], .obsp['distances'] for connectivities and\n distances respectively (default storage places for pp.neighbors).\n If specified, dpt looks .uns[neighbors_key] for neighbors settings and\n .obsp[.uns[neighbors_key]['connectivities_key']],\n .obsp[.uns[neighbors_key]['distances_key']] for connectivities and distances\n respectively.\n copy\n Copy instance before computation and return a copy.\n Otherwise, perform computation inplace and return `None`.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields (If `n_branchings==0`, no field `adata.obs['dpt_groups']` will be written):\n\n `adata.obs['dpt_pseudotime']` : :class:`pandas.Series` (dtype `float`)\n Array of dim (number of samples) that stores the pseudotime of each\n cell, that is, the DPT distance with respect to the root cell.\n `adata.obs['dpt_groups']` : :class:`pandas.Series` (dtype `category`)\n Array of dim (number of samples) that stores the subgroup id ('0',\n '1', ...) for each cell. The groups typically correspond to\n 'progenitor cells', 'undecided cells' or 'branches' of a process.\n\n Notes\n -----\n The tool is similar to the R package `destiny` of :cite:t:`Angerer2015`.\n \"\"\"\n # standard errors, warnings etc.\n adata = adata.copy() if copy else adata\n\n if neighbors_key is None:\n neighbors_key = \"neighbors\"\n if neighbors_key not in adata.uns:\n raise ValueError(\"You need to run `pp.neighbors` and `tl.diffmap` first.\")\n if \"iroot\" not in adata.uns and \"xroot\" not in adata.var:\n logg.warning(\n \"No root cell found. To compute pseudotime, pass the index or \"\n \"expression vector of a root cell, one of:\\n\"\n \" adata.uns['iroot'] = root_cell_index\\n\"\n \" adata.var['xroot'] = adata[root_cell_name, :].X\"\n )\n if \"X_diffmap\" not in adata.obsm:\n logg.warning(\n \"Trying to run `tl.dpt` without prior call of `tl.diffmap`. \"\n \"Falling back to `tl.diffmap` with default parameters.\"\n )\n _diffmap(adata, neighbors_key=neighbors_key)\n # start with the actual computation\n dpt = DPT(\n adata,\n n_dcs=n_dcs,\n min_group_size=min_group_size,\n n_branchings=n_branchings,\n allow_kendall_tau_shift=allow_kendall_tau_shift,\n neighbors_key=neighbors_key,\n )\n start = logg.info(f\"computing Diffusion Pseudotime using n_dcs={n_dcs}\")\n if n_branchings > 1:\n logg.info(\" this uses a hierarchical implementation\")\n if dpt.iroot is not None:\n dpt._set_pseudotime() # pseudotimes are distances from root point\n adata.uns[\"iroot\"] = (\n dpt.iroot\n ) # update iroot, might have changed when subsampling, for example\n adata.obs[\"dpt_pseudotime\"] = dpt.pseudotime\n # detect branchings and partition the data into segments\n if n_branchings > 0:\n dpt.branchings_segments()\n adata.obs[\"dpt_groups\"] = pd.Categorical(\n values=dpt.segs_names.astype(\"U\"),\n categories=natsorted(np.array(dpt.segs_names_unique).astype(\"U\")),\n )\n # the \"change points\" separate segments in the ordering above\n adata.uns[\"dpt_changepoints\"] = dpt.changepoints\n # the tip points of segments\n adata.uns[\"dpt_grouptips\"] = dpt.segs_tips\n # the ordering according to segments and pseudotime\n ordering_id = np.zeros(adata.n_obs, dtype=int)\n for count, idx in enumerate(dpt.indices):\n ordering_id[idx] = count\n adata.obs[\"dpt_order\"] = ordering_id\n adata.obs[\"dpt_order_indices\"] = dpt.indices\n logg.info(\n \" finished\",\n time=start,\n deep=(\n \"added\\n\"\n + (\n \" 'dpt_pseudotime', the pseudotime (adata.obs)\"\n if dpt.iroot is not None\n else \"\"\n )\n + (\n \"\\n 'dpt_groups', the branching subgroups of dpt (adata.obs)\"\n \"\\n 'dpt_order', cell order (adata.obs)\"\n if n_branchings > 0\n else \"\"\n )\n ),\n )\n return adata if copy else None\n\n\nclass DPT(Neighbors):\n \"\"\"\\\n Hierarchical Diffusion Pseudotime.\n \"\"\"\n\n def __init__(\n self,\n adata: AnnData,\n *,\n n_dcs: int | None = None,\n min_group_size: float = 0.01,\n n_branchings: int = 0,\n allow_kendall_tau_shift: bool = False,\n neighbors_key: str | None = None,\n ):\n super().__init__(adata, n_dcs=n_dcs, neighbors_key=neighbors_key)\n self.flavor = \"haghverdi16\"\n self.n_branchings = n_branchings\n self.min_group_size = (\n min_group_size\n if min_group_size >= 1\n else int(min_group_size * self._adata.shape[0])\n )\n self.passed_adata = adata # just for debugging purposes\n self.choose_largest_segment = False\n self.allow_kendall_tau_shift = allow_kendall_tau_shift\n\n def branchings_segments(self):\n \"\"\"\\\n Detect branchings and partition the data into corresponding segments.\n\n Detect all branchings up to `n_branchings`.\n\n Writes\n ------\n segs : :class:`~numpy.ndarray`\n Array of dimension (number of segments) × (number of data\n points). Each row stores a mask array that defines a segment.\n segs_tips : :class:`~numpy.ndarray`\n Array of dimension (number of segments) × 2. Each row stores the\n indices of the two tip points of each segment.\n segs_names : :class:`~numpy.ndarray`\n Array of dimension (number of data points). Stores an integer label\n for each segment.\n \"\"\"\n self.detect_branchings()\n self.postprocess_segments()\n self.set_segs_names()\n self.order_pseudotime()\n\n def detect_branchings(self):\n \"\"\"\\\n Detect all branchings up to `n_branchings`.\n\n Writes Attributes\n -----------------\n segs : :class:`~numpy.ndarray`\n List of integer index arrays.\n segs_tips : :class:`~numpy.ndarray`\n List of indices of the tips of segments.\n \"\"\"\n logg.debug(\n f\" detect {self.n_branchings} \"\n f'branching{\"\" if self.n_branchings == 1 else \"s\"}',\n )\n # a segment is a subset of points of the data set (defined by the\n # indices of the points in the segment)\n # initialize the search for branchings with a single segment,\n # that is, get the indices of the whole data set\n indices_all = np.arange(self._adata.shape[0], dtype=int)\n # let's keep a list of segments, the first segment to add is the\n # whole data set\n segs = [indices_all]\n # a segment can as well be defined by the two points that have maximal\n # distance in the segment, the \"tips\" of the segment\n #\n # the rest of the points in the segment is then defined by demanding\n # them to \"be close to the line segment that connects the tips\", that\n # is, for such a point, the normalized added distance to both tips is\n # smaller than one:\n # (D[tips[0],i] + D[tips[1],i])/D[tips[0],tips[1] < 1\n # of course, this condition is fulfilled by the full cylindrical\n # subspace surrounding that line segment, where the radius of the\n # cylinder can be infinite\n #\n # if D denotes a euclidian distance matrix, a line segment is a linear\n # object, and the name \"line\" is justified. if we take the\n # diffusion-based distance matrix Dchosen, which approximates geodesic\n # distance, with \"line\", we mean the shortest path between two points,\n # which can be highly non-linear in the original space\n #\n # let us define the tips of the whole data set\n if False: # this is safe, but not compatible with on-the-fly computation\n tips_all = np.array(\n np.unravel_index(\n np.argmax(self.distances_dpt), self.distances_dpt.shape\n )\n )\n else:\n if self.iroot is not None:\n tip_0 = np.argmax(self.distances_dpt[self.iroot])\n else:\n tip_0 = np.argmax(self.distances_dpt[0])\n tips_all = np.array([tip_0, np.argmax(self.distances_dpt[tip_0])])\n # we keep a list of the tips of each segment\n segs_tips = [tips_all]\n segs_connects = [[]]\n segs_undecided = [True]\n segs_adjacency = [[]]\n logg.debug(\n \" do not consider groups with less than \"\n f\"{self.min_group_size} points for splitting\"\n )\n for ibranch in range(self.n_branchings):\n iseg, tips3 = self.select_segment(segs, segs_tips, segs_undecided)\n if iseg == -1:\n logg.debug(\" partitioning converged\")\n break\n logg.debug(\n f\" branching {ibranch + 1}: split group {iseg}\",\n ) # [third start end]\n # detect branching and update segs and segs_tips\n self.detect_branching(\n segs=segs,\n segs_tips=segs_tips,\n segs_connects=segs_connects,\n segs_undecided=segs_undecided,\n segs_adjacency=segs_adjacency,\n iseg=iseg,\n tips3=tips3,\n )\n # store as class members\n self.segs = segs\n self.segs_tips = segs_tips\n self.segs_undecided = segs_undecided\n # the following is a bit too much, but this allows easy storage\n self.segs_adjacency = sp.sparse.lil_matrix((len(segs), len(segs)), dtype=float)\n self.segs_connects = sp.sparse.lil_matrix((len(segs), len(segs)), dtype=int)\n for i, seg_adjacency in enumerate(segs_adjacency):\n self.segs_connects[i, seg_adjacency] = segs_connects[i]\n for i in range(len(segs)):\n for j in range(len(segs)):\n self.segs_adjacency[i, j] = self.distances_dpt[\n self.segs_connects[i, j], self.segs_connects[j, i]\n ]\n self.segs_adjacency = self.segs_adjacency.tocsr()\n self.segs_connects = self.segs_connects.tocsr()\n\n def check_adjacency(self):\n n_edges_per_seg = np.sum(self.segs_adjacency > 0, axis=1).A1\n for n_edges in range(1, np.max(n_edges_per_seg) + 1):\n for iseg in range(self.segs_adjacency.shape[0]):\n if n_edges_per_seg[iseg] == n_edges:\n neighbor_segs = ( # noqa: F841 TODO Evaluate whether to assign the variable or not\n self.segs_adjacency[iseg].todense().A1\n )\n closest_points_other_segs = [\n seg[np.argmin(self.distances_dpt[self.segs_tips[iseg][0], seg])]\n for seg in self.segs\n ]\n seg = self.segs[iseg]\n closest_points_in_segs = [\n seg[np.argmin(self.distances_dpt[tips[0], seg])]\n for tips in self.segs_tips\n ]\n distance_segs = [\n self.distances_dpt[closest_points_other_segs[ipoint], point]\n for ipoint, point in enumerate(closest_points_in_segs)\n ]\n # exclude the first point, the segment itself\n closest_segs = np.argsort(distance_segs)[1 : n_edges + 1]\n # update adjacency matrix within the loop!\n # self.segs_adjacency[iseg, neighbor_segs > 0] = 0\n # self.segs_adjacency[iseg, closest_segs] = np.array(distance_segs)[closest_segs]\n # self.segs_adjacency[neighbor_segs > 0, iseg] = 0\n # self.segs_adjacency[closest_segs, iseg] = np.array(distance_segs)[closest_segs].reshape(len(closest_segs), 1)\n # n_edges_per_seg = np.sum(self.segs_adjacency > 0, axis=1).A1\n print(iseg, distance_segs, closest_segs)\n # print(self.segs_adjacency)\n # self.segs_adjacency.eliminate_zeros()\n\n def select_segment(self, segs, segs_tips, segs_undecided) -> tuple[int, int]:\n \"\"\"\\\n Out of a list of line segments, choose segment that has the most\n distant second data point.\n\n Assume the distance matrix Ddiff is sorted according to seg_idcs.\n Compute all the distances.\n\n Returns\n -------\n iseg\n Index identifying the position within the list of line segments.\n tips3\n Positions of tips within chosen segment.\n \"\"\"\n scores_tips = np.zeros((len(segs), 4))\n allindices = np.arange(self._adata.shape[0], dtype=int)\n for iseg, seg in enumerate(segs):\n # do not consider too small segments\n if segs_tips[iseg][0] == -1:\n continue\n # restrict distance matrix to points in segment\n if not isinstance(self.distances_dpt, OnFlySymMatrix):\n Dseg = self.distances_dpt[np.ix_(seg, seg)]\n else:\n Dseg = self.distances_dpt.restrict(seg)\n third_maximizer = None\n if segs_undecided[iseg]:\n # check that none of our tips \"connects\" with a tip of the\n # other segments\n for jseg in range(len(segs)):\n if jseg != iseg:\n # take the inner tip, the \"second tip\" of the segment\n for itip in range(2):\n if (\n self.distances_dpt[\n segs_tips[jseg][1], segs_tips[iseg][itip]\n ]\n < 0.5\n * self.distances_dpt[\n segs_tips[iseg][~itip], segs_tips[iseg][itip]\n ]\n ):\n # logg.debug(\n # ' group', iseg, 'with tip', segs_tips[iseg][itip],\n # 'connects with', jseg, 'with tip', segs_tips[jseg][1],\n # )\n # logg.debug(' do not use the tip for \"triangulation\"')\n third_maximizer = itip\n # map the global position to the position within the segment\n tips = [np.where(allindices[seg] == tip)[0][0] for tip in segs_tips[iseg]]\n # find the third point on the segment that has maximal\n # added distance from the two tip points\n dseg = Dseg[tips[0]] + Dseg[tips[1]]\n if not np.isfinite(dseg).any():\n continue\n # add this point to tips, it's a third tip, we store it at the first\n # position in an array called tips3\n third_tip = np.argmax(dseg)\n if third_maximizer is not None:\n # find a fourth point that has maximal distance to all three\n dseg += Dseg[third_tip]\n fourth_tip = np.argmax(dseg)\n if fourth_tip != tips[0] and fourth_tip != third_tip:\n tips[1] = fourth_tip\n dseg -= Dseg[tips[1]]\n else:\n dseg -= Dseg[third_tip]\n tips3 = np.append(tips, third_tip)\n # compute the score as ratio of the added distance to the third tip,\n # to what it would be if it were on the straight line between the\n # two first tips, given by Dseg[tips[:2]]\n # if we did not normalize, there would be a danger of simply\n # assigning the highest score to the longest segment\n score = dseg[tips3[2]] / Dseg[tips3[0], tips3[1]]\n score = (\n len(seg) if self.choose_largest_segment else score\n ) # simply the number of points\n logg.debug(\n f\" group {iseg} score {score} n_points {len(seg)} \" + \"(too small)\"\n if len(seg) < self.min_group_size\n else \"\",\n )\n if len(seg) <= self.min_group_size:\n score = 0\n # write result\n scores_tips[iseg, 0] = score\n scores_tips[iseg, 1:] = tips3\n iseg = np.argmax(scores_tips[:, 0])\n if scores_tips[iseg, 0] == 0:\n return -1, None\n tips3 = scores_tips[iseg, 1:].astype(int)\n return iseg, tips3\n\n def postprocess_segments(self):\n \"\"\"Convert the format of the segment class members.\"\"\"\n # make segs a list of mask arrays, it's easier to store\n # as there is a hdf5 equivalent\n for iseg, seg in enumerate(self.segs):\n mask = np.zeros(self._adata.shape[0], dtype=bool)\n mask[seg] = True\n self.segs[iseg] = mask\n # convert to arrays\n self.segs = np.array(self.segs)\n self.segs_tips = np.array(self.segs_tips)\n\n def set_segs_names(self):\n \"\"\"Return a single array that stores integer segment labels.\"\"\"\n segs_names = np.zeros(self._adata.shape[0], dtype=np.int8)\n self.segs_names_unique = []\n for iseg, seg in enumerate(self.segs):\n segs_names[seg] = iseg\n self.segs_names_unique.append(iseg)\n self.segs_names = segs_names\n\n def order_pseudotime(self):\n \"\"\"\\\n Define indices that reflect segment and pseudotime order.\n\n Writes\n ------\n indices : :class:`~numpy.ndarray`\n Index array of shape n, which stores an ordering of the data points\n with respect to increasing segment index and increasing pseudotime.\n changepoints : :class:`~numpy.ndarray`\n Index array of shape len(ssegs)-1, which stores the indices of\n points where the segment index changes, with respect to the ordering\n of indices.\n \"\"\"\n # within segs_tips, order tips according to pseudotime\n if self.iroot is not None:\n for itips, tips in enumerate(self.segs_tips):\n if tips[0] != -1:\n indices = np.argsort(self.pseudotime[tips])\n self.segs_tips[itips] = self.segs_tips[itips][indices]\n else:\n logg.debug(f\" group {itips} is very small\")\n # sort indices according to segments\n indices = np.argsort(self.segs_names)\n segs_names = self.segs_names[indices]\n # find changepoints of segments\n changepoints = np.arange(indices.size - 1)[np.diff(segs_names) == 1] + 1\n if self.iroot is not None:\n pseudotime = self.pseudotime[indices]\n for iseg, seg in enumerate(self.segs):\n # only consider one segment, it's already ordered by segment\n seg_sorted = seg[indices]\n # consider the pseudotime on this segment and sort them\n seg_indices = np.argsort(pseudotime[seg_sorted])\n # within the segment, order indices according to increasing pseudotime\n indices[seg_sorted] = indices[seg_sorted][seg_indices]\n # define class members\n self.indices = indices\n self.changepoints = changepoints\n\n def detect_branching(\n self,\n *,\n segs: Sequence[np.ndarray],\n segs_tips: Sequence[np.ndarray],\n segs_connects,\n segs_undecided,\n segs_adjacency,\n iseg: int,\n tips3: np.ndarray,\n ):\n \"\"\"\\\n Detect branching on given segment.\n\n Updates all list parameters inplace.\n\n Call function _detect_branching and perform bookkeeping on segs and\n segs_tips.\n\n Parameters\n ----------\n segs\n Dchosen distance matrix restricted to segment.\n segs_tips\n Stores all tip points for the segments in segs.\n iseg\n Position of segment under study in segs.\n tips3\n The three tip points. They form a 'triangle' that contains the data.\n \"\"\"\n seg = segs[iseg]\n # restrict distance matrix to points in segment\n if not isinstance(self.distances_dpt, OnFlySymMatrix):\n Dseg = self.distances_dpt[np.ix_(seg, seg)]\n else:\n Dseg = self.distances_dpt.restrict(seg)\n # given the three tip points and the distance matrix detect the\n # branching on the segment, return the list ssegs of segments that\n # are defined by splitting this segment\n result = self._detect_branching(Dseg, tips3, seg)\n ssegs, ssegs_tips, ssegs_adjacency, ssegs_connects, trunk = result\n # map back to global indices\n for iseg_new, seg_new in enumerate(ssegs):\n ssegs[iseg_new] = seg[seg_new]\n ssegs_tips[iseg_new] = seg[ssegs_tips[iseg_new]]\n ssegs_connects[iseg_new] = list(seg[ssegs_connects[iseg_new]])\n # remove previous segment\n segs.pop(iseg)\n segs_tips.pop(iseg)\n # insert trunk/undecided_cells at same position\n segs.insert(iseg, ssegs[trunk])\n segs_tips.insert(iseg, ssegs_tips[trunk])\n # append other segments\n segs += [seg for iseg, seg in enumerate(ssegs) if iseg != trunk]\n segs_tips += [\n seg_tips for iseg, seg_tips in enumerate(ssegs_tips) if iseg != trunk\n ]\n if len(ssegs) == 4:\n # insert undecided cells at same position\n segs_undecided.pop(iseg)\n segs_undecided.insert(iseg, True)\n # correct edges in adjacency matrix\n n_add = len(ssegs) - 1\n prev_connecting_segments = segs_adjacency[iseg].copy()\n if self.flavor == \"haghverdi16\":\n segs_adjacency += [[iseg] for i in range(n_add)]\n segs_connects += [\n seg_connects\n for iseg, seg_connects in enumerate(ssegs_connects)\n if iseg != trunk\n ]\n # TODO Evaluate whether to assign the variable or not\n prev_connecting_points = segs_connects[iseg] # noqa: F841\n for jseg_cnt, jseg in enumerate(prev_connecting_segments):\n iseg_cnt = 0\n for iseg_new, seg_new in enumerate(ssegs):\n if iseg_new != trunk:\n pos = segs_adjacency[jseg].index(iseg)\n connection_to_iseg = segs_connects[jseg][pos]\n if connection_to_iseg in seg_new:\n kseg = len(segs) - n_add + iseg_cnt\n segs_adjacency[jseg][pos] = kseg\n pos_2 = segs_adjacency[iseg].index(jseg)\n segs_adjacency[iseg].pop(pos_2)\n idx = segs_connects[iseg].pop(pos_2)\n segs_adjacency[kseg].append(jseg)\n segs_connects[kseg].append(idx)\n break\n iseg_cnt += 1\n segs_adjacency[iseg] += list(\n range(len(segs_adjacency) - n_add, len(segs_adjacency))\n )\n segs_connects[iseg] += ssegs_connects[trunk]\n else:\n import networkx as nx\n\n segs_adjacency += [[] for i in range(n_add)]\n segs_connects += [[] for i in range(n_add)]\n kseg_list = [iseg] + list(range(len(segs) - n_add, len(segs)))\n for jseg in prev_connecting_segments:\n pos = segs_adjacency[jseg].index(iseg)\n distances = []\n closest_points_in_jseg = []\n closest_points_in_kseg = []\n for kseg in kseg_list:\n reference_point_in_k = segs_tips[kseg][0]\n closest_points_in_jseg.append(\n segs[jseg][\n np.argmin(\n self.distances_dpt[reference_point_in_k, segs[jseg]]\n )\n ]\n )\n # do not use the tip in the large segment j, instead, use the closest point\n reference_point_in_j = closest_points_in_jseg[\n -1\n ] # segs_tips[jseg][0]\n closest_points_in_kseg.append(\n segs[kseg][\n np.argmin(\n self.distances_dpt[reference_point_in_j, segs[kseg]]\n )\n ]\n )\n distances.append(\n self.distances_dpt[\n closest_points_in_jseg[-1], closest_points_in_kseg[-1]\n ]\n )\n # print(jseg, '(', segs_tips[jseg][0], closest_points_in_jseg[-1], ')',\n # kseg, '(', segs_tips[kseg][0], closest_points_in_kseg[-1], ') :', distances[-1])\n idx = np.argmin(distances)\n kseg_min = kseg_list[idx]\n segs_adjacency[jseg][pos] = kseg_min\n segs_connects[jseg][pos] = closest_points_in_kseg[idx]\n pos_2 = segs_adjacency[iseg].index(jseg)\n segs_adjacency[iseg].pop(pos_2)\n segs_connects[iseg].pop(pos_2)\n segs_adjacency[kseg_min].append(jseg)\n segs_connects[kseg_min].append(closest_points_in_jseg[idx])\n # if we split two clusters, we need to check whether the new segments connect to any of the other\n # old segments\n # if not, we add a link between the new segments, if yes, we add two links to connect them at the\n # correct old segments\n do_not_attach_kseg = False\n for kseg in kseg_list:\n distances = []\n closest_points_in_jseg = []\n closest_points_in_kseg = []\n jseg_list = [\n jseg\n for jseg in range(len(segs))\n if jseg != kseg and jseg not in prev_connecting_segments\n ]\n for jseg in jseg_list:\n reference_point_in_k = segs_tips[kseg][0]\n closest_points_in_jseg.append(\n segs[jseg][\n np.argmin(\n self.distances_dpt[reference_point_in_k, segs[jseg]]\n )\n ]\n )\n # do not use the tip in the large segment j, instead, use the closest point\n reference_point_in_j = closest_points_in_jseg[\n -1\n ] # segs_tips[jseg][0]\n closest_points_in_kseg.append(\n segs[kseg][\n np.argmin(\n self.distances_dpt[reference_point_in_j, segs[kseg]]\n )\n ]\n )\n distances.append(\n self.distances_dpt[\n closest_points_in_jseg[-1], closest_points_in_kseg[-1]\n ]\n )\n idx = np.argmin(distances)\n jseg_min = jseg_list[idx]\n if jseg_min not in kseg_list:\n segs_adjacency_sparse = sp.sparse.lil_matrix(\n (len(segs), len(segs)), dtype=float\n )\n for i, seg_adjacency in enumerate(segs_adjacency):\n segs_adjacency_sparse[i, seg_adjacency] = 1\n G = nx.Graph(segs_adjacency_sparse)\n paths_all = nx.single_source_dijkstra_path(G, source=kseg)\n if jseg_min not in paths_all:\n segs_adjacency[jseg_min].append(kseg)\n segs_connects[jseg_min].append(closest_points_in_kseg[idx])\n segs_adjacency[kseg].append(jseg_min)\n segs_connects[kseg].append(closest_points_in_jseg[idx])\n logg.debug(f\" attaching new segment {kseg} at {jseg_min}\")\n # if we split the cluster, we should not attach kseg\n do_not_attach_kseg = True\n else:\n logg.debug(\n f\" cannot attach new segment {kseg} at {jseg_min} \"\n \"(would produce cycle)\"\n )\n if kseg != kseg_list[-1]:\n logg.debug(\" continue\")\n continue\n else:\n logg.debug(\" do not add another link\")\n break\n if jseg_min in kseg_list and not do_not_attach_kseg:\n segs_adjacency[jseg_min].append(kseg)\n segs_connects[jseg_min].append(closest_points_in_kseg[idx])\n segs_adjacency[kseg].append(jseg_min)\n segs_connects[kseg].append(closest_points_in_jseg[idx])\n break\n segs_undecided += [False for i in range(n_add)]\n\n def _detect_branching(\n self,\n Dseg: np.ndarray,\n tips: np.ndarray,\n seg_reference=None,\n ) -> tuple[\n list[np.ndarray],\n list[np.ndarray],\n list[list[int]],\n list[list[int]],\n int,\n ]:\n \"\"\"\\\n Detect branching on given segment.\n\n Call function __detect_branching three times for all three orderings of\n tips. Points that do not belong to the same segment in all three\n orderings are assigned to a fourth segment. The latter is, by Haghverdi\n et al. (2016) referred to as 'undecided cells'.\n\n Parameters\n ----------\n Dseg\n Dchosen distance matrix restricted to segment.\n tips\n The three tip points. They form a 'triangle' that contains the data.\n\n Returns\n -------\n ssegs\n List of segments obtained from splitting the single segment defined\n via the first two tip cells.\n ssegs_tips\n List of tips of segments in ssegs.\n ssegs_adjacency\n ?\n ssegs_connects\n ?\n trunk\n ?\n \"\"\"\n if self.flavor == \"haghverdi16\":\n ssegs = self._detect_branching_single_haghverdi16(Dseg, tips)\n elif self.flavor == \"wolf17_tri\":\n ssegs = self._detect_branching_single_wolf17_tri(Dseg, tips)\n elif self.flavor == \"wolf17_bi\" or self.flavor == \"wolf17_bi_un\":\n ssegs = self._detect_branching_single_wolf17_bi(Dseg, tips)\n else:\n raise ValueError(\n '`flavor` needs to be in {\"haghverdi16\", \"wolf17_tri\", \"wolf17_bi\"}.'\n )\n # make sure that each data point has a unique association with a segment\n masks = np.zeros((len(ssegs), Dseg.shape[0]), dtype=bool)\n for iseg, seg in enumerate(ssegs):\n masks[iseg][seg] = True\n nonunique = np.sum(masks, axis=0) > 1\n ssegs = []\n for iseg, mask in enumerate(masks):\n mask[nonunique] = False\n ssegs.append(np.arange(Dseg.shape[0], dtype=int)[mask])\n # compute new tips within new segments\n ssegs_tips = []\n for inewseg, newseg in enumerate(ssegs):\n if len(np.flatnonzero(newseg)) <= 1:\n logg.warning(f\"detected group with only {np.flatnonzero(newseg)} cells\")\n secondtip = newseg[np.argmax(Dseg[tips[inewseg]][newseg])]\n ssegs_tips.append([tips[inewseg], secondtip])\n undecided_cells = np.arange(Dseg.shape[0], dtype=int)[nonunique]\n if len(undecided_cells) > 0:\n ssegs.append(undecided_cells)\n # establish the connecting points with the other segments\n ssegs_connects = [[], [], [], []]\n for inewseg, newseg_tips in enumerate(ssegs_tips):\n reference_point = newseg_tips[0]\n # closest cell to the new segment within undecided cells\n closest_cell = undecided_cells[\n np.argmin(Dseg[reference_point][undecided_cells])\n ]\n ssegs_connects[inewseg].append(closest_cell)\n # closest cell to the undecided cells within new segment\n closest_cell = ssegs[inewseg][\n np.argmin(Dseg[closest_cell][ssegs[inewseg]])\n ]\n ssegs_connects[-1].append(closest_cell)\n # also compute tips for the undecided cells\n tip_0 = undecided_cells[\n np.argmax(Dseg[undecided_cells[0]][undecided_cells])\n ]\n tip_1 = undecided_cells[np.argmax(Dseg[tip_0][undecided_cells])]\n ssegs_tips.append([tip_0, tip_1])\n ssegs_adjacency = [[3], [3], [3], [0, 1, 2]]\n trunk = 3\n elif len(ssegs) == 3:\n reference_point = np.zeros(3, dtype=int)\n reference_point[0] = ssegs_tips[0][0]\n reference_point[1] = ssegs_tips[1][0]\n reference_point[2] = ssegs_tips[2][0]\n closest_points = np.zeros((3, 3), dtype=int)\n # this is another strategy than for the undecided_cells\n # here it's possible to use the more symmetric procedure\n # shouldn't make much of a difference\n closest_points[0, 1] = ssegs[1][\n np.argmin(Dseg[reference_point[0]][ssegs[1]])\n ]\n closest_points[1, 0] = ssegs[0][\n np.argmin(Dseg[reference_point[1]][ssegs[0]])\n ]\n closest_points[0, 2] = ssegs[2][\n np.argmin(Dseg[reference_point[0]][ssegs[2]])\n ]\n closest_points[2, 0] = ssegs[0][\n np.argmin(Dseg[reference_point[2]][ssegs[0]])\n ]\n closest_points[1, 2] = ssegs[2][\n np.argmin(Dseg[reference_point[1]][ssegs[2]])\n ]\n closest_points[2, 1] = ssegs[1][\n np.argmin(Dseg[reference_point[2]][ssegs[1]])\n ]\n added_dist = np.zeros(3)\n added_dist[0] = (\n Dseg[closest_points[1, 0], closest_points[0, 1]]\n + Dseg[closest_points[2, 0], closest_points[0, 2]]\n )\n added_dist[1] = (\n Dseg[closest_points[0, 1], closest_points[1, 0]]\n + Dseg[closest_points[2, 1], closest_points[1, 2]]\n )\n added_dist[2] = (\n Dseg[closest_points[1, 2], closest_points[2, 1]]\n + Dseg[closest_points[0, 2], closest_points[2, 0]]\n )\n trunk = np.argmin(added_dist)\n ssegs_adjacency = [\n [trunk] if i != trunk else [j for j in range(3) if j != trunk]\n for i in range(3)\n ]\n ssegs_connects = [\n [closest_points[i, trunk]]\n if i != trunk\n else [closest_points[trunk, j] for j in range(3) if j != trunk]\n for i in range(3)\n ]\n else:\n trunk = 0\n ssegs_adjacency = [[1], [0]]\n reference_point_in_0 = ssegs_tips[0][0]\n closest_point_in_1 = ssegs[1][\n np.argmin(Dseg[reference_point_in_0][ssegs[1]])\n ]\n reference_point_in_1 = closest_point_in_1 # ssegs_tips[1][0]\n closest_point_in_0 = ssegs[0][\n np.argmin(Dseg[reference_point_in_1][ssegs[0]])\n ]\n ssegs_connects = [[closest_point_in_1], [closest_point_in_0]]\n return ssegs, ssegs_tips, ssegs_adjacency, ssegs_connects, trunk\n\n def _detect_branching_single_haghverdi16(self, Dseg, tips):\n \"\"\"Detect branching on given segment.\"\"\"\n # compute branchings using different starting points the first index of\n # tips is the starting point for the other two, the order does not\n # matter\n ssegs = []\n # permutations of tip cells\n ps = [\n [0, 1, 2], # start by computing distances from the first tip\n [1, 2, 0], # -\"- second tip\n [2, 0, 1], # -\"- third tip\n ]\n for i, p in enumerate(ps):\n ssegs.append(self.__detect_branching_haghverdi16(Dseg, tips[p]))\n return ssegs\n\n def _detect_branching_single_wolf17_tri(self, Dseg, tips):\n # all pairwise distances\n dist_from_0 = Dseg[tips[0]]\n dist_from_1 = Dseg[tips[1]]\n dist_from_2 = Dseg[tips[2]]\n closer_to_0_than_to_1 = dist_from_0 < dist_from_1\n closer_to_0_than_to_2 = dist_from_0 < dist_from_2\n closer_to_1_than_to_2 = dist_from_1 < dist_from_2\n masks = np.zeros((2, Dseg.shape[0]), dtype=bool)\n masks[0] = closer_to_0_than_to_1\n masks[1] = closer_to_0_than_to_2\n segment_0 = np.sum(masks, axis=0) == 2\n masks = np.zeros((2, Dseg.shape[0]), dtype=bool)\n masks[0] = ~closer_to_0_than_to_1\n masks[1] = closer_to_1_than_to_2\n segment_1 = np.sum(masks, axis=0) == 2\n masks = np.zeros((2, Dseg.shape[0]), dtype=bool)\n masks[0] = ~closer_to_0_than_to_2\n masks[1] = ~closer_to_1_than_to_2\n segment_2 = np.sum(masks, axis=0) == 2\n ssegs = [segment_0, segment_1, segment_2]\n return ssegs\n\n def _detect_branching_single_wolf17_bi(self, Dseg, tips):\n dist_from_0 = Dseg[tips[0]]\n dist_from_1 = Dseg[tips[1]]\n closer_to_0_than_to_1 = dist_from_0 < dist_from_1\n ssegs = [closer_to_0_than_to_1, ~closer_to_0_than_to_1]\n return ssegs\n\n def __detect_branching_haghverdi16(\n self, Dseg: np.ndarray, tips: np.ndarray\n ) -> np.ndarray:\n \"\"\"\\\n Detect branching on given segment.\n\n Compute point that maximizes kendall tau correlation of the sequences of\n distances to the second and the third tip, respectively, when 'moving\n away' from the first tip: tips[0]. 'Moving away' means moving in the\n direction of increasing distance from the first tip.\n\n Parameters\n ----------\n Dseg\n Dchosen distance matrix restricted to segment.\n tips\n The three tip points. They form a 'triangle' that contains the data.\n\n Returns\n -------\n Segments obtained from \"splitting away the first tip cell\".\n \"\"\"\n # sort distance from first tip point\n # then the sequence of distances Dseg[tips[0]][idcs] increases\n idcs = np.argsort(Dseg[tips[0]])\n # consider now the sequence of distances from the other\n # two tip points, which only increase when being close to `tips[0]`\n # where they become correlated\n # at the point where this happens, we define a branching point\n if True:\n imax = self.kendall_tau_split(\n Dseg[tips[1]][idcs],\n Dseg[tips[2]][idcs],\n )\n if False:\n # if we were in euclidian space, the following should work\n # as well, but here, it doesn't because the scales in Dseg are\n # highly different, one would need to write the following equation\n # in terms of an ordering, such as exploited by the kendall\n # correlation method above\n imax = np.argmin(\n Dseg[tips[0]][idcs] + Dseg[tips[1]][idcs] + Dseg[tips[2]][idcs]\n )\n # init list to store new segments\n ssegs = [] # noqa: F841 # TODO Look into this\n # first new segment: all points until, but excluding the branching point\n # increasing the following slightly from imax is a more conservative choice\n # as the criterion based on normalized distances, which follows below,\n # is less stable\n if imax > 0.95 * len(idcs) and self.allow_kendall_tau_shift:\n # if \"everything\" is correlated (very large value of imax), a more\n # conservative choice amounts to reducing this\n logg.warning(\n \"shifting branching point away from maximal kendall-tau \"\n \"correlation (suppress this with `allow_kendall_tau_shift=False`)\"\n )\n ibranch = int(0.95 * imax)\n else:\n # otherwise, a more conservative choice is the following\n ibranch = imax + 1\n return idcs[:ibranch]\n\n def kendall_tau_split(self, a: np.ndarray, b: np.ndarray) -> int:\n \"\"\"Return splitting index that maximizes correlation in the sequences.\n\n Compute difference in Kendall tau for all splitted sequences.\n\n For each splitting index i, compute the difference of the two\n correlation measures kendalltau(a[:i], b[:i]) and\n kendalltau(a[i:], b[i:]).\n\n Returns the splitting index that maximizes\n kendalltau(a[:i], b[:i]) - kendalltau(a[i:], b[i:])\n\n Parameters\n ----------\n a\n b\n One dimensional sequences.\n\n Returns\n -------\n Splitting index according to above description.\n \"\"\"\n if a.size != b.size:\n raise ValueError(\"a and b need to have the same size\")\n if a.ndim != b.ndim != 1:\n raise ValueError(\"a and b need to be one-dimensional arrays\")\n import scipy as sp\n\n min_length = 5\n n = a.size\n idx_range = np.arange(min_length, a.size - min_length - 1, dtype=int)\n corr_coeff = np.zeros(idx_range.size)\n pos_old = sp.stats.kendalltau(a[:min_length], b[:min_length])[0]\n neg_old = sp.stats.kendalltau(a[min_length:], b[min_length:])[0]\n for ii, i in enumerate(idx_range):\n if True:\n # compute differences in concordance when adding a[i] and b[i]\n # to the first subsequence, and removing these elements from\n # the second subsequence\n diff_pos, diff_neg = self._kendall_tau_diff(a, b, i)\n pos = pos_old + self._kendall_tau_add(i, diff_pos, pos_old)\n neg = neg_old + self._kendall_tau_subtract(n - i, diff_neg, neg_old)\n pos_old = pos\n neg_old = neg\n if False:\n # computation using sp.stats.kendalltau, takes much longer!\n # just for debugging purposes\n pos = sp.stats.kendalltau(a[: i + 1], b[: i + 1])[0]\n neg = sp.stats.kendalltau(a[i + 1 :], b[i + 1 :])[0]\n if False:\n # the following is much slower than using sp.stats.kendalltau,\n # it is only good for debugging because it allows to compute the\n # tau-a version, which does not account for ties, whereas\n # sp.stats.kendalltau computes tau-b version, which accounts for\n # ties\n pos = sp.stats.mstats.kendalltau(a[:i], b[:i], use_ties=False)[0]\n neg = sp.stats.mstats.kendalltau(a[i:], b[i:], use_ties=False)[0]\n corr_coeff[ii] = pos - neg\n iimax = np.argmax(corr_coeff)\n imax = min_length + iimax\n corr_coeff_max = corr_coeff[iimax]\n if corr_coeff_max < 0.3:\n logg.debug(\" is root itself, never obtain significant correlation\")\n return imax\n\n def _kendall_tau_add(self, len_old: int, diff_pos: int, tau_old: float):\n \"\"\"Compute Kendall tau delta.\n\n The new sequence has length len_old + 1.\n\n Parameters\n ----------\n len_old\n The length of the old sequence, used to compute tau_old.\n diff_pos\n Difference between concordant and non-concordant pairs.\n tau_old\n Kendall rank correlation of the old sequence.\n \"\"\"\n return 2.0 / (len_old + 1) * (float(diff_pos) / len_old - tau_old)\n\n def _kendall_tau_subtract(self, len_old: int, diff_neg: int, tau_old: float):\n \"\"\"Compute Kendall tau delta.\n\n The new sequence has length len_old - 1.\n\n Parameters\n ----------\n len_old\n The length of the old sequence, used to compute tau_old.\n diff_neg\n Difference between concordant and non-concordant pairs.\n tau_old\n Kendall rank correlation of the old sequence.\n \"\"\"\n return 2.0 / (len_old - 2) * (-float(diff_neg) / (len_old - 1) + tau_old)\n\n def _kendall_tau_diff(self, a: np.ndarray, b: np.ndarray, i) -> tuple[int, int]:\n \"\"\"Compute difference in concordance of pairs in split sequences.\n\n Consider splitting a and b at index i.\n\n Parameters\n ----------\n a\n ?\n b\n ?\n\n Returns\n -------\n diff_pos\n Difference between concordant pairs for both subsequences.\n diff_neg\n Difference between non-concordant pairs for both subsequences.\n \"\"\"\n # compute ordering relation of the single points a[i] and b[i]\n # with all previous points of the sequences a and b, respectively\n a_pos = np.zeros(a[:i].size, dtype=int)\n a_pos[a[:i] > a[i]] = 1\n a_pos[a[:i] < a[i]] = -1\n b_pos = np.zeros(b[:i].size, dtype=int)\n b_pos[b[:i] > b[i]] = 1\n b_pos[b[:i] < b[i]] = -1\n diff_pos = np.dot(a_pos, b_pos).astype(float)\n\n # compute ordering relation of the single points a[i] and b[i]\n # with all later points of the sequences\n a_neg = np.zeros(a[i:].size, dtype=int)\n a_neg[a[i:] > a[i]] = 1\n a_neg[a[i:] < a[i]] = -1\n b_neg = np.zeros(b[i:].size, dtype=int)\n b_neg[b[i:] > b[i]] = 1\n b_neg[b[i:] < b[i]] = -1\n diff_neg = np.dot(a_neg, b_neg)\n\n return diff_pos, diff_neg\n\n\nfrom __future__ import annotations\n\nimport random\nfrom importlib.util import find_spec\nfrom typing import TYPE_CHECKING, Literal, get_args\n\nimport numpy as np\n\nfrom .. import _utils\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._utils import _choose_graph\nfrom ._utils import get_init_pos_from_paga\n\nif TYPE_CHECKING:\n from typing import LiteralString, TypeVar\n\n from anndata import AnnData\n from scipy.sparse import spmatrix\n\n from .._utils import AnyRandom\n\n S = TypeVar(\"S\", bound=LiteralString)\n\n\n_Layout = Literal[\"fr\", \"drl\", \"kk\", \"grid_fr\", \"lgl\", \"rt\", \"rt_circular\", \"fa\"]\n_LAYOUTS = get_args(_Layout)\n\n\n@old_positionals(\n \"init_pos\",\n \"root\",\n \"random_state\",\n \"n_jobs\",\n \"adjacency\",\n \"key_added_ext\",\n \"neighbors_key\",\n \"obsp\",\n \"copy\",\n)\ndef draw_graph(\n adata: AnnData,\n layout: _Layout = \"fa\",\n *,\n init_pos: str | bool | None = None,\n root: int | None = None,\n random_state: AnyRandom = 0,\n n_jobs: int | None = None,\n adjacency: spmatrix | None = None,\n key_added_ext: str | None = None,\n neighbors_key: str | None = None,\n obsp: str | None = None,\n copy: bool = False,\n **kwds,\n) -> AnnData | None:\n \"\"\"\\\n Force-directed graph drawing :cite:p:`Islam2011,Jacomy2014,Chippada2018`.\n\n An alternative to tSNE that often preserves the topology of the data\n better. This requires to run :func:`~scanpy.pp.neighbors`, first.\n\n The default layout ('fa', `ForceAtlas2`, :cite:t:`Jacomy2014`) uses the package |fa2-modified|_\n :cite:p:`Chippada2018`, which can be installed via `pip install fa2-modified`.\n\n `Force-directed graph drawing`_ describes a class of long-established\n algorithms for visualizing graphs.\n It has been suggested for visualizing single-cell data by :cite:t:`Islam2011`.\n Many other layouts as implemented in igraph :cite:p:`Csardi2006` are available.\n Similar approaches have been used by :cite:t:`Zunder2015` or :cite:t:`Weinreb2017`.\n\n .. |fa2-modified| replace:: `fa2-modified`\n .. _fa2-modified: https://github.com/AminAlam/fa2_modified\n .. _Force-directed graph drawing: https://en.wikipedia.org/wiki/Force-directed_graph_drawing\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n layout\n 'fa' (`ForceAtlas2`) or any valid `igraph layout\n `__. Of particular interest\n are 'fr' (Fruchterman Reingold), 'grid_fr' (Grid Fruchterman Reingold,\n faster than 'fr'), 'kk' (Kamadi Kawai', slower than 'fr'), 'lgl' (Large\n Graph, very fast), 'drl' (Distributed Recursive Layout, pretty fast) and\n 'rt' (Reingold Tilford tree layout).\n root\n Root for tree layouts.\n random_state\n For layouts with random initialization like 'fr', change this to use\n different intial states for the optimization. If `None`, no seed is set.\n adjacency\n Sparse adjacency matrix of the graph, defaults to neighbors connectivities.\n key_added_ext\n By default, append `layout`.\n proceed\n Continue computation, starting off with 'X_draw_graph_`layout`'.\n init_pos\n `'paga'`/`True`, `None`/`False`, or any valid 2d-`.obsm` key.\n Use precomputed coordinates for initialization.\n If `False`/`None` (the default), initialize randomly.\n neighbors_key\n If not specified, draw_graph looks .obsp['connectivities'] for connectivities\n (default storage place for pp.neighbors).\n If specified, draw_graph looks\n .obsp[.uns[neighbors_key]['connectivities_key']] for connectivities.\n obsp\n Use .obsp[obsp] as adjacency. You can't specify both\n `obsp` and `neighbors_key` at the same time.\n copy\n Return a copy instead of writing to adata.\n **kwds\n Parameters of chosen igraph layout. See e.g.\n :meth:`~igraph.GraphBase.layout_fruchterman_reingold` :cite:p:`Fruchterman1991`.\n One of the most important ones is `maxiter`.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.obsm['X_draw_graph_[layout | key_added_ext]']` : :class:`numpy.ndarray` (dtype `float`)\n Coordinates of graph layout. E.g. for `layout='fa'` (the default),\n the field is called `'X_draw_graph_fa'`. `key_added_ext` overwrites `layout`.\n `adata.uns['draw_graph']`: :class:`dict`\n `draw_graph` parameters.\n \"\"\"\n start = logg.info(f\"drawing single-cell graph using layout {layout!r}\")\n if layout not in _LAYOUTS:\n raise ValueError(f\"Provide a valid layout, one of {_LAYOUTS}.\")\n adata = adata.copy() if copy else adata\n if adjacency is None:\n adjacency = _choose_graph(adata, obsp, neighbors_key)\n # init coordinates\n if init_pos in adata.obsm:\n init_coords = adata.obsm[init_pos]\n elif init_pos == \"paga\" or init_pos:\n init_coords = get_init_pos_from_paga(\n adata,\n adjacency,\n random_state=random_state,\n neighbors_key=neighbors_key,\n obsp=obsp,\n )\n else:\n np.random.seed(random_state)\n init_coords = np.random.random((adjacency.shape[0], 2))\n layout = coerce_fa2_layout(layout)\n # actual drawing\n if layout == \"fa\":\n positions = np.array(fa2_positions(adjacency, init_coords, **kwds))\n else:\n # igraph doesn't use numpy seed\n random.seed(random_state)\n\n g = _utils.get_igraph_from_adjacency(adjacency)\n if layout in {\"fr\", \"drl\", \"kk\", \"grid_fr\"}:\n ig_layout = g.layout(layout, seed=init_coords.tolist(), **kwds)\n elif \"rt\" in layout:\n if root is not None:\n root = [root]\n ig_layout = g.layout(layout, root=root, **kwds)\n else:\n ig_layout = g.layout(layout, **kwds)\n positions = np.array(ig_layout.coords)\n adata.uns[\"draw_graph\"] = {}\n adata.uns[\"draw_graph\"][\"params\"] = dict(layout=layout, random_state=random_state)\n key_added = f\"X_draw_graph_{key_added_ext or layout}\"\n adata.obsm[key_added] = positions\n logg.info(\n \" finished\",\n time=start,\n deep=f\"added\\n {key_added!r}, graph_drawing coordinates (adata.obsm)\",\n )\n return adata if copy else None\n\n\ndef fa2_positions(\n adjacency: spmatrix | np.ndarray, init_coords: np.ndarray, **kwds\n) -> list[tuple[float, float]]:\n from fa2_modified import ForceAtlas2\n\n forceatlas2 = ForceAtlas2(\n # Behavior alternatives\n outboundAttractionDistribution=False, # Dissuade hubs\n linLogMode=False, # NOT IMPLEMENTED\n adjustSizes=False, # Prevent overlap (NOT IMPLEMENTED)\n edgeWeightInfluence=1.0,\n # Performance\n jitterTolerance=1.0, # Tolerance\n barnesHutOptimize=True,\n barnesHutTheta=1.2,\n multiThreaded=False, # NOT IMPLEMENTED\n # Tuning\n scalingRatio=2.0,\n strongGravityMode=False,\n gravity=1.0,\n # Log\n verbose=False,\n )\n if \"maxiter\" in kwds:\n iterations = kwds[\"maxiter\"]\n elif \"iterations\" in kwds:\n iterations = kwds[\"iterations\"]\n else:\n iterations = 500\n return forceatlas2.forceatlas2(adjacency, pos=init_coords, iterations=iterations)\n\n\ndef coerce_fa2_layout(layout: S) -> S | Literal[\"fa\", \"fr\"]:\n # see whether fa2 is installed\n if layout != \"fa\":\n return layout\n\n if find_spec(\"fa2_modified\") is None:\n logg.warning(\n \"Package 'fa2-modified' is not installed, falling back to layout 'fr'.\"\n \"To use the faster and better ForceAtlas2 layout, \"\n \"install package 'fa2-modified' (`pip install fa2-modified`).\"\n )\n return \"fr\"\n\n return \"fa\"\n\n\n# Author: T. Callies\n#\n\"\"\"\\\nThis modules provides all non-visualization tools for advanced gene ranking and exploration of genes\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport pandas as pd\nfrom scipy.sparse import issparse\nfrom sklearn import metrics\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._utils import select_groups\n\nif TYPE_CHECKING:\n from collections.abc import Collection\n from typing import Literal\n\n from anndata import AnnData\n\n\n@old_positionals(\"group\", \"n_genes\", \"data\", \"method\", \"annotation_key\")\ndef correlation_matrix(\n adata: AnnData,\n name_list: Collection[str] | None = None,\n groupby: str | None = None,\n *,\n group: int | None = None,\n n_genes: int = 20,\n data: Literal[\"Complete\", \"Group\", \"Rest\"] = \"Complete\",\n method: Literal[\"pearson\", \"kendall\", \"spearman\"] = \"pearson\",\n annotation_key: str | None = None,\n) -> None:\n \"\"\"\\\n Calculate correlation matrix.\n\n Calculate a correlation matrix for genes strored in sample annotation\n using :func:`~scanpy.tl.rank_genes_groups`.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n name_list\n Takes a list of genes for which to calculate the correlation matrix\n groupby\n If no name list is passed, genes are selected from the\n results of rank_gene_groups. Then this is the key of the sample grouping to consider.\n Note that in this case also a group index has to be specified.\n group\n Group index for which the correlation matrix for top_ranked genes should be calculated.\n Currently only int is supported, will change very soon\n n_genes\n For how many genes to calculate correlation matrix? If specified, cuts the name list\n (in whatever order it is passed).\n data\n At the moment, this is only relevant for the case that name_list is drawn from rank_gene_groups results.\n If specified, collects mask for the called group and then takes only those cells specified.\n If 'Complete', calculate correlation using full data\n If 'Group', calculate correlation within the selected group.\n If 'Rest', calculate corrlation for everything except the group\n method\n Which kind of correlation coefficient to use\n\n pearson\n standard correlation coefficient\n kendall\n Kendall Tau correlation coefficient\n spearman\n Spearman rank correlation\n annotation_key\n Allows to define the name of the anndata entry where results are stored.\n \"\"\"\n\n # TODO: At the moment, only works for int identifiers\n\n # If no genes are passed, selects ranked genes from sample annotation.\n # At the moment, only calculate one table (Think about what comes next)\n if name_list is None:\n name_list = list()\n for j, k in enumerate(adata.uns[\"rank_genes_groups_gene_names\"]):\n if j >= n_genes:\n break\n name_list.append(adata.uns[\"rank_genes_groups_gene_names\"][j][group])\n else:\n if len(name_list) > n_genes:\n name_list = name_list[0:n_genes]\n\n # If special method (later) , truncate\n adata_relevant = adata[:, name_list]\n # This line just makes group_mask access easier. Nothing else but 'all' will stand here.\n groups = \"all\"\n if data == \"Complete\" or groupby is None:\n if issparse(adata_relevant.X):\n Data_array = adata_relevant.X.todense()\n else:\n Data_array = adata_relevant.X\n else:\n # get group_mask\n groups_order, groups_masks = select_groups(adata, groups, groupby)\n if data == \"Group\":\n if issparse(adata_relevant.X):\n Data_array = adata_relevant.X[groups_masks[group], :].todense()\n else:\n Data_array = adata_relevant.X[groups_masks[group], :]\n elif data == \"Rest\":\n if issparse(adata_relevant.X):\n Data_array = adata_relevant.X[~groups_masks[group], :].todense()\n else:\n Data_array = adata_relevant.X[~groups_masks[group], :]\n else:\n logg.error(\"data argument should be either or or \")\n\n # Distinguish between sparse and non-sparse data\n\n DF_array = pd.DataFrame(Data_array, columns=name_list)\n cor_table = DF_array.corr(method=method)\n if annotation_key is None:\n if groupby is None:\n adata.uns[\"Correlation_matrix\"] = cor_table\n else:\n adata.uns[\"Correlation_matrix\" + groupby + str(group)] = cor_table\n else:\n adata.uns[annotation_key] = cor_table\n\n\ndef ROC_AUC_analysis(\n adata: AnnData,\n groupby: str,\n group: str | None = None,\n n_genes: int = 100,\n):\n \"\"\"\\\n Calculate correlation matrix.\n\n Calculate a correlation matrix for genes strored in sample annotation\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n groupby\n The key of the sample grouping to consider.\n group\n Group name or index for which the correlation matrix for top ranked\n genes should be calculated.\n If no parameter is passed, ROC/AUC is calculated for all groups\n n_genes\n For how many genes to calculate ROC and AUC. If no parameter is passed,\n calculation is done for all stored top ranked genes.\n \"\"\"\n if group is None:\n pass\n # TODO: Loop over all groups instead of just taking one.\n\n # Assume group takes an int value for one group for the moment.\n name_list = list()\n for j, k in enumerate(adata.uns[\"rank_genes_groups_gene_names\"]):\n if j >= n_genes:\n break\n name_list.append(adata.uns[\"rank_genes_groups_gene_names\"][j][group])\n\n # TODO: For the moment, see that everything works for comparison against the rest. Resolve issues later.\n groups = \"all\"\n groups_order, groups_masks = select_groups(adata, groups, groupby)\n\n # Use usual convention, better for looping later.\n mask = groups_masks[group]\n\n # TODO: Allow for sample weighting requires better mask access... later\n\n # We store calculated data in dict, access it via dict to dict. Check if this is the best way.\n fpr = {}\n tpr = {}\n thresholds = {}\n roc_auc = {}\n y_true = mask\n for i, j in enumerate(name_list):\n vec = adata[:, [j]].X\n y_score = vec.todense() if issparse(vec) else vec\n\n (\n fpr[name_list[i]],\n tpr[name_list[i]],\n thresholds[name_list[i]],\n ) = metrics.roc_curve(\n y_true, y_score, pos_label=None, sample_weight=None, drop_intermediate=False\n )\n roc_auc[name_list[i]] = metrics.auc(fpr[name_list[i]], tpr[name_list[i]])\n adata.uns[\"ROCfpr\" + groupby + str(group)] = fpr\n adata.uns[\"ROCtpr\" + groupby + str(group)] = tpr\n adata.uns[\"ROCthresholds\" + groupby + str(group)] = thresholds\n adata.uns[\"ROC_AUC\" + groupby + str(group)] = roc_auc\n\n\ndef subsampled_estimates(mask, mask_rest=None, precision=0.01, probability=0.99):\n # Simple method that can be called by rank_gene_group. It uses masks that have been passed to the function and\n # calculates how much has to be subsampled in order to reach a certain precision with a certain probability\n # Then it subsamples for mask, mask rest\n # Since convergence speed varies, we take the slower one, i.e. the variance. This might have future speed-up\n # potential\n if mask_rest is None:\n mask_rest = ~mask\n # TODO: DO precision calculation for mean variance shared\n\n # TODO: Subsample\n\n\ndef dominated_ROC_elimination(adata, grouby):\n # This tool has the purpose to take a set of genes (possibly already pre-selected) and analyze AUC.\n # Those and only those are eliminated who are dominated completely\n # TODO: Potentially (But not till tomorrow), this can be adapted to only consider the AUC in the given\n # TODO: optimization frame\n pass\n\n\ndef _gene_preselection(adata, mask, thresholds):\n # This tool serves to\n # It is not thought to be addressed directly but rather using rank_genes_group or ROC analysis or comparable\n # TODO: Pass back a truncated adata object with only those genes that fullfill thresholding criterias\n # This function should be accessible by both rank_genes_groups and ROC_curve analysis\n pass\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING, NamedTuple\n\nimport numpy as np\nimport scipy as sp\nfrom scipy.sparse.csgraph import minimum_spanning_tree\n\nfrom .. import _utils\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom ..neighbors import Neighbors\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from anndata import AnnData\n\n_AVAIL_MODELS = {\"v1.0\", \"v1.2\"}\n\n\n@old_positionals(\"use_rna_velocity\", \"model\", \"neighbors_key\", \"copy\")\ndef paga(\n adata: AnnData,\n groups: str | None = None,\n *,\n use_rna_velocity: bool = False,\n model: Literal[\"v1.2\", \"v1.0\"] = \"v1.2\",\n neighbors_key: str | None = None,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Mapping out the coarse-grained connectivity structures of complex manifolds :cite:p:`Wolf2019`.\n\n By quantifying the connectivity of partitions (groups, clusters) of the\n single-cell graph, partition-based graph abstraction (PAGA) generates a much\n simpler abstracted graph (*PAGA graph*) of partitions, in which edge weights\n represent confidence in the presence of connections. By thresholding this\n confidence in :func:`~scanpy.pl.paga`, a much simpler representation of the\n manifold data is obtained, which is nonetheless faithful to the topology of\n the manifold.\n\n The confidence should be interpreted as the ratio of the actual versus the\n expected value of connections under the null model of randomly connecting\n partitions. We do not provide a p-value as this null model does not\n precisely capture what one would consider \"connected\" in real data, hence it\n strongly overestimates the expected value. See an extensive discussion of\n this in :cite:t:`Wolf2019`.\n\n .. note::\n Note that you can use the result of :func:`~scanpy.pl.paga` in\n :func:`~scanpy.tl.umap` and :func:`~scanpy.tl.draw_graph` via\n `init_pos='paga'` to get single-cell embeddings that are typically more\n faithful to the global topology.\n\n Parameters\n ----------\n adata\n An annotated data matrix.\n groups\n Key for categorical in `adata.obs`. You can pass your predefined groups\n by choosing any categorical annotation of observations. Default:\n The first present key of `'leiden'` or `'louvain'`.\n use_rna_velocity\n Use RNA velocity to orient edges in the abstracted graph and estimate\n transitions. Requires that `adata.uns` contains a directed single-cell\n graph with key `['velocity_graph']`. This feature might be subject\n to change in the future.\n model\n The PAGA connectivity model.\n neighbors_key\n If not specified, paga looks `.uns['neighbors']` for neighbors settings\n and `.obsp['connectivities']`, `.obsp['distances']` for connectivities and\n distances respectively (default storage places for `pp.neighbors`).\n If specified, paga looks `.uns[neighbors_key]` for neighbors settings and\n `.obsp[.uns[neighbors_key]['connectivities_key']]`,\n `.obsp[.uns[neighbors_key]['distances_key']]` for connectivities and distances\n respectively.\n copy\n Copy `adata` before computation and return a copy. Otherwise, perform\n computation inplace and return `None`.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.uns['connectivities']` : :class:`numpy.ndarray` (dtype `float`)\n The full adjacency matrix of the abstracted graph, weights correspond to\n confidence in the connectivities of partitions.\n `adata.uns['connectivities_tree']` : :class:`scipy.sparse.csr_matrix` (dtype `float`)\n The adjacency matrix of the tree-like subgraph that best explains\n the topology.\n\n Notes\n -----\n Together with a random walk-based distance measure\n (e.g. :func:`scanpy.tl.dpt`) this generates a partial coordinatization of\n data useful for exploring and explaining its variation.\n\n .. currentmodule:: scanpy\n\n See Also\n --------\n pl.paga\n pl.paga_path\n pl.paga_compare\n \"\"\"\n check_neighbors = \"neighbors\" if neighbors_key is None else neighbors_key\n if check_neighbors not in adata.uns:\n raise ValueError(\n \"You need to run `pp.neighbors` first to compute a neighborhood graph.\"\n )\n if groups is None:\n for k in (\"leiden\", \"louvain\"):\n if k in adata.obs.columns:\n groups = k\n break\n if groups is None:\n raise ValueError(\n \"You need to run `tl.leiden` or `tl.louvain` to compute \"\n \"community labels, or specify `groups='an_existing_key'`\"\n )\n elif groups not in adata.obs.columns:\n raise KeyError(f\"`groups` key {groups!r} not found in `adata.obs`.\")\n\n adata = adata.copy() if copy else adata\n _utils.sanitize_anndata(adata)\n start = logg.info(\"running PAGA\")\n paga = PAGA(adata, groups, model=model, neighbors_key=neighbors_key)\n # only add if not present\n if \"paga\" not in adata.uns:\n adata.uns[\"paga\"] = {}\n if not use_rna_velocity:\n paga.compute_connectivities()\n adata.uns[\"paga\"][\"connectivities\"] = paga.connectivities\n adata.uns[\"paga\"][\"connectivities_tree\"] = paga.connectivities_tree\n # adata.uns['paga']['expected_n_edges_random'] = paga.expected_n_edges_random\n adata.uns[groups + \"_sizes\"] = np.array(paga.ns)\n else:\n paga.compute_transitions()\n adata.uns[\"paga\"][\"transitions_confidence\"] = paga.transitions_confidence\n # adata.uns['paga']['transitions_ttest'] = paga.transitions_ttest\n adata.uns[\"paga\"][\"groups\"] = groups\n logg.info(\n \" finished\",\n time=start,\n deep=\"added\\n\"\n + (\n \" 'paga/transitions_confidence', connectivities adjacency (adata.uns)\"\n # \" 'paga/transitions_ttest', t-test on transitions (adata.uns)\"\n if use_rna_velocity\n else \" 'paga/connectivities', connectivities adjacency (adata.uns)\\n\"\n \" 'paga/connectivities_tree', connectivities subtree (adata.uns)\"\n ),\n )\n return adata if copy else None\n\n\nclass PAGA:\n def __init__(self, adata, groups, model=\"v1.2\", neighbors_key=None):\n assert groups in adata.obs.columns\n self._adata = adata\n self._neighbors = Neighbors(adata, neighbors_key=neighbors_key)\n self._model = model\n self._groups_key = groups\n\n def compute_connectivities(self):\n if self._model == \"v1.2\":\n return self._compute_connectivities_v1_2()\n elif self._model == \"v1.0\":\n return self._compute_connectivities_v1_0()\n else:\n raise ValueError(\n f\"`model` {self._model} needs to be one of {_AVAIL_MODELS}.\"\n )\n\n def _compute_connectivities_v1_2(self):\n import igraph\n\n ones = self._neighbors.distances.copy()\n ones.data = np.ones(len(ones.data))\n # should be directed if we deal with distances\n g = _utils.get_igraph_from_adjacency(ones, directed=True)\n vc = igraph.VertexClustering(\n g, membership=self._adata.obs[self._groups_key].cat.codes.values\n )\n ns = vc.sizes()\n n = sum(ns)\n es_inner_cluster = [vc.subgraph(i).ecount() for i in range(len(ns))]\n cg = vc.cluster_graph(combine_edges=\"sum\")\n inter_es = cg.get_adjacency_sparse(attribute=\"weight\")\n es = np.array(es_inner_cluster) + inter_es.sum(axis=1).A1\n inter_es = inter_es + inter_es.T # \\epsilon_i + \\epsilon_j\n connectivities = inter_es.copy()\n expected_n_edges = inter_es.copy()\n inter_es = inter_es.tocoo()\n for i, j, v in zip(inter_es.row, inter_es.col, inter_es.data):\n expected_random_null = (es[i] * ns[j] + es[j] * ns[i]) / (n - 1)\n scaled_value = v / expected_random_null if expected_random_null != 0 else 1\n if scaled_value > 1:\n scaled_value = 1\n connectivities[i, j] = scaled_value\n expected_n_edges[i, j] = expected_random_null\n # set attributes\n self.ns = ns\n self.expected_n_edges_random = expected_n_edges\n self.connectivities = connectivities\n self.connectivities_tree = self._get_connectivities_tree_v1_2()\n return inter_es.tocsr(), connectivities\n\n def _compute_connectivities_v1_0(self):\n import igraph\n\n ones = self._neighbors.connectivities.copy()\n ones.data = np.ones(len(ones.data))\n g = _utils.get_igraph_from_adjacency(ones)\n vc = igraph.VertexClustering(\n g, membership=self._adata.obs[self._groups_key].cat.codes.values\n )\n ns = vc.sizes()\n cg = vc.cluster_graph(combine_edges=\"sum\")\n inter_es = cg.get_adjacency_sparse(attribute=\"weight\") / 2\n connectivities = inter_es.copy()\n inter_es = inter_es.tocoo()\n n_neighbors_sq = self._neighbors.n_neighbors**2\n for i, j, v in zip(inter_es.row, inter_es.col, inter_es.data):\n # have n_neighbors**2 inside sqrt for backwards compat\n geom_mean_approx_knn = np.sqrt(n_neighbors_sq * ns[i] * ns[j])\n scaled_value = v / geom_mean_approx_knn if geom_mean_approx_knn != 0 else 1\n connectivities[i, j] = scaled_value\n # set attributes\n self.ns = ns\n self.connectivities = connectivities\n self.connectivities_tree = self._get_connectivities_tree_v1_0(inter_es)\n return inter_es.tocsr(), connectivities\n\n def _get_connectivities_tree_v1_2(self):\n inverse_connectivities = self.connectivities.copy()\n inverse_connectivities.data = 1.0 / inverse_connectivities.data\n connectivities_tree = minimum_spanning_tree(inverse_connectivities)\n connectivities_tree_indices = [\n connectivities_tree[i].nonzero()[1]\n for i in range(connectivities_tree.shape[0])\n ]\n connectivities_tree = sp.sparse.lil_matrix(\n self.connectivities.shape, dtype=float\n )\n for i, neighbors in enumerate(connectivities_tree_indices):\n if len(neighbors) > 0:\n connectivities_tree[i, neighbors] = self.connectivities[i, neighbors]\n return connectivities_tree.tocsr()\n\n def _get_connectivities_tree_v1_0(self, inter_es):\n inverse_inter_es = inter_es.copy()\n inverse_inter_es.data = 1.0 / inverse_inter_es.data\n connectivities_tree = minimum_spanning_tree(inverse_inter_es)\n connectivities_tree_indices = [\n connectivities_tree[i].nonzero()[1]\n for i in range(connectivities_tree.shape[0])\n ]\n connectivities_tree = sp.sparse.lil_matrix(inter_es.shape, dtype=float)\n for i, neighbors in enumerate(connectivities_tree_indices):\n if len(neighbors) > 0:\n connectivities_tree[i, neighbors] = self.connectivities[i, neighbors]\n return connectivities_tree.tocsr()\n\n def compute_transitions(self):\n vkey = \"velocity_graph\"\n if vkey not in self._adata.uns:\n if \"velocyto_transitions\" in self._adata.uns:\n self._adata.uns[vkey] = self._adata.uns[\"velocyto_transitions\"]\n logg.debug(\n \"The key 'velocyto_transitions' has been changed to 'velocity_graph'.\"\n )\n else:\n raise ValueError(\n \"The passed AnnData needs to have an `uns` annotation \"\n \"with key 'velocity_graph' - a sparse matrix from RNA velocity.\"\n )\n if self._adata.uns[vkey].shape != (self._adata.n_obs, self._adata.n_obs):\n raise ValueError(\n f\"The passed 'velocity_graph' have shape {self._adata.uns[vkey].shape} \"\n f\"but shoud have shape {(self._adata.n_obs, self._adata.n_obs)}\"\n )\n # restore this at some point\n # if 'expected_n_edges_random' not in self._adata.uns['paga']:\n # raise ValueError(\n # 'Before running PAGA with `use_rna_velocity=True`, run it with `False`.')\n import igraph\n\n g = _utils.get_igraph_from_adjacency(\n self._adata.uns[vkey].astype(\"bool\"),\n directed=True,\n )\n vc = igraph.VertexClustering(\n g, membership=self._adata.obs[self._groups_key].cat.codes.values\n )\n # set combine_edges to False if you want self loops\n cg_full = vc.cluster_graph(combine_edges=\"sum\")\n transitions = cg_full.get_adjacency_sparse(attribute=\"weight\")\n transitions = transitions - transitions.T\n transitions_conf = transitions.copy()\n transitions = transitions.tocoo()\n total_n = self._neighbors.n_neighbors * np.array(vc.sizes())\n # total_n_sum = sum(total_n)\n # expected_n_edges_random = self._adata.uns['paga']['expected_n_edges_random']\n for i, j, v in zip(transitions.row, transitions.col, transitions.data):\n # if expected_n_edges_random[i, j] != 0:\n # # factor 0.5 because of asymmetry\n # reference = 0.5 * expected_n_edges_random[i, j]\n # else:\n # # approximate\n # reference = self._neighbors.n_neighbors * total_n[i] * total_n[j] / total_n_sum\n reference = np.sqrt(total_n[i] * total_n[j])\n transitions_conf[i, j] = 0 if v < 0 else v / reference\n transitions_conf.eliminate_zeros()\n # transpose in order to match convention of stochastic matrices\n # entry ij means transition from j to i\n self.transitions_confidence = transitions_conf.T\n\n def compute_transitions_old(self):\n import igraph\n\n g = _utils.get_igraph_from_adjacency(\n self._adata.uns[\"velocyto_transitions\"],\n directed=True,\n )\n vc = igraph.VertexClustering(\n g, membership=self._adata.obs[self._groups_key].cat.codes.values\n )\n # this stores all single-cell edges in the cluster graph\n cg_full = vc.cluster_graph(combine_edges=False)\n # this is the boolean version that simply counts edges in the clustered graph\n g_bool = _utils.get_igraph_from_adjacency(\n self._adata.uns[\"velocyto_transitions\"].astype(\"bool\"),\n directed=True,\n )\n vc_bool = igraph.VertexClustering(\n g_bool, membership=self._adata.obs[self._groups_key].cat.codes.values\n )\n cg_bool = vc_bool.cluster_graph(combine_edges=\"sum\") # collapsed version\n transitions = cg_bool.get_adjacency_sparse(attribute=\"weight\")\n total_n = self._neighbors.n_neighbors * np.array(vc_bool.sizes())\n transitions_ttest = transitions.copy()\n transitions_confidence = transitions.copy()\n from scipy.stats import ttest_1samp\n\n for i in range(transitions.shape[0]):\n neighbors = transitions[i].nonzero()[1]\n for j in neighbors:\n forward = cg_full.es.select(_source=i, _target=j)[\"weight\"]\n backward = cg_full.es.select(_source=j, _target=i)[\"weight\"]\n # backward direction: add minus sign\n values = np.array(list(forward) + list(-np.array(backward)))\n # require some minimal number of observations\n if len(values) < 5:\n transitions_ttest[i, j] = 0\n transitions_ttest[j, i] = 0\n transitions_confidence[i, j] = 0\n transitions_confidence[j, i] = 0\n continue\n t, prob = ttest_1samp(values, 0.0)\n if t > 0:\n # number of outgoing edges greater than number of ingoing edges\n # i.e., transition from i to j\n transitions_ttest[i, j] = -np.log10(max(prob, 1e-10))\n transitions_ttest[j, i] = 0\n else:\n transitions_ttest[j, i] = -np.log10(max(prob, 1e-10))\n transitions_ttest[i, j] = 0\n # geom_mean\n geom_mean = np.sqrt(total_n[i] * total_n[j])\n diff = (len(forward) - len(backward)) / geom_mean\n if diff > 0:\n transitions_confidence[i, j] = diff\n transitions_confidence[j, i] = 0\n else:\n transitions_confidence[j, i] = -diff\n transitions_confidence[i, j] = 0\n transitions_ttest.eliminate_zeros()\n transitions_confidence.eliminate_zeros()\n # transpose in order to match convention of stochastic matrices\n # entry ij means transition from j to i\n self.transitions_ttest = transitions_ttest.T\n self.transitions_confidence = transitions_confidence.T\n\n\ndef paga_degrees(adata: AnnData) -> list[int]:\n \"\"\"Compute the degree of each node in the abstracted graph.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n\n Returns\n -------\n List of degrees for each node.\n \"\"\"\n import networkx as nx\n\n g = nx.Graph(adata.uns[\"paga\"][\"connectivities\"])\n degrees = [d for _, d in g.degree(weight=\"weight\")]\n return degrees\n\n\ndef paga_expression_entropies(adata: AnnData) -> list[float]:\n \"\"\"Compute the median expression entropy for each node-group.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n\n Returns\n -------\n Entropies of median expressions for each node.\n \"\"\"\n from scipy.stats import entropy\n\n groups_order, groups_masks = _utils.select_groups(\n adata, key=adata.uns[\"paga\"][\"groups\"]\n )\n entropies = []\n for mask in groups_masks:\n X_mask = adata.X[mask].todense()\n x_median = np.nanmedian(X_mask, axis=1, overwrite_input=True)\n x_probs = (x_median - np.nanmin(x_median)) / (\n np.nanmax(x_median) - np.nanmin(x_median)\n )\n entropies.append(entropy(x_probs))\n return entropies\n\n\nclass PAGAComparePathsResult(NamedTuple):\n frac_steps: float\n n_steps: int\n frac_paths: float\n n_paths: int\n\n\ndef paga_compare_paths(\n adata1: AnnData,\n adata2: AnnData,\n adjacency_key: str = \"connectivities\",\n adjacency_key2: str | None = None,\n) -> PAGAComparePathsResult:\n \"\"\"Compare paths in abstracted graphs in two datasets.\n\n Compute the fraction of consistent paths between leafs, a measure for the\n topological similarity between graphs.\n\n By increasing the verbosity to level 4 and 5, the paths that do not agree\n and the paths that agree are written to the output, respectively.\n\n The PAGA \"groups key\" needs to be the same in both objects.\n\n Parameters\n ----------\n adata1, adata2\n Annotated data matrices to compare.\n adjacency_key\n Key for indexing the adjacency matrices in `.uns['paga']` to be used in\n adata1 and adata2.\n adjacency_key2\n If provided, used for adata2.\n\n Returns\n -------\n NamedTuple with attributes\n\n frac_steps\n fraction of consistent steps\n n_steps\n total number of steps in paths\n frac_paths\n Fraction of consistent paths\n n_paths\n Number of paths\n \"\"\"\n import networkx as nx\n\n g1 = nx.Graph(adata1.uns[\"paga\"][adjacency_key])\n g2 = nx.Graph(\n adata2.uns[\"paga\"][\n adjacency_key2 if adjacency_key2 is not None else adjacency_key\n ]\n )\n leaf_nodes1 = [str(x) for x in g1.nodes() if g1.degree(x) == 1]\n logg.debug(f\"leaf nodes in graph 1: {leaf_nodes1}\")\n paga_groups = adata1.uns[\"paga\"][\"groups\"]\n asso_groups1 = _utils.identify_groups(\n adata1.obs[paga_groups].values,\n adata2.obs[paga_groups].values,\n )\n asso_groups2 = _utils.identify_groups(\n adata2.obs[paga_groups].values,\n adata1.obs[paga_groups].values,\n )\n orig_names1 = adata1.obs[paga_groups].cat.categories\n orig_names2 = adata2.obs[paga_groups].cat.categories\n\n import itertools\n\n n_steps = 0\n n_agreeing_steps = 0\n n_paths = 0\n n_agreeing_paths = 0\n # loop over all pairs of leaf nodes in the reference adata1\n for r, s in itertools.combinations(leaf_nodes1, r=2):\n r2, s2 = asso_groups1[r][0], asso_groups1[s][0]\n on1_g1, on2_g1 = (orig_names1[int(i)] for i in [r, s])\n on1_g2, on2_g2 = (orig_names2[int(i)] for i in [r2, s2])\n logg.debug(\n f\"compare shortest paths between leafs ({on1_g1}, {on2_g1}) \"\n f\"in graph1 and ({on1_g2}, {on2_g2}) in graph2:\"\n )\n try:\n path1 = [str(x) for x in nx.shortest_path(g1, int(r), int(s))]\n except nx.NetworkXNoPath:\n path1 = None\n try:\n path2 = [str(x) for x in nx.shortest_path(g2, int(r2), int(s2))]\n except nx.NetworkXNoPath:\n path2 = None\n if path1 is None and path2 is None:\n # consistent behavior\n n_paths += 1\n n_agreeing_paths += 1\n n_steps += 1\n n_agreeing_steps += 1\n logg.debug(\"there are no connecting paths in both graphs\")\n continue\n elif path1 is None or path2 is None:\n # non-consistent result\n n_paths += 1\n n_steps += 1\n continue\n if len(path1) >= len(path2):\n path_mapped = [asso_groups1[l] for l in path1]\n path_compare = path2\n path_compare_id = 2\n path_compare_orig_names = [\n [orig_names2[int(s)] for s in l] for l in path_compare\n ]\n path_mapped_orig_names = [\n [orig_names2[int(s)] for s in l] for l in path_mapped\n ]\n else:\n path_mapped = [asso_groups2[l] for l in path2]\n path_compare = path1\n path_compare_id = 1\n path_compare_orig_names = [\n [orig_names1[int(s)] for s in l] for l in path_compare\n ]\n path_mapped_orig_names = [\n [orig_names1[int(s)] for s in l] for l in path_mapped\n ]\n n_agreeing_steps_path = 0\n ip_progress = 0\n for il, l in enumerate(path_compare[:-1]):\n for ip, p in enumerate(path_mapped):\n if (\n ip < ip_progress\n or l not in p\n or not (\n ip + 1 < len(path_mapped)\n and path_compare[il + 1] in path_mapped[ip + 1]\n )\n ):\n continue\n # make sure that a step backward leads us to the same value of l\n # in case we \"jumped\"\n logg.debug(\n f\"found matching step ({l} -> {path_compare_orig_names[il + 1]}) \"\n f\"at position {il} in path{path_compare_id} and position {ip} in path_mapped\"\n )\n consistent_history = True\n for iip in range(ip, ip_progress, -1):\n if l not in path_mapped[iip - 1]:\n consistent_history = False\n if consistent_history:\n # here, we take one step further back (ip_progress - 1); it's implied that this\n # was ok in the previous step\n poss = list(range(ip - 1, ip_progress - 2, -1))\n logg.debug(\n f\" step(s) backward to position(s) {poss} \"\n \"in path_mapped are fine, too: valid step\"\n )\n n_agreeing_steps_path += 1\n ip_progress = ip + 1\n break\n n_steps_path = len(path_compare) - 1\n n_agreeing_steps += n_agreeing_steps_path\n n_steps += n_steps_path\n n_paths += 1\n if n_agreeing_steps_path == n_steps_path:\n n_agreeing_paths += 1\n\n # only for the output, use original names\n path1_orig_names = [orig_names1[int(s)] for s in path1]\n path2_orig_names = [orig_names2[int(s)] for s in path2]\n logg.debug(\n f\" path1 = {path1_orig_names},\\n\"\n f\"path_mapped = {[list(p) for p in path_mapped_orig_names]},\\n\"\n f\" path2 = {path2_orig_names},\\n\"\n f\"-> n_agreeing_steps = {n_agreeing_steps_path} / n_steps = {n_steps_path}.\",\n )\n return PAGAComparePathsResult(\n frac_steps=n_agreeing_steps / n_steps if n_steps > 0 else np.nan,\n n_steps=n_steps if n_steps > 0 else np.nan,\n frac_paths=n_agreeing_paths / n_paths if n_steps > 0 else np.nan,\n n_paths=n_paths if n_steps > 0 else np.nan,\n )\n\n\nfrom __future__ import annotations\n\nfrom collections.abc import MutableMapping\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom packaging.version import Version\nfrom scipy.sparse import issparse\nfrom sklearn.utils import check_random_state\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals, pkg_version\nfrom .._settings import settings\nfrom .._utils import NeighborsView, raise_not_implemented_error_if_backed_type\nfrom .._utils._doctests import doctest_skip\nfrom ..neighbors import FlatTree\n\nif TYPE_CHECKING:\n from collections.abc import Generator, Iterable\n\n from anndata import AnnData\n\n from ..neighbors import RPForestDict\n\nANNDATA_MIN_VERSION = Version(\"0.7rc1\")\n\n\n@old_positionals(\n \"obs\",\n \"embedding_method\",\n \"labeling_method\",\n \"neighbors_key\",\n \"neighbors_key\",\n \"inplace\",\n)\n@doctest_skip(\"illustrative short example but not runnable\")\ndef ingest(\n adata: AnnData,\n adata_ref: AnnData,\n *,\n obs: str | Iterable[str] | None = None,\n embedding_method: str | Iterable[str] = (\"umap\", \"pca\"),\n labeling_method: str = \"knn\",\n neighbors_key: str | None = None,\n inplace: bool = True,\n **kwargs,\n):\n \"\"\"\\\n Map labels and embeddings from reference data to new data.\n\n :doc:`/tutorials/basics/integrating-data-using-ingest`\n\n Integrates embeddings and annotations of an `adata` with a reference dataset\n `adata_ref` through projecting on a PCA (or alternate\n model) that has been fitted on the reference data. The function uses a knn\n classifier for mapping labels and the UMAP package :cite:p:`McInnes2018` for mapping\n the embeddings.\n\n .. note::\n\n We refer to this *asymmetric* dataset integration as *ingesting*\n annotations from reference data to new data. This is different from\n learning a joint representation that integrates both datasets in an\n unbiased way, as CCA (e.g. in Seurat) or a conditional VAE (e.g. in\n scVI) would do.\n\n You need to run :func:`~scanpy.pp.neighbors` on `adata_ref` before\n passing it.\n\n Parameters\n ----------\n adata\n The annotated data matrix of shape `n_obs` × `n_vars`. Rows correspond\n to cells and columns to genes. This is the dataset without labels and\n embeddings.\n adata_ref\n The annotated data matrix of shape `n_obs` × `n_vars`. Rows correspond\n to cells and columns to genes.\n Variables (`n_vars` and `var_names`) of `adata_ref` should be the same\n as in `adata`.\n This is the dataset with labels and embeddings\n which need to be mapped to `adata`.\n obs\n Labels' keys in `adata_ref.obs` which need to be mapped to `adata.obs`\n (inferred for observation of `adata`).\n embedding_method\n Embeddings in `adata_ref` which need to be mapped to `adata`.\n The only supported values are 'umap' and 'pca'.\n labeling_method\n The method to map labels in `adata_ref.obs` to `adata.obs`.\n The only supported value is 'knn'.\n neighbors_key\n If not specified, ingest looks adata_ref.uns['neighbors']\n for neighbors settings and adata_ref.obsp['distances'] for\n distances (default storage places for pp.neighbors).\n If specified, ingest looks adata_ref.uns[neighbors_key] for\n neighbors settings and\n adata_ref.obsp[adata_ref.uns[neighbors_key]['distances_key']] for distances.\n inplace\n Only works if `return_joint=False`.\n Add labels and embeddings to the passed `adata` (if `True`)\n or return a copy of `adata` with mapped embeddings and labels.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.obs[obs]` : :class:`pandas.Series` (dtype ``category``)\n Mapped labels.\n `adata.obsm['X_umap' | 'X_pca']` : :class:`numpy.ndarray` (dtype ``float``)\n Mapped embeddings. `'X_umap'` if `embedding_method` is `'umap'`, `'X_pca'` if `embedding_method` is `'pca'`.\n\n Example\n -------\n Call sequence:\n\n >>> import scanpy as sc\n >>> sc.pp.neighbors(adata_ref)\n >>> sc.tl.umap(adata_ref)\n >>> sc.tl.ingest(adata, adata_ref, obs='cell_type')\n \"\"\"\n # anndata version check\n anndata_version = pkg_version(\"anndata\")\n if anndata_version < ANNDATA_MIN_VERSION:\n raise ValueError(\n f\"ingest only works correctly with anndata>={ANNDATA_MIN_VERSION} \"\n f\"(you have {anndata_version}) as prior to {ANNDATA_MIN_VERSION}, \"\n \"`AnnData.concatenate` did not concatenate `.obsm`.\"\n )\n\n start = logg.info(\"running ingest\")\n obs = [obs] if isinstance(obs, str) else obs\n embedding_method = (\n [embedding_method] if isinstance(embedding_method, str) else embedding_method\n )\n labeling_method = (\n [labeling_method] if isinstance(labeling_method, str) else labeling_method\n )\n\n if len(labeling_method) == 1 and len(obs or []) > 1:\n labeling_method = labeling_method * len(obs)\n\n ing = Ingest(adata_ref, neighbors_key)\n ing.fit(adata)\n\n for method in embedding_method:\n ing.map_embedding(method)\n\n if obs is not None:\n ing.neighbors(**kwargs)\n for i, col in enumerate(obs):\n ing.map_labels(col, labeling_method[i])\n\n logg.info(\" finished\", time=start)\n return ing.to_adata(inplace=inplace)\n\n\ndef _rp_forest_generate(\n rp_forest_dict: RPForestDict,\n) -> Generator[FlatTree, None, None]:\n props = FlatTree._fields\n num_trees = len(rp_forest_dict[props[0]][\"start\"]) - 1\n\n for i in range(num_trees):\n tree = []\n for prop in props:\n start = rp_forest_dict[prop][\"start\"][i]\n end = rp_forest_dict[prop][\"start\"][i + 1]\n tree.append(rp_forest_dict[prop][\"data\"][start:end])\n yield FlatTree(*tree)\n\n tree = []\n for prop in props:\n start = rp_forest_dict[prop][\"start\"][num_trees]\n tree.append(rp_forest_dict[prop][\"data\"][start:])\n yield FlatTree(*tree)\n\n\nclass _DimDict(MutableMapping):\n def __init__(self, dim, axis=0, vals=None):\n self._data = {}\n self._dim = dim\n self._axis = axis\n if vals is not None:\n self.update(vals)\n\n def __setitem__(self, key, value):\n if value.shape[self._axis] != self._dim:\n raise ValueError(\n f\"Value passed for key '{key}' is of incorrect shape. \"\n f\"Value has shape {value.shape[self._axis]} \"\n f\"for dimension {self._axis} while \"\n f\"it should have {self._dim}.\"\n )\n self._data[key] = value\n\n def __getitem__(self, key):\n return self._data[key]\n\n def __delitem__(self, key):\n del self._data[key]\n\n def __iter__(self):\n return iter(self._data)\n\n def __len__(self):\n return len(self._data)\n\n def __repr__(self):\n return f\"{type(self).__name__}({self._data})\"\n\n\nclass Ingest:\n \"\"\"\\\n Class to map labels and embeddings from existing data to new data.\n\n You need to run :func:`~scanpy.pp.neighbors` on `adata` before\n initializing Ingest with it.\n\n Parameters\n ----------\n adata : :class:`~anndata.AnnData`\n The annotated data matrix of shape `n_obs` × `n_vars`\n with embeddings and labels.\n \"\"\"\n\n def _init_umap(self, adata):\n from umap import UMAP\n\n self._umap = UMAP(\n metric=self._metric,\n random_state=adata.uns[\"umap\"][\"params\"].get(\"random_state\", 0),\n )\n\n self._umap._initial_alpha = self._umap.learning_rate\n self._umap._raw_data = self._rep\n self._umap.knn_dists = None\n\n self._umap._validate_parameters()\n\n self._umap.embedding_ = adata.obsm[\"X_umap\"]\n self._umap._sparse_data = issparse(self._rep)\n self._umap._small_data = self._rep.shape[0] < 4096\n self._umap._metric_kwds = self._metric_kwds\n\n self._umap._n_neighbors = self._n_neighbors\n self._umap.n_neighbors = self._n_neighbors\n\n self._umap._knn_search_index = self._nnd_idx\n\n self._umap._a = adata.uns[\"umap\"][\"params\"][\"a\"]\n self._umap._b = adata.uns[\"umap\"][\"params\"][\"b\"]\n\n self._umap._input_hash = None\n\n def _init_pynndescent(self, distances):\n from pynndescent import NNDescent\n\n first_col = np.arange(distances.shape[0])[:, None]\n init_indices = np.hstack((first_col, np.stack(distances.tolil().rows)))\n\n self._nnd_idx = NNDescent(\n data=self._rep,\n metric=self._metric,\n metric_kwds=self._metric_kwds,\n n_neighbors=self._n_neighbors,\n init_graph=init_indices,\n random_state=self._neigh_random_state,\n )\n\n # temporary hack for the broken forest storage\n from pynndescent.rp_trees import make_forest\n\n current_random_state = check_random_state(self._nnd_idx.random_state)\n self._nnd_idx._rp_forest = make_forest(\n self._nnd_idx._raw_data,\n self._nnd_idx.n_neighbors,\n self._nnd_idx.n_search_trees,\n self._nnd_idx.leaf_size,\n self._nnd_idx.rng_state,\n current_random_state,\n self._nnd_idx.n_jobs,\n self._nnd_idx._angular_trees,\n )\n\n def _init_neighbors(self, adata, neighbors_key):\n neighbors = NeighborsView(adata, neighbors_key)\n\n self._n_neighbors = neighbors[\"params\"][\"n_neighbors\"]\n\n if \"use_rep\" in neighbors[\"params\"]:\n self._use_rep = neighbors[\"params\"][\"use_rep\"]\n self._rep = adata.X if self._use_rep == \"X\" else adata.obsm[self._use_rep]\n elif \"n_pcs\" in neighbors[\"params\"]:\n self._use_rep = \"X_pca\"\n self._n_pcs = neighbors[\"params\"][\"n_pcs\"]\n self._rep = adata.obsm[\"X_pca\"][:, : self._n_pcs]\n elif adata.n_vars > settings.N_PCS and \"X_pca\" in adata.obsm:\n self._use_rep = \"X_pca\"\n self._rep = adata.obsm[\"X_pca\"][:, : settings.N_PCS]\n self._n_pcs = self._rep.shape[1]\n\n self._metric_kwds = neighbors[\"params\"].get(\"metric_kwds\", {})\n self._metric = neighbors[\"params\"][\"metric\"]\n\n self._neigh_random_state = neighbors[\"params\"].get(\"random_state\", 0)\n self._init_pynndescent(neighbors[\"distances\"])\n\n def _init_pca(self, adata):\n self._pca_centered = adata.uns[\"pca\"][\"params\"][\"zero_center\"]\n self._pca_use_hvg = adata.uns[\"pca\"][\"params\"][\"use_highly_variable\"]\n\n mask = \"highly_variable\"\n if self._pca_use_hvg and mask not in adata.var.columns:\n msg = f\"Did not find `adata.var[{mask!r}']`.\"\n raise ValueError(msg)\n\n if self._pca_use_hvg:\n self._pca_basis = adata.varm[\"PCs\"][adata.var[mask]]\n else:\n self._pca_basis = adata.varm[\"PCs\"]\n\n def __init__(self, adata: AnnData, neighbors_key: str | None = None):\n # assume rep is X if all initializations fail to identify it\n self._rep = adata.X\n self._use_rep = \"X\"\n\n self._n_pcs = None\n\n self._adata_ref = adata\n self._adata_new = None\n\n if \"pca\" in adata.uns:\n self._init_pca(adata)\n\n if neighbors_key is None:\n neighbors_key = \"neighbors\"\n\n if neighbors_key in adata.uns:\n self._init_neighbors(adata, neighbors_key)\n else:\n raise ValueError(\n f'There is no neighbors data in `adata.uns[\"{neighbors_key}\"]`.\\n'\n \"Please run pp.neighbors.\"\n )\n\n if \"X_umap\" in adata.obsm:\n self._init_umap(adata)\n\n self._obsm = None\n self._obs = None\n self._labels = None\n\n self._indices = None\n self._distances = None\n\n def _pca(self, n_pcs=None):\n X = self._adata_new.X\n X = X.toarray() if issparse(X) else X.copy()\n if self._pca_use_hvg:\n X = X[:, self._adata_ref.var[\"highly_variable\"]]\n if self._pca_centered:\n X -= X.mean(axis=0)\n X_pca = np.dot(X, self._pca_basis[:, :n_pcs])\n return X_pca\n\n def _same_rep(self):\n adata = self._adata_new\n if self._n_pcs is not None:\n return self._pca(self._n_pcs)\n if self._use_rep == \"X\":\n return adata.X\n if self._use_rep in adata.obsm:\n return adata.obsm[self._use_rep]\n return adata.X\n\n def fit(self, adata_new):\n \"\"\"\\\n Map `adata_new` to the same representation as `adata`.\n\n This function identifies the representation which was used to\n calculate neighbors in 'adata' and maps `adata_new` to\n this representation.\n Variables (`n_vars` and `var_names`) of `adata_new` should be the same\n as in `adata`.\n\n `adata` refers to the :class:`~anndata.AnnData` object\n that is passed during the initialization of an Ingest instance.\n \"\"\"\n raise_not_implemented_error_if_backed_type(adata_new.X, \"Ingest.fit\")\n ref_var_names = self._adata_ref.var_names.str.upper()\n new_var_names = adata_new.var_names.str.upper()\n\n if not ref_var_names.equals(new_var_names):\n raise ValueError(\n \"Variables in the new adata are different \"\n \"from variables in the reference adata\"\n )\n\n self._obs = pd.DataFrame(index=adata_new.obs.index)\n self._obsm = _DimDict(adata_new.n_obs, axis=0)\n\n self._adata_new = adata_new\n self._obsm[\"rep\"] = self._same_rep()\n\n def neighbors(self, k=None, queue_size=5, epsilon=0.1, random_state=0):\n \"\"\"\\\n Calculate neighbors of `adata_new` observations in `adata`.\n\n This function calculates `k` neighbors in `adata` for\n each observation of `adata_new`.\n \"\"\"\n from umap.umap_ import INT32_MAX, INT32_MIN\n\n random_state = check_random_state(random_state)\n rng_state = random_state.randint(INT32_MIN, INT32_MAX, 3).astype(np.int64)\n\n test = self._obsm[\"rep\"]\n\n if k is None:\n k = self._n_neighbors\n\n self._nnd_idx.search_rng_state = rng_state\n self._indices, self._distances = self._nnd_idx.query(test, k, epsilon)\n\n def _umap_transform(self):\n return self._umap.transform(self._obsm[\"rep\"])\n\n def map_embedding(self, method):\n \"\"\"\\\n Map embeddings of `adata` to `adata_new`.\n\n This function infers embeddings, specified by `method`,\n for `adata_new` from existing embeddings in `adata`.\n `method` can be 'umap' or 'pca'.\n \"\"\"\n if method == \"umap\":\n self._obsm[\"X_umap\"] = self._umap_transform()\n elif method == \"pca\":\n self._obsm[\"X_pca\"] = self._pca()\n else:\n raise NotImplementedError(\n \"Ingest supports only umap and pca embeddings for now.\"\n )\n\n def _knn_classify(self, labels):\n # ensure it's categorical\n cat_array: pd.Series = self._adata_ref.obs[labels].astype(\"category\")\n values = [cat_array.iloc[inds].mode()[0] for inds in self._indices]\n return pd.Categorical(values=values, categories=cat_array.cat.categories)\n\n def map_labels(self, labels, method):\n \"\"\"\\\n Map labels of `adata` to `adata_new`.\n\n This function infers `labels` for `adata_new.obs`\n from existing labels in `adata.obs`.\n `method` can be only 'knn'.\n \"\"\"\n if method == \"knn\":\n self._obs[labels] = self._knn_classify(labels)\n else:\n raise NotImplementedError(\"Ingest supports knn labeling for now.\")\n\n @old_positionals(\"inplace\")\n def to_adata(self, *, inplace: bool = False) -> AnnData | None:\n \"\"\"\\\n Returns `adata_new` with mapped embeddings and labels.\n\n If `inplace=False` returns a copy of `adata_new`\n with mapped embeddings and labels in `obsm` and `obs` correspondingly.\n If `inplace=True` returns nothing and updates `adata_new.obsm`\n and `adata_new.obs` with mapped embeddings and labels.\n \"\"\"\n adata = self._adata_new if inplace else self._adata_new.copy()\n\n adata.obsm.update(self._obsm)\n\n for key in self._obs:\n adata.obs[key] = self._obs[key]\n\n if not inplace:\n return adata\n\n def to_adata_joint(\n self, batch_key=\"batch\", batch_categories=None, index_unique=\"-\"\n ):\n \"\"\"\\\n Returns concatenated object.\n\n This function returns the new :class:`~anndata.AnnData` object\n with concatenated existing embeddings and labels of 'adata'\n and inferred embeddings and labels for `adata_new`.\n \"\"\"\n adata = self._adata_ref.concatenate(\n self._adata_new,\n batch_key=batch_key,\n batch_categories=batch_categories,\n index_unique=index_unique,\n )\n\n obs_update = self._obs.copy()\n obs_update.index = adata[adata.obs[batch_key] == \"1\"].obs_names\n adata.obs.update(obs_update)\n\n for key in self._obsm:\n if key in self._adata_ref.obsm:\n adata.obsm[key] = np.vstack(\n (self._adata_ref.obsm[key], self._obsm[key])\n )\n\n if self._use_rep not in (\"X_pca\", \"X\"):\n adata.obsm[self._use_rep] = np.vstack(\n (self._adata_ref.obsm[self._use_rep], self._obsm[\"rep\"])\n )\n\n if \"X_umap\" in self._obsm:\n adata.uns[\"umap\"] = self._adata_ref.uns[\"umap\"]\n if \"X_pca\" in self._obsm:\n adata.uns[\"pca\"] = self._adata_ref.uns[\"pca\"]\n adata.varm[\"PCs\"] = self._adata_ref.varm[\"PCs\"]\n\n return adata\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nfrom sklearn.utils import check_array, check_random_state\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._settings import settings\nfrom .._utils import NeighborsView\nfrom ._utils import _choose_representation, get_init_pos_from_paga\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from anndata import AnnData\n\n from .._utils import AnyRandom\n\n _InitPos = Literal[\"paga\", \"spectral\", \"random\"]\n\n\n@old_positionals(\n \"min_dist\",\n \"spread\",\n \"n_components\",\n \"maxiter\",\n \"alpha\",\n \"gamma\",\n \"negative_sample_rate\",\n \"init_pos\",\n \"random_state\",\n \"a\",\n \"b\",\n \"copy\",\n \"method\",\n \"neighbors_key\",\n)\ndef umap(\n adata: AnnData,\n *,\n min_dist: float = 0.5,\n spread: float = 1.0,\n n_components: int = 2,\n maxiter: int | None = None,\n alpha: float = 1.0,\n gamma: float = 1.0,\n negative_sample_rate: int = 5,\n init_pos: _InitPos | np.ndarray | None = \"spectral\",\n random_state: AnyRandom = 0,\n a: float | None = None,\n b: float | None = None,\n method: Literal[\"umap\", \"rapids\"] = \"umap\",\n key_added: str | None = None,\n neighbors_key: str = \"neighbors\",\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Embed the neighborhood graph using UMAP :cite:p:`McInnes2018`.\n\n UMAP (Uniform Manifold Approximation and Projection) is a manifold learning\n technique suitable for visualizing high-dimensional data. Besides tending to\n be faster than tSNE, it optimizes the embedding such that it best reflects\n the topology of the data, which we represent throughout Scanpy using a\n neighborhood graph. tSNE, by contrast, optimizes the distribution of\n nearest-neighbor distances in the embedding such that these best match the\n distribution of distances in the high-dimensional space.\n We use the implementation of umap-learn_ :cite:p:`McInnes2018`.\n For a few comparisons of UMAP with tSNE, see :cite:t:`Becht2018`.\n\n .. _umap-learn: https://github.com/lmcinnes/umap\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n min_dist\n The effective minimum distance between embedded points. Smaller values\n will result in a more clustered/clumped embedding where nearby points on\n the manifold are drawn closer together, while larger values will result\n on a more even dispersal of points. The value should be set relative to\n the ``spread`` value, which determines the scale at which embedded\n points will be spread out. The default of in the `umap-learn` package is\n 0.1.\n spread\n The effective scale of embedded points. In combination with `min_dist`\n this determines how clustered/clumped the embedded points are.\n n_components\n The number of dimensions of the embedding.\n maxiter\n The number of iterations (epochs) of the optimization. Called `n_epochs`\n in the original UMAP.\n alpha\n The initial learning rate for the embedding optimization.\n gamma\n Weighting applied to negative samples in low dimensional embedding\n optimization. Values higher than one will result in greater weight\n being given to negative samples.\n negative_sample_rate\n The number of negative edge/1-simplex samples to use per positive\n edge/1-simplex sample in optimizing the low dimensional embedding.\n init_pos\n How to initialize the low dimensional embedding. Called `init` in the\n original UMAP. Options are:\n\n * Any key for `adata.obsm`.\n * 'paga': positions from :func:`~scanpy.pl.paga`.\n * 'spectral': use a spectral embedding of the graph.\n * 'random': assign initial embedding positions at random.\n * A numpy array of initial embedding positions.\n random_state\n If `int`, `random_state` is the seed used by the random number generator;\n If `RandomState` or `Generator`, `random_state` is the random number generator;\n If `None`, the random number generator is the `RandomState` instance used\n by `np.random`.\n a\n More specific parameters controlling the embedding. If `None` these\n values are set automatically as determined by `min_dist` and\n `spread`.\n b\n More specific parameters controlling the embedding. If `None` these\n values are set automatically as determined by `min_dist` and\n `spread`.\n method\n Chosen implementation.\n\n ``'umap'``\n Umap’s simplical set embedding.\n ``'rapids'``\n GPU accelerated implementation.\n\n .. deprecated:: 1.10.0\n Use :func:`rapids_singlecell.tl.umap` instead.\n key_added\n If not specified, the embedding is stored as\n :attr:`~anndata.AnnData.obsm`\\\\ `['X_umap']` and the the parameters in\n :attr:`~anndata.AnnData.uns`\\\\ `['umap']`.\n If specified, the embedding is stored as\n :attr:`~anndata.AnnData.obsm`\\\\ ``[key_added]`` and the the parameters in\n :attr:`~anndata.AnnData.uns`\\\\ ``[key_added]``.\n neighbors_key\n Umap looks in\n :attr:`~anndata.AnnData.uns`\\\\ ``[neighbors_key]`` for neighbors settings and\n :attr:`~anndata.AnnData.obsp`\\\\ ``[.uns[neighbors_key]['connectivities_key']]`` for connectivities.\n copy\n Return a copy instead of writing to adata.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.obsm['X_umap' | key_added]` : :class:`numpy.ndarray` (dtype `float`)\n UMAP coordinates of data.\n `adata.uns['umap' | key_added]` : :class:`dict`\n UMAP parameters.\n\n \"\"\"\n adata = adata.copy() if copy else adata\n\n key_obsm, key_uns = (\"X_umap\", \"umap\") if key_added is None else [key_added] * 2\n\n if neighbors_key is None: # backwards compat\n neighbors_key = \"neighbors\"\n if neighbors_key not in adata.uns:\n raise ValueError(\n f\"Did not find .uns[{neighbors_key!r}]. Run `sc.pp.neighbors` first.\"\n )\n\n start = logg.info(\"computing UMAP\")\n\n neighbors = NeighborsView(adata, neighbors_key)\n\n if \"params\" not in neighbors or neighbors[\"params\"][\"method\"] != \"umap\":\n logg.warning(\n f'.obsp[\"{neighbors[\"connectivities_key\"]}\"] have not been computed using umap'\n )\n\n with warnings.catch_warnings():\n # umap 0.5.0\n warnings.filterwarnings(\"ignore\", message=r\"Tensorflow not installed\")\n import umap\n\n from umap.umap_ import find_ab_params, simplicial_set_embedding\n\n if a is None or b is None:\n a, b = find_ab_params(spread, min_dist)\n adata.uns[key_uns] = dict(params=dict(a=a, b=b))\n if isinstance(init_pos, str) and init_pos in adata.obsm:\n init_coords = adata.obsm[init_pos]\n elif isinstance(init_pos, str) and init_pos == \"paga\":\n init_coords = get_init_pos_from_paga(\n adata, random_state=random_state, neighbors_key=neighbors_key\n )\n else:\n init_coords = init_pos # Let umap handle it\n if hasattr(init_coords, \"dtype\"):\n init_coords = check_array(init_coords, dtype=np.float32, accept_sparse=False)\n\n if random_state != 0:\n adata.uns[key_uns][\"params\"][\"random_state\"] = random_state\n random_state = check_random_state(random_state)\n\n neigh_params = neighbors[\"params\"]\n X = _choose_representation(\n adata,\n use_rep=neigh_params.get(\"use_rep\", None),\n n_pcs=neigh_params.get(\"n_pcs\", None),\n silent=True,\n )\n if method == \"umap\":\n # the data matrix X is really only used for determining the number of connected components\n # for the init condition in the UMAP embedding\n default_epochs = 500 if neighbors[\"connectivities\"].shape[0] <= 10000 else 200\n n_epochs = default_epochs if maxiter is None else maxiter\n X_umap, _ = simplicial_set_embedding(\n data=X,\n graph=neighbors[\"connectivities\"].tocoo(),\n n_components=n_components,\n initial_alpha=alpha,\n a=a,\n b=b,\n gamma=gamma,\n negative_sample_rate=negative_sample_rate,\n n_epochs=n_epochs,\n init=init_coords,\n random_state=random_state,\n metric=neigh_params.get(\"metric\", \"euclidean\"),\n metric_kwds=neigh_params.get(\"metric_kwds\", {}),\n densmap=False,\n densmap_kwds={},\n output_dens=False,\n verbose=settings.verbosity > 3,\n )\n elif method == \"rapids\":\n msg = (\n \"`method='rapids'` is deprecated. \"\n \"Use `rapids_singlecell.tl.louvain` instead.\"\n )\n warnings.warn(msg, FutureWarning)\n metric = neigh_params.get(\"metric\", \"euclidean\")\n if metric != \"euclidean\":\n raise ValueError(\n f\"`sc.pp.neighbors` was called with `metric` {metric!r}, \"\n \"but umap `method` 'rapids' only supports the 'euclidean' metric.\"\n )\n from cuml import UMAP\n\n n_neighbors = neighbors[\"params\"][\"n_neighbors\"]\n n_epochs = (\n 500 if maxiter is None else maxiter\n ) # 0 is not a valid value for rapids, unlike original umap\n X_contiguous = np.ascontiguousarray(X, dtype=np.float32)\n umap = UMAP(\n n_neighbors=n_neighbors,\n n_components=n_components,\n n_epochs=n_epochs,\n learning_rate=alpha,\n init=init_pos,\n min_dist=min_dist,\n spread=spread,\n negative_sample_rate=negative_sample_rate,\n a=a,\n b=b,\n verbose=settings.verbosity > 3,\n random_state=random_state,\n )\n X_umap = umap.fit_transform(X_contiguous)\n adata.obsm[key_obsm] = X_umap # annotate samples with UMAP coordinates\n logg.info(\n \" finished\",\n time=start,\n deep=(\n \"added\\n\"\n f\" {key_obsm!r}, UMAP coordinates (adata.obsm)\\n\"\n f\" {key_uns!r}, UMAP parameters (adata.uns)\"\n ),\n )\n return adata if copy else None\n\n\n\"\"\"\\\nCalculate overlaps of rank_genes_groups marker genes with marker gene dictionaries\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom collections.abc import Set\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\n\nfrom .. import logging as logg\nfrom .._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from anndata import AnnData\n\n _Method = Literal[\"overlap_count\", \"overlap_coef\", \"jaccard\"]\n\n\ndef _calc_overlap_count(markers1: dict, markers2: dict):\n \"\"\"\\\n Calculate overlap count between the values of two dictionaries\n\n Note: dict values must be sets\n \"\"\"\n overlaps = np.zeros((len(markers1), len(markers2)))\n\n for j, marker_group in enumerate(markers1):\n tmp = [len(markers2[i].intersection(markers1[marker_group])) for i in markers2]\n overlaps[j, :] = tmp\n\n return overlaps\n\n\ndef _calc_overlap_coef(markers1: dict, markers2: dict):\n \"\"\"\\\n Calculate overlap coefficient between the values of two dictionaries\n\n Note: dict values must be sets\n \"\"\"\n overlap_coef = np.zeros((len(markers1), len(markers2)))\n\n for j, marker_group in enumerate(markers1):\n tmp = [\n len(markers2[i].intersection(markers1[marker_group]))\n / max(min(len(markers2[i]), len(markers1[marker_group])), 1)\n for i in markers2\n ]\n overlap_coef[j, :] = tmp\n\n return overlap_coef\n\n\ndef _calc_jaccard(markers1: dict, markers2: dict):\n \"\"\"\\\n Calculate jaccard index between the values of two dictionaries\n\n Note: dict values must be sets\n \"\"\"\n jacc_results = np.zeros((len(markers1), len(markers2)))\n\n for j, marker_group in enumerate(markers1):\n tmp = [\n len(markers2[i].intersection(markers1[marker_group]))\n / len(markers2[i].union(markers1[marker_group]))\n for i in markers2\n ]\n jacc_results[j, :] = tmp\n\n return jacc_results\n\n\n@doctest_needs(\"leidenalg\")\ndef marker_gene_overlap(\n adata: AnnData,\n reference_markers: dict[str, set] | dict[str, list],\n *,\n key: str = \"rank_genes_groups\",\n method: _Method = \"overlap_count\",\n normalize: Literal[\"reference\", \"data\"] | None = None,\n top_n_markers: int | None = None,\n adj_pval_threshold: float | None = None,\n key_added: str = \"marker_gene_overlap\",\n inplace: bool = False,\n):\n \"\"\"\\\n Calculate an overlap score between data-deriven marker genes and\n provided markers\n\n Marker gene overlap scores can be quoted as overlap counts, overlap\n coefficients, or jaccard indices. The method returns a pandas dataframe\n which can be used to annotate clusters based on marker gene overlaps.\n\n This function was written by Malte Luecken.\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n reference_markers\n A marker gene dictionary object. Keys should be strings with the\n cell identity name and values are sets or lists of strings which match\n format of `adata.var_name`.\n key\n The key in `adata.uns` where the rank_genes_groups output is stored.\n By default this is `'rank_genes_groups'`.\n method\n (default: `overlap_count`)\n Method to calculate marker gene overlap. `'overlap_count'` uses the\n intersection of the gene set, `'overlap_coef'` uses the overlap\n coefficient, and `'jaccard'` uses the Jaccard index.\n normalize\n Normalization option for the marker gene overlap output. This parameter\n can only be set when `method` is set to `'overlap_count'`. `'reference'`\n normalizes the data by the total number of marker genes given in the\n reference annotation per group. `'data'` normalizes the data by the\n total number of marker genes used for each cluster.\n top_n_markers\n The number of top data-derived marker genes to use. By default the top\n 100 marker genes are used. If `adj_pval_threshold` is set along with\n `top_n_markers`, then `adj_pval_threshold` is ignored.\n adj_pval_threshold\n A significance threshold on the adjusted p-values to select marker\n genes. This can only be used when adjusted p-values are calculated by\n `sc.tl.rank_genes_groups()`. If `adj_pval_threshold` is set along with\n `top_n_markers`, then `adj_pval_threshold` is ignored.\n key_added\n Name of the `.uns` field that will contain the marker overlap scores.\n inplace\n Return a marker gene dataframe or store it inplace in `adata.uns`.\n\n Returns\n -------\n Returns :class:`pandas.DataFrame` if `inplace=True`, else returns an `AnnData` object where it sets the following field:\n\n `adata.uns[key_added]` : :class:`pandas.DataFrame` (dtype `float`)\n Marker gene overlap scores. Default for `key_added` is `'marker_gene_overlap'`.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> sc.pp.pca(adata, svd_solver='arpack')\n >>> sc.pp.neighbors(adata)\n >>> sc.tl.leiden(adata)\n >>> sc.tl.rank_genes_groups(adata, groupby='leiden')\n >>> marker_genes = {\n ... 'CD4 T cells': {'IL7R'},\n ... 'CD14+ Monocytes': {'CD14', 'LYZ'},\n ... 'B cells': {'MS4A1'},\n ... 'CD8 T cells': {'CD8A'},\n ... 'NK cells': {'GNLY', 'NKG7'},\n ... 'FCGR3A+ Monocytes': {'FCGR3A', 'MS4A7'},\n ... 'Dendritic Cells': {'FCER1A', 'CST3'},\n ... 'Megakaryocytes': {'PPBP'}\n ... }\n >>> marker_matches = sc.tl.marker_gene_overlap(adata, marker_genes)\n \"\"\"\n # Test user inputs\n if inplace:\n raise NotImplementedError(\n \"Writing Pandas dataframes to h5ad is currently under development.\"\n \"\\nPlease use `inplace=False`.\"\n )\n\n if key not in adata.uns:\n raise ValueError(\n \"Could not find marker gene data. \"\n \"Please run `sc.tl.rank_genes_groups()` first.\"\n )\n\n avail_methods = {\"overlap_count\", \"overlap_coef\", \"jaccard\", \"enrich\"}\n if method not in avail_methods:\n raise ValueError(f\"Method must be one of {avail_methods}.\")\n\n if normalize == \"None\":\n normalize = None\n\n avail_norm = {\"reference\", \"data\", None}\n if normalize not in avail_norm:\n raise ValueError(f\"Normalize must be one of {avail_norm}.\")\n\n if normalize is not None and method != \"overlap_count\":\n raise ValueError(\"Can only normalize with method=`overlap_count`.\")\n\n if not all(isinstance(val, Set) for val in reference_markers.values()):\n try:\n reference_markers = {\n key: set(val) for key, val in reference_markers.items()\n }\n except Exception:\n raise ValueError(\n \"Please ensure that `reference_markers` contains \"\n \"sets or lists of markers as values.\"\n )\n\n if adj_pval_threshold is not None:\n if \"pvals_adj\" not in adata.uns[key]:\n raise ValueError(\n \"Could not find adjusted p-value data. \"\n \"Please run `sc.tl.rank_genes_groups()` with a \"\n \"method that outputs adjusted p-values.\"\n )\n\n if adj_pval_threshold < 0:\n logg.warning(\n \"`adj_pval_threshold` was set below 0. Threshold will be set to 0.\"\n )\n adj_pval_threshold = 0\n elif adj_pval_threshold > 1:\n logg.warning(\n \"`adj_pval_threshold` was set above 1. Threshold will be set to 1.\"\n )\n adj_pval_threshold = 1\n\n if top_n_markers is not None:\n logg.warning(\n \"Both `adj_pval_threshold` and `top_n_markers` is set. \"\n \"`adj_pval_threshold` will be ignored.\"\n )\n\n if top_n_markers is not None and top_n_markers < 1:\n logg.warning(\n \"`top_n_markers` was set below 1. `top_n_markers` will be set to 1.\"\n )\n top_n_markers = 1\n\n # Get data-derived marker genes in a dictionary of sets\n data_markers = dict()\n cluster_ids = adata.uns[key][\"names\"].dtype.names\n\n for group in cluster_ids:\n if top_n_markers is not None:\n n_genes = min(top_n_markers, adata.uns[key][\"names\"].shape[0])\n data_markers[group] = set(adata.uns[key][\"names\"][group][:n_genes])\n elif adj_pval_threshold is not None:\n n_genes = (adata.uns[key][\"pvals_adj\"][group] < adj_pval_threshold).sum()\n data_markers[group] = set(adata.uns[key][\"names\"][group][:n_genes])\n if n_genes == 0:\n logg.warning(\n \"No marker genes passed the significance threshold of \"\n f\"{adj_pval_threshold} for cluster {group!r}.\"\n )\n # Use top 100 markers as default if top_n_markers = None\n else:\n data_markers[group] = set(adata.uns[key][\"names\"][group][:100])\n\n # Find overlaps\n if method == \"overlap_count\":\n marker_match = _calc_overlap_count(reference_markers, data_markers)\n if normalize == \"reference\":\n # Ensure rows sum to 1\n ref_lengths = np.array(\n [len(reference_markers[m_group]) for m_group in reference_markers]\n )\n marker_match = marker_match / ref_lengths[:, np.newaxis]\n marker_match = np.nan_to_num(marker_match)\n elif normalize == \"data\":\n # Ensure columns sum to 1\n data_lengths = np.array(\n [len(data_markers[dat_group]) for dat_group in data_markers]\n )\n marker_match = marker_match / data_lengths\n marker_match = np.nan_to_num(marker_match)\n elif method == \"overlap_coef\":\n marker_match = _calc_overlap_coef(reference_markers, data_markers)\n elif method == \"jaccard\":\n marker_match = _calc_jaccard(reference_markers, data_markers)\n\n # Note:\n # Could add an 'enrich' option here\n # (fisher's exact test or hypergeometric test),\n # but that would require knowledge of the size of the space from which\n # the reference marker gene set was taken.\n # This is at best approximately known.\n\n # Create a pandas dataframe with the results\n marker_groups = list(reference_markers.keys())\n clusters = list(cluster_ids)\n marker_matching_df = pd.DataFrame(\n marker_match, index=marker_groups, columns=clusters\n )\n\n # Store the results\n if inplace:\n adata.uns[key_added] = marker_matching_df\n logg.hint(f\"added\\n {key_added!r}, marker overlap scores (adata.uns)\")\n else:\n return marker_matching_df\n\n\n\"\"\"\\\nCalculate density of cells in embeddings\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._utils import sanitize_anndata\n\nif TYPE_CHECKING:\n from collections.abc import Sequence\n\n from anndata import AnnData\n\n\ndef _calc_density(x: np.ndarray, y: np.ndarray):\n \"\"\"\\\n Calculates the density of points in 2 dimensions.\n \"\"\"\n from scipy.stats import gaussian_kde\n\n # Calculate the point density\n xy = np.vstack([x, y])\n z = gaussian_kde(xy)(xy)\n\n min_z = np.min(z)\n max_z = np.max(z)\n\n # Scale between 0 and 1\n scaled_z = (z - min_z) / (max_z - min_z)\n\n return scaled_z\n\n\n@old_positionals(\"groupby\", \"key_added\", \"components\")\ndef embedding_density(\n adata: AnnData,\n basis: str = \"umap\",\n *,\n groupby: str | None = None,\n key_added: str | None = None,\n components: str | Sequence[str] | None = None,\n) -> None:\n \"\"\"\\\n Calculate the density of cells in an embedding (per condition).\n\n Gaussian kernel density estimation is used to calculate the density of\n cells in an embedded space. This can be performed per category over a\n categorical cell annotation. The cell density can be plotted using the\n `pl.embedding_density` function.\n\n Note that density values are scaled to be between 0 and 1. Thus, the\n density value at each cell is only comparable to densities in\n the same category.\n\n Beware that the KDE estimate used (`scipy.stats.gaussian_kde`) becomes\n unreliable if you don't have enough cells in a category.\n\n This function was written by Sophie Tritschler and implemented into\n Scanpy by Malte Luecken.\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n basis\n The embedding over which the density will be calculated. This embedded\n representation should be found in `adata.obsm['X_[basis]']``.\n groupby\n Key for categorical observation/cell annotation for which densities\n are calculated per category.\n key_added\n Name of the `.obs` covariate that will be added with the density\n estimates.\n components\n The embedding dimensions over which the density should be calculated.\n This is limited to two components.\n\n Returns\n -------\n Sets the following fields (`key_added` defaults to `[basis]_density_[groupby]`, where `[basis]` is one of `umap`, `diffmap`, `pca`, `tsne`, or `draw_graph_fa` and `[groupby]` denotes the parameter input):\n\n `adata.obs[key_added]` : :class:`numpy.ndarray` (dtype `float`)\n Embedding density values for each cell.\n `adata.uns['[key_added]_params']` : :class:`dict`\n A dict with the values for the parameters `covariate` (for the `groupby` parameter) and `components`.\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.tl.umap(adata)\n sc.tl.embedding_density(adata, basis='umap', groupby='phase')\n sc.pl.embedding_density(\n adata, basis='umap', key='umap_density_phase', group='G1'\n )\n\n .. plot::\n :context: close-figs\n\n sc.pl.embedding_density(\n adata, basis='umap', key='umap_density_phase', group='S'\n )\n\n .. currentmodule:: scanpy\n\n See also\n --------\n pl.embedding_density\n \"\"\"\n # to ensure that newly created covariates are categorical\n # to test for category numbers\n sanitize_anndata(adata)\n\n logg.info(f\"computing density on {basis!r}\")\n\n # Test user inputs\n basis = basis.lower()\n\n if basis == \"fa\":\n basis = \"draw_graph_fa\"\n\n if f\"X_{basis}\" not in adata.obsm_keys():\n raise ValueError(\n \"Cannot find the embedded representation \"\n f\"`adata.obsm['X_{basis}']`. Compute the embedding first.\"\n )\n\n if components is None:\n components = \"1,2\"\n if isinstance(components, str):\n components = components.split(\",\")\n components = np.array(components).astype(int) - 1\n\n if len(components) != 2:\n raise ValueError(\"Please specify exactly 2 components, or `None`.\")\n\n if basis == \"diffmap\":\n components += 1\n\n if groupby is not None:\n if groupby not in adata.obs:\n raise ValueError(f\"Could not find {groupby!r} `.obs` column.\")\n\n if adata.obs[groupby].dtype.name != \"category\":\n raise ValueError(f\"{groupby!r} column does not contain categorical data\")\n\n # Define new covariate name\n if key_added is not None:\n density_covariate = key_added\n elif groupby is not None:\n density_covariate = f\"{basis}_density_{groupby}\"\n else:\n density_covariate = f\"{basis}_density\"\n\n # Calculate the densities over each category in the groupby column\n if groupby is not None:\n categories = adata.obs[groupby].cat.categories\n\n density_values = np.zeros(adata.n_obs)\n\n for cat in categories:\n cat_mask = adata.obs[groupby] == cat\n embed_x = adata.obsm[f\"X_{basis}\"][cat_mask, components[0]]\n embed_y = adata.obsm[f\"X_{basis}\"][cat_mask, components[1]]\n\n dens_embed = _calc_density(embed_x, embed_y)\n density_values[cat_mask] = dens_embed\n\n adata.obs[density_covariate] = density_values\n else: # if groupby is None\n # Calculate the density over the whole embedding without subsetting\n embed_x = adata.obsm[f\"X_{basis}\"][:, components[0]]\n embed_y = adata.obsm[f\"X_{basis}\"][:, components[1]]\n\n adata.obs[density_covariate] = _calc_density(embed_x, embed_y)\n\n # Reduce diffmap components for labeling\n # Note: plot_scatter takes care of correcting diffmap components\n # for plotting automatically\n if basis != \"diffmap\":\n components += 1\n\n adata.uns[f\"{density_covariate}_params\"] = dict(\n covariate=groupby, components=components.tolist()\n )\n\n logg.hint(\n f\"added\\n\"\n f\" '{density_covariate}', densities (adata.obs)\\n\"\n f\" '{density_covariate}_params', parameter (adata.uns)\"\n )\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\n\nfrom .. import logging as logg\nfrom .._settings import settings\nfrom .._utils import _choose_graph\n\nif TYPE_CHECKING:\n from anndata import AnnData\n from scipy.sparse import csr_matrix\n\n\ndef _choose_representation(\n adata: AnnData,\n *,\n use_rep: str | None = None,\n n_pcs: int | None = None,\n silent: bool = False,\n) -> np.ndarray | csr_matrix: # TODO: what else?\n from ..preprocessing import pca\n\n verbosity = settings.verbosity\n if silent and settings.verbosity > 1:\n settings.verbosity = 1\n if use_rep is None and n_pcs == 0: # backwards compat for specifying `.X`\n use_rep = \"X\"\n if use_rep is None:\n if adata.n_vars > settings.N_PCS:\n if \"X_pca\" in adata.obsm:\n if n_pcs is not None and n_pcs > adata.obsm[\"X_pca\"].shape[1]:\n raise ValueError(\n \"`X_pca` does not have enough PCs. Rerun `sc.pp.pca` with adjusted `n_comps`.\"\n )\n X = adata.obsm[\"X_pca\"][:, :n_pcs]\n logg.info(f\" using 'X_pca' with n_pcs = {X.shape[1]}\")\n else:\n warnings.warn(\n f\"You’re trying to run this on {adata.n_vars} dimensions of `.X`, \"\n \"if you really want this, set `use_rep='X'`.\\n \"\n \"Falling back to preprocessing with `sc.pp.pca` and default params.\"\n )\n n_pcs_pca = n_pcs if n_pcs is not None else settings.N_PCS\n pca(adata, n_comps=n_pcs_pca)\n X = adata.obsm[\"X_pca\"]\n else:\n logg.info(\" using data matrix X directly\")\n X = adata.X\n else:\n if use_rep in adata.obsm and n_pcs is not None:\n if n_pcs > adata.obsm[use_rep].shape[1]:\n raise ValueError(\n f\"{use_rep} does not have enough Dimensions. Provide a \"\n \"Representation with equal or more dimensions than\"\n \"`n_pcs` or lower `n_pcs` \"\n )\n X = adata.obsm[use_rep][:, :n_pcs]\n elif use_rep in adata.obsm and n_pcs is None:\n X = adata.obsm[use_rep]\n elif use_rep == \"X\":\n X = adata.X\n else:\n raise ValueError(\n f\"Did not find {use_rep} in `.obsm.keys()`. \"\n \"You need to compute it first.\"\n )\n settings.verbosity = verbosity # resetting verbosity\n return X\n\n\ndef preprocess_with_pca(adata, n_pcs: int | None = None, random_state=0):\n \"\"\"\n Parameters\n ----------\n n_pcs\n If `n_pcs=0`, do not preprocess with PCA.\n If `None` and there is a PCA version of the data, use this.\n If an integer, compute the PCA.\n \"\"\"\n from ..preprocessing import pca\n\n if n_pcs == 0:\n logg.info(\" using data matrix X directly (no PCA)\")\n return adata.X\n elif n_pcs is None and \"X_pca\" in adata.obsm_keys():\n logg.info(f' using \\'X_pca\\' with n_pcs = {adata.obsm[\"X_pca\"].shape[1]}')\n return adata.obsm[\"X_pca\"]\n elif \"X_pca\" in adata.obsm_keys() and adata.obsm[\"X_pca\"].shape[1] >= n_pcs:\n logg.info(f\" using 'X_pca' with n_pcs = {n_pcs}\")\n return adata.obsm[\"X_pca\"][:, :n_pcs]\n else:\n n_pcs = settings.N_PCS if n_pcs is None else n_pcs\n if adata.X.shape[1] > n_pcs:\n logg.info(f\" computing 'X_pca' with n_pcs = {n_pcs}\")\n logg.hint(\"avoid this by setting n_pcs = 0\")\n X = pca(adata.X, n_comps=n_pcs, random_state=random_state)\n adata.obsm[\"X_pca\"] = X\n return X\n else:\n logg.info(\" using data matrix X directly (no PCA)\")\n return adata.X\n\n\ndef get_init_pos_from_paga(\n adata, adjacency=None, random_state=0, neighbors_key=None, obsp=None\n):\n np.random.seed(random_state)\n if adjacency is None:\n adjacency = _choose_graph(adata, obsp, neighbors_key)\n if \"paga\" in adata.uns and \"pos\" in adata.uns[\"paga\"]:\n groups = adata.obs[adata.uns[\"paga\"][\"groups\"]]\n pos = adata.uns[\"paga\"][\"pos\"]\n connectivities_coarse = adata.uns[\"paga\"][\"connectivities\"]\n init_pos = np.ones((adjacency.shape[0], 2))\n for i, group_pos in enumerate(pos):\n subset = (groups == groups.cat.categories[i]).values\n neighbors = connectivities_coarse[i].nonzero()\n if len(neighbors[1]) > 0:\n connectivities = connectivities_coarse[i][neighbors]\n nearest_neighbor = neighbors[1][np.argmax(connectivities)]\n noise = np.random.random((len(subset[subset]), 2))\n dist = pos[i] - pos[nearest_neighbor]\n noise = noise * dist\n init_pos[subset] = group_pos - 0.5 * dist + noise\n else:\n init_pos[subset] = group_pos\n else:\n raise ValueError(\"Plot PAGA first, so that adata.uns['paga']\" \"with key 'pos'.\")\n return init_pos\n\n\n\"\"\"Calculate scores based on the expression of gene lists.\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom scipy.sparse import issparse\n\nfrom scanpy._utils import _check_use_raw, is_backed_type\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom ..get import _get_obs_rep\n\nif TYPE_CHECKING:\n from collections.abc import Callable, Generator, Sequence\n from typing import Literal\n\n from anndata import AnnData\n from numpy.typing import DTypeLike, NDArray\n from scipy.sparse import csc_matrix, csr_matrix\n\n from .._utils import AnyRandom\n\n try:\n _StrIdx = pd.Index[str]\n except TypeError: # Sphinx\n _StrIdx = pd.Index\n _GetSubset = Callable[[_StrIdx], np.ndarray | csr_matrix | csc_matrix]\n\n\ndef _sparse_nanmean(\n X: csr_matrix | csc_matrix, axis: Literal[0, 1]\n) -> NDArray[np.float64]:\n \"\"\"\n np.nanmean equivalent for sparse matrices\n \"\"\"\n if not issparse(X):\n raise TypeError(\"X must be a sparse matrix\")\n\n # count the number of nan elements per row/column (dep. on axis)\n Z = X.copy()\n Z.data = np.isnan(Z.data)\n Z.eliminate_zeros()\n n_elements = Z.shape[axis] - Z.sum(axis)\n\n # set the nans to 0, so that a normal .sum() works\n Y = X.copy()\n Y.data[np.isnan(Y.data)] = 0\n Y.eliminate_zeros()\n\n # the average\n s = Y.sum(axis, dtype=\"float64\") # float64 for score_genes function compatibility)\n m = s / n_elements\n\n return m\n\n\n@old_positionals(\n \"ctrl_size\", \"gene_pool\", \"n_bins\", \"score_name\", \"random_state\", \"copy\", \"use_raw\"\n)\ndef score_genes(\n adata: AnnData,\n gene_list: Sequence[str] | pd.Index[str],\n *,\n ctrl_as_ref: bool = True,\n ctrl_size: int = 50,\n gene_pool: Sequence[str] | pd.Index[str] | None = None,\n n_bins: int = 25,\n score_name: str = \"score\",\n random_state: AnyRandom = 0,\n copy: bool = False,\n use_raw: bool | None = None,\n layer: str | None = None,\n) -> AnnData | None:\n \"\"\"\\\n Score a set of genes :cite:p:`Satija2015`.\n\n The score is the average expression of a set of genes subtracted with the\n average expression of a reference set of genes. The reference set is\n randomly sampled from the `gene_pool` for each binned expression value.\n\n This reproduces the approach in Seurat :cite:p:`Satija2015` and has been implemented\n for Scanpy by Davide Cittaro.\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n gene_list\n The list of gene names used for score calculation.\n ctrl_as_ref\n Allow the algorithm to use the control genes as reference.\n Will be changed to `False` in scanpy 2.0.\n ctrl_size\n Number of reference genes to be sampled from each bin. If `len(gene_list)` is not too\n low, you can set `ctrl_size=len(gene_list)`.\n gene_pool\n Genes for sampling the reference set. Default is all genes.\n n_bins\n Number of expression level bins for sampling.\n score_name\n Name of the field to be added in `.obs`.\n random_state\n The random seed for sampling.\n copy\n Copy `adata` or modify it inplace.\n use_raw\n Whether to use `raw` attribute of `adata`. Defaults to `True` if `.raw` is present.\n\n .. versionchanged:: 1.4.5\n Default value changed from `False` to `None`.\n layer\n Key from `adata.layers` whose value will be used to perform tests on.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following field:\n\n `adata.obs[score_name]` : :class:`numpy.ndarray` (dtype `float`)\n Scores of each cell.\n\n Examples\n --------\n See this `notebook `__.\n \"\"\"\n start = logg.info(f\"computing score {score_name!r}\")\n adata = adata.copy() if copy else adata\n use_raw = _check_use_raw(adata, use_raw, layer=layer)\n if is_backed_type(adata.X) and not use_raw:\n raise NotImplementedError(\n f\"score_genes is not implemented for matrices of type {type(adata.X)}\"\n )\n\n if random_state is not None:\n np.random.seed(random_state)\n\n gene_list, gene_pool, get_subset = _check_score_genes_args(\n adata, gene_list, gene_pool, use_raw=use_raw, layer=layer\n )\n del use_raw, layer, random_state\n\n # Trying here to match the Seurat approach in scoring cells.\n # Basically we need to compare genes against random genes in a matched\n # interval of expression.\n\n control_genes = pd.Index([], dtype=\"string\")\n for r_genes in _score_genes_bins(\n gene_list,\n gene_pool,\n ctrl_as_ref=ctrl_as_ref,\n ctrl_size=ctrl_size,\n n_bins=n_bins,\n get_subset=get_subset,\n ):\n control_genes = control_genes.union(r_genes)\n\n if len(control_genes) == 0:\n msg = \"No control genes found in any cut.\"\n if ctrl_as_ref:\n msg += \" Try setting `ctrl_as_ref=False`.\"\n raise RuntimeError(msg)\n\n means_list, means_control = (\n _nan_means(get_subset(genes), axis=1, dtype=\"float64\")\n for genes in (gene_list, control_genes)\n )\n score = means_list - means_control\n\n adata.obs[score_name] = pd.Series(\n np.array(score).ravel(), index=adata.obs_names, dtype=\"float64\"\n )\n\n logg.info(\n \" finished\",\n time=start,\n deep=(\n \"added\\n\"\n f\" {score_name!r}, score of gene set (adata.obs).\\n\"\n f\" {len(control_genes)} total control genes are used.\"\n ),\n )\n return adata if copy else None\n\n\ndef _check_score_genes_args(\n adata: AnnData,\n gene_list: pd.Index[str] | Sequence[str],\n gene_pool: pd.Index[str] | Sequence[str] | None,\n *,\n layer: str | None,\n use_raw: bool,\n) -> tuple[pd.Index[str], pd.Index[str], _GetSubset]:\n \"\"\"Restrict `gene_list` and `gene_pool` to present genes in `adata`.\n\n Also returns a function to get subset of `adata.X` based on a set of genes passed.\n \"\"\"\n var_names = adata.raw.var_names if use_raw else adata.var_names\n gene_list = pd.Index([gene_list] if isinstance(gene_list, str) else gene_list)\n genes_to_ignore = gene_list.difference(var_names, sort=False) # first get missing\n gene_list = gene_list.intersection(var_names) # then restrict to present\n if len(genes_to_ignore) > 0:\n logg.warning(f\"genes are not in var_names and ignored: {genes_to_ignore}\")\n if len(gene_list) == 0:\n raise ValueError(\"No valid genes were passed for scoring.\")\n\n if gene_pool is None:\n gene_pool = var_names.astype(\"string\")\n else:\n gene_pool = pd.Index(gene_pool, dtype=\"string\").intersection(var_names)\n if len(gene_pool) == 0:\n raise ValueError(\"No valid genes were passed for reference set.\")\n\n def get_subset(genes: pd.Index[str]):\n x = _get_obs_rep(adata, use_raw=use_raw, layer=layer)\n if len(genes) == len(var_names):\n return x\n idx = var_names.get_indexer(genes)\n return x[:, idx]\n\n return gene_list, gene_pool, get_subset\n\n\ndef _score_genes_bins(\n gene_list: pd.Index[str],\n gene_pool: pd.Index[str],\n *,\n ctrl_as_ref: bool,\n ctrl_size: int,\n n_bins: int,\n get_subset: _GetSubset,\n) -> Generator[pd.Index[str], None, None]:\n # average expression of genes\n obs_avg = pd.Series(_nan_means(get_subset(gene_pool), axis=0), index=gene_pool)\n # Sometimes (and I don’t know how) missing data may be there, with NaNs for missing entries\n obs_avg = obs_avg[np.isfinite(obs_avg)]\n\n n_items = int(np.round(len(obs_avg) / (n_bins - 1)))\n obs_cut = obs_avg.rank(method=\"min\") // n_items\n keep_ctrl_in_obs_cut = False if ctrl_as_ref else obs_cut.index.isin(gene_list)\n\n # now pick `ctrl_size` genes from every cut\n for cut in np.unique(obs_cut.loc[gene_list]):\n r_genes: pd.Index[str] = obs_cut[(obs_cut == cut) & ~keep_ctrl_in_obs_cut].index\n if len(r_genes) == 0:\n msg = (\n f\"No control genes for {cut=}. You might want to increase \"\n f\"gene_pool size (current size: {len(gene_pool)})\"\n )\n logg.warning(msg)\n if ctrl_size < len(r_genes):\n r_genes = r_genes.to_series().sample(ctrl_size).index\n if ctrl_as_ref: # otherwise `r_genes` is already filtered\n r_genes = r_genes.difference(gene_list)\n yield r_genes\n\n\ndef _nan_means(\n x, *, axis: Literal[0, 1], dtype: DTypeLike | None = None\n) -> NDArray[np.float64]:\n if issparse(x):\n return np.array(_sparse_nanmean(x, axis=axis)).flatten()\n return np.nanmean(x, axis=axis, dtype=dtype)\n\n\n@old_positionals(\"s_genes\", \"g2m_genes\", \"copy\")\ndef score_genes_cell_cycle(\n adata: AnnData,\n *,\n s_genes: Sequence[str],\n g2m_genes: Sequence[str],\n copy: bool = False,\n **kwargs,\n) -> AnnData | None:\n \"\"\"\\\n Score cell cycle genes :cite:p:`Satija2015`.\n\n Given two lists of genes associated to S phase and G2M phase, calculates\n scores and assigns a cell cycle phase (G1, S or G2M). See\n :func:`~scanpy.tl.score_genes` for more explanation.\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n s_genes\n List of genes associated with S phase.\n g2m_genes\n List of genes associated with G2M phase.\n copy\n Copy `adata` or modify it inplace.\n **kwargs\n Are passed to :func:`~scanpy.tl.score_genes`. `ctrl_size` is not\n possible, as it's set as `min(len(s_genes), len(g2m_genes))`.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.obs['S_score']` : :class:`pandas.Series` (dtype `object`)\n The score for S phase for each cell.\n `adata.obs['G2M_score']` : :class:`pandas.Series` (dtype `object`)\n The score for G2M phase for each cell.\n `adata.obs['phase']` : :class:`pandas.Series` (dtype `object`)\n The cell cycle phase (`S`, `G2M` or `G1`) for each cell.\n\n See also\n --------\n score_genes\n\n Examples\n --------\n See this `notebook `__.\n \"\"\"\n logg.info(\"calculating cell cycle phase\")\n\n adata = adata.copy() if copy else adata\n ctrl_size = min(len(s_genes), len(g2m_genes))\n for genes, name in [(s_genes, \"S_score\"), (g2m_genes, \"G2M_score\")]:\n score_genes(adata, genes, score_name=name, ctrl_size=ctrl_size, **kwargs)\n scores = adata.obs[[\"S_score\", \"G2M_score\"]]\n\n # default phase is S\n phase = pd.Series(\"S\", index=scores.index)\n\n # if G2M is higher than S, it's G2M\n phase[scores[\"G2M_score\"] > scores[\"S_score\"]] = \"G2M\"\n\n # if all scores are negative, it's G1...\n phase[np.all(scores < 0, axis=1)] = \"G1\"\n\n adata.obs[\"phase\"] = phase\n logg.hint(\" 'phase', cell cycle phase (adata.obs)\")\n return adata if copy else None\n\n\n\"\"\"Rank genes according to differential expression.\"\"\"\n\nfrom __future__ import annotations\n\nfrom math import floor\nfrom typing import TYPE_CHECKING, Literal, get_args\n\nimport numpy as np\nimport pandas as pd\nfrom scipy.sparse import issparse, vstack\n\nfrom .. import _utils\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._utils import (\n check_nonnegative_integers,\n raise_not_implemented_error_if_backed_type,\n)\nfrom ..get import _check_mask\nfrom ..preprocessing._utils import _get_mean_var\n\nif TYPE_CHECKING:\n from collections.abc import Generator, Iterable\n\n from anndata import AnnData\n from numpy.typing import NDArray\n from scipy import sparse\n\n _CorrMethod = Literal[\"benjamini-hochberg\", \"bonferroni\"]\n\n# Used with get_args\n_Method = Literal[\"logreg\", \"t-test\", \"wilcoxon\", \"t-test_overestim_var\"]\n\n\ndef _select_top_n(scores: NDArray, n_top: int):\n n_from = scores.shape[0]\n reference_indices = np.arange(n_from, dtype=int)\n partition = np.argpartition(scores, -n_top)[-n_top:]\n partial_indices = np.argsort(scores[partition])[::-1]\n global_indices = reference_indices[partition][partial_indices]\n\n return global_indices\n\n\ndef _ranks(\n X: np.ndarray | sparse.csr_matrix | sparse.csc_matrix,\n mask_obs: NDArray[np.bool_] | None = None,\n mask_obs_rest: NDArray[np.bool_] | None = None,\n):\n CONST_MAX_SIZE = 10000000\n\n n_genes = X.shape[1]\n\n if issparse(X):\n merge = lambda tpl: vstack(tpl).toarray()\n adapt = lambda X: X.toarray()\n else:\n merge = np.vstack\n adapt = lambda X: X\n\n masked = mask_obs is not None and mask_obs_rest is not None\n\n if masked:\n n_cells = np.count_nonzero(mask_obs) + np.count_nonzero(mask_obs_rest)\n get_chunk = lambda X, left, right: merge(\n (X[mask_obs, left:right], X[mask_obs_rest, left:right])\n )\n else:\n n_cells = X.shape[0]\n get_chunk = lambda X, left, right: adapt(X[:, left:right])\n\n # Calculate chunk frames\n max_chunk = floor(CONST_MAX_SIZE / n_cells)\n\n for left in range(0, n_genes, max_chunk):\n right = min(left + max_chunk, n_genes)\n\n df = pd.DataFrame(data=get_chunk(X, left, right))\n ranks = df.rank()\n yield ranks, left, right\n\n\ndef _tiecorrect(ranks):\n size = np.float64(ranks.shape[0])\n if size < 2:\n return np.repeat(ranks.shape[1], 1.0)\n\n arr = np.sort(ranks, axis=0)\n tf = np.insert(arr[1:] != arr[:-1], (0, arr.shape[0] - 1), True, axis=0)\n idx = np.where(tf, np.arange(tf.shape[0])[:, None], 0)\n idx = np.sort(idx, axis=0)\n cnt = np.diff(idx, axis=0).astype(np.float64)\n\n return 1.0 - (cnt**3 - cnt).sum(axis=0) / (size**3 - size)\n\n\nclass _RankGenes:\n def __init__(\n self,\n adata: AnnData,\n groups: list[str] | Literal[\"all\"],\n groupby: str,\n *,\n mask_var: NDArray[np.bool_] | None = None,\n reference: Literal[\"rest\"] | str = \"rest\",\n use_raw: bool = True,\n layer: str | None = None,\n comp_pts: bool = False,\n ) -> None:\n self.mask_var = mask_var\n if (base := adata.uns.get(\"log1p\", {}).get(\"base\")) is not None:\n self.expm1_func = lambda x: np.expm1(x * np.log(base))\n else:\n self.expm1_func = np.expm1\n\n self.groups_order, self.groups_masks_obs = _utils.select_groups(\n adata, groups, groupby\n )\n\n # Singlet groups cause division by zero errors\n invalid_groups_selected = set(self.groups_order) & set(\n adata.obs[groupby].value_counts().loc[lambda x: x < 2].index\n )\n\n if len(invalid_groups_selected) > 0:\n raise ValueError(\n \"Could not calculate statistics for groups {} since they only \"\n \"contain one sample.\".format(\", \".join(invalid_groups_selected))\n )\n\n adata_comp = adata\n if layer is not None:\n if use_raw:\n raise ValueError(\"Cannot specify `layer` and have `use_raw=True`.\")\n X = adata_comp.layers[layer]\n else:\n if use_raw and adata.raw is not None:\n adata_comp = adata.raw\n X = adata_comp.X\n raise_not_implemented_error_if_backed_type(X, \"rank_genes_groups\")\n\n # for correct getnnz calculation\n if issparse(X):\n X.eliminate_zeros()\n\n if self.mask_var is not None:\n self.X = X[:, self.mask_var]\n self.var_names = adata_comp.var_names[self.mask_var]\n\n else:\n self.X = X\n self.var_names = adata_comp.var_names\n\n self.ireference = None\n if reference != \"rest\":\n self.ireference = np.where(self.groups_order == reference)[0][0]\n\n self.means = None\n self.vars = None\n\n self.means_rest = None\n self.vars_rest = None\n\n self.comp_pts = comp_pts\n self.pts = None\n self.pts_rest = None\n\n self.stats = None\n\n # for logreg only\n self.grouping_mask = adata.obs[groupby].isin(self.groups_order)\n self.grouping = adata.obs.loc[self.grouping_mask, groupby]\n\n def _basic_stats(self) -> None:\n \"\"\"Set self.{means,vars,pts}{,_rest} depending on X.\"\"\"\n n_genes = self.X.shape[1]\n n_groups = self.groups_masks_obs.shape[0]\n\n self.means = np.zeros((n_groups, n_genes))\n self.vars = np.zeros((n_groups, n_genes))\n self.pts = np.zeros((n_groups, n_genes)) if self.comp_pts else None\n\n if self.ireference is None:\n self.means_rest = np.zeros((n_groups, n_genes))\n self.vars_rest = np.zeros((n_groups, n_genes))\n self.pts_rest = np.zeros((n_groups, n_genes)) if self.comp_pts else None\n else:\n mask_rest = self.groups_masks_obs[self.ireference]\n X_rest = self.X[mask_rest]\n self.means[self.ireference], self.vars[self.ireference] = _get_mean_var(\n X_rest\n )\n # deleting the next line causes a memory leak for some reason\n del X_rest\n\n if issparse(self.X):\n get_nonzeros = lambda X: X.getnnz(axis=0)\n else:\n get_nonzeros = lambda X: np.count_nonzero(X, axis=0)\n\n for group_index, mask_obs in enumerate(self.groups_masks_obs):\n X_mask = self.X[mask_obs]\n\n if self.comp_pts:\n self.pts[group_index] = get_nonzeros(X_mask) / X_mask.shape[0]\n\n if self.ireference is not None and group_index == self.ireference:\n continue\n\n self.means[group_index], self.vars[group_index] = _get_mean_var(X_mask)\n\n if self.ireference is None:\n mask_rest = ~mask_obs\n X_rest = self.X[mask_rest]\n (\n self.means_rest[group_index],\n self.vars_rest[group_index],\n ) = _get_mean_var(X_rest)\n # this can be costly for sparse data\n if self.comp_pts:\n self.pts_rest[group_index] = get_nonzeros(X_rest) / X_rest.shape[0]\n # deleting the next line causes a memory leak for some reason\n del X_rest\n\n def t_test(\n self, method: Literal[\"t-test\", \"t-test_overestim_var\"]\n ) -> Generator[tuple[int, NDArray[np.floating], NDArray[np.floating]], None, None]:\n from scipy import stats\n\n self._basic_stats()\n\n for group_index, (mask_obs, mean_group, var_group) in enumerate(\n zip(self.groups_masks_obs, self.means, self.vars)\n ):\n if self.ireference is not None and group_index == self.ireference:\n continue\n\n ns_group = np.count_nonzero(mask_obs)\n\n if self.ireference is not None:\n mean_rest = self.means[self.ireference]\n var_rest = self.vars[self.ireference]\n ns_other = np.count_nonzero(self.groups_masks_obs[self.ireference])\n else:\n mean_rest = self.means_rest[group_index]\n var_rest = self.vars_rest[group_index]\n ns_other = self.X.shape[0] - ns_group\n\n if method == \"t-test\":\n ns_rest = ns_other\n elif method == \"t-test_overestim_var\":\n # hack for overestimating the variance for small groups\n ns_rest = ns_group\n else:\n raise ValueError(\"Method does not exist.\")\n\n # TODO: Come up with better solution. Mask unexpressed genes?\n # See https://github.com/scipy/scipy/issues/10269\n with np.errstate(invalid=\"ignore\"):\n scores, pvals = stats.ttest_ind_from_stats(\n mean1=mean_group,\n std1=np.sqrt(var_group),\n nobs1=ns_group,\n mean2=mean_rest,\n std2=np.sqrt(var_rest),\n nobs2=ns_rest,\n equal_var=False, # Welch's\n )\n\n # I think it's only nan when means are the same and vars are 0\n scores[np.isnan(scores)] = 0\n # This also has to happen for Benjamini Hochberg\n pvals[np.isnan(pvals)] = 1\n\n yield group_index, scores, pvals\n\n def wilcoxon(\n self, *, tie_correct: bool\n ) -> Generator[tuple[int, NDArray[np.floating], NDArray[np.floating]], None, None]:\n from scipy import stats\n\n self._basic_stats()\n\n n_genes = self.X.shape[1]\n # First loop: Loop over all genes\n if self.ireference is not None:\n # initialize space for z-scores\n scores = np.zeros(n_genes)\n # initialize space for tie correction coefficients\n T = np.zeros(n_genes) if tie_correct else 1\n\n for group_index, mask_obs in enumerate(self.groups_masks_obs):\n if group_index == self.ireference:\n continue\n\n mask_obs_rest = self.groups_masks_obs[self.ireference]\n\n n_active = np.count_nonzero(mask_obs)\n m_active = np.count_nonzero(mask_obs_rest)\n\n if n_active <= 25 or m_active <= 25:\n logg.hint(\n \"Few observations in a group for \"\n \"normal approximation (<=25). Lower test accuracy.\"\n )\n\n # Calculate rank sums for each chunk for the current mask\n for ranks, left, right in _ranks(self.X, mask_obs, mask_obs_rest):\n scores[left:right] = ranks.iloc[0:n_active, :].sum(axis=0)\n if tie_correct:\n T[left:right] = _tiecorrect(ranks)\n\n std_dev = np.sqrt(\n T * n_active * m_active * (n_active + m_active + 1) / 12.0\n )\n\n scores = (\n scores - (n_active * ((n_active + m_active + 1) / 2.0))\n ) / std_dev\n scores[np.isnan(scores)] = 0\n pvals = 2 * stats.distributions.norm.sf(np.abs(scores))\n\n yield group_index, scores, pvals\n # If no reference group exists,\n # ranking needs only to be done once (full mask)\n else:\n n_groups = self.groups_masks_obs.shape[0]\n scores = np.zeros((n_groups, n_genes))\n n_cells = self.X.shape[0]\n\n if tie_correct:\n T = np.zeros((n_groups, n_genes))\n\n for ranks, left, right in _ranks(self.X):\n # sum up adjusted_ranks to calculate W_m,n\n for group_index, mask_obs in enumerate(self.groups_masks_obs):\n scores[group_index, left:right] = ranks.iloc[mask_obs, :].sum(\n axis=0\n )\n if tie_correct:\n T[group_index, left:right] = _tiecorrect(ranks)\n\n for group_index, mask_obs in enumerate(self.groups_masks_obs):\n n_active = np.count_nonzero(mask_obs)\n\n T_i = T[group_index] if tie_correct else 1\n\n std_dev = np.sqrt(\n T_i * n_active * (n_cells - n_active) * (n_cells + 1) / 12.0\n )\n\n scores[group_index, :] = (\n scores[group_index, :] - (n_active * (n_cells + 1) / 2.0)\n ) / std_dev\n scores[np.isnan(scores)] = 0\n pvals = 2 * stats.distributions.norm.sf(np.abs(scores[group_index, :]))\n\n yield group_index, scores[group_index], pvals\n\n def logreg(\n self, **kwds\n ) -> Generator[tuple[int, NDArray[np.floating], None], None, None]:\n # if reference is not set, then the groups listed will be compared to the rest\n # if reference is set, then the groups listed will be compared only to the other groups listed\n from sklearn.linear_model import LogisticRegression\n\n # Indexing with a series causes issues, possibly segfault\n X = self.X[self.grouping_mask.values, :]\n\n if len(self.groups_order) == 1:\n raise ValueError(\"Cannot perform logistic regression on a single cluster.\")\n\n clf = LogisticRegression(**kwds)\n clf.fit(X, self.grouping.cat.codes)\n scores_all = clf.coef_\n # not all codes necessarily appear in data\n existing_codes = np.unique(self.grouping.cat.codes)\n for igroup, cat in enumerate(self.groups_order):\n if len(self.groups_order) <= 2: # binary logistic regression\n scores = scores_all[0]\n else:\n # cat code is index of cat value in .categories\n cat_code: int = np.argmax(self.grouping.cat.categories == cat)\n # index of scores row is index of cat code in array of existing codes\n scores_idx: int = np.argmax(existing_codes == cat_code)\n scores = scores_all[scores_idx]\n yield igroup, scores, None\n\n if len(self.groups_order) <= 2:\n break\n\n def compute_statistics(\n self,\n method: _Method,\n *,\n corr_method: _CorrMethod = \"benjamini-hochberg\",\n n_genes_user: int | None = None,\n rankby_abs: bool = False,\n tie_correct: bool = False,\n **kwds,\n ) -> None:\n if method in {\"t-test\", \"t-test_overestim_var\"}:\n generate_test_results = self.t_test(method)\n elif method == \"wilcoxon\":\n generate_test_results = self.wilcoxon(tie_correct=tie_correct)\n elif method == \"logreg\":\n generate_test_results = self.logreg(**kwds)\n\n self.stats = None\n\n n_genes = self.X.shape[1]\n\n for group_index, scores, pvals in generate_test_results:\n group_name = str(self.groups_order[group_index])\n\n if n_genes_user is not None:\n scores_sort = np.abs(scores) if rankby_abs else scores\n global_indices = _select_top_n(scores_sort, n_genes_user)\n first_col = \"names\"\n else:\n global_indices = slice(None)\n first_col = \"scores\"\n\n if self.stats is None:\n idx = pd.MultiIndex.from_tuples([(group_name, first_col)])\n self.stats = pd.DataFrame(columns=idx)\n\n if n_genes_user is not None:\n self.stats[group_name, \"names\"] = self.var_names[global_indices]\n\n self.stats[group_name, \"scores\"] = scores[global_indices]\n\n if pvals is not None:\n self.stats[group_name, \"pvals\"] = pvals[global_indices]\n if corr_method == \"benjamini-hochberg\":\n from statsmodels.stats.multitest import multipletests\n\n pvals[np.isnan(pvals)] = 1\n _, pvals_adj, _, _ = multipletests(\n pvals, alpha=0.05, method=\"fdr_bh\"\n )\n elif corr_method == \"bonferroni\":\n pvals_adj = np.minimum(pvals * n_genes, 1.0)\n self.stats[group_name, \"pvals_adj\"] = pvals_adj[global_indices]\n\n if self.means is not None:\n mean_group = self.means[group_index]\n if self.ireference is None:\n mean_rest = self.means_rest[group_index]\n else:\n mean_rest = self.means[self.ireference]\n foldchanges = (self.expm1_func(mean_group) + 1e-9) / (\n self.expm1_func(mean_rest) + 1e-9\n ) # add small value to remove 0's\n self.stats[group_name, \"logfoldchanges\"] = np.log2(\n foldchanges[global_indices]\n )\n\n if n_genes_user is None:\n self.stats.index = self.var_names\n\n\n@old_positionals(\n \"mask\",\n \"use_raw\",\n \"groups\",\n \"reference\",\n \"n_genes\",\n \"rankby_abs\",\n \"pts\",\n \"key_added\",\n \"copy\",\n \"method\",\n \"corr_method\",\n \"tie_correct\",\n \"layer\",\n)\ndef rank_genes_groups(\n adata: AnnData,\n groupby: str,\n *,\n mask_var: NDArray[np.bool_] | str | None = None,\n use_raw: bool | None = None,\n groups: Literal[\"all\"] | Iterable[str] = \"all\",\n reference: str = \"rest\",\n n_genes: int | None = None,\n rankby_abs: bool = False,\n pts: bool = False,\n key_added: str | None = None,\n copy: bool = False,\n method: _Method | None = None,\n corr_method: _CorrMethod = \"benjamini-hochberg\",\n tie_correct: bool = False,\n layer: str | None = None,\n **kwds,\n) -> AnnData | None:\n \"\"\"\\\n Rank genes for characterizing groups.\n\n Expects logarithmized data.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n groupby\n The key of the observations grouping to consider.\n mask_var\n Select subset of genes to use in statistical tests.\n use_raw\n Use `raw` attribute of `adata` if present.\n layer\n Key from `adata.layers` whose value will be used to perform tests on.\n groups\n Subset of groups, e.g. [`'g1'`, `'g2'`, `'g3'`], to which comparison\n shall be restricted, or `'all'` (default), for all groups. Note that if\n `reference='rest'` all groups will still be used as the reference, not\n just those specified in `groups`.\n reference\n If `'rest'`, compare each group to the union of the rest of the group.\n If a group identifier, compare with respect to this group.\n n_genes\n The number of genes that appear in the returned tables.\n Defaults to all genes.\n method\n The default method is `'t-test'`,\n `'t-test_overestim_var'` overestimates variance of each group,\n `'wilcoxon'` uses Wilcoxon rank-sum,\n `'logreg'` uses logistic regression. See :cite:t:`Ntranos2019`,\n `here `__ and `here\n `__,\n for why this is meaningful.\n corr_method\n p-value correction method.\n Used only for `'t-test'`, `'t-test_overestim_var'`, and `'wilcoxon'`.\n tie_correct\n Use tie correction for `'wilcoxon'` scores.\n Used only for `'wilcoxon'`.\n rankby_abs\n Rank genes by the absolute value of the score, not by the\n score. The returned scores are never the absolute values.\n pts\n Compute the fraction of cells expressing the genes.\n key_added\n The key in `adata.uns` information is saved to.\n copy\n Whether to copy `adata` or modify it inplace.\n kwds\n Are passed to test methods. Currently this affects only parameters that\n are passed to :class:`sklearn.linear_model.LogisticRegression`.\n For instance, you can pass `penalty='l1'` to try to come up with a\n minimal set of genes that are good predictors (sparse solution meaning\n few non-zero fitted coefficients).\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.uns['rank_genes_groups' | key_added]['names']` : structured :class:`numpy.ndarray` (dtype `object`)\n Structured array to be indexed by group id storing the gene\n names. Ordered according to scores.\n `adata.uns['rank_genes_groups' | key_added]['scores']` : structured :class:`numpy.ndarray` (dtype `object`)\n Structured array to be indexed by group id storing the z-score\n underlying the computation of a p-value for each gene for each\n group. Ordered according to scores.\n `adata.uns['rank_genes_groups' | key_added]['logfoldchanges']` : structured :class:`numpy.ndarray` (dtype `object`)\n Structured array to be indexed by group id storing the log2\n fold change for each gene for each group. Ordered according to\n scores. Only provided if method is 't-test' like.\n Note: this is an approximation calculated from mean-log values.\n `adata.uns['rank_genes_groups' | key_added]['pvals']` : structured :class:`numpy.ndarray` (dtype `float`)\n p-values.\n `adata.uns['rank_genes_groups' | key_added]['pvals_adj']` : structured :class:`numpy.ndarray` (dtype `float`)\n Corrected p-values.\n `adata.uns['rank_genes_groups' | key_added]['pts']` : :class:`pandas.DataFrame` (dtype `float`)\n Fraction of cells expressing the genes for each group.\n `adata.uns['rank_genes_groups' | key_added]['pts_rest']` : :class:`pandas.DataFrame` (dtype `float`)\n Only if `reference` is set to `'rest'`.\n Fraction of cells from the union of the rest of each group\n expressing the genes.\n\n Notes\n -----\n There are slight inconsistencies depending on whether sparse\n or dense data are passed. See `here `__.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> sc.tl.rank_genes_groups(adata, 'bulk_labels', method='wilcoxon')\n >>> # to visualize the results\n >>> sc.pl.rank_genes_groups(adata)\n \"\"\"\n if mask_var is not None:\n mask_var = _check_mask(adata, mask_var, \"var\")\n\n if use_raw is None:\n use_raw = adata.raw is not None\n elif use_raw is True and adata.raw is None:\n raise ValueError(\"Received `use_raw=True`, but `adata.raw` is empty.\")\n\n if method is None:\n method = \"t-test\"\n\n if \"only_positive\" in kwds:\n rankby_abs = not kwds.pop(\"only_positive\") # backwards compat\n\n start = logg.info(\"ranking genes\")\n avail_methods = set(get_args(_Method))\n if method not in avail_methods:\n raise ValueError(f\"Method must be one of {avail_methods}.\")\n\n avail_corr = {\"benjamini-hochberg\", \"bonferroni\"}\n if corr_method not in avail_corr:\n raise ValueError(f\"Correction method must be one of {avail_corr}.\")\n\n adata = adata.copy() if copy else adata\n _utils.sanitize_anndata(adata)\n # for clarity, rename variable\n if groups == \"all\":\n groups_order = \"all\"\n elif isinstance(groups, (str, int)):\n raise ValueError(\"Specify a sequence of groups\")\n else:\n groups_order = list(groups)\n if isinstance(groups_order[0], int):\n groups_order = [str(n) for n in groups_order]\n if reference != \"rest\" and reference not in set(groups_order):\n groups_order += [reference]\n if reference != \"rest\" and reference not in adata.obs[groupby].cat.categories:\n cats = adata.obs[groupby].cat.categories.tolist()\n raise ValueError(\n f\"reference = {reference} needs to be one of groupby = {cats}.\"\n )\n\n if key_added is None:\n key_added = \"rank_genes_groups\"\n adata.uns[key_added] = {}\n adata.uns[key_added][\"params\"] = dict(\n groupby=groupby,\n reference=reference,\n method=method,\n use_raw=use_raw,\n layer=layer,\n corr_method=corr_method,\n )\n\n test_obj = _RankGenes(\n adata,\n groups_order,\n groupby,\n mask_var=mask_var,\n reference=reference,\n use_raw=use_raw,\n layer=layer,\n comp_pts=pts,\n )\n\n if check_nonnegative_integers(test_obj.X) and method != \"logreg\":\n logg.warning(\n \"It seems you use rank_genes_groups on the raw count data. \"\n \"Please logarithmize your data before calling rank_genes_groups.\"\n )\n\n # for clarity, rename variable\n n_genes_user = n_genes\n # make sure indices are not OoB in case there are less genes than n_genes\n # defaults to all genes\n if n_genes_user is None or n_genes_user > test_obj.X.shape[1]:\n n_genes_user = test_obj.X.shape[1]\n\n logg.debug(f\"consider {groupby!r} groups:\")\n logg.debug(f\"with sizes: {np.count_nonzero(test_obj.groups_masks_obs, axis=1)}\")\n\n test_obj.compute_statistics(\n method,\n corr_method=corr_method,\n n_genes_user=n_genes_user,\n rankby_abs=rankby_abs,\n tie_correct=tie_correct,\n **kwds,\n )\n\n if test_obj.pts is not None:\n groups_names = [str(name) for name in test_obj.groups_order]\n adata.uns[key_added][\"pts\"] = pd.DataFrame(\n test_obj.pts.T, index=test_obj.var_names, columns=groups_names\n )\n if test_obj.pts_rest is not None:\n adata.uns[key_added][\"pts_rest\"] = pd.DataFrame(\n test_obj.pts_rest.T, index=test_obj.var_names, columns=groups_names\n )\n\n test_obj.stats.columns = test_obj.stats.columns.swaplevel()\n\n dtypes = {\n \"names\": \"O\",\n \"scores\": \"float32\",\n \"logfoldchanges\": \"float32\",\n \"pvals\": \"float64\",\n \"pvals_adj\": \"float64\",\n }\n\n for col in test_obj.stats.columns.levels[0]:\n adata.uns[key_added][col] = test_obj.stats[col].to_records(\n index=False, column_dtypes=dtypes[col]\n )\n\n logg.info(\n \" finished\",\n time=start,\n deep=(\n f\"added to `.uns[{key_added!r}]`\\n\"\n \" 'names', sorted np.recarray to be indexed by group ids\\n\"\n \" 'scores', sorted np.recarray to be indexed by group ids\\n\"\n + (\n \" 'logfoldchanges', sorted np.recarray to be indexed by group ids\\n\"\n \" 'pvals', sorted np.recarray to be indexed by group ids\\n\"\n \" 'pvals_adj', sorted np.recarray to be indexed by group ids\"\n if method in {\"t-test\", \"t-test_overestim_var\", \"wilcoxon\"}\n else \"\"\n )\n ),\n )\n return adata if copy else None\n\n\ndef _calc_frac(X):\n n_nonzero = X.getnnz(axis=0) if issparse(X) else np.count_nonzero(X, axis=0)\n return n_nonzero / X.shape[0]\n\n\n@old_positionals(\n \"key\",\n \"groupby\",\n \"use_raw\",\n \"key_added\",\n \"min_in_group_fraction\",\n \"min_fold_change\",\n \"max_out_group_fraction\",\n \"compare_abs\",\n)\ndef filter_rank_genes_groups(\n adata: AnnData,\n *,\n key: str | None = None,\n groupby: str | None = None,\n use_raw: bool | None = None,\n key_added: str = \"rank_genes_groups_filtered\",\n min_in_group_fraction: float = 0.25,\n min_fold_change: int | float = 1,\n max_out_group_fraction: float = 0.5,\n compare_abs: bool = False,\n) -> None:\n \"\"\"\\\n Filters out genes based on log fold change and fraction of genes expressing the\n gene within and outside the `groupby` categories.\n\n See :func:`~scanpy.tl.rank_genes_groups`.\n\n Results are stored in `adata.uns[key_added]`\n (default: 'rank_genes_groups_filtered').\n\n To preserve the original structure of adata.uns['rank_genes_groups'],\n filtered genes are set to `NaN`.\n\n Parameters\n ----------\n adata\n key\n groupby\n use_raw\n key_added\n min_in_group_fraction\n min_fold_change\n max_out_group_fraction\n compare_abs\n If `True`, compare absolute values of log fold change with `min_fold_change`.\n\n Returns\n -------\n Same output as :func:`scanpy.tl.rank_genes_groups` but with filtered genes names set to\n `nan`\n\n Examples\n --------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> sc.tl.rank_genes_groups(adata, 'bulk_labels', method='wilcoxon')\n >>> sc.tl.filter_rank_genes_groups(adata, min_fold_change=3)\n >>> # visualize results\n >>> sc.pl.rank_genes_groups(adata, key='rank_genes_groups_filtered')\n >>> # visualize results using dotplot\n >>> sc.pl.rank_genes_groups_dotplot(adata, key='rank_genes_groups_filtered')\n \"\"\"\n if key is None:\n key = \"rank_genes_groups\"\n\n if groupby is None:\n groupby = adata.uns[key][\"params\"][\"groupby\"]\n\n if use_raw is None:\n use_raw = adata.uns[key][\"params\"][\"use_raw\"]\n\n same_params = (\n adata.uns[key][\"params\"][\"groupby\"] == groupby\n and adata.uns[key][\"params\"][\"reference\"] == \"rest\"\n and adata.uns[key][\"params\"][\"use_raw\"] == use_raw\n )\n\n use_logfolds = same_params and \"logfoldchanges\" in adata.uns[key]\n use_fraction = same_params and \"pts_rest\" in adata.uns[key]\n\n # convert structured numpy array into DataFrame\n gene_names = pd.DataFrame(adata.uns[key][\"names\"])\n\n fraction_in_cluster_matrix = pd.DataFrame(\n np.zeros(gene_names.shape),\n columns=gene_names.columns,\n index=gene_names.index,\n )\n fraction_out_cluster_matrix = pd.DataFrame(\n np.zeros(gene_names.shape),\n columns=gene_names.columns,\n index=gene_names.index,\n )\n\n if use_logfolds:\n fold_change_matrix = pd.DataFrame(adata.uns[key][\"logfoldchanges\"])\n else:\n fold_change_matrix = pd.DataFrame(\n np.zeros(gene_names.shape),\n columns=gene_names.columns,\n index=gene_names.index,\n )\n\n if (base := adata.uns.get(\"log1p\", {}).get(\"base\")) is not None:\n expm1_func = lambda x: np.expm1(x * np.log(base))\n else:\n expm1_func = np.expm1\n\n logg.info(\n f\"Filtering genes using: \"\n f\"min_in_group_fraction: {min_in_group_fraction} \"\n f\"min_fold_change: {min_fold_change}, \"\n f\"max_out_group_fraction: {max_out_group_fraction}\"\n )\n\n for cluster in gene_names.columns:\n # iterate per column\n var_names = gene_names[cluster].values\n\n if not use_logfolds or not use_fraction:\n sub_X = adata.raw[:, var_names].X if use_raw else adata[:, var_names].X\n in_group = adata.obs[groupby] == cluster\n X_in = sub_X[in_group]\n X_out = sub_X[~in_group]\n\n if use_fraction:\n fraction_in_cluster_matrix.loc[:, cluster] = (\n adata.uns[key][\"pts\"][cluster].loc[var_names].values\n )\n fraction_out_cluster_matrix.loc[:, cluster] = (\n adata.uns[key][\"pts_rest\"][cluster].loc[var_names].values\n )\n else:\n fraction_in_cluster_matrix.loc[:, cluster] = _calc_frac(X_in)\n fraction_out_cluster_matrix.loc[:, cluster] = _calc_frac(X_out)\n\n if not use_logfolds:\n # compute mean value\n mean_in_cluster = np.ravel(X_in.mean(0))\n mean_out_cluster = np.ravel(X_out.mean(0))\n # compute fold change\n fold_change_matrix.loc[:, cluster] = np.log2(\n (expm1_func(mean_in_cluster) + 1e-9)\n / (expm1_func(mean_out_cluster) + 1e-9)\n )\n\n if compare_abs:\n fold_change_matrix = fold_change_matrix.abs()\n # filter original_matrix\n gene_names = gene_names[\n (fraction_in_cluster_matrix > min_in_group_fraction)\n & (fraction_out_cluster_matrix < max_out_group_fraction)\n & (fold_change_matrix > min_fold_change)\n ]\n # create new structured array using 'key_added'.\n adata.uns[key_added] = adata.uns[key].copy()\n adata.uns[key_added][\"names\"] = gene_names.to_records(index=False)\n\n\n# Author: Alex Wolf (https://falexwolf.de)\n\"\"\"Simulate Data\n\nSimulate stochastic dynamic systems to model gene expression dynamics and\ncause-effect data.\n\nTODO\n----\nBeta Version. The code will be reorganized soon.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport itertools\nimport shutil\nimport sys\nfrom pathlib import Path\nfrom types import MappingProxyType\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport scipy as sp\n\nfrom .. import _utils, readwrite\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._settings import settings\n\nif TYPE_CHECKING:\n from collections.abc import Mapping\n from typing import Literal\n\n from anndata import AnnData\n\n\n@old_positionals(\n \"params_file\",\n \"tmax\",\n \"branching\",\n \"nrRealizations\",\n \"noiseObs\",\n \"noiseDyn\",\n \"step\",\n \"seed\",\n \"writedir\",\n)\ndef sim(\n model: Literal[\"krumsiek11\", \"toggleswitch\"],\n *,\n params_file: bool = True,\n tmax: int | None = None,\n branching: bool | None = None,\n nrRealizations: int | None = None,\n noiseObs: float | None = None,\n noiseDyn: float | None = None,\n step: int | None = None,\n seed: int | None = None,\n writedir: Path | str | None = None,\n) -> AnnData:\n \"\"\"\\\n Simulate dynamic gene expression data :cite:p:`Wittmann2009` :cite:p:`Wolf2018`.\n\n Sample from a stochastic differential equation model built from\n literature-curated boolean gene regulatory networks, as suggested by\n :cite:t:`Wittmann2009`. The Scanpy implementation is due to :cite:t:`Wolf2018`.\n\n Parameters\n ----------\n model\n Model file in 'sim_models' directory.\n params_file\n Read default params from file.\n tmax\n Number of time steps per realization of time series.\n branching\n Only write realizations that contain new branches.\n nrRealizations\n Number of realizations.\n noiseObs\n Observatory/Measurement noise.\n noiseDyn\n Dynamic noise.\n step\n Interval for saving state of system.\n seed\n Seed for generation of random numbers.\n writedir\n Path to directory for writing output files.\n\n Returns\n -------\n Annotated data matrix.\n\n Examples\n --------\n See this `use case `__\n \"\"\"\n params = locals()\n if params_file:\n model_key = Path(model).with_suffix(\"\").name\n from .. import sim_models\n\n pfile_sim = Path(sim_models.__file__).parent / f\"{model_key}_params.txt\"\n default_params = readwrite.read_params(pfile_sim)\n params = _utils.update_params(default_params, params)\n adata = sample_dynamic_data(**params)\n adata.uns[\"iroot\"] = 0\n return adata\n\n\ndef add_args(p):\n \"\"\"\n Update parser with tool specific arguments.\n\n This overwrites was is done in utils.uns_args.\n \"\"\"\n # dictionary for adding arguments\n dadd_args = {\n \"--opfile\": {\n \"default\": \"\",\n \"metavar\": \"f\",\n \"type\": str,\n \"help\": \"Specify a parameter file \" '(default: \"sim/${exkey}_params.txt\")',\n }\n }\n p = _utils.add_args(p, dadd_args)\n return p\n\n\ndef sample_dynamic_data(**params):\n \"\"\"\n Helper function.\n \"\"\"\n model_key = Path(params[\"model\"]).with_suffix(\"\").name\n writedir = params.get(\"writedir\")\n if writedir is None:\n writedir = settings.writedir / (model_key + \"_sim\")\n else:\n writedir = Path(writedir)\n writedir.mkdir(parents=True, exist_ok=True)\n readwrite.write_params(writedir / \"params.txt\", params)\n # init variables\n tmax = params[\"tmax\"]\n branching = params[\"branching\"]\n noiseObs = params[\"noiseObs\"]\n noiseDyn = params[\"noiseDyn\"]\n nrRealizations = params[\"nrRealizations\"]\n step = params[\"step\"] # step size for saving the figure\n\n nrSamples = 1 # how many files?\n maxRestarts = 1000\n maxNrSamples = 1\n\n # simple vector auto regressive process or\n # hill kinetics process simulation\n if \"krumsiek11\" not in model_key:\n # create instance, set seed\n grnsim = GRNsim(model=model_key, params=params)\n nrOffEdges_list = np.zeros(nrSamples)\n for sample in range(nrSamples):\n # random topology / for a given edge density\n if \"hill\" not in model_key:\n Coupl = np.array(grnsim.Coupl)\n for sampleCoupl in range(10):\n nrOffEdges = 0\n for gp in range(grnsim.dim):\n for g in range(grnsim.dim):\n # only consider off-diagonal edges\n if g != gp:\n Coupl[gp, g] = 0.7 if np.random.rand() < 0.4 else 0\n nrOffEdges += 1 if Coupl[gp, g] > 0 else 0\n else:\n Coupl[gp, g] = 0.7\n # check that the coupling matrix does not have eigenvalues\n # greater than 1, which would lead to an exploding var process\n if max(sp.linalg.eig(Coupl)[0]) < 1:\n break\n nrOffEdges_list[sample] = nrOffEdges\n grnsim.set_coupl(Coupl)\n # init type\n real = 0\n X0 = np.random.rand(grnsim.dim)\n Xsamples = []\n for restart in range(nrRealizations + maxRestarts):\n # slightly break symmetry in initial conditions\n if \"toggleswitch\" in model_key:\n X0 = np.array(\n [0.8 for i in range(grnsim.dim)]\n ) + 0.01 * np.random.randn(grnsim.dim)\n X = grnsim.sim_model(tmax=tmax, X0=X0, noiseDyn=noiseDyn)\n # check branching\n check = True\n if branching:\n check, Xsamples = _check_branching(X, Xsamples, restart)\n if check:\n real += 1\n grnsim.write_data(\n X[::step],\n dir=writedir,\n noiseObs=noiseObs,\n append=restart != 0,\n branching=branching,\n nrRealizations=nrRealizations,\n )\n # append some zeros\n if \"zeros\" in writedir.name and real == 2:\n grnsim.write_data(\n noiseDyn * np.random.randn(500, 3),\n dir=writedir,\n noiseObs=noiseObs,\n append=restart != 0,\n branching=branching,\n nrRealizations=nrRealizations,\n )\n if real >= nrRealizations:\n break\n logg.debug(\n f\"mean nr of offdiagonal edges {nrOffEdges_list.mean()} \"\n f\"compared to total nr {grnsim.dim * (grnsim.dim - 1) / 2.}\"\n )\n\n # more complex models\n else:\n initType = \"random\"\n\n dim = 11\n step = 5\n\n grnsim = GRNsim(dim=dim, initType=initType, model=model_key, params=params)\n Xsamples = []\n for sample in range(maxNrSamples):\n # choose initial conditions such that branchings result\n if initType == \"branch\":\n X0mean = grnsim.branch_init_model1(tmax)\n if X0mean is None:\n grnsim.set_coupl()\n continue\n real = 0\n for restart in range(nrRealizations + maxRestarts):\n if initType == \"branch\":\n # vary initial conditions around mean\n X0 = X0mean + (0.05 * np.random.rand(dim) - 0.025 * np.ones(dim))\n else:\n # generate random initial conditions within [0.3,0.7]\n X0 = 0.4 * np.random.rand(dim) + 0.3\n if model_key in [5, 6]:\n X0 = np.array([0.3, 0.3, 0, 0, 0, 0])\n if model_key in [7, 8, 9, 10]:\n X0 = 0.6 * np.random.rand(dim) + 0.2\n X0[2:] = np.zeros(4)\n if \"krumsiek11\" in model_key:\n X0 = np.zeros(dim)\n X0[grnsim.varNames[\"Gata2\"]] = 0.8\n X0[grnsim.varNames[\"Pu.1\"]] = 0.8\n X0[grnsim.varNames[\"Cebpa\"]] = 0.8\n X0 += 0.001 * np.random.randn(dim)\n if False:\n switch_gene = restart - (nrRealizations - dim)\n if switch_gene >= dim:\n break\n X0[switch_gene] = 0 if X0[switch_gene] > 0.1 else 0.8\n X = grnsim.sim_model(tmax, X0=X0, noiseDyn=noiseDyn, restart=restart)\n # check branching\n check = True\n if branching:\n check, Xsamples = _check_branching(X, Xsamples, restart)\n if check:\n real += 1\n grnsim.write_data(\n X[::step],\n dir=writedir,\n noiseObs=noiseObs,\n append=restart != 0,\n branching=branching,\n nrRealizations=nrRealizations,\n )\n if real >= nrRealizations:\n break\n # load the last simulation file\n filename = None\n for filename in writedir.glob(\"sim*.txt\"):\n pass\n logg.info(f\"reading simulation results {filename}\")\n adata = readwrite._read(\n filename, first_column_names=True, suppress_cache_warning=True\n )\n adata.uns[\"tmax_write\"] = tmax / step\n return adata\n\n\ndef write_data(\n X,\n *,\n dir=Path(\"sim/test\"),\n append=False,\n header=\"\",\n varNames: Mapping[str, int] = MappingProxyType({}),\n Adj=np.array([]),\n Coupl=np.array([]),\n boolRules: Mapping[str, str] = MappingProxyType({}),\n model=\"\",\n modelType=\"\",\n invTimeStep=1,\n):\n \"\"\"Write simulated data.\n\n Accounts for saving at the same time an ID\n and a model file.\n \"\"\"\n dir.mkdir(parents=True, exist_ok=True)\n # update file with sample ids\n filename = dir / \"id.txt\"\n if filename.is_file():\n with filename.open(\"r\") as f:\n id = int(f.read()) + (0 if append else 1)\n else:\n id = 0\n with filename.open(\"w\") as f:\n id = f\"{id:0>6}\"\n f.write(str(id))\n # dimension\n dim = X.shape[1]\n # write files with adjacancy and coupling matrices\n if not append:\n if False:\n if Adj.size > 0:\n # due to 'update formulation' of model, there\n # is always a diagonal dependence\n Adj = np.copy(Adj)\n if \"hill\" in model:\n for i in range(Adj.shape[0]):\n Adj[i, i] = 1\n np.savetxt(dir + \"/adj_\" + id + \".txt\", Adj, header=header, fmt=\"%d\")\n if Coupl.size > 0:\n np.savetxt(\n dir + \"/coupl_\" + id + \".txt\", Coupl, header=header, fmt=\"%10.6f\"\n )\n # write model file\n if varNames and Coupl.size > 0:\n with (dir / f\"model_{id}.txt\").open(\"w\") as f:\n f.write('# For each \"variable = \", there must be a right hand side: \\n')\n f.write(\n \"# either an empty string or a python-style logical expression \\n\"\n )\n f.write('# involving variable names, \"or\", \"and\", \"(\", \")\". \\n')\n f.write(\"# The order of equations matters! \\n\")\n f.write(\"# \\n\")\n f.write(\"# modelType = \" + modelType + \"\\n\")\n f.write(\"# invTimeStep = \" + str(invTimeStep) + \"\\n\")\n f.write(\"# \\n\")\n f.write(\"# boolean update rules: \\n\")\n for k, v in boolRules.items():\n f.write(f\"{k} = {v}\\n\")\n # write coupling via names\n f.write(\"# coupling list: \\n\")\n names = list(varNames.keys())\n for gp in range(dim):\n for g in range(dim):\n if np.abs(Coupl[gp, g]) > 1e-10:\n f.write(\n f\"{names[gp]:10} \"\n f\"{names[g]:10} \"\n f\"{Coupl[gp, g]:10.3} \\n\"\n )\n # write simulated data\n # the binary mode option in the following line is a fix for python 3\n # variable names\n if varNames:\n header += f'{\"it\":>2} '\n for v in varNames:\n header += f\"{v:>7} \"\n with (dir / f\"sim_{id}.txt\").open(\"ab\" if append else \"wb\") as f:\n np.savetxt(\n f,\n np.c_[np.arange(0, X.shape[0]), X],\n header=(\"\" if append else header),\n fmt=[\"%4.f\"] + [\"%7.4f\" for i in range(dim)],\n )\n\n\nclass GRNsim:\n \"\"\"\n Simlulation of stochastic dynamic systems.\n\n Main application: simulation of gene expression dynamics.\n\n Also standard models are implemented.\n \"\"\"\n\n availModels = dict(\n krumsiek11=(\n \"myeloid progenitor network, Krumsiek et al., PLOS One 6, e22649, \"\n \"\\n equations from Table 1 on page 3, \"\n \"doi:10.1371/journal.pone.0022649 \\n\"\n ),\n var=\"vector autoregressive process \\n\",\n hill=\"process with hill kinetics \\n\",\n )\n\n writeOutputOnce = True\n\n def __init__(\n self,\n *,\n dim=3,\n model=\"ex0\",\n modelType=\"var\",\n initType=\"random\",\n show=False,\n verbosity=0,\n Coupl=None,\n params=MappingProxyType({}),\n ):\n \"\"\"\n Params\n ------\n model\n either string for predefined model,\n or directory with a model file and a couple matrix files\n \"\"\"\n # number of nodes / dimension of system\n self.dim = dim if Coupl is None else Coupl.shape[0]\n self.maxnpar = 1 # maximal number of parents\n self.p_indep = 0.4 # fraction of independent genes\n self.model = model\n self.modelType = modelType\n self.initType = initType # string characterizing a specific initial\n self.show = show\n self.verbosity = verbosity\n # checks\n if initType not in [\"branch\", \"random\"]:\n raise RuntimeError(\"initType must be either: branch, random\")\n if model not in self.availModels:\n message = \"model not among predefined models \\n\" # noqa: F841 # TODO FIX\n # read from file\n from .. import sim_models\n\n model = Path(sim_models.__file__).parent / f\"{model}.txt\"\n if not model.is_file():\n raise RuntimeError(f\"Model file {model} does not exist\")\n self.model = model\n # set the coupling matrix, and with that the adjacency matrix\n self.set_coupl(Coupl=Coupl)\n # seed\n np.random.seed(params[\"seed\"])\n # header\n self.header = \"model = \" + self.model.name + \" \\n\"\n # params\n self.params = params\n\n def sim_model(self, tmax, X0, noiseDyn=0, restart=0):\n \"\"\"Simulate the model.\"\"\"\n self.noiseDyn = noiseDyn\n #\n X = np.zeros((tmax, self.dim))\n X[0] = X0 + noiseDyn * np.random.randn(self.dim)\n # run simulation\n for t in range(1, tmax):\n if self.modelType == \"hill\":\n Xdiff = self.Xdiff_hill(X[t - 1])\n elif self.modelType == \"var\":\n Xdiff = self.Xdiff_var(X[t - 1])\n else:\n raise ValueError(f\"Unknown modelType {self.modelType!r}\")\n X[t] = X[t - 1] + Xdiff\n # add dynamic noise\n X[t] += noiseDyn * np.random.randn(self.dim)\n return X\n\n def Xdiff_hill(self, Xt):\n \"\"\"Build Xdiff from coefficients of boolean network,\n that is, using self.boolCoeff. The employed functions\n are Hill type activation and deactivation functions.\n\n See Wittmann et al., BMC Syst. Biol. 3, 98 (2009),\n doi:10.1186/1752-0509-3-98 for more details.\n \"\"\"\n verbosity = self.verbosity > 0 and self.writeOutputOnce\n self.writeOutputOnce = False\n Xdiff = np.zeros(self.dim)\n for ichild, child in enumerate(self.pas.keys()):\n # check whether list of parents is non-empty,\n # otherwise continue\n if self.pas[child]:\n Xdiff_syn = 0 # synthesize term\n if verbosity > 0:\n Xdiff_syn_str = \"\"\n else:\n continue\n # loop over all tuples for which the boolean update\n # rule returns true, these are stored in self.boolCoeff\n for ituple, tuple in enumerate(self.boolCoeff[child]):\n Xdiff_syn_tuple = 1\n Xdiff_syn_tuple_str = \"\"\n for iv, v in enumerate(tuple):\n iparent = self.varNames[self.pas[child][iv]]\n x = Xt[iparent]\n threshold = 0.1 / np.abs(self.Coupl[ichild, iparent])\n Xdiff_syn_tuple *= (\n self.hill_a(x, threshold) if v else self.hill_i(x, threshold)\n )\n if verbosity > 0:\n Xdiff_syn_tuple_str += (\n f'{\"a\" if v else \"i\"}'\n f\"({self.pas[child][iv]}, {threshold:.2})\"\n )\n Xdiff_syn += Xdiff_syn_tuple\n if verbosity > 0:\n Xdiff_syn_str += (\"+\" if ituple != 0 else \"\") + Xdiff_syn_tuple_str\n # multiply with degradation term\n Xdiff[ichild] = self.invTimeStep * (Xdiff_syn - Xt[ichild])\n if verbosity > 0:\n Xdiff_str = (\n f\"{child}_{child}-{child} = \"\n f\"{self.invTimeStep}*({Xdiff_syn_str}-{child})\"\n )\n settings.m(0, Xdiff_str)\n return Xdiff\n\n def Xdiff_var(self, Xt, verbosity=0):\n \"\"\"\"\"\"\n # subtract the current state\n Xdiff = -Xt\n # add the information from the past\n Xdiff += np.dot(self.Coupl, Xt)\n return Xdiff\n\n def hill_a(self, x, threshold=0.1, power=2):\n \"\"\"Activating hill function.\"\"\"\n x_pow = np.power(x, power)\n threshold_pow = np.power(threshold, power)\n return x_pow / (x_pow + threshold_pow)\n\n def hill_i(self, x, threshold=0.1, power=2):\n \"\"\"Inhibiting hill function.\n\n Is equivalent to 1-hill_a(self,x,power,threshold).\n \"\"\"\n x_pow = np.power(x, power)\n threshold_pow = np.power(threshold, power)\n return threshold_pow / (x_pow + threshold_pow)\n\n def nhill_a(self, x, threshold=0.1, power=2, ichild=2):\n \"\"\"Normalized activating hill function.\"\"\"\n x_pow = np.power(x, power)\n threshold_pow = np.power(threshold, power)\n return x_pow / (x_pow + threshold_pow) * (1 + threshold_pow)\n\n def nhill_i(self, x, threshold=0.1, power=2):\n \"\"\"Normalized inhibiting hill function.\n\n Is equivalent to 1-nhill_a(self,x,power,threshold).\n \"\"\"\n x_pow = np.power(x, power)\n threshold_pow = np.power(threshold, power)\n return threshold_pow / (x_pow + threshold_pow) * (1 - x_pow)\n\n def read_model(self):\n \"\"\"Read the model and the couplings from the model file.\"\"\"\n if self.verbosity > 0:\n settings.m(0, \"reading model\", self.model)\n # read model\n boolRules = []\n for line in self.model.open():\n if line.startswith(\"#\") and \"modelType =\" in line:\n keyval = line\n if \"|\" in line:\n keyval, type = line.split(\"|\")[:2]\n self.modelType = keyval.split(\"=\")[1].strip()\n if line.startswith(\"#\") and \"invTimeStep =\" in line:\n keyval = line\n if \"|\" in line:\n keyval, type = line.split(\"|\")[:2]\n self.invTimeStep = float(keyval.split(\"=\")[1].strip())\n if not line.startswith(\"#\"):\n boolRules.append([s.strip() for s in line.split(\"=\")])\n if line.startswith(\"# coupling list:\"):\n break\n self.dim = len(boolRules)\n self.boolRules = dict(boolRules)\n self.varNames = {s: i for i, s in enumerate(self.boolRules.keys())}\n names = self.varNames\n # read couplings via names\n self.Coupl = np.zeros((self.dim, self.dim))\n boolContinue = True\n for (\n line\n ) in self.model.open(): # open(self.model.replace('/model','/couplList')):\n if line.startswith(\"# coupling list:\"):\n boolContinue = False\n if boolContinue:\n continue\n if not line.startswith(\"#\"):\n gps, gs, val = line.strip().split()\n self.Coupl[int(names[gps]), int(names[gs])] = float(val)\n # adjancecy matrices\n self.Adj_signed = np.sign(self.Coupl)\n self.Adj = np.abs(np.array(self.Adj_signed))\n # build bool coefficients (necessary for odefy type\n # version of the discrete model)\n self.build_boolCoeff()\n\n def set_coupl(self, Coupl=None):\n \"\"\"Construct the coupling matrix (and adjacancy matrix) from predefined models\n or via sampling.\n \"\"\"\n self.varNames = {str(i): i for i in range(self.dim)}\n if self.model not in self.availModels and Coupl is None:\n self.read_model()\n elif \"var\" in self.model.name:\n # vector auto regressive process\n self.Coupl = Coupl\n self.boolRules = {s: \"\" for s in self.varNames}\n names = list(self.varNames.keys())\n for gp in range(self.dim):\n pas = []\n for g in range(self.dim):\n if np.abs(self.Coupl[gp, g] > 1e-10):\n pas.append(names[g])\n self.boolRules[names[gp]] = \"\".join(\n pas[:1] + [\" or \" + pa for pa in pas[1:]]\n )\n self.Adj_signed = np.sign(Coupl)\n elif self.model in [\"6\", \"7\", \"8\", \"9\", \"10\"]:\n self.Adj_signed = np.zeros((self.dim, self.dim))\n n_sinknodes = 2\n # sinknodes = np.random.choice(np.arange(0,self.dim),\n # size=n_sinknodes,replace=False)\n sinknodes = np.array([0, 1])\n # assume sinknodes have feeback\n self.Adj_signed[sinknodes, sinknodes] = np.ones(n_sinknodes)\n # # allow negative feedback\n # if self.model == 10:\n # plus_minus = (np.random.randint(0,2,n_sinknodes) - 0.5)*2\n # self.Adj_signed[sinknodes,sinknodes] = plus_minus\n leafnodes = np.array(sinknodes)\n availnodes = np.array([i for i in range(self.dim) if i not in sinknodes])\n # settings.m(0,leafnodes,availnodes)\n while len(availnodes) != 0:\n # parent\n parent_idx = np.random.choice(\n np.arange(0, len(leafnodes)), size=1, replace=False\n )\n parent = leafnodes[parent_idx]\n # children\n children_ids = np.random.choice(\n np.arange(0, len(availnodes)), size=2, replace=False\n )\n children = availnodes[children_ids]\n settings.m(0, parent, children)\n self.Adj_signed[children, parent] = np.ones(2)\n if self.model == 8:\n self.Adj_signed[children[0], children[1]] = -1\n if self.model in [9, 10]:\n self.Adj_signed[children[0], children[1]] = -1\n self.Adj_signed[children[1], children[0]] = -1\n # update leafnodes\n leafnodes = np.delete(leafnodes, parent_idx)\n leafnodes = np.append(leafnodes, children)\n # update availnodes\n availnodes = np.delete(availnodes, children_ids)\n # settings.m(0,availnodes)\n # settings.m(0,leafnodes)\n # settings.m(0,self.Adj)\n # settings.m(0,'-')\n else:\n self.Adj = np.zeros((self.dim, self.dim))\n for i in range(self.dim):\n indep = np.random.binomial(1, self.p_indep)\n if indep == 0:\n # this number includes parents (other variables)\n # and the variable itself, therefore its\n # self.maxnpar+2 in the following line\n nr = np.random.randint(1, self.maxnpar + 2)\n j_par = np.random.choice(\n np.arange(0, self.dim), size=nr, replace=False\n )\n self.Adj[i, j_par] = 1\n else:\n self.Adj[i, i] = 1\n #\n self.Adj = np.abs(np.array(self.Adj_signed))\n # settings.m(0,self.Adj)\n\n def set_coupl_old(self):\n \"\"\"Using the adjacency matrix, sample a coupling matrix.\"\"\"\n if self.model == \"krumsiek11\" or self.model == \"var\":\n # we already built the coupling matrix in set_coupl20()\n return\n self.Coupl = np.zeros((self.dim, self.dim))\n for i in range(self.Adj.shape[0]):\n for j, a in enumerate(self.Adj[i]):\n # if there is a 1 in Adj, specify co and antiregulation\n # and strength of regulation\n if a != 0:\n co_anti = np.random.randint(2)\n # set a lower bound for the coupling parameters\n # they ought not to be smaller than 0.1\n # and not be larger than 0.4\n self.Coupl[i, j] = 0.0 * np.random.rand() + 0.1\n # set sign for coupling\n if co_anti == 1:\n self.Coupl[i, j] *= -1\n # enforce certain requirements on models\n if self.model == 1:\n self.coupl_model1()\n elif self.model == 5:\n self.coupl_model5()\n elif self.model in [6, 7]:\n self.coupl_model6()\n elif self.model in [8, 9, 10]:\n self.coupl_model8()\n # output\n if self.verbosity > 1:\n settings.m(0, self.Coupl)\n\n def coupl_model1(self):\n \"\"\"In model 1, we want enforce the following signs\n on the couplings. Model 2 has the same couplings\n but arbitrary signs.\n \"\"\"\n self.Coupl[0, 0] = np.abs(self.Coupl[0, 0])\n self.Coupl[0, 1] = -np.abs(self.Coupl[0, 1])\n self.Coupl[1, 1] = np.abs(self.Coupl[1, 1])\n\n def coupl_model5(self):\n \"\"\"Toggle switch.\"\"\"\n self.Coupl = -0.2 * self.Adj\n self.Coupl[2, 0] *= -1\n self.Coupl[3, 0] *= -1\n self.Coupl[4, 1] *= -1\n self.Coupl[5, 1] *= -1\n\n def coupl_model6(self):\n \"\"\"Variant of toggle switch.\"\"\"\n self.Coupl = 0.5 * self.Adj_signed\n\n def coupl_model8(self):\n \"\"\"Variant of toggle switch.\"\"\"\n self.Coupl = 0.5 * self.Adj_signed\n # reduce the value of the coupling of the repressing genes\n # otherwise completely unstable solutions are obtained\n for x in np.nditer(self.Coupl, op_flags=[\"readwrite\"]):\n if x < -1e-6:\n x[...] = -0.2\n\n def coupl_model_krumsiek11(self):\n \"\"\"Variant of toggle switch.\"\"\"\n self.Coupl = self.Adj_signed\n\n def sim_model_back_help(self, Xt, Xt1):\n \"\"\"Yields zero when solved for X_t\n given X_{t+1}.\n \"\"\"\n return -Xt1 + Xt + self.Xdiff(Xt)\n\n def sim_model_backwards(self, tmax, X0):\n \"\"\"Simulate the model backwards in time.\"\"\"\n X = np.zeros((tmax, self.dim))\n X[tmax - 1] = X0\n for t in range(tmax - 2, -1, -1):\n sol = sp.optimize.root(\n self.sim_model_back_help, X[t + 1], args=(X[t + 1]), method=\"hybr\"\n )\n X[t] = sol.x\n return X\n\n def branch_init_model1(self, tmax=100):\n # check whether we can define trajectories\n Xfix = np.array([self.Coupl[0, 1] / self.Coupl[0, 0], 1])\n if Xfix[0] > 0.97 or Xfix[0] < 0.03:\n settings.m(\n 0,\n \"... either no fixed point in [0,1]^2! \\n\"\n + \" or fixed point is too close to bounds\",\n )\n return None\n #\n XbackUp = self.sim_model_backwards(\n tmax=tmax / 3, X0=Xfix + np.array([0.02, -0.02])\n )\n XbackDo = self.sim_model_backwards(\n tmax=tmax / 3, X0=Xfix + np.array([-0.02, -0.02])\n )\n #\n Xup = self.sim_model(tmax=tmax, X0=XbackUp[0])\n Xdo = self.sim_model(tmax=tmax, X0=XbackDo[0])\n # compute mean\n X0mean = 0.5 * (Xup[0] + Xdo[0])\n #\n if np.min(X0mean) < 0.025 or np.max(X0mean) > 0.975:\n settings.m(0, \"... initial point is too close to bounds\")\n return None\n if self.show and self.verbosity > 1:\n pl.figure() # noqa: F821 TODO Fix me\n pl.plot(XbackUp[:, 0], \".b\", XbackUp[:, 1], \".g\") # noqa: F821 TODO Fix me\n pl.plot(XbackDo[:, 0], \".b\", XbackDo[:, 1], \".g\") # noqa: F821 TODO Fix me\n pl.plot(Xup[:, 0], \"b\", Xup[:, 1], \"g\") # noqa: F821 TODO Fix me\n pl.plot(Xdo[:, 0], \"b\", Xdo[:, 1], \"g\") # noqa: F821 TODO Fix me\n return X0mean\n\n def parents_from_boolRule(self, rule):\n \"\"\"Determine parents based on boolean updaterule.\n\n Returns list of parents.\n \"\"\"\n rule_pa = (\n rule.replace(\"(\", \"\")\n .replace(\")\", \"\")\n .replace(\"or\", \"\")\n .replace(\"and\", \"\")\n .replace(\"not\", \"\")\n )\n rule_pa = rule_pa.split()\n # if there are no parents, continue\n if not rule_pa:\n return []\n # check whether these are meaningful parents\n pa_old = []\n pa_delete = []\n for pa in rule_pa:\n if pa not in self.varNames:\n settings.m(0, \"list of available variables:\")\n settings.m(0, list(self.varNames.keys()))\n message = (\n 'processing of rule \"'\n + rule\n + \" yields an invalid parent: \"\n + pa\n + \" | check whether the syntax is correct: \\n\"\n + 'only python expressions \"(\",\")\",\"or\",\"and\",\"not\" '\n + \"are allowed, variable names and expressions have to be separated \"\n + \"by white spaces\"\n )\n raise ValueError(message)\n if pa in pa_old:\n pa_delete.append(pa)\n for pa in pa_delete:\n rule_pa.remove(pa)\n return rule_pa\n\n def build_boolCoeff(self):\n \"\"\"Compute coefficients for tuple space.\"\"\"\n # coefficients for hill functions from boolean update rules\n self.boolCoeff = {s: [] for s in self.varNames}\n # parents\n self.pas = {s: [] for s in self.varNames}\n #\n for key, rule in self.boolRules.items():\n self.pas[key] = self.parents_from_boolRule(rule)\n pasIndices = [self.varNames[pa] for pa in self.pas[key]]\n # check whether there are coupling matrix entries for each parent\n for g in range(self.dim):\n if g in pasIndices:\n if np.abs(self.Coupl[self.varNames[key], g]) < 1e-10:\n raise ValueError(f\"specify coupling value for {key} <- {g}\")\n else:\n if np.abs(self.Coupl[self.varNames[key], g]) > 1e-10:\n raise ValueError(\n \"there should be no coupling value for \" f\"{key} <- {g}\"\n )\n if self.verbosity > 1:\n settings.m(0, \"...\" + key)\n settings.m(0, rule)\n settings.m(0, rule_pa) # noqa: F821\n # now evaluate coefficients\n for tuple in list(\n itertools.product([False, True], repeat=len(self.pas[key]))\n ):\n if self.process_rule(rule, self.pas[key], tuple):\n self.boolCoeff[key].append(tuple)\n #\n if self.verbosity > 1:\n settings.m(0, self.boolCoeff[key])\n\n def process_rule(self, rule, pa, tuple):\n \"\"\"Process a string that denotes a boolean rule.\"\"\"\n for i, v in enumerate(tuple):\n rule = rule.replace(pa[i], str(v))\n return eval(rule)\n\n def write_data(\n self,\n X,\n *,\n dir=Path(\"sim/test\"),\n noiseObs=0.0,\n append=False,\n branching=False,\n nrRealizations=1,\n seed=0,\n ):\n header = self.header\n tmax = int(X.shape[0])\n header += \"tmax = \" + str(tmax) + \"\\n\"\n header += \"branching = \" + str(branching) + \"\\n\"\n header += \"nrRealizations = \" + str(nrRealizations) + \"\\n\"\n header += \"noiseObs = \" + str(noiseObs) + \"\\n\"\n header += \"noiseDyn = \" + str(self.noiseDyn) + \"\\n\"\n header += \"seed = \" + str(seed) + \"\\n\"\n # add observational noise\n X += noiseObs * np.random.randn(tmax, self.dim)\n # call helper function\n write_data(\n X,\n dir=dir,\n append=append,\n header=header,\n varNames=self.varNames,\n Adj=self.Adj,\n Coupl=self.Coupl,\n model=self.model,\n modelType=self.modelType,\n boolRules=self.boolRules,\n invTimeStep=self.invTimeStep,\n )\n\n\ndef _check_branching(\n X: np.ndarray, Xsamples: np.ndarray, restart: int, threshold: float = 0.25\n) -> tuple[bool, list[np.ndarray]]:\n \"\"\"\\\n Check whether time series branches.\n\n Parameters\n ----------\n X\n current time series data.\n Xsamples\n list of previous branching samples.\n restart\n counts number of restart trials.\n threshold\n sets threshold for attractor identification.\n\n Returns\n -------\n check\n true if branching realization\n Xsamples\n updated list\n \"\"\"\n check = True\n Xsamples = list(Xsamples)\n if restart == 0:\n Xsamples.append(X)\n else:\n for Xcompare in Xsamples:\n Xtmax_diff = np.absolute(X[-1, :] - Xcompare[-1, :])\n # If the second largest element is smaller than threshold\n # set check to False, i.e. at least two elements\n # need to change in order to have a branching.\n # If we observe all parameters of the system,\n # a new attractor state must involve changes in two\n # variables.\n if np.partition(Xtmax_diff, -2)[-2] < threshold:\n check = False\n if check:\n Xsamples.append(X)\n logg.debug(f'realization {restart}: {\"\" if check else \"no\"} new branch')\n return check, Xsamples\n\n\ndef check_nocycles(Adj: np.ndarray, verbosity: int = 2) -> bool:\n \"\"\"\\\n Checks that there are no cycles in graph described by adjacancy matrix.\n\n Parameters\n ----------\n Adj\n adjancancy matrix of dimension (dim, dim)\n\n Returns\n -------\n True if there is no cycle, False otherwise.\n \"\"\"\n dim = Adj.shape[0]\n for g in range(dim):\n v = np.zeros(dim)\n v[g] = 1\n for i in range(dim):\n v = Adj.dot(v)\n if v[g] > 1e-10:\n if verbosity > 2:\n settings.m(0, Adj)\n settings.m(\n 0,\n \"contains a cycle of length\",\n i + 1,\n \"starting from node\",\n g,\n \"-> reject\",\n )\n return False\n return True\n\n\ndef sample_coupling_matrix(\n dim: int = 3, connectivity: float = 0.5\n) -> tuple[np.ndarray, np.ndarray, np.ndarray, int]:\n \"\"\"\\\n Sample coupling matrix.\n\n Checks that returned graphs contain no self-cycles.\n\n Parameters\n ----------\n dim\n dimension of coupling matrix.\n connectivity\n fraction of connectivity, fully connected means 1.,\n not-connected means 0, in the case of fully connected, one has\n dim*(dim-1)/2 edges in the graph.\n\n Returns\n -------\n coupl\n coupling matrix\n adj\n adjancancy matrix\n adj_signed\n signed adjacancy matrix\n n_edges\n Number of edges\n \"\"\"\n max_trial = 10\n check = False\n for trial in range(max_trial):\n # random topology for a given connectivity / edge density\n Coupl = np.zeros((dim, dim))\n n_edges = 0\n for gp in range(dim):\n for g in range(dim):\n if gp == g:\n continue\n # need to have the factor 0.5, otherwise\n # connectivity=1 would lead to dim*(dim-1) edges\n if np.random.rand() < 0.5 * connectivity:\n Coupl[gp, g] = 0.7\n n_edges += 1\n # obtain adjacancy matrix\n Adj_signed = np.zeros((dim, dim), dtype=\"int_\")\n Adj_signed = np.sign(Coupl)\n Adj = np.abs(Adj_signed)\n # check for cycles and whether there is at least one edge\n if check_nocycles(Adj) and n_edges > 0:\n check = True\n break\n if not check:\n raise ValueError(\n \"did not find graph without cycles after\" f\"{max_trial} trials\"\n )\n return Coupl, Adj, Adj_signed, n_edges\n\n\nclass StaticCauseEffect:\n \"\"\"\n Simulates static data to investigate structure learning.\n \"\"\"\n\n availModels = dict(\n line=\"y = αx \\n\",\n noise=\"y = noise \\n\",\n absline=\"y = |x| \\n\",\n parabola=\"y = αx² \\n\",\n sawtooth=\"y = x - |x| \\n\",\n tanh=\"y = tanh(x) \\n\",\n combi=\"combinatorial regulation \\n\",\n )\n\n def __init__(self):\n # define a set of available functions\n self.funcs = dict(\n line=lambda x: x,\n noise=lambda x: 0,\n absline=np.abs,\n parabola=lambda x: x**2,\n sawtooth=lambda x: 0.5 * x - np.floor(0.5 * x),\n tanh=lambda x: np.tanh(2 * x),\n )\n\n def sim_givenAdj(self, Adj: np.ndarray, model=\"line\"):\n \"\"\"\\\n Simulate data given only an adjacancy matrix and a model.\n\n The model is a bivariate funtional dependence. The adjacancy matrix\n needs to be acyclic.\n\n Parameters\n ----------\n Adj\n adjacancy matrix of shape (dim,dim).\n\n Returns\n -------\n Data array of shape (n_samples,dim).\n \"\"\"\n # nice examples\n examples = [ # noqa: F841 TODO We are really unsure whether this is needed.\n dict(\n func=\"sawtooth\",\n gdist=\"uniform\",\n sigma_glob=1.8,\n sigma_noise=0.1,\n )\n ]\n\n # nr of samples\n n_samples = 100\n\n # noise\n sigma_glob = 1.8\n sigma_noise = 0.4\n\n # coupling function / model\n func = self.funcs[model]\n\n # glob distribution\n sourcedist = \"uniform\"\n\n # loop over source nodes\n dim = Adj.shape[0]\n X = np.zeros((n_samples, dim))\n # source nodes have no parents themselves\n nrpar = 0\n children = list(range(dim))\n parents = []\n for gp in range(dim):\n if Adj[gp, :].sum() == nrpar:\n if sourcedist == \"gaussian\":\n X[:, gp] = np.random.normal(0, sigma_glob, n_samples)\n if sourcedist == \"uniform\":\n X[:, gp] = np.random.uniform(-sigma_glob, sigma_glob, n_samples)\n parents.append(gp)\n children.remove(gp)\n\n # all of the following guarantees for 3 dim, that we generate the data\n # in the correct sequence\n # then compute all nodes that have 1 parent, then those with 2 parents\n children_sorted = []\n nrchildren_par = np.zeros(dim)\n nrchildren_par[0] = len(parents)\n for nrpar in range(1, dim):\n # loop over child nodes\n for gp in children:\n if Adj[gp, :].sum() == nrpar:\n children_sorted.append(gp)\n nrchildren_par[nrpar] += 1\n # if there is more than a child with a single parent\n # order these children (there are two in three dim)\n # by distance to the source/parent\n if nrchildren_par[1] > 1 and Adj[children_sorted[0], parents[0]] == 0:\n help = children_sorted[0]\n children_sorted[0] = children_sorted[1]\n children_sorted[1] = help\n\n for gp in children_sorted:\n for g in range(dim):\n if Adj[gp, g] > 0:\n X[:, gp] += 1.0 / Adj[gp, :].sum() * func(X[:, g])\n X[:, gp] += np.random.normal(0, sigma_noise, n_samples)\n\n # fig = pl.figure()\n # fig.add_subplot(311)\n # pl.plot(X[:,0],X[:,1],'.',mec='white')\n # fig.add_subplot(312)\n # pl.plot(X[:,1],X[:,2],'.',mec='white')\n # fig.add_subplot(313)\n # pl.plot(X[:,2],X[:,0],'.',mec='white')\n # pl.show()\n\n return X\n\n def sim_combi(self):\n \"\"\"Simulate data to model combi regulation.\"\"\"\n n_samples = 500\n sigma_glob = 1.8\n\n X = np.zeros((n_samples, 3))\n\n X[:, 0] = np.random.uniform(-sigma_glob, sigma_glob, n_samples)\n X[:, 1] = np.random.uniform(-sigma_glob, sigma_glob, n_samples)\n\n func = self.funcs[\"tanh\"]\n\n # XOR type\n # X[:,2] = (func(X[:,0])*sp.stats.norm.pdf(X[:,1],0,0.2)\n # + func(X[:,1])*sp.stats.norm.pdf(X[:,0],0,0.2))\n # AND type / diagonal\n # X[:,2] = (func(X[:,0]+X[:,1])*sp.stats.norm.pdf(X[:,1]-X[:,0],0,0.2))\n # AND type / horizontal\n X[:, 2] = func(X[:, 0]) * sp.stats.norm.cdf(X[:, 1], 1, 0.2)\n\n pl.scatter( # noqa: F821 TODO Fix me\n X[:, 0], X[:, 1], c=X[:, 2], edgecolor=\"face\"\n )\n pl.show() # noqa: F821 TODO Fix me\n\n pl.plot(X[:, 1], X[:, 2], \".\") # noqa: F821 TODO Fix me\n pl.show() # noqa: F821 TODO Fix me\n\n return X\n\n\ndef sample_static_data(model, dir, verbosity=0):\n # fraction of connectivity as compared to fully connected\n # in one direction, which amounts to dim*(dim-1)/2 edges\n connectivity = 0.8\n dim = 3\n n_Coupls = 50\n model = model.replace(\"static-\", \"\")\n np.random.seed(0)\n\n if model != \"combi\":\n n_edges = np.zeros(n_Coupls)\n for icoupl in range(n_Coupls):\n Coupl, Adj, Adj_signed, n_e = sample_coupling_matrix(dim, connectivity)\n if verbosity > 1:\n settings.m(0, icoupl)\n settings.m(0, Adj)\n n_edges[icoupl] = n_e\n # sample data\n X = StaticCauseEffect().sim_givenAdj(Adj, model)\n write_data(X, dir, Adj=Adj)\n settings.m(0, \"mean edge number:\", n_edges.mean())\n\n else:\n X = StaticCauseEffect().sim_combi()\n Adj = np.zeros((3, 3))\n Adj[2, 0] = Adj[2, 1] = 0\n write_data(X, dir, Adj=Adj)\n\n\nif __name__ == \"__main__\":\n import argparse\n\n # epilog = (' 1: 2dim, causal direction X_1 -> X_0, constraint signs\\n'\n # + ' 2: 2dim, causal direction X_1 -> X_0, arbitrary signs\\n'\n # + ' 3: 2dim, causal direction X_1 <-> X_0, arbitrary signs\\n'\n # + ' 4: 2dim, mix of model 2 and 3\\n'\n # + ' 5: 6dim double toggle switch\\n'\n # + ' 6: two independent evolutions without repression, sync.\\n'\n # + ' 7: two independent evolutions without repression, random init\\n'\n # + ' 8: two independent evolutions directed repression, random init\\n'\n # + ' 9: two independent evolutions mutual repression, random init\\n'\n # + ' 10: two indep. evol., diff. self-loops possible, mut. repr., rand init\\n')\n epilog = \"\"\n for k, v in StaticCauseEffect.availModels.items():\n epilog += \" static-\" + k + \": \" + v\n for k, v in GRNsim.availModels.items():\n epilog += \" \" + k + \": \" + v\n # command line options\n p = argparse.ArgumentParser(\n description=(\n \"Simulate stochastic discrete-time dynamical systems,\\n\"\n \"in particular gene regulatory networks.\"\n ),\n formatter_class=argparse.RawDescriptionHelpFormatter,\n epilog=(\n \" MODEL: specify one of the following models, or one of \\n\"\n ' the filenames (without \".txt\") in the directory \"models\" \\n' + epilog\n ),\n )\n aa = p.add_argument\n dir_arg = aa(\n \"--dir\",\n required=True,\n type=str,\n default=\"\",\n help=(\n \"specify directory to store data, \"\n + ' must start with \"sim/MODEL_...\", see possible values for MODEL below '\n ),\n )\n aa(\"--show\", action=\"store_true\", help=\"show plots\")\n aa(\n \"--verbosity\",\n type=int,\n default=0,\n help=\"specify integer > 0 to get more output [default 0]\",\n )\n args = p.parse_args()\n\n # run checks on output directory\n dir = Path(args.dir)\n if not dir.resolve().parent.name == \"sim\":\n raise argparse.ArgumentError(\n dir_arg,\n \"The parent directory of the --dir argument needs to be named 'sim'\",\n )\n else:\n model = dir.name.split(\"_\")[0]\n settings.m(0, f\"...model is: {model!r}\")\n if dir.is_dir() and \"test\" not in str(dir):\n message = (\n f\"directory {dir} already exists, \"\n \"remove it and continue? [y/n, press enter]\"\n )\n if str(input(message)) != \"y\":\n settings.m(0, \" ...quit program execution\")\n sys.exit()\n else:\n settings.m(0, \" ...removing directory and continuing...\")\n shutil.rmtree(dir)\n\n settings.m(0, model)\n settings.m(0, dir)\n\n # sample data\n if \"static\" in model:\n sample_static_data(model=model, dir=dir, verbosity=args.verbosity)\n else:\n sample_dynamic_data(model=model, dir=dir)\n\n\n\"\"\"\nComputes a dendrogram based on a given categorical observation.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport pandas as pd\nfrom pandas.api.types import CategoricalDtype\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._utils import _doc_params, raise_not_implemented_error_if_backed_type\nfrom ..neighbors._doc import doc_n_pcs, doc_use_rep\nfrom ._utils import _choose_representation\n\nif TYPE_CHECKING:\n from collections.abc import Sequence\n from typing import Any\n\n from anndata import AnnData\n\n\n@old_positionals(\n \"n_pcs\",\n \"use_rep\",\n \"var_names\",\n \"use_raw\",\n \"cor_method\",\n \"linkage_method\",\n \"optimal_ordering\",\n \"key_added\",\n \"inplace\",\n)\n@_doc_params(n_pcs=doc_n_pcs, use_rep=doc_use_rep)\ndef dendrogram(\n adata: AnnData,\n groupby: str | Sequence[str],\n *,\n n_pcs: int | None = None,\n use_rep: str | None = None,\n var_names: Sequence[str] | None = None,\n use_raw: bool | None = None,\n cor_method: str = \"pearson\",\n linkage_method: str = \"complete\",\n optimal_ordering: bool = False,\n key_added: str | None = None,\n inplace: bool = True,\n) -> dict[str, Any] | None:\n \"\"\"\\\n Computes a hierarchical clustering for the given `groupby` categories.\n\n By default, the PCA representation is used unless `.X`\n has less than 50 variables.\n\n Alternatively, a list of `var_names` (e.g. genes) can be given.\n\n Average values of either `var_names` or components are used\n to compute a correlation matrix.\n\n The hierarchical clustering can be visualized using\n :func:`scanpy.pl.dendrogram` or multiple other visualizations that can\n include a dendrogram: :func:`~scanpy.pl.matrixplot`,\n :func:`~scanpy.pl.heatmap`, :func:`~scanpy.pl.dotplot`,\n and :func:`~scanpy.pl.stacked_violin`.\n\n .. note::\n The computation of the hierarchical clustering is based on predefined\n groups and not per cell. The correlation matrix is computed using by\n default pearson but other methods are available.\n\n Parameters\n ----------\n adata\n Annotated data matrix\n {n_pcs}\n {use_rep}\n var_names\n List of var_names to use for computing the hierarchical clustering.\n If `var_names` is given, then `use_rep` and `n_pcs` is ignored.\n use_raw\n Only when `var_names` is not None.\n Use `raw` attribute of `adata` if present.\n cor_method\n correlation method to use.\n Options are 'pearson', 'kendall', and 'spearman'\n linkage_method\n linkage method to use. See :func:`scipy.cluster.hierarchy.linkage`\n for more information.\n optimal_ordering\n Same as the optimal_ordering argument of :func:`scipy.cluster.hierarchy.linkage`\n which reorders the linkage matrix so that the distance between successive\n leaves is minimal.\n key_added\n By default, the dendrogram information is added to\n `.uns[f'dendrogram_{{groupby}}']`.\n Notice that the `groupby` information is added to the dendrogram.\n inplace\n If `True`, adds dendrogram information to `adata.uns[key_added]`,\n else this function returns the information.\n\n Returns\n -------\n Returns `None` if `inplace=True`, else returns a `dict` with dendrogram information. Sets the following field if `inplace=True`:\n\n `adata.uns[f'dendrogram_{{group_by}}' | key_added]` : :class:`dict`\n Dendrogram information.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> sc.tl.dendrogram(adata, groupby='bulk_labels')\n >>> sc.pl.dendrogram(adata, groupby='bulk_labels') # doctest: +SKIP\n \n >>> markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n >>> sc.pl.dotplot(adata, markers, groupby='bulk_labels', dendrogram=True)\n \"\"\"\n\n raise_not_implemented_error_if_backed_type(adata.X, \"dendrogram\")\n if isinstance(groupby, str):\n # if not a list, turn into a list\n groupby = [groupby]\n for group in groupby:\n if group not in adata.obs_keys():\n raise ValueError(\n \"groupby has to be a valid observation. \"\n f\"Given value: {group}, valid observations: {adata.obs_keys()}\"\n )\n if not isinstance(adata.obs[group].dtype, CategoricalDtype):\n raise ValueError(\n \"groupby has to be a categorical observation. \"\n f\"Given value: {group}, Column type: {adata.obs[group].dtype}\"\n )\n\n if var_names is None:\n rep_df = pd.DataFrame(\n _choose_representation(adata, use_rep=use_rep, n_pcs=n_pcs)\n )\n categorical = adata.obs[groupby[0]]\n if len(groupby) > 1:\n for group in groupby[1:]:\n # create new category by merging the given groupby categories\n categorical = (\n categorical.astype(str) + \"_\" + adata.obs[group].astype(str)\n ).astype(\"category\")\n categorical.name = \"_\".join(groupby)\n\n rep_df.set_index(categorical, inplace=True)\n categories: pd.Index = rep_df.index.categories\n else:\n gene_names = adata.raw.var_names if use_raw else adata.var_names\n from ..plotting._anndata import _prepare_dataframe\n\n categories, rep_df = _prepare_dataframe(\n adata, gene_names, groupby, use_raw=use_raw\n )\n\n # aggregate values within categories using 'mean'\n mean_df = (\n rep_df.groupby(level=0, observed=True)\n .mean()\n .loc[categories] # Fixed ordering for pandas < 2\n )\n\n import scipy.cluster.hierarchy as sch\n from scipy.spatial import distance\n\n corr_matrix = mean_df.T.corr(method=cor_method).clip(-1, 1)\n corr_condensed = distance.squareform(1 - corr_matrix)\n z_var = sch.linkage(\n corr_condensed, method=linkage_method, optimal_ordering=optimal_ordering\n )\n dendro_info = sch.dendrogram(z_var, labels=list(categories), no_plot=True)\n\n dat = dict(\n linkage=z_var,\n groupby=groupby,\n use_rep=use_rep,\n cor_method=cor_method,\n linkage_method=linkage_method,\n categories_ordered=dendro_info[\"ivl\"],\n categories_idx_ordered=dendro_info[\"leaves\"],\n dendrogram_info=dendro_info,\n correlation_matrix=corr_matrix.values,\n )\n\n if inplace:\n if key_added is None:\n key_added = f'dendrogram_{\"_\".join(groupby)}'\n logg.info(f\"Storing dendrogram info using `.uns[{key_added!r}]`\")\n adata.uns[key_added] = dat\n else:\n return dat\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom natsort import natsorted\n\nfrom .. import _utils\nfrom .. import logging as logg\nfrom ._utils_clustering import rename_groups, restrict_adjacency\n\nif TYPE_CHECKING:\n from collections.abc import Sequence\n from typing import Literal\n\n from anndata import AnnData\n from scipy import sparse\n\ntry:\n from leidenalg.VertexPartition import MutableVertexPartition\nexcept ImportError:\n\n class MutableVertexPartition:\n pass\n\n MutableVertexPartition.__module__ = \"leidenalg.VertexPartition\"\n\n\ndef leiden(\n adata: AnnData,\n resolution: float = 1,\n *,\n restrict_to: tuple[str, Sequence[str]] | None = None,\n random_state: _utils.AnyRandom = 0,\n key_added: str = \"leiden\",\n adjacency: sparse.spmatrix | None = None,\n directed: bool | None = None,\n use_weights: bool = True,\n n_iterations: int = -1,\n partition_type: type[MutableVertexPartition] | None = None,\n neighbors_key: str | None = None,\n obsp: str | None = None,\n copy: bool = False,\n flavor: Literal[\"leidenalg\", \"igraph\"] = \"leidenalg\",\n **clustering_args,\n) -> AnnData | None:\n \"\"\"\\\n Cluster cells into subgroups :cite:p:`Traag2019`.\n\n Cluster cells using the Leiden algorithm :cite:p:`Traag2019`,\n an improved version of the Louvain algorithm :cite:p:`Blondel2008`.\n It has been proposed for single-cell analysis by :cite:t:`Levine2015`.\n\n This requires having ran :func:`~scanpy.pp.neighbors` or\n :func:`~scanpy.external.pp.bbknn` first.\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n resolution\n A parameter value controlling the coarseness of the clustering.\n Higher values lead to more clusters.\n Set to `None` if overriding `partition_type`\n to one that doesn’t accept a `resolution_parameter`.\n random_state\n Change the initialization of the optimization.\n restrict_to\n Restrict the clustering to the categories within the key for sample\n annotation, tuple needs to contain `(obs_key, list_of_categories)`.\n key_added\n `adata.obs` key under which to add the cluster labels.\n adjacency\n Sparse adjacency matrix of the graph, defaults to neighbors connectivities.\n directed\n Whether to treat the graph as directed or undirected.\n use_weights\n If `True`, edge weights from the graph are used in the computation\n (placing more emphasis on stronger edges).\n n_iterations\n How many iterations of the Leiden clustering algorithm to perform.\n Positive values above 2 define the total number of iterations to perform,\n -1 has the algorithm run until it reaches its optimal clustering.\n 2 is faster and the default for underlying packages.\n partition_type\n Type of partition to use.\n Defaults to :class:`~leidenalg.RBConfigurationVertexPartition`.\n For the available options, consult the documentation for\n :func:`~leidenalg.find_partition`.\n neighbors_key\n Use neighbors connectivities as adjacency.\n If not specified, leiden looks .obsp['connectivities'] for connectivities\n (default storage place for pp.neighbors).\n If specified, leiden looks\n .obsp[.uns[neighbors_key]['connectivities_key']] for connectivities.\n obsp\n Use .obsp[obsp] as adjacency. You can't specify both\n `obsp` and `neighbors_key` at the same time.\n copy\n Whether to copy `adata` or modify it inplace.\n flavor\n Which package's implementation to use.\n **clustering_args\n Any further arguments to pass to :func:`~leidenalg.find_partition` (which in turn passes arguments to the `partition_type`)\n or :meth:`igraph.Graph.community_leiden` from `igraph`.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.obs['leiden' | key_added]` : :class:`pandas.Series` (dtype ``category``)\n Array of dim (number of samples) that stores the subgroup id\n (``'0'``, ``'1'``, ...) for each cell.\n\n `adata.uns['leiden' | key_added]['params']` : :class:`dict`\n A dict with the values for the parameters `resolution`, `random_state`,\n and `n_iterations`.\n \"\"\"\n if flavor not in {\"igraph\", \"leidenalg\"}:\n raise ValueError(\n f\"flavor must be either 'igraph' or 'leidenalg', but '{flavor}' was passed\"\n )\n _utils.ensure_igraph()\n if flavor == \"igraph\":\n if directed:\n raise ValueError(\n \"Cannot use igraph’s leiden implementation with a directed graph.\"\n )\n if partition_type is not None:\n raise ValueError(\n \"Do not pass in partition_type argument when using igraph.\"\n )\n else:\n try:\n import leidenalg\n\n msg = 'In the future, the default backend for leiden will be igraph instead of leidenalg.\\n\\n To achieve the future defaults please pass: flavor=\"igraph\" and n_iterations=2. directed must also be False to work with igraph\\'s implementation.'\n _utils.warn_once(msg, FutureWarning, stacklevel=3)\n except ImportError:\n raise ImportError(\n \"Please install the leiden algorithm: `conda install -c conda-forge leidenalg` or `pip3 install leidenalg`.\"\n )\n clustering_args = dict(clustering_args)\n\n start = logg.info(\"running Leiden clustering\")\n adata = adata.copy() if copy else adata\n # are we clustering a user-provided graph or the default AnnData one?\n if adjacency is None:\n adjacency = _utils._choose_graph(adata, obsp, neighbors_key)\n if restrict_to is not None:\n restrict_key, restrict_categories = restrict_to\n adjacency, restrict_indices = restrict_adjacency(\n adata,\n restrict_key,\n restrict_categories=restrict_categories,\n adjacency=adjacency,\n )\n # Prepare find_partition arguments as a dictionary,\n # appending to whatever the user provided. It needs to be this way\n # as this allows for the accounting of a None resolution\n # (in the case of a partition variant that doesn't take it on input)\n clustering_args[\"n_iterations\"] = n_iterations\n if flavor == \"leidenalg\":\n if resolution is not None:\n clustering_args[\"resolution_parameter\"] = resolution\n directed = True if directed is None else directed\n g = _utils.get_igraph_from_adjacency(adjacency, directed=directed)\n if partition_type is None:\n partition_type = leidenalg.RBConfigurationVertexPartition\n if use_weights:\n clustering_args[\"weights\"] = np.array(g.es[\"weight\"]).astype(np.float64)\n clustering_args[\"seed\"] = random_state\n part = leidenalg.find_partition(g, partition_type, **clustering_args)\n else:\n g = _utils.get_igraph_from_adjacency(adjacency, directed=False)\n if use_weights:\n clustering_args[\"weights\"] = \"weight\"\n if resolution is not None:\n clustering_args[\"resolution\"] = resolution\n clustering_args.setdefault(\"objective_function\", \"modularity\")\n with _utils.set_igraph_random_state(random_state):\n part = g.community_leiden(**clustering_args)\n # store output into adata.obs\n groups = np.array(part.membership)\n if restrict_to is not None:\n if key_added == \"leiden\":\n key_added += \"_R\"\n groups = rename_groups(\n adata,\n key_added=key_added,\n restrict_key=restrict_key,\n restrict_categories=restrict_categories,\n restrict_indices=restrict_indices,\n groups=groups,\n )\n adata.obs[key_added] = pd.Categorical(\n values=groups.astype(\"U\"),\n categories=natsorted(map(str, np.unique(groups))),\n )\n # store information on the clustering parameters\n adata.uns[key_added] = {}\n adata.uns[key_added][\"params\"] = dict(\n resolution=resolution,\n random_state=random_state,\n n_iterations=n_iterations,\n )\n logg.info(\n \" finished\",\n time=start,\n deep=(\n f\"found {len(np.unique(groups))} clusters and added\\n\"\n f\" {key_added!r}, the cluster labels (adata.obs, categorical)\"\n ),\n )\n return adata if copy else None\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nfrom .._compat import old_positionals\nfrom ._dpt import _diffmap\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n from .._utils import AnyRandom\n\n\n@old_positionals(\"neighbors_key\", \"random_state\", \"copy\")\ndef diffmap(\n adata: AnnData,\n n_comps: int = 15,\n *,\n neighbors_key: str | None = None,\n random_state: AnyRandom = 0,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Diffusion Maps :cite:p:`Coifman2005,Haghverdi2015,Wolf2018`.\n\n Diffusion maps :cite:p:`Coifman2005` has been proposed for visualizing single-cell\n data by :cite:t:`Haghverdi2015`. The tool uses the adapted Gaussian kernel suggested\n by :cite:t:`Haghverdi2016` in the implementation of :cite:t:`Wolf2018`.\n\n The width (\"sigma\") of the connectivity kernel is implicitly determined by\n the number of neighbors used to compute the single-cell graph in\n :func:`~scanpy.pp.neighbors`. To reproduce the original implementation\n using a Gaussian kernel, use `method=='gauss'` in\n :func:`~scanpy.pp.neighbors`. To use an exponential kernel, use the default\n `method=='umap'`. Differences between these options shouldn't usually be\n dramatic.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n n_comps\n The number of dimensions of the representation.\n neighbors_key\n If not specified, diffmap looks .uns['neighbors'] for neighbors settings\n and .obsp['connectivities'], .obsp['distances'] for connectivities and\n distances respectively (default storage places for pp.neighbors).\n If specified, diffmap looks .uns[neighbors_key] for neighbors settings and\n .obsp[.uns[neighbors_key]['connectivities_key']],\n .obsp[.uns[neighbors_key]['distances_key']] for connectivities and distances\n respectively.\n random_state\n A numpy random seed\n copy\n Return a copy instead of writing to adata.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.obsm['X_diffmap']` : :class:`numpy.ndarray` (dtype `float`)\n Diffusion map representation of data, which is the right eigen basis of\n the transition matrix with eigenvectors as columns.\n\n `adata.uns['diffmap_evals']` : :class:`numpy.ndarray` (dtype `float`)\n Array of size (number of eigen vectors).\n Eigenvalues of transition matrix.\n\n Notes\n -----\n The 0-th column in `adata.obsm[\"X_diffmap\"]` is the steady-state solution,\n which is non-informative in diffusion maps.\n Therefore, the first diffusion component is at index 1,\n e.g. `adata.obsm[\"X_diffmap\"][:,1]`\n \"\"\"\n if neighbors_key is None:\n neighbors_key = \"neighbors\"\n\n if neighbors_key not in adata.uns:\n raise ValueError(\n \"You need to run `pp.neighbors` first to compute a neighborhood graph.\"\n )\n if n_comps <= 2:\n raise ValueError(\"Provide any value greater than 2 for `n_comps`. \")\n adata = adata.copy() if copy else adata\n _diffmap(\n adata, n_comps=n_comps, neighbors_key=neighbors_key, random_state=random_state\n )\n return adata if copy else None\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom types import MappingProxyType\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom natsort import natsorted\nfrom packaging.version import Version\n\nfrom .. import _utils\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._utils import _choose_graph\nfrom ._utils_clustering import rename_groups, restrict_adjacency\n\nif TYPE_CHECKING:\n from collections.abc import Mapping, Sequence\n from typing import Any, Literal\n\n from anndata import AnnData\n from scipy.sparse import spmatrix\n\ntry:\n from louvain.VertexPartition import MutableVertexPartition\nexcept ImportError:\n\n class MutableVertexPartition:\n pass\n\n MutableVertexPartition.__module__ = \"louvain.VertexPartition\"\n\n\n@old_positionals(\n \"random_state\",\n \"restrict_to\",\n \"key_added\",\n \"adjacency\",\n \"flavor\",\n \"directed\",\n \"use_weights\",\n \"partition_type\",\n \"partition_kwargs\",\n \"neighbors_key\",\n \"obsp\",\n \"copy\",\n)\ndef louvain(\n adata: AnnData,\n resolution: float | None = None,\n *,\n random_state: _utils.AnyRandom = 0,\n restrict_to: tuple[str, Sequence[str]] | None = None,\n key_added: str = \"louvain\",\n adjacency: spmatrix | None = None,\n flavor: Literal[\"vtraag\", \"igraph\", \"rapids\"] = \"vtraag\",\n directed: bool = True,\n use_weights: bool = False,\n partition_type: type[MutableVertexPartition] | None = None,\n partition_kwargs: Mapping[str, Any] = MappingProxyType({}),\n neighbors_key: str | None = None,\n obsp: str | None = None,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Cluster cells into subgroups :cite:p:`Blondel2008,Levine2015,Traag2017`.\n\n Cluster cells using the Louvain algorithm :cite:p:`Blondel2008` in the implementation\n of :cite:t:`Traag2017`. The Louvain algorithm has been proposed for single-cell\n analysis by :cite:t:`Levine2015`.\n\n This requires having ran :func:`~scanpy.pp.neighbors` or\n :func:`~scanpy.external.pp.bbknn` first,\n or explicitly passing a ``adjacency`` matrix.\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n resolution\n For the default flavor (``'vtraag'``) or for ```RAPIDS```, you can provide a\n resolution (higher resolution means finding more and smaller clusters),\n which defaults to 1.0.\n See “Time as a resolution parameter” in :cite:t:`Lambiotte2014`.\n random_state\n Change the initialization of the optimization.\n restrict_to\n Restrict the clustering to the categories within the key for sample\n annotation, tuple needs to contain ``(obs_key, list_of_categories)``.\n key_added\n Key under which to add the cluster labels. (default: ``'louvain'``)\n adjacency\n Sparse adjacency matrix of the graph, defaults to neighbors connectivities.\n flavor\n Choose between to packages for computing the clustering.\n\n ``'vtraag'``\n Much more powerful than ``'igraph'``, and the default.\n ``'igraph'``\n Built in ``igraph`` method.\n ``'rapids'``\n GPU accelerated implementation.\n\n .. deprecated:: 1.10.0\n Use :func:`rapids_singlecell.tl.louvain` instead.\n directed\n Interpret the ``adjacency`` matrix as directed graph?\n use_weights\n Use weights from knn graph.\n partition_type\n Type of partition to use.\n Only a valid argument if ``flavor`` is ``'vtraag'``.\n partition_kwargs\n Key word arguments to pass to partitioning,\n if ``vtraag`` method is being used.\n neighbors_key\n Use neighbors connectivities as adjacency.\n If not specified, louvain looks .obsp['connectivities'] for connectivities\n (default storage place for pp.neighbors).\n If specified, louvain looks\n .obsp[.uns[neighbors_key]['connectivities_key']] for connectivities.\n obsp\n Use .obsp[obsp] as adjacency. You can't specify both\n `obsp` and `neighbors_key` at the same time.\n copy\n Copy adata or modify it inplace.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.obs['louvain' | key_added]` : :class:`pandas.Series` (dtype ``category``)\n Array of dim (number of samples) that stores the subgroup id\n (``'0'``, ``'1'``, ...) for each cell.\n\n `adata.uns['louvain' | key_added]['params']` : :class:`dict`\n A dict with the values for the parameters `resolution`, `random_state`,\n and `n_iterations`.\n \"\"\"\n partition_kwargs = dict(partition_kwargs)\n start = logg.info(\"running Louvain clustering\")\n if (flavor != \"vtraag\") and (partition_type is not None):\n raise ValueError(\n \"`partition_type` is only a valid argument \" 'when `flavour` is \"vtraag\"'\n )\n adata = adata.copy() if copy else adata\n if adjacency is None:\n adjacency = _choose_graph(adata, obsp, neighbors_key)\n if restrict_to is not None:\n restrict_key, restrict_categories = restrict_to\n adjacency, restrict_indices = restrict_adjacency(\n adata,\n restrict_key,\n restrict_categories=restrict_categories,\n adjacency=adjacency,\n )\n if flavor in {\"vtraag\", \"igraph\"}:\n if flavor == \"igraph\" and resolution is not None:\n logg.warning('`resolution` parameter has no effect for flavor \"igraph\"')\n if directed and flavor == \"igraph\":\n directed = False\n if not directed:\n logg.debug(\" using the undirected graph\")\n g = _utils.get_igraph_from_adjacency(adjacency, directed=directed)\n weights = np.array(g.es[\"weight\"]).astype(np.float64) if use_weights else None\n if flavor == \"vtraag\":\n import louvain\n\n if partition_type is None:\n partition_type = louvain.RBConfigurationVertexPartition\n if resolution is not None:\n partition_kwargs[\"resolution_parameter\"] = resolution\n if use_weights:\n partition_kwargs[\"weights\"] = weights\n if Version(louvain.__version__) < Version(\"0.7.0\"):\n louvain.set_rng_seed(random_state)\n else:\n partition_kwargs[\"seed\"] = random_state\n logg.info(' using the \"louvain\" package of Traag (2017)')\n part = louvain.find_partition(\n g,\n partition_type,\n **partition_kwargs,\n )\n # adata.uns['louvain_quality'] = part.quality()\n else:\n part = g.community_multilevel(weights=weights)\n groups = np.array(part.membership)\n elif flavor == \"rapids\":\n msg = (\n \"`flavor='rapids'` is deprecated. \"\n \"Use `rapids_singlecell.tl.louvain` instead.\"\n )\n warnings.warn(msg, FutureWarning)\n # nvLouvain only works with undirected graphs,\n # and `adjacency` must have a directed edge in both directions\n import cudf\n import cugraph\n\n offsets = cudf.Series(adjacency.indptr)\n indices = cudf.Series(adjacency.indices)\n if use_weights:\n sources, targets = adjacency.nonzero()\n weights = adjacency[sources, targets]\n if isinstance(weights, np.matrix):\n weights = weights.A1\n weights = cudf.Series(weights)\n else:\n weights = None\n g = cugraph.Graph()\n\n if hasattr(g, \"add_adj_list\"):\n g.add_adj_list(offsets, indices, weights)\n else:\n g.from_cudf_adjlist(offsets, indices, weights)\n\n logg.info(' using the \"louvain\" package of rapids')\n if resolution is not None:\n louvain_parts, _ = cugraph.louvain(g, resolution=resolution)\n else:\n louvain_parts, _ = cugraph.louvain(g)\n groups = (\n louvain_parts.to_pandas()\n .sort_values(\"vertex\")[[\"partition\"]]\n .to_numpy()\n .ravel()\n )\n elif flavor == \"taynaud\":\n # this is deprecated\n import community\n import networkx as nx\n\n g = nx.Graph(adjacency)\n partition = community.best_partition(g)\n groups = np.zeros(len(partition), dtype=int)\n for k, v in partition.items():\n groups[k] = v\n else:\n raise ValueError('`flavor` needs to be \"vtraag\" or \"igraph\" or \"taynaud\".')\n if restrict_to is not None:\n if key_added == \"louvain\":\n key_added += \"_R\"\n groups = rename_groups(\n adata,\n key_added=key_added,\n restrict_key=restrict_key,\n restrict_categories=restrict_categories,\n restrict_indices=restrict_indices,\n groups=groups,\n )\n adata.obs[key_added] = pd.Categorical(\n values=groups.astype(\"U\"),\n categories=natsorted(map(str, np.unique(groups))),\n )\n adata.uns[key_added] = {}\n adata.uns[key_added][\"params\"] = dict(\n resolution=resolution,\n random_state=random_state,\n )\n logg.info(\n \" finished\",\n time=start,\n deep=(\n f\"found {len(np.unique(groups))} clusters and added\\n\"\n f\" {key_added!r}, the cluster labels (adata.obs, categorical)\"\n ),\n )\n return adata if copy else None\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nfrom ._dendrogram import dendrogram\nfrom ._diffmap import diffmap\nfrom ._dpt import dpt\nfrom ._draw_graph import draw_graph\nfrom ._embedding_density import embedding_density\nfrom ._ingest import (\n Ingest, # noqa: F401\n ingest,\n)\nfrom ._leiden import leiden\nfrom ._louvain import louvain\nfrom ._marker_gene_overlap import marker_gene_overlap\nfrom ._paga import (\n paga,\n paga_compare_paths, # noqa: F401\n paga_degrees, # noqa: F401\n paga_expression_entropies, # noqa: F401\n)\nfrom ._rank_genes_groups import filter_rank_genes_groups, rank_genes_groups\nfrom ._score_genes import score_genes, score_genes_cell_cycle\nfrom ._sim import sim\nfrom ._tsne import tsne\nfrom ._umap import umap\n\nif TYPE_CHECKING:\n from typing import Any\n\n\ndef __getattr__(name: str) -> Any:\n if name == \"pca\":\n from ..preprocessing import pca\n\n return pca\n raise AttributeError(name)\n\n\n__all__ = [\n \"dendrogram\",\n \"diffmap\",\n \"dpt\",\n \"draw_graph\",\n \"embedding_density\",\n \"ingest\",\n \"leiden\",\n \"louvain\",\n \"marker_gene_overlap\",\n \"paga\",\n \"filter_rank_genes_groups\",\n \"rank_genes_groups\",\n \"score_genes\",\n \"score_genes_cell_cycle\",\n \"sim\",\n \"tsne\",\n \"umap\",\n]\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom typing import TYPE_CHECKING\n\nfrom packaging.version import Version\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._settings import settings\nfrom .._utils import _doc_params, raise_not_implemented_error_if_backed_type\nfrom ..neighbors._doc import doc_n_pcs, doc_use_rep\nfrom ._utils import _choose_representation\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n from .._utils import AnyRandom\n\n\n@old_positionals(\n \"use_rep\",\n \"perplexity\",\n \"early_exaggeration\",\n \"learning_rate\",\n \"random_state\",\n \"use_fast_tsne\",\n \"n_jobs\",\n \"copy\",\n)\n@_doc_params(doc_n_pcs=doc_n_pcs, use_rep=doc_use_rep)\ndef tsne(\n adata: AnnData,\n n_pcs: int | None = None,\n *,\n use_rep: str | None = None,\n perplexity: float | int = 30,\n metric: str = \"euclidean\",\n early_exaggeration: float | int = 12,\n learning_rate: float | int = 1000,\n random_state: AnyRandom = 0,\n use_fast_tsne: bool = False,\n n_jobs: int | None = None,\n key_added: str | None = None,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n t-SNE :cite:p:`vanDerMaaten2008,Amir2013,Pedregosa2011`.\n\n t-distributed stochastic neighborhood embedding (tSNE, :cite:t:`vanDerMaaten2008`) has been\n proposed for visualizating single-cell data by :cite:t:`Amir2013`. Here, by default,\n we use the implementation of *scikit-learn* :cite:p:`Pedregosa2011`. You can achieve\n a huge speedup and better convergence if you install Multicore-tSNE_\n by :cite:t:`Ulyanov2016`, which will be automatically detected by Scanpy.\n\n .. _multicore-tsne: https://github.com/DmitryUlyanov/Multicore-TSNE\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n {doc_n_pcs}\n {use_rep}\n perplexity\n The perplexity is related to the number of nearest neighbors that\n is used in other manifold learning algorithms. Larger datasets\n usually require a larger perplexity. Consider selecting a value\n between 5 and 50. The choice is not extremely critical since t-SNE\n is quite insensitive to this parameter.\n metric\n Distance metric calculate neighbors on.\n early_exaggeration\n Controls how tight natural clusters in the original space are in the\n embedded space and how much space will be between them. For larger\n values, the space between natural clusters will be larger in the\n embedded space. Again, the choice of this parameter is not very\n critical. If the cost function increases during initial optimization,\n the early exaggeration factor or the learning rate might be too high.\n learning_rate\n Note that the R-package \"Rtsne\" uses a default of 200.\n The learning rate can be a critical parameter. It should be\n between 100 and 1000. If the cost function increases during initial\n optimization, the early exaggeration factor or the learning rate\n might be too high. If the cost function gets stuck in a bad local\n minimum increasing the learning rate helps sometimes.\n random_state\n Change this to use different intial states for the optimization.\n If `None`, the initial state is not reproducible.\n n_jobs\n Number of jobs for parallel computation.\n `None` means using :attr:`scanpy._settings.ScanpyConfig.n_jobs`.\n key_added\n If not specified, the embedding is stored as\n :attr:`~anndata.AnnData.obsm`\\\\ `['X_tsne']` and the the parameters in\n :attr:`~anndata.AnnData.uns`\\\\ `['tsne']`.\n If specified, the embedding is stored as\n :attr:`~anndata.AnnData.obsm`\\\\ ``[key_added]`` and the the parameters in\n :attr:`~anndata.AnnData.uns`\\\\ ``[key_added]``.\n copy\n Return a copy instead of writing to `adata`.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.obsm['X_tsne' | key_added]` : :class:`numpy.ndarray` (dtype `float`)\n tSNE coordinates of data.\n `adata.uns['tsne' | key_added]` : :class:`dict`\n tSNE parameters.\n\n \"\"\"\n import sklearn\n\n start = logg.info(\"computing tSNE\")\n adata = adata.copy() if copy else adata\n X = _choose_representation(adata, use_rep=use_rep, n_pcs=n_pcs)\n raise_not_implemented_error_if_backed_type(X, \"tsne\")\n # params for sklearn\n n_jobs = settings.n_jobs if n_jobs is None else n_jobs\n params_sklearn = dict(\n perplexity=perplexity,\n random_state=random_state,\n verbose=settings.verbosity > 3,\n early_exaggeration=early_exaggeration,\n learning_rate=learning_rate,\n n_jobs=n_jobs,\n metric=metric,\n )\n if metric != \"euclidean\" and (Version(sklearn.__version__) < Version(\"1.3.0rc1\")):\n params_sklearn[\"square_distances\"] = True\n\n # Backwards compat handling: Remove in scanpy 1.9.0\n if n_jobs != 1 and not use_fast_tsne:\n warnings.warn(\n UserWarning(\n \"In previous versions of scanpy, calling tsne with n_jobs > 1 would use \"\n \"MulticoreTSNE. Now this uses the scikit-learn version of TSNE by default. \"\n \"If you'd like the old behaviour (which is deprecated), pass \"\n \"'use_fast_tsne=True'. Note, MulticoreTSNE is not actually faster anymore.\"\n )\n )\n if use_fast_tsne:\n warnings.warn(\n FutureWarning(\n \"Argument `use_fast_tsne` is deprecated, and support for MulticoreTSNE \"\n \"will be dropped in a future version of scanpy.\"\n )\n )\n\n # deal with different tSNE implementations\n if use_fast_tsne:\n try:\n from MulticoreTSNE import MulticoreTSNE as TSNE\n\n tsne = TSNE(**params_sklearn)\n logg.info(\" using the 'MulticoreTSNE' package by Ulyanov (2017)\")\n # need to transform to float64 for MulticoreTSNE...\n X_tsne = tsne.fit_transform(X.astype(\"float64\"))\n except ImportError:\n use_fast_tsne = False\n warnings.warn(\n UserWarning(\n \"Could not import 'MulticoreTSNE'. Falling back to scikit-learn.\"\n )\n )\n if use_fast_tsne is False: # In case MultiCore failed to import\n from sklearn.manifold import TSNE\n\n # unfortunately, sklearn does not allow to set a minimum number\n # of iterations for barnes-hut tSNE\n tsne = TSNE(**params_sklearn)\n logg.info(\" using sklearn.manifold.TSNE\")\n X_tsne = tsne.fit_transform(X)\n\n # update AnnData instance\n params = dict(\n perplexity=perplexity,\n early_exaggeration=early_exaggeration,\n learning_rate=learning_rate,\n n_jobs=n_jobs,\n metric=metric,\n use_rep=use_rep,\n )\n key_uns, key_obsm = (\"tsne\", \"X_tsne\") if key_added is None else [key_added] * 2\n adata.obsm[key_obsm] = X_tsne # annotate samples with tSNE coordinates\n adata.uns[key_uns] = dict(params={k: v for k, v in params.items() if v is not None})\n\n logg.info(\n \" finished\",\n time=start,\n deep=(\n f\"added\\n\"\n f\" {key_obsm!r}, tSNE coordinates (adata.obsm)\\n\"\n f\" {key_uns!r}, tSNE parameters (adata.uns)\"\n ),\n )\n\n return adata if copy else None\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom anndata import AnnData\n\nfrom .. import _utils\nfrom .._compat import old_positionals\nfrom .._settings import settings\nfrom .._utils._doctests import doctest_internet, doctest_needs\nfrom ..readwrite import read, read_visium\nfrom ._utils import check_datasetdir_exists, filter_oldformatwarning\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from .._utils import AnyRandom\n\n VisiumSampleID = Literal[\n \"V1_Breast_Cancer_Block_A_Section_1\",\n \"V1_Breast_Cancer_Block_A_Section_2\",\n \"V1_Human_Heart\",\n \"V1_Human_Lymph_Node\",\n \"V1_Mouse_Kidney\",\n \"V1_Adult_Mouse_Brain\",\n \"V1_Mouse_Brain_Sagittal_Posterior\",\n \"V1_Mouse_Brain_Sagittal_Posterior_Section_2\",\n \"V1_Mouse_Brain_Sagittal_Anterior\",\n \"V1_Mouse_Brain_Sagittal_Anterior_Section_2\",\n \"V1_Human_Brain_Section_1\",\n \"V1_Human_Brain_Section_2\",\n \"V1_Adult_Mouse_Brain_Coronal_Section_1\",\n \"V1_Adult_Mouse_Brain_Coronal_Section_2\",\n # spaceranger version 1.2.0\n \"Targeted_Visium_Human_Cerebellum_Neuroscience\",\n \"Parent_Visium_Human_Cerebellum\",\n \"Targeted_Visium_Human_SpinalCord_Neuroscience\",\n \"Parent_Visium_Human_SpinalCord\",\n \"Targeted_Visium_Human_Glioblastoma_Pan_Cancer\",\n \"Parent_Visium_Human_Glioblastoma\",\n \"Targeted_Visium_Human_BreastCancer_Immunology\",\n \"Parent_Visium_Human_BreastCancer\",\n \"Targeted_Visium_Human_OvarianCancer_Pan_Cancer\",\n \"Targeted_Visium_Human_OvarianCancer_Immunology\",\n \"Parent_Visium_Human_OvarianCancer\",\n \"Targeted_Visium_Human_ColorectalCancer_GeneSignature\",\n \"Parent_Visium_Human_ColorectalCancer\",\n ]\n\nHERE = Path(__file__).parent\n\n\n@old_positionals(\n \"n_variables\", \"n_centers\", \"cluster_std\", \"n_observations\", \"random_state\"\n)\ndef blobs(\n *,\n n_variables: int = 11,\n n_centers: int = 5,\n cluster_std: float = 1.0,\n n_observations: int = 640,\n random_state: AnyRandom = 0,\n) -> AnnData:\n \"\"\"\\\n Gaussian Blobs.\n\n Parameters\n ----------\n n_variables\n Dimension of feature space.\n n_centers\n Number of cluster centers.\n cluster_std\n Standard deviation of clusters.\n n_observations\n Number of observations. By default, this is the same observation number\n as in :func:`scanpy.datasets.krumsiek11`.\n random_state\n Determines random number generation for dataset creation.\n\n Returns\n -------\n Annotated data matrix containing a observation annotation 'blobs' that\n indicates cluster identity.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> sc.datasets.blobs()\n AnnData object with n_obs × n_vars = 640 × 11\n obs: 'blobs'\n \"\"\"\n import sklearn.datasets\n\n X, y = sklearn.datasets.make_blobs(\n n_samples=n_observations,\n n_features=n_variables,\n centers=n_centers,\n cluster_std=cluster_std,\n random_state=random_state,\n )\n return AnnData(X, obs=dict(blobs=y.astype(str)))\n\n\n@doctest_internet\n@check_datasetdir_exists\ndef burczynski06() -> AnnData:\n \"\"\"\\\n Bulk data with conditions ulcerative colitis (UC) and Crohn’s disease (CD) :cite:p:`Burczynski2006`.\n\n The study assesses transcriptional profiles in peripheral blood mononuclear\n cells from 42 healthy individuals, 59 CD patients, and 26 UC patients by\n hybridization to microarrays interrogating more than 22,000 sequences.\n\n Returns\n -------\n Annotated data matrix.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> sc.datasets.burczynski06()\n AnnData object with n_obs × n_vars = 127 × 22283\n obs: 'groups'\n \"\"\"\n filename = settings.datasetdir / \"burczynski06/GDS1615_full.soft.gz\"\n url = \"ftp://ftp.ncbi.nlm.nih.gov/geo/datasets/GDS1nnn/GDS1615/soft/GDS1615_full.soft.gz\"\n return read(filename, backup_url=url)\n\n\ndef krumsiek11() -> AnnData:\n \"\"\"\\\n Simulated myeloid progenitors :cite:p:`Krumsiek2011`.\n\n The literature-curated boolean network from :cite:t:`Krumsiek2011` was used to\n simulate the data. It describes development to four cell fates annotated in\n :attr:`~anndata.AnnData.obs`\\\\ `[\"cell_type\"]`:\n “monocyte” (`Mo`), “erythrocyte” (`Ery`), “megakaryocyte” (`Mk`) and “neutrophil” (`Neu`).\n\n See also the discussion of this data in :cite:t:`Wolf2019`.\n\n Simulate via :func:`~scanpy.tl.sim`.\n\n Returns\n -------\n Annotated data matrix.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> sc.datasets.krumsiek11()\n UserWarning: Observation names are not unique. To make them unique, call `.obs_names_make_unique`.\n utils.warn_names_duplicates(\"obs\")\n AnnData object with n_obs × n_vars = 640 × 11\n obs: 'cell_type'\n uns: 'iroot', 'highlights'\n \"\"\"\n with settings.verbosity.override(\"error\"): # suppress output...\n adata = read(HERE / \"krumsiek11.txt\", first_column_names=True)\n adata.uns[\"iroot\"] = 0\n fate_labels = {0: \"Stem\", 159: \"Mo\", 319: \"Ery\", 459: \"Mk\", 619: \"Neu\"}\n adata.uns[\"highlights\"] = fate_labels\n cell_type = pd.array([\"progenitor\"]).repeat(adata.n_obs)\n cell_type[80:160] = \"Mo\"\n cell_type[240:320] = \"Ery\"\n cell_type[400:480] = \"Mk\"\n cell_type[560:640] = \"Neu\"\n adata.obs[\"cell_type\"] = cell_type\n _utils.sanitize_anndata(adata)\n return adata\n\n\n@doctest_internet\n@doctest_needs(\"openpyxl\")\n@check_datasetdir_exists\ndef moignard15() -> AnnData:\n \"\"\"\\\n Hematopoiesis in early mouse embryos :cite:p:`Moignard2015`.\n\n The data was obtained using qRT–PCR.\n :attr:`~anndata.AnnData.X` contains the normalized dCt values from supp. table 7 of the publication.\n\n :attr:`~anndata.AnnData.obs`\\\\ `[\"exp_groups\"]` contains the stages derived by\n flow sorting and GFP marker status:\n “primitive streak” (`PS`), “neural plate” (`NP`), “head fold (`HF`),\n “four somite” blood/GFP⁺ (4SG), and “four somite” endothelial/GFP¯ (`4SFG`).\n\n Returns\n -------\n Annotated data matrix.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> sc.datasets.moignard15()\n AnnData object with n_obs × n_vars = 3934 × 42\n obs: 'exp_groups'\n uns: 'iroot', 'exp_groups_colors'\n \"\"\"\n filename = settings.datasetdir / \"moignard15/nbt.3154-S3.xlsx\"\n backup_url = \"https://static-content.springer.com/esm/art%3A10.1038%2Fnbt.3154/MediaObjects/41587_2015_BFnbt3154_MOESM4_ESM.xlsx\"\n adata = read(filename, sheet=\"dCt_values.txt\", backup_url=backup_url)\n # filter out 4 genes as in Haghverdi et al. (2016)\n gene_subset = ~np.in1d(adata.var_names, [\"Eif2b1\", \"Mrpl19\", \"Polr2a\", \"Ubc\"])\n adata = adata[:, gene_subset].copy() # retain non-removed genes\n # choose root cell for DPT analysis as in Haghverdi et al. (2016)\n adata.uns[\"iroot\"] = 532 # note that in Matlab/R, counting starts at 1\n # annotate with Moignard et al. (2015) experimental cell groups\n groups = {\n \"HF\": \"#D7A83E\",\n \"NP\": \"#7AAE5D\",\n \"PS\": \"#497ABC\",\n \"4SG\": \"#AF353A\",\n \"4SFG\": \"#765099\",\n }\n # annotate each observation/cell\n adata.obs[\"exp_groups\"] = [\n next(gname for gname in groups if sname.startswith(gname))\n for sname in adata.obs_names\n ]\n # fix the order and colors of names in \"groups\"\n adata.obs[\"exp_groups\"] = pd.Categorical(\n adata.obs[\"exp_groups\"], categories=list(groups.keys())\n )\n adata.uns[\"exp_groups_colors\"] = list(groups.values())\n return adata\n\n\n@doctest_internet\n@check_datasetdir_exists\ndef paul15() -> AnnData:\n \"\"\"\\\n Development of Myeloid Progenitors :cite:p:`Paul2015`.\n\n Non-logarithmized raw data.\n\n The data has been sent out by Email from the Amit Lab. An R version for\n loading the data can be found `here\n `_.\n\n Returns\n -------\n Annotated data matrix.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> sc.datasets.paul15()\n AnnData object with n_obs × n_vars = 2730 × 3451\n obs: 'paul15_clusters'\n uns: 'iroot'\n \"\"\"\n import h5py\n\n filename = settings.datasetdir / \"paul15/paul15.h5\"\n filename.parent.mkdir(exist_ok=True)\n backup_url = \"https://falexwolf.de/data/paul15.h5\"\n _utils.check_presence_download(filename, backup_url)\n with h5py.File(filename, \"r\") as f:\n # Coercing to float32 for backwards compatibility\n X = f[\"data.debatched\"][()].astype(np.float32)\n gene_names = f[\"data.debatched_rownames\"][()].astype(str)\n cell_names = f[\"data.debatched_colnames\"][()].astype(str)\n clusters = f[\"cluster.id\"][()].flatten().astype(int)\n infogenes_names = f[\"info.genes_strings\"][()].astype(str)\n # each row has to correspond to a observation, therefore transpose\n adata = AnnData(X.transpose())\n adata.var_names = gene_names\n adata.obs_names = cell_names\n # names reflecting the cell type identifications from the paper\n cell_type = 6 * [\"Ery\"]\n cell_type += \"MEP Mk GMP GMP DC Baso Baso Mo Mo Neu Neu Eos Lymph\".split()\n adata.obs[\"paul15_clusters\"] = [f\"{i}{cell_type[i - 1]}\" for i in clusters]\n # make string annotations categorical (optional)\n _utils.sanitize_anndata(adata)\n # just keep the first of the two equivalent names per gene\n adata.var_names = [gn.split(\";\")[0] for gn in adata.var_names]\n # remove 10 corrupted gene names\n infogenes_names = np.intersect1d(infogenes_names, adata.var_names)\n # restrict data array to the 3461 informative genes\n adata = adata[:, infogenes_names].copy()\n # usually we'd set the root cell to an arbitrary cell in the MEP cluster\n # adata.uns['iroot'] = np.flatnonzero(adata.obs['paul15_clusters'] == '7MEP')[0]\n # here, set the root cell as in Haghverdi et al. (2016)\n # note that other than in Matlab/R, counting starts at 0\n adata.uns[\"iroot\"] = 840\n return adata\n\n\ndef toggleswitch() -> AnnData:\n \"\"\"\\\n Simulated toggleswitch.\n\n Data obtained simulating a simple toggleswitch :cite:p:`Gardner2000`\n\n Simulate via :func:`~scanpy.tl.sim`.\n\n Returns\n -------\n Annotated data matrix.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> sc.datasets.toggleswitch()\n UserWarning: Observation names are not unique. To make them unique, call `.obs_names_make_unique`.\n utils.warn_names_duplicates(\"obs\")\n AnnData object with n_obs × n_vars = 200 × 2\n uns: 'iroot'\n \"\"\"\n filename = HERE / \"toggleswitch.txt\"\n adata = read(filename, first_column_names=True)\n adata.uns[\"iroot\"] = 0\n return adata\n\n\n@filter_oldformatwarning\ndef pbmc68k_reduced() -> AnnData:\n \"\"\"\\\n Subsampled and processed 68k PBMCs.\n\n `PBMC 68k dataset`_ from 10x Genomics.\n\n The original PBMC 68k dataset was preprocessed with steps including\n :func:`~scanpy.pp.normalize_total`\\\\ [#norm]_ and :func:`~scanpy.pp.scale`.\n It was saved keeping only 724 cells and 221 highly variable genes.\n\n The saved file contains the annotation of cell types (key: `'bulk_labels'`),\n UMAP coordinates, louvain clustering and gene rankings based on the\n `bulk_labels`.\n\n .. [#norm] Back when the dataset was created, :func:`~scanpy.pp.normalize_per_cell` was used instead.\n .. _PBMC 68k dataset: https://www.10xgenomics.com/datasets/fresh-68-k-pbm-cs-donor-a-1-standard-1-1-0\n\n Returns\n -------\n Annotated data matrix.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> sc.datasets.pbmc68k_reduced()\n AnnData object with n_obs × n_vars = 700 × 765\n obs: 'bulk_labels', 'n_genes', 'percent_mito', 'n_counts', 'S_score', 'G2M_score', 'phase', 'louvain'\n var: 'n_counts', 'means', 'dispersions', 'dispersions_norm', 'highly_variable'\n uns: 'bulk_labels_colors', 'louvain', 'louvain_colors', 'neighbors', 'pca', 'rank_genes_groups'\n obsm: 'X_pca', 'X_umap'\n varm: 'PCs'\n obsp: 'distances', 'connectivities'\n \"\"\"\n\n filename = HERE / \"10x_pbmc68k_reduced.h5ad\"\n with warnings.catch_warnings():\n warnings.filterwarnings(\"ignore\", category=FutureWarning, module=\"anndata\")\n return read(filename)\n\n\n@doctest_internet\n@filter_oldformatwarning\n@check_datasetdir_exists\ndef pbmc3k() -> AnnData:\n \"\"\"\\\n 3k PBMCs from 10x Genomics.\n\n The data consists in 3k PBMCs from a Healthy Donor and is freely available\n from 10x Genomics (file_ from this webpage_).\n\n The exact same data is also used in Seurat’s `basic clustering tutorial`_.\n\n .. _file: https://cf.10xgenomics.com/samples/cell-exp/1.1.0/pbmc3k/pbmc3k_filtered_gene_bc_matrices.tar.gz\n .. _webpage: https://support.10xgenomics.com/single-cell-gene-expression/datasets/1.1.0/pbmc3k\n .. _basic clustering tutorial: https://satijalab.org/seurat/articles/pbmc3k_tutorial.html\n\n .. note::\n This downloads 5.9 MB of data upon the first call of the function and stores it in\n :attr:`~scanpy._settings.ScanpyConfig.datasetdir`\\\\ `/pbmc3k_raw.h5ad`.\n\n The following code was run to produce the file.\n\n .. code:: python\n\n adata = sc.read_10x_mtx(\n # the directory with the `.mtx` file\n './data/filtered_gene_bc_matrices/hg19/',\n # use gene symbols for the variable names (variables-axis index)\n var_names='gene_symbols',\n # write a cache file for faster subsequent reading\n cache=True,\n )\n\n adata.var_names_make_unique() # this is unnecessary if using 'gene_ids'\n adata.write('write/pbmc3k_raw.h5ad', compression='gzip')\n\n Returns\n -------\n Annotated data matrix.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> sc.datasets.pbmc3k()\n AnnData object with n_obs × n_vars = 2700 × 32738\n var: 'gene_ids'\n \"\"\"\n url = \"https://falexwolf.de/data/pbmc3k_raw.h5ad\"\n adata = read(settings.datasetdir / \"pbmc3k_raw.h5ad\", backup_url=url)\n return adata\n\n\n@doctest_internet\n@filter_oldformatwarning\n@check_datasetdir_exists\ndef pbmc3k_processed() -> AnnData:\n \"\"\"\\\n Processed 3k PBMCs from 10x Genomics.\n\n Processed using the basic tutorial :doc:`/tutorials/basics/clustering-2017`.\n\n For preprocessing, cells are filtered out that have few gene counts or too high a `percent_mito`.\n The counts are logarithmized and only genes marked by :func:`~scanpy.pp.highly_variable_genes` are retained.\n The :attr:`~anndata.AnnData.obs` variables `n_counts` and `percent_mito` are corrected for\n using :func:`~scanpy.pp.regress_out`, and values are scaled and clipped by :func:`~scanpy.pp.scale`.\n Finally, :func:`~scanpy.pp.pca` and :func:`~scanpy.pp.neighbors` are calculated.\n\n As analysis steps, the embeddings :func:`~scanpy.tl.tsne` and :func:`~scanpy.tl.umap` are performed.\n Communities are identified using :func:`~scanpy.tl.louvain` and marker genes using :func:`~scanpy.tl.rank_genes_groups`.\n\n Returns\n -------\n Annotated data matrix.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> sc.datasets.pbmc3k_processed()\n AnnData object with n_obs × n_vars = 2638 × 1838\n obs: 'n_genes', 'percent_mito', 'n_counts', 'louvain'\n var: 'n_cells'\n uns: 'draw_graph', 'louvain', 'louvain_colors', 'neighbors', 'pca', 'rank_genes_groups'\n obsm: 'X_pca', 'X_tsne', 'X_umap', 'X_draw_graph_fr'\n varm: 'PCs'\n obsp: 'distances', 'connectivities'\n \"\"\"\n url = \"https://raw.githubusercontent.com/chanzuckerberg/cellxgene/main/example-dataset/pbmc3k.h5ad\"\n\n with warnings.catch_warnings():\n warnings.filterwarnings(\"ignore\", category=FutureWarning, module=\"anndata\")\n return read(settings.datasetdir / \"pbmc3k_processed.h5ad\", backup_url=url)\n\n\ndef _download_visium_dataset(\n sample_id: VisiumSampleID,\n spaceranger_version: Literal[\"1.1.0\", \"1.2.0\"],\n *,\n base_dir: Path | None = None,\n download_image: bool = False,\n) -> Path:\n \"\"\"\\\n Download Visium spatial data from 10x Genomics’ database.\n\n Params\n ------\n sample_id\n String name of example visium dataset.\n base_dir\n Where to download the dataset to.\n download_image\n Whether to download the high-resolution tissue section.\n \"\"\"\n import tarfile\n\n if base_dir is None:\n base_dir = settings.datasetdir\n\n url_prefix = f\"https://cf.10xgenomics.com/samples/spatial-exp/{spaceranger_version}/{sample_id}\"\n\n sample_dir = base_dir / sample_id\n sample_dir.mkdir(exist_ok=True)\n\n # Download spatial data\n tar_filename = f\"{sample_id}_spatial.tar.gz\"\n tar_pth = sample_dir / tar_filename\n _utils.check_presence_download(\n filename=tar_pth, backup_url=f\"{url_prefix}/{tar_filename}\"\n )\n with tarfile.open(tar_pth) as f:\n f.extraction_filter = tarfile.data_filter\n for el in f:\n if not (sample_dir / el.name).exists():\n f.extract(el, sample_dir)\n\n # Download counts\n _utils.check_presence_download(\n filename=sample_dir / \"filtered_feature_bc_matrix.h5\",\n backup_url=f\"{url_prefix}/{sample_id}_filtered_feature_bc_matrix.h5\",\n )\n\n # Download image\n if download_image:\n _utils.check_presence_download(\n filename=sample_dir / \"image.tif\",\n backup_url=f\"{url_prefix}/{sample_id}_image.tif\",\n )\n\n return sample_dir\n\n\n@doctest_internet\n@check_datasetdir_exists\ndef visium_sge(\n sample_id: VisiumSampleID = \"V1_Breast_Cancer_Block_A_Section_1\",\n *,\n include_hires_tiff: bool = False,\n) -> AnnData:\n \"\"\"\\\n Processed Visium Spatial Gene Expression data from 10x Genomics’ database.\n\n The database_ can be browsed online to find the ``sample_id`` you want.\n\n .. _database: https://support.10xgenomics.com/spatial-gene-expression/datasets\n\n Parameters\n ----------\n sample_id\n The ID of the data sample in 10x’s spatial database.\n include_hires_tiff\n Download and include the high-resolution tissue image (tiff) in\n `adata.uns[\"spatial\"][sample_id][\"metadata\"][\"source_image_path\"]`.\n\n Returns\n -------\n Annotated data matrix.\n\n Examples\n --------\n\n >>> import scanpy as sc\n >>> sc.datasets.visium_sge(sample_id='V1_Breast_Cancer_Block_A_Section_1')\n AnnData object with n_obs × n_vars = 3798 × 36601\n obs: 'in_tissue', 'array_row', 'array_col'\n var: 'gene_ids', 'feature_types', 'genome'\n uns: 'spatial'\n obsm: 'spatial'\n \"\"\"\n spaceranger_version = \"1.1.0\" if \"V1_\" in sample_id else \"1.2.0\"\n sample_dir = _download_visium_dataset(\n sample_id, spaceranger_version, download_image=include_hires_tiff\n )\n source_image_path = sample_dir / \"image.tif\" if include_hires_tiff else None\n return read_visium(sample_dir, source_image_path=source_image_path)\n\n\n# model = /Users/alexwolf/hholtz/01_projects/1512_scanpy/scanpy/scanpy/sim_models/krumsiek11.txt\n# tmax = 160\n# branching = True\n# nrRealizations = 4\n# noiseObs = 0\n# noiseDyn = 0.001\n# seed = 0\n# it Gata2 Gata1 Fog1 EKLF Fli1 SCL Cebpa Pu.1 cJun EgrNab Gfi1\n 0 0.8032 -0.0005 -0.0001 0.0003 0.0013 0.0011 0.7997 0.8017 0.0006 0.0009 0.0002\n 1 0.7239 0.0016 -0.0003 -0.0010 0.0011 0.0022 0.8195 0.7256 0.0784 0.0077 0.0941\n 2 0.6578 0.0013 -0.0010 -0.0027 0.0025 0.0011 0.8358 0.6624 0.1105 0.0239 0.1497\n 3 0.5972 0.0038 -0.0009 -0.0064 0.0007 -0.0036 0.8534 0.5992 0.1271 0.0311 0.1702\n 4 0.5396 0.0071 -0.0029 -0.0063 0.0011 -0.0070 0.8656 0.5470 0.1385 0.0379 0.1752\n 5 0.4897 0.0110 -0.0023 -0.0061 0.0013 -0.0034 0.8759 0.4998 0.1496 0.0459 0.1747\n 6 0.4449 0.0153 -0.0020 -0.0089 0.0003 -0.0057 0.8854 0.4516 0.1569 0.0489 0.1727\n 7 0.4078 0.0131 -0.0013 -0.0121 -0.0024 -0.0076 0.8986 0.4096 0.1666 0.0556 0.1661\n 8 0.3719 0.0156 0.0021 -0.0105 -0.0007 -0.0066 0.9062 0.3742 0.1731 0.0631 0.1623\n 9 0.3424 0.0175 0.0027 -0.0083 -0.0045 -0.0017 0.9135 0.3433 0.1781 0.0698 0.1565\n 10 0.3143 0.0179 -0.0001 -0.0100 -0.0042 -0.0039 0.9191 0.3175 0.1891 0.0785 0.1473\n 11 0.2900 0.0215 -0.0007 -0.0069 -0.0034 -0.0018 0.9280 0.2934 0.1988 0.0844 0.1371\n 12 0.2674 0.0255 -0.0020 -0.0074 -0.0030 -0.0013 0.9373 0.2749 0.2103 0.0942 0.1301\n 13 0.2519 0.0284 -0.0011 -0.0029 -0.0037 -0.0022 0.9424 0.2600 0.2211 0.1020 0.1193\n 14 0.2353 0.0371 0.0009 -0.0017 -0.0018 0.0000 0.9492 0.2439 0.2336 0.1114 0.1089\n 15 0.2210 0.0413 0.0009 -0.0028 0.0019 0.0025 0.9503 0.2353 0.2473 0.1219 0.1018\n 16 0.2077 0.0423 0.0003 -0.0016 0.0047 0.0057 0.9527 0.2248 0.2645 0.1389 0.0915\n 17 0.1986 0.0464 -0.0005 0.0012 0.0025 0.0081 0.9551 0.2187 0.2866 0.1572 0.0846\n 18 0.1914 0.0531 0.0020 0.0035 0.0022 0.0143 0.9573 0.2117 0.3103 0.1756 0.0761\n 19 0.1818 0.0613 0.0037 0.0063 0.0018 0.0179 0.9579 0.2034 0.3332 0.2000 0.0722\n 20 0.1734 0.0666 0.0069 0.0066 0.0053 0.0218 0.9585 0.2051 0.3505 0.2227 0.0672\n 21 0.1720 0.0715 0.0052 0.0112 0.0066 0.0260 0.9586 0.2025 0.3742 0.2433 0.0567\n 22 0.1638 0.0762 0.0084 0.0177 0.0096 0.0289 0.9655 0.1981 0.3971 0.2726 0.0492\n 23 0.1556 0.0826 0.0090 0.0151 0.0125 0.0374 0.9661 0.1974 0.4208 0.2965 0.0461\n 24 0.1515 0.0831 0.0117 0.0169 0.0101 0.0385 0.9724 0.1936 0.4450 0.3236 0.0395\n 25 0.1475 0.0821 0.0123 0.0174 0.0150 0.0429 0.9757 0.1898 0.4697 0.3503 0.0346\n 26 0.1480 0.0842 0.0137 0.0186 0.0133 0.0468 0.9751 0.1900 0.4953 0.3773 0.0297\n 27 0.1440 0.0870 0.0101 0.0184 0.0152 0.0544 0.9734 0.1893 0.5165 0.4037 0.0264\n 28 0.1463 0.0892 0.0090 0.0231 0.0156 0.0551 0.9760 0.1906 0.5320 0.4323 0.0203\n 29 0.1413 0.0919 0.0085 0.0255 0.0193 0.0594 0.9795 0.1887 0.5534 0.4581 0.0198\n 30 0.1415 0.0919 0.0067 0.0273 0.0218 0.0629 0.9853 0.1891 0.5680 0.4826 0.0224\n 31 0.1385 0.0952 0.0072 0.0306 0.0209 0.0687 0.9866 0.1904 0.5856 0.5025 0.0213\n 32 0.1386 0.0993 0.0075 0.0265 0.0268 0.0730 0.9878 0.1916 0.6031 0.5224 0.0198\n 33 0.1337 0.1011 0.0061 0.0237 0.0242 0.0751 0.9897 0.1885 0.6190 0.5403 0.0177\n 34 0.1318 0.1009 0.0046 0.0278 0.0267 0.0793 0.9925 0.1892 0.6339 0.5546 0.0173\n 35 0.1295 0.1035 0.0074 0.0300 0.0266 0.0817 0.9967 0.1855 0.6451 0.5754 0.0171\n 36 0.1284 0.1001 0.0063 0.0299 0.0293 0.0868 0.9949 0.1841 0.6546 0.5917 0.0112\n 37 0.1246 0.1018 0.0112 0.0354 0.0264 0.0901 0.9949 0.1886 0.6651 0.6068 0.0119\n 38 0.1230 0.1057 0.0079 0.0353 0.0300 0.0972 0.9890 0.1858 0.6758 0.6210 0.0118\n 39 0.1184 0.1040 0.0087 0.0357 0.0303 0.0996 0.9889 0.1861 0.6811 0.6300 0.0092\n 40 0.1191 0.1036 0.0092 0.0322 0.0303 0.1016 0.9881 0.1853 0.6913 0.6417 0.0084\n 41 0.1198 0.1035 0.0052 0.0332 0.0306 0.1001 0.9874 0.1870 0.7005 0.6513 0.0069\n 42 0.1171 0.1055 0.0055 0.0339 0.0303 0.1041 0.9877 0.1889 0.7091 0.6590 0.0077\n 43 0.1157 0.1066 0.0084 0.0349 0.0303 0.1031 0.9882 0.1905 0.7142 0.6705 0.0064\n 44 0.1160 0.1097 0.0087 0.0341 0.0321 0.1078 0.9902 0.1886 0.7201 0.6802 0.0046\n 45 0.1141 0.1096 0.0116 0.0362 0.0363 0.1088 0.9904 0.1884 0.7225 0.6883 0.0035\n 46 0.1185 0.1081 0.0096 0.0351 0.0363 0.1055 0.9888 0.1909 0.7314 0.6989 0.0051\n 47 0.1174 0.1046 0.0098 0.0370 0.0369 0.1061 0.9890 0.1883 0.7348 0.7054 0.0022\n 48 0.1135 0.1022 0.0110 0.0408 0.0330 0.1091 0.9909 0.1905 0.7427 0.7131 -0.0011\n 49 0.1121 0.1003 0.0074 0.0365 0.0365 0.1053 0.9905 0.1945 0.7472 0.7213 -0.0017\n 50 0.1081 0.1033 0.0071 0.0346 0.0347 0.1052 0.9898 0.1982 0.7510 0.7302 -0.0023\n 51 0.1103 0.1027 0.0079 0.0363 0.0340 0.1067 0.9896 0.1997 0.7534 0.7336 -0.0017\n 52 0.1047 0.1011 0.0097 0.0364 0.0325 0.1058 0.9923 0.1992 0.7573 0.7389 -0.0003\n 53 0.1025 0.0991 0.0093 0.0366 0.0337 0.1076 0.9941 0.2040 0.7612 0.7454 -0.0040\n 54 0.1005 0.0926 0.0114 0.0321 0.0336 0.1090 0.9901 0.2098 0.7634 0.7517 -0.0053\n 55 0.0997 0.0921 0.0115 0.0339 0.0349 0.1073 0.9903 0.2147 0.7698 0.7554 -0.0021\n 56 0.0951 0.0846 0.0111 0.0332 0.0301 0.1033 0.9878 0.2235 0.7738 0.7626 0.0029\n 57 0.0924 0.0828 0.0075 0.0315 0.0260 0.1016 0.9870 0.2298 0.7837 0.7651 0.0008\n 58 0.0834 0.0783 0.0061 0.0319 0.0253 0.0989 0.9901 0.2409 0.7928 0.7706 0.0024\n 59 0.0813 0.0749 0.0031 0.0287 0.0255 0.0924 0.9896 0.2546 0.7998 0.7774 -0.0016\n 60 0.0768 0.0739 0.0021 0.0278 0.0231 0.0906 0.9857 0.2699 0.8083 0.7835 -0.0024\n 61 0.0722 0.0680 0.0031 0.0277 0.0217 0.0858 0.9876 0.2841 0.8176 0.7923 0.0003\n 62 0.0674 0.0627 0.0038 0.0277 0.0194 0.0808 0.9897 0.3011 0.8278 0.8008 -0.0021\n 63 0.0631 0.0582 0.0021 0.0274 0.0150 0.0737 0.9927 0.3201 0.8357 0.8083 0.0001\n 64 0.0613 0.0532 0.0057 0.0297 0.0130 0.0709 0.9939 0.3414 0.8409 0.8186 -0.0032\n 65 0.0579 0.0485 0.0053 0.0274 0.0107 0.0615 0.9973 0.3643 0.8514 0.8257 -0.0039\n 66 0.0535 0.0452 0.0050 0.0238 0.0132 0.0566 0.9986 0.3888 0.8545 0.8368 0.0008\n 67 0.0477 0.0400 0.0042 0.0218 0.0116 0.0498 0.9957 0.4158 0.8616 0.8432 0.0028\n 68 0.0469 0.0364 0.0036 0.0232 0.0097 0.0487 0.9951 0.4458 0.8701 0.8508 0.0007\n 69 0.0389 0.0306 0.0076 0.0225 0.0100 0.0461 0.9950 0.4745 0.8801 0.8617 -0.0010\n 70 0.0382 0.0282 0.0055 0.0210 0.0079 0.0401 0.9959 0.5078 0.8884 0.8711 -0.0010\n 71 0.0349 0.0258 0.0073 0.0155 0.0071 0.0379 0.9961 0.5373 0.8909 0.8802 -0.0034\n 72 0.0320 0.0208 0.0082 0.0151 0.0080 0.0346 0.9989 0.5653 0.8929 0.8879 -0.0034\n 73 0.0293 0.0187 0.0041 0.0176 0.0085 0.0329 1.0026 0.5954 0.9017 0.8970 -0.0010\n 74 0.0240 0.0158 0.0022 0.0167 0.0050 0.0303 1.0038 0.6247 0.9094 0.9030 0.0002\n 75 0.0220 0.0090 -0.0012 0.0165 0.0034 0.0272 1.0070 0.6518 0.9158 0.9111 0.0011\n 76 0.0211 0.0078 0.0003 0.0161 0.0044 0.0248 1.0038 0.6807 0.9249 0.9182 -0.0004\n 77 0.0172 0.0098 -0.0039 0.0135 0.0038 0.0226 1.0018 0.7108 0.9311 0.9203 0.0011\n 78 0.0169 0.0092 -0.0026 0.0159 0.0003 0.0222 1.0008 0.7342 0.9365 0.9253 -0.0024\n 79 0.0167 0.0094 -0.0009 0.0104 0.0012 0.0208 1.0022 0.7559 0.9428 0.9311 0.0006\n 80 0.0185 0.0098 -0.0003 0.0101 -0.0013 0.0189 0.9999 0.7761 0.9460 0.9362 0.0030\n 81 0.0154 0.0114 0.0026 0.0074 -0.0037 0.0172 0.9994 0.7940 0.9499 0.9424 -0.0009\n 82 0.0126 0.0088 0.0024 0.0053 0.0001 0.0156 1.0002 0.8087 0.9503 0.9475 -0.0000\n 83 0.0134 0.0109 0.0041 0.0088 -0.0014 0.0132 0.9984 0.8265 0.9490 0.9501 -0.0007\n 84 0.0125 0.0123 0.0074 0.0102 -0.0061 0.0118 0.9963 0.8391 0.9515 0.9545 -0.0025\n 85 0.0107 0.0062 0.0096 0.0118 -0.0063 0.0124 0.9984 0.8514 0.9559 0.9538 -0.0040\n 86 0.0059 0.0064 0.0085 0.0098 -0.0053 0.0137 0.9971 0.8692 0.9585 0.9517 -0.0023\n 87 0.0021 0.0086 0.0082 0.0063 -0.0020 0.0124 0.9969 0.8816 0.9620 0.9576 -0.0047\n 88 0.0020 0.0108 0.0068 0.0048 -0.0022 0.0107 0.9977 0.8923 0.9618 0.9594 -0.0029\n 89 -0.0007 0.0139 0.0069 0.0037 0.0010 0.0110 0.9976 0.9006 0.9620 0.9634 -0.0037\n 90 -0.0030 0.0166 0.0068 0.0059 0.0021 0.0133 0.9994 0.9091 0.9675 0.9675 -0.0059\n 91 0.0008 0.0137 0.0064 0.0065 0.0014 0.0107 1.0005 0.9129 0.9680 0.9715 -0.0038\n 92 0.0013 0.0093 0.0072 0.0065 0.0038 0.0074 0.9998 0.9200 0.9722 0.9696 -0.0059\n 93 -0.0029 0.0144 0.0078 0.0070 0.0060 0.0031 1.0009 0.9250 0.9704 0.9735 -0.0061\n 94 -0.0020 0.0116 0.0036 0.0052 0.0057 0.0038 0.9996 0.9346 0.9716 0.9735 -0.0105\n 95 -0.0033 0.0080 0.0035 0.0036 0.0017 0.0006 1.0053 0.9462 0.9707 0.9710 -0.0066\n 96 -0.0007 0.0108 0.0061 0.0022 0.0002 0.0006 1.0030 0.9457 0.9736 0.9694 -0.0085\n 97 -0.0025 0.0108 0.0093 0.0052 -0.0024 -0.0026 1.0027 0.9527 0.9752 0.9695 -0.0089\n 98 -0.0053 0.0130 0.0116 0.0052 -0.0019 -0.0026 1.0038 0.9568 0.9781 0.9704 -0.0043\n 99 -0.0068 0.0112 0.0093 -0.0005 -0.0010 -0.0036 1.0003 0.9568 0.9795 0.9688 -0.0049\n 100 -0.0074 0.0139 0.0063 -0.0010 -0.0047 -0.0024 0.9981 0.9605 0.9802 0.9699 -0.0049\n 101 -0.0061 0.0137 0.0035 0.0012 -0.0037 -0.0012 0.9979 0.9641 0.9809 0.9701 -0.0066\n 102 -0.0059 0.0165 0.0025 0.0017 -0.0053 0.0005 0.9960 0.9642 0.9795 0.9754 -0.0068\n 103 -0.0045 0.0184 0.0014 0.0040 -0.0033 0.0004 0.9965 0.9577 0.9801 0.9744 -0.0065\n 104 -0.0043 0.0163 0.0025 0.0038 0.0009 -0.0003 0.9945 0.9593 0.9806 0.9747 -0.0044\n 105 -0.0045 0.0147 0.0030 0.0044 0.0015 0.0000 0.9942 0.9609 0.9830 0.9734 -0.0003\n 106 -0.0047 0.0131 0.0037 -0.0018 0.0018 0.0049 0.9941 0.9631 0.9853 0.9728 0.0013\n 107 0.0002 0.0101 0.0029 -0.0029 0.0058 0.0068 0.9959 0.9660 0.9824 0.9748 -0.0022\n 108 0.0016 0.0111 0.0001 -0.0078 0.0081 0.0064 0.9959 0.9716 0.9835 0.9735 -0.0042\n 109 0.0025 0.0106 -0.0023 -0.0085 0.0085 0.0027 0.9943 0.9753 0.9868 0.9730 -0.0021\n 110 0.0027 0.0092 -0.0021 -0.0051 0.0070 0.0048 0.9943 0.9794 0.9826 0.9674 -0.0006\n 111 0.0042 0.0062 0.0020 -0.0054 0.0056 0.0059 0.9933 0.9829 0.9873 0.9668 0.0006\n 112 0.0039 0.0068 0.0009 -0.0043 0.0046 0.0025 0.9921 0.9870 0.9891 0.9643 0.0000\n 113 0.0028 0.0096 -0.0007 -0.0031 0.0057 0.0042 0.9922 0.9837 0.9896 0.9661 -0.0009\n 114 0.0019 0.0047 -0.0007 -0.0082 0.0065 0.0035 0.9948 0.9885 0.9905 0.9749 0.0027\n 115 0.0028 0.0038 0.0001 -0.0087 0.0040 0.0014 0.9908 0.9901 0.9883 0.9756 0.0049\n 116 0.0039 0.0005 0.0006 -0.0096 -0.0003 0.0012 0.9952 0.9904 0.9888 0.9733 0.0042\n 117 0.0050 -0.0023 -0.0001 -0.0071 0.0023 0.0027 0.9925 0.9893 0.9932 0.9762 0.0070\n 118 0.0012 -0.0036 -0.0023 -0.0031 -0.0015 0.0026 0.9942 0.9892 0.9905 0.9751 0.0051\n 119 0.0040 -0.0063 -0.0019 -0.0054 0.0018 -0.0013 0.9924 0.9932 0.9899 0.9769 0.0043\n 120 0.0063 -0.0024 -0.0017 -0.0035 0.0039 -0.0023 0.9944 0.9919 0.9901 0.9737 0.0036\n 121 0.0058 -0.0052 -0.0013 -0.0040 0.0065 -0.0003 0.9961 0.9918 0.9902 0.9757 0.0085\n 122 0.0018 -0.0063 0.0011 -0.0058 0.0093 0.0018 0.9963 0.9893 0.9866 0.9750 0.0094\n 123 0.0022 -0.0093 0.0020 -0.0062 0.0066 0.0022 0.9980 0.9903 0.9829 0.9705 0.0088\n 124 0.0019 -0.0062 0.0016 -0.0047 0.0062 0.0021 0.9974 0.9918 0.9846 0.9692 0.0120\n 125 -0.0005 -0.0015 0.0002 -0.0034 0.0061 0.0036 0.9996 0.9918 0.9850 0.9668 0.0124\n 126 -0.0016 -0.0014 -0.0024 -0.0052 0.0070 0.0029 1.0000 0.9946 0.9850 0.9632 0.0118\n 127 -0.0015 -0.0043 -0.0008 -0.0037 0.0049 -0.0014 0.9971 0.9967 0.9822 0.9612 0.0098\n 128 -0.0000 -0.0043 -0.0004 -0.0035 0.0081 -0.0023 1.0001 0.9934 0.9818 0.9589 0.0120\n 129 0.0027 -0.0074 -0.0033 -0.0043 0.0075 0.0013 0.9986 0.9908 0.9843 0.9579 0.0108\n 130 0.0019 -0.0079 -0.0030 -0.0035 0.0092 0.0040 0.9993 0.9894 0.9820 0.9598 0.0080\n 131 0.0006 -0.0051 -0.0023 -0.0014 0.0091 0.0008 0.9975 0.9897 0.9811 0.9603 0.0096\n 132 -0.0002 -0.0048 -0.0008 0.0030 0.0075 0.0011 1.0006 0.9884 0.9795 0.9586 0.0092\n 133 0.0023 -0.0038 0.0014 0.0040 0.0067 0.0015 1.0028 0.9882 0.9801 0.9605 0.0036\n 134 0.0014 -0.0065 0.0020 0.0036 0.0056 0.0016 1.0036 0.9893 0.9826 0.9663 0.0010\n 135 -0.0007 -0.0055 -0.0049 0.0039 0.0080 0.0025 1.0024 0.9888 0.9850 0.9644 0.0024\n 136 -0.0024 -0.0029 -0.0052 0.0036 0.0068 -0.0005 1.0003 0.9927 0.9850 0.9666 -0.0009\n 137 -0.0031 -0.0021 -0.0020 0.0037 0.0069 -0.0019 0.9958 0.9910 0.9841 0.9664 -0.0010\n 138 -0.0030 -0.0011 -0.0076 0.0040 0.0056 0.0010 0.9939 0.9920 0.9847 0.9643 0.0027\n 139 0.0022 -0.0001 -0.0061 0.0021 0.0071 0.0049 0.9942 0.9918 0.9844 0.9643 0.0071\n 140 0.0023 -0.0038 -0.0019 -0.0010 0.0113 0.0033 0.9937 0.9944 0.9869 0.9642 0.0058\n 141 -0.0003 -0.0077 -0.0024 -0.0017 0.0093 0.0013 0.9973 0.9974 0.9832 0.9691 0.0047\n 142 -0.0002 -0.0075 -0.0002 -0.0038 0.0071 0.0033 0.9980 0.9991 0.9821 0.9672 0.0077\n 143 -0.0030 -0.0126 0.0015 -0.0014 0.0051 0.0053 0.9955 0.9991 0.9796 0.9675 0.0066\n 144 -0.0034 -0.0110 0.0002 -0.0015 0.0030 0.0000 1.0002 0.9958 0.9807 0.9680 0.0036\n 145 -0.0023 -0.0103 0.0044 -0.0015 0.0015 0.0004 0.9980 0.9940 0.9825 0.9632 0.0000\n 146 0.0016 -0.0055 0.0046 -0.0005 -0.0004 0.0019 0.9962 0.9964 0.9850 0.9651 -0.0030\n 147 -0.0024 -0.0046 0.0050 -0.0000 -0.0025 -0.0009 0.9988 0.9948 0.9831 0.9667 -0.0070\n 148 -0.0019 -0.0047 0.0046 0.0044 -0.0027 -0.0020 0.9983 0.9936 0.9876 0.9659 -0.0017\n 149 0.0003 -0.0060 0.0020 0.0068 -0.0005 -0.0003 1.0017 0.9924 0.9882 0.9700 0.0009\n 150 0.0028 -0.0084 0.0001 0.0052 -0.0012 0.0010 1.0011 0.9943 0.9870 0.9724 0.0020\n 151 0.0062 -0.0111 -0.0006 0.0063 -0.0020 -0.0004 1.0000 0.9932 0.9869 0.9714 -0.0032\n 152 0.0039 -0.0129 -0.0038 0.0022 -0.0010 -0.0039 0.9987 0.9930 0.9905 0.9704 -0.0045\n 153 0.0051 -0.0115 -0.0064 0.0016 -0.0044 -0.0044 0.9956 0.9956 0.9873 0.9671 -0.0024\n 154 0.0063 -0.0117 -0.0049 0.0019 0.0039 -0.0029 0.9967 0.9937 0.9887 0.9639 -0.0018\n 155 0.0054 -0.0098 -0.0060 0.0015 0.0029 -0.0064 0.9971 0.9938 0.9855 0.9656 0.0008\n 156 0.0053 -0.0098 -0.0015 -0.0005 0.0002 -0.0080 0.9963 0.9955 0.9835 0.9653 0.0025\n 157 0.0047 -0.0100 -0.0010 -0.0038 0.0043 -0.0108 0.9998 0.9949 0.9820 0.9645 0.0019\n 158 0.0023 -0.0092 0.0008 -0.0045 0.0035 -0.0064 1.0021 0.9949 0.9839 0.9718 -0.0002\n 159 0.0046 -0.0072 0.0018 -0.0036 0.0021 -0.0090 1.0048 0.9932 0.9893 0.9727 0.0001\n 0 0.8010 -0.0022 0.0006 0.0015 0.0011 0.0010 0.8006 0.8023 0.0012 -0.0001 0.0001\n 1 0.7284 0.0041 -0.0003 0.0006 -0.0033 -0.0026 0.8190 0.7276 0.0838 0.0093 0.0925\n 2 0.6616 0.0035 -0.0021 -0.0001 -0.0011 0.0020 0.8405 0.6612 0.1179 0.0241 0.1436\n 3 0.5999 0.0101 -0.0041 0.0001 0.0013 -0.0001 0.8545 0.6028 0.1350 0.0346 0.1620\n 4 0.5440 0.0099 -0.0027 -0.0016 0.0025 0.0042 0.8693 0.5464 0.1468 0.0408 0.1701\n 5 0.4967 0.0113 -0.0014 -0.0018 0.0002 0.0052 0.8819 0.4959 0.1593 0.0417 0.1744\n 6 0.4502 0.0148 -0.0003 -0.0066 -0.0010 0.0037 0.8939 0.4550 0.1674 0.0488 0.1762\n 7 0.4116 0.0179 -0.0021 -0.0026 -0.0013 0.0058 0.9043 0.4148 0.1741 0.0538 0.1727\n 8 0.3719 0.0200 -0.0040 -0.0024 0.0002 0.0039 0.9130 0.3749 0.1811 0.0593 0.1686\n 9 0.3395 0.0204 -0.0037 -0.0020 0.0015 0.0070 0.9220 0.3441 0.1851 0.0629 0.1611\n 10 0.3157 0.0243 -0.0035 -0.0038 -0.0005 0.0090 0.9312 0.3184 0.1962 0.0672 0.1511\n 11 0.2927 0.0311 -0.0008 -0.0016 0.0038 0.0071 0.9339 0.2921 0.2057 0.0781 0.1386\n 12 0.2708 0.0356 -0.0049 -0.0009 0.0034 0.0060 0.9413 0.2739 0.2158 0.0845 0.1295\n 13 0.2521 0.0379 -0.0031 0.0022 0.0022 0.0073 0.9490 0.2528 0.2296 0.0945 0.1177\n 14 0.2362 0.0405 -0.0026 0.0034 0.0010 0.0086 0.9547 0.2394 0.2437 0.1072 0.1092\n 15 0.2234 0.0443 -0.0039 0.0056 0.0020 0.0098 0.9583 0.2288 0.2601 0.1218 0.0992\n 16 0.2127 0.0473 -0.0038 0.0068 0.0020 0.0085 0.9623 0.2219 0.2744 0.1386 0.0929\n 17 0.2047 0.0499 -0.0026 0.0067 0.0057 0.0107 0.9698 0.2114 0.2937 0.1583 0.0828\n 18 0.1978 0.0517 -0.0033 0.0080 0.0028 0.0132 0.9742 0.2057 0.3168 0.1786 0.0740\n 19 0.1868 0.0594 -0.0054 0.0069 0.0041 0.0163 0.9829 0.2020 0.3436 0.2011 0.0645\n 20 0.1831 0.0645 -0.0092 0.0061 0.0048 0.0177 0.9855 0.1986 0.3686 0.2213 0.0587\n 21 0.1770 0.0707 -0.0080 0.0079 0.0065 0.0212 0.9879 0.1967 0.3865 0.2469 0.0539\n 22 0.1693 0.0770 -0.0089 0.0105 0.0038 0.0276 0.9885 0.1916 0.4128 0.2742 0.0429\n 23 0.1646 0.0818 -0.0076 0.0122 0.0067 0.0306 0.9909 0.1897 0.4387 0.3045 0.0405\n 24 0.1654 0.0858 -0.0078 0.0137 0.0069 0.0383 0.9925 0.1840 0.4608 0.3276 0.0386\n 25 0.1629 0.0917 -0.0041 0.0137 0.0059 0.0448 0.9932 0.1801 0.4801 0.3554 0.0369\n 26 0.1613 0.0982 -0.0017 0.0184 0.0095 0.0469 0.9971 0.1779 0.4980 0.3820 0.0307\n 27 0.1598 0.1014 -0.0001 0.0234 0.0092 0.0524 1.0007 0.1767 0.5127 0.4030 0.0274\n 28 0.1588 0.1023 0.0033 0.0253 0.0128 0.0617 1.0001 0.1709 0.5313 0.4282 0.0229\n 29 0.1623 0.1061 0.0010 0.0270 0.0107 0.0690 1.0003 0.1667 0.5437 0.4455 0.0197\n 30 0.1617 0.1115 0.0017 0.0279 0.0131 0.0742 1.0010 0.1629 0.5597 0.4675 0.0149\n 31 0.1599 0.1162 0.0044 0.0294 0.0140 0.0815 1.0009 0.1556 0.5704 0.4850 0.0126\n 32 0.1598 0.1257 0.0023 0.0321 0.0166 0.0899 0.9989 0.1496 0.5863 0.5039 0.0108\n 33 0.1605 0.1277 0.0051 0.0355 0.0171 0.1021 0.9973 0.1451 0.5942 0.5198 0.0097\n 34 0.1617 0.1322 0.0035 0.0388 0.0186 0.1144 0.9964 0.1406 0.5955 0.5337 0.0065\n 35 0.1653 0.1372 0.0027 0.0400 0.0233 0.1250 0.9950 0.1402 0.6044 0.5442 0.0062\n 36 0.1687 0.1432 0.0038 0.0460 0.0264 0.1354 0.9937 0.1354 0.6070 0.5519 0.0067\n 37 0.1702 0.1497 0.0081 0.0453 0.0276 0.1436 0.9949 0.1308 0.6099 0.5583 0.0058\n 38 0.1773 0.1585 0.0093 0.0488 0.0323 0.1551 0.9969 0.1283 0.6105 0.5676 0.0024\n 39 0.1812 0.1662 0.0093 0.0523 0.0362 0.1681 0.9986 0.1217 0.6106 0.5711 0.0060\n 40 0.1854 0.1749 0.0119 0.0520 0.0373 0.1837 0.9998 0.1168 0.6087 0.5704 0.0063\n 41 0.1962 0.1837 0.0195 0.0567 0.0465 0.2012 1.0025 0.1120 0.6016 0.5694 0.0076\n 42 0.2050 0.1944 0.0221 0.0611 0.0473 0.2189 1.0032 0.1037 0.5965 0.5631 0.0105\n 43 0.2176 0.2054 0.0232 0.0677 0.0500 0.2334 0.9975 0.1006 0.5854 0.5531 0.0115\n 44 0.2394 0.2212 0.0237 0.0774 0.0556 0.2492 0.9962 0.0924 0.5755 0.5463 0.0131\n 45 0.2559 0.2381 0.0266 0.0839 0.0599 0.2693 0.9923 0.0862 0.5638 0.5329 0.0090\n 46 0.2749 0.2586 0.0313 0.0914 0.0620 0.2953 0.9886 0.0758 0.5468 0.5163 0.0103\n 47 0.3003 0.2814 0.0351 0.0976 0.0650 0.3204 0.9811 0.0689 0.5257 0.4982 0.0078\n 48 0.3248 0.3078 0.0416 0.1060 0.0717 0.3479 0.9760 0.0650 0.5069 0.4811 0.0089\n 49 0.3500 0.3345 0.0466 0.1107 0.0782 0.3781 0.9665 0.0576 0.4836 0.4571 0.0118\n 50 0.3788 0.3665 0.0501 0.1190 0.0827 0.4093 0.9559 0.0486 0.4563 0.4348 0.0107\n 51 0.4105 0.4007 0.0588 0.1308 0.0871 0.4417 0.9433 0.0436 0.4301 0.4067 0.0113\n 52 0.4407 0.4345 0.0662 0.1399 0.0947 0.4768 0.9247 0.0417 0.4051 0.3803 0.0087\n 53 0.4716 0.4666 0.0773 0.1478 0.0956 0.5090 0.9014 0.0396 0.3802 0.3551 0.0117\n 54 0.5031 0.4984 0.0885 0.1567 0.0993 0.5384 0.8763 0.0360 0.3573 0.3319 0.0075\n 55 0.5235 0.5300 0.1004 0.1643 0.1034 0.5712 0.8445 0.0351 0.3344 0.3097 0.0052\n 56 0.5463 0.5585 0.1134 0.1712 0.1054 0.5991 0.8107 0.0316 0.3141 0.2901 0.0037\n 57 0.5675 0.5858 0.1291 0.1819 0.1096 0.6259 0.7738 0.0278 0.2932 0.2701 0.0024\n 58 0.5832 0.6125 0.1426 0.1915 0.1081 0.6487 0.7343 0.0250 0.2737 0.2516 0.0026\n 59 0.5993 0.6388 0.1556 0.1987 0.1113 0.6799 0.6984 0.0259 0.2520 0.2289 0.0054\n 60 0.6089 0.6667 0.1741 0.2063 0.1154 0.6995 0.6580 0.0240 0.2325 0.2175 0.0087\n 61 0.6186 0.6921 0.1895 0.2115 0.1147 0.7219 0.6228 0.0242 0.2181 0.1990 0.0112\n 62 0.6270 0.7108 0.2027 0.2202 0.1125 0.7420 0.5894 0.0259 0.2023 0.1847 0.0103\n 63 0.6305 0.7311 0.2166 0.2245 0.1168 0.7652 0.5562 0.0201 0.1872 0.1696 0.0132\n 64 0.6301 0.7527 0.2349 0.2343 0.1153 0.7824 0.5216 0.0156 0.1736 0.1609 0.0122\n 65 0.6341 0.7738 0.2456 0.2387 0.1153 0.7985 0.4897 0.0148 0.1577 0.1457 0.0154\n 66 0.6352 0.7905 0.2633 0.2428 0.1168 0.8144 0.4607 0.0138 0.1452 0.1345 0.0141\n 67 0.6360 0.8070 0.2763 0.2520 0.1138 0.8305 0.4335 0.0150 0.1350 0.1254 0.0140\n 68 0.6322 0.8231 0.2887 0.2595 0.1102 0.8415 0.4037 0.0115 0.1216 0.1172 0.0174\n 69 0.6290 0.8384 0.2979 0.2678 0.1081 0.8536 0.3766 0.0149 0.1121 0.1032 0.0175\n 70 0.6238 0.8474 0.3051 0.2776 0.1040 0.8605 0.3496 0.0165 0.1039 0.0940 0.0215\n 71 0.6192 0.8593 0.3165 0.2851 0.1014 0.8706 0.3240 0.0165 0.0949 0.0856 0.0212\n 72 0.6156 0.8678 0.3298 0.2935 0.0962 0.8800 0.3031 0.0139 0.0867 0.0802 0.0261\n 73 0.6117 0.8787 0.3425 0.3021 0.0981 0.8878 0.2860 0.0131 0.0786 0.0700 0.0293\n 74 0.6071 0.8869 0.3557 0.3125 0.0958 0.8941 0.2657 0.0145 0.0749 0.0632 0.0355\n 75 0.5986 0.8922 0.3632 0.3190 0.0955 0.9000 0.2488 0.0159 0.0694 0.0570 0.0374\n 76 0.5905 0.8954 0.3657 0.3241 0.0931 0.9087 0.2322 0.0146 0.0654 0.0543 0.0427\n 77 0.5838 0.8963 0.3741 0.3335 0.0956 0.9119 0.2226 0.0167 0.0628 0.0495 0.0489\n 78 0.5813 0.9027 0.3806 0.3409 0.0882 0.9175 0.2105 0.0183 0.0597 0.0471 0.0581\n 79 0.5767 0.9050 0.3907 0.3539 0.0875 0.9220 0.1973 0.0135 0.0556 0.0435 0.0668\n 80 0.5717 0.9083 0.3967 0.3635 0.0866 0.9295 0.1883 0.0145 0.0530 0.0374 0.0707\n 81 0.5698 0.9155 0.4027 0.3710 0.0831 0.9328 0.1767 0.0136 0.0535 0.0327 0.0822\n 82 0.5670 0.9177 0.4084 0.3811 0.0787 0.9375 0.1645 0.0127 0.0462 0.0305 0.0959\n 83 0.5664 0.9246 0.4147 0.3923 0.0778 0.9399 0.1561 0.0129 0.0410 0.0263 0.1112\n 84 0.5642 0.9319 0.4157 0.4011 0.0762 0.9401 0.1501 0.0144 0.0385 0.0212 0.1246\n 85 0.5624 0.9355 0.4219 0.4165 0.0733 0.9446 0.1423 0.0119 0.0330 0.0140 0.1447\n 86 0.5555 0.9437 0.4247 0.4277 0.0723 0.9476 0.1344 0.0151 0.0299 0.0117 0.1783\n 87 0.5533 0.9452 0.4292 0.4351 0.0692 0.9504 0.1257 0.0092 0.0296 0.0066 0.2120\n 88 0.5549 0.9466 0.4325 0.4457 0.0652 0.9532 0.1207 0.0106 0.0254 0.0041 0.2469\n 89 0.5509 0.9508 0.4387 0.4559 0.0621 0.9564 0.1144 0.0085 0.0224 0.0069 0.2793\n 90 0.5523 0.9542 0.4399 0.4610 0.0596 0.9553 0.1056 0.0090 0.0215 0.0059 0.2985\n 91 0.5477 0.9519 0.4398 0.4680 0.0595 0.9630 0.1035 0.0088 0.0198 0.0044 0.3173\n 92 0.5465 0.9524 0.4417 0.4841 0.0552 0.9631 0.1006 0.0093 0.0212 0.0030 0.3342\n 93 0.5435 0.9538 0.4457 0.4936 0.0556 0.9652 0.0974 0.0125 0.0174 0.0028 0.3524\n 94 0.5425 0.9534 0.4488 0.5020 0.0526 0.9637 0.0956 0.0104 0.0148 0.0031 0.3659\n 95 0.5394 0.9545 0.4515 0.5094 0.0483 0.9685 0.0921 0.0113 0.0145 0.0020 0.3746\n 96 0.5364 0.9577 0.4528 0.5258 0.0466 0.9713 0.0902 0.0141 0.0145 0.0047 0.3804\n 97 0.5350 0.9600 0.4567 0.5378 0.0462 0.9690 0.0861 0.0127 0.0136 0.0033 0.3855\n 98 0.5336 0.9612 0.4535 0.5530 0.0445 0.9699 0.0849 0.0090 0.0122 0.0040 0.3898\n 99 0.5333 0.9610 0.4578 0.5634 0.0441 0.9697 0.0875 0.0096 0.0137 0.0021 0.3943\n 100 0.5296 0.9628 0.4592 0.5770 0.0424 0.9745 0.0851 0.0102 0.0123 0.0046 0.3997\n 101 0.5262 0.9617 0.4584 0.5886 0.0419 0.9755 0.0840 0.0094 0.0111 0.0058 0.4003\n 102 0.5256 0.9610 0.4602 0.5961 0.0401 0.9768 0.0840 0.0109 0.0106 0.0056 0.3967\n 103 0.5252 0.9620 0.4632 0.6069 0.0380 0.9755 0.0798 0.0154 0.0100 0.0049 0.3958\n 104 0.5247 0.9625 0.4659 0.6176 0.0363 0.9756 0.0743 0.0110 0.0114 0.0058 0.3909\n 105 0.5224 0.9650 0.4695 0.6248 0.0371 0.9738 0.0750 0.0136 0.0087 0.0053 0.3830\n 106 0.5218 0.9620 0.4710 0.6345 0.0366 0.9751 0.0727 0.0114 0.0077 0.0023 0.3819\n 107 0.5211 0.9625 0.4738 0.6465 0.0395 0.9738 0.0731 0.0121 0.0059 0.0071 0.3777\n 108 0.5192 0.9622 0.4750 0.6501 0.0394 0.9743 0.0735 0.0085 0.0038 0.0036 0.3706\n 109 0.5196 0.9635 0.4745 0.6546 0.0369 0.9774 0.0751 0.0081 0.0055 0.0059 0.3718\n 110 0.5189 0.9642 0.4776 0.6605 0.0359 0.9782 0.0741 0.0079 0.0039 0.0083 0.3682\n 111 0.5158 0.9633 0.4768 0.6613 0.0353 0.9781 0.0703 0.0039 0.0044 0.0073 0.3640\n 112 0.5188 0.9659 0.4794 0.6638 0.0304 0.9792 0.0691 0.0046 -0.0003 0.0008 0.3562\n 113 0.5187 0.9628 0.4797 0.6698 0.0310 0.9807 0.0692 0.0056 0.0027 0.0012 0.3519\n 114 0.5191 0.9646 0.4808 0.6776 0.0297 0.9780 0.0673 0.0093 0.0003 0.0019 0.3478\n 115 0.5201 0.9615 0.4860 0.6857 0.0263 0.9789 0.0673 0.0076 0.0026 0.0059 0.3432\n 116 0.5221 0.9605 0.4821 0.6905 0.0279 0.9820 0.0644 0.0051 0.0029 0.0050 0.3348\n 117 0.5223 0.9593 0.4826 0.6953 0.0199 0.9872 0.0655 0.0035 0.0057 0.0041 0.3308\n 118 0.5240 0.9646 0.4817 0.7012 0.0184 0.9859 0.0629 0.0057 0.0082 0.0007 0.3269\n 119 0.5260 0.9628 0.4831 0.7102 0.0151 0.9831 0.0661 0.0048 0.0063 -0.0021 0.3230\n 120 0.5256 0.9669 0.4795 0.7140 0.0127 0.9820 0.0635 0.0029 0.0019 0.0005 0.3170\n 121 0.5250 0.9660 0.4803 0.7181 0.0105 0.9801 0.0643 -0.0024 0.0008 -0.0015 0.3144\n 122 0.5251 0.9690 0.4778 0.7233 0.0094 0.9789 0.0608 -0.0002 0.0003 -0.0021 0.3110\n 123 0.5278 0.9697 0.4804 0.7299 0.0099 0.9808 0.0577 -0.0039 -0.0016 -0.0034 0.3051\n 124 0.5278 0.9687 0.4840 0.7376 0.0099 0.9774 0.0536 -0.0037 0.0013 -0.0010 0.2971\n 125 0.5252 0.9712 0.4847 0.7412 0.0118 0.9801 0.0540 -0.0032 0.0037 -0.0014 0.2914\n 126 0.5276 0.9702 0.4834 0.7455 0.0146 0.9826 0.0547 -0.0051 0.0052 0.0022 0.2848\n 127 0.5290 0.9717 0.4856 0.7501 0.0118 0.9798 0.0557 -0.0033 0.0037 -0.0016 0.2780\n 128 0.5282 0.9729 0.4852 0.7516 0.0124 0.9792 0.0557 -0.0026 0.0044 -0.0032 0.2760\n 129 0.5260 0.9709 0.4882 0.7558 0.0124 0.9771 0.0583 -0.0037 0.0044 -0.0049 0.2701\n 130 0.5270 0.9734 0.4886 0.7595 0.0113 0.9790 0.0550 -0.0032 0.0028 -0.0047 0.2644\n 131 0.5281 0.9765 0.4870 0.7605 0.0106 0.9772 0.0593 -0.0024 0.0011 -0.0020 0.2591\n 132 0.5247 0.9772 0.4868 0.7643 0.0130 0.9794 0.0593 -0.0026 -0.0014 0.0009 0.2572\n 133 0.5252 0.9779 0.4875 0.7647 0.0084 0.9793 0.0621 -0.0038 0.0029 0.0010 0.2552\n 134 0.5239 0.9783 0.4885 0.7701 0.0086 0.9812 0.0608 -0.0039 0.0008 0.0032 0.2561\n 135 0.5209 0.9749 0.4852 0.7708 0.0099 0.9829 0.0556 -0.0010 0.0007 0.0029 0.2567\n 136 0.5231 0.9754 0.4852 0.7692 0.0112 0.9844 0.0540 0.0041 -0.0015 0.0010 0.2559\n 137 0.5220 0.9749 0.4858 0.7704 0.0111 0.9839 0.0556 -0.0001 0.0022 -0.0014 0.2516\n 138 0.5224 0.9775 0.4882 0.7698 0.0144 0.9854 0.0552 0.0001 0.0014 -0.0027 0.2459\n 139 0.5235 0.9796 0.4915 0.7709 0.0118 0.9852 0.0528 0.0006 -0.0012 -0.0054 0.2453\n 140 0.5250 0.9828 0.4919 0.7703 0.0099 0.9866 0.0515 -0.0036 -0.0029 -0.0067 0.2405\n 141 0.5229 0.9851 0.4914 0.7683 0.0099 0.9875 0.0525 -0.0019 -0.0034 -0.0049 0.2358\n 142 0.5235 0.9850 0.4915 0.7691 0.0071 0.9885 0.0508 -0.0006 -0.0017 -0.0026 0.2347\n 143 0.5216 0.9836 0.4919 0.7739 0.0078 0.9912 0.0548 -0.0036 -0.0007 -0.0012 0.2338\n 144 0.5204 0.9768 0.4917 0.7724 0.0089 0.9875 0.0534 -0.0071 -0.0011 -0.0009 0.2304\n 145 0.5205 0.9735 0.4889 0.7759 0.0078 0.9862 0.0521 -0.0042 -0.0050 0.0017 0.2289\n 146 0.5192 0.9777 0.4866 0.7793 0.0085 0.9875 0.0526 -0.0046 -0.0030 -0.0031 0.2290\n 147 0.5206 0.9777 0.4859 0.7787 0.0155 0.9841 0.0529 -0.0047 -0.0065 -0.0018 0.2277\n 148 0.5178 0.9770 0.4878 0.7762 0.0166 0.9844 0.0550 -0.0068 -0.0041 -0.0046 0.2303\n 149 0.5216 0.9768 0.4887 0.7730 0.0136 0.9828 0.0566 -0.0071 -0.0026 -0.0045 0.2297\n 150 0.5237 0.9734 0.4914 0.7712 0.0157 0.9866 0.0585 -0.0056 0.0014 -0.0047 0.2330\n 151 0.5226 0.9705 0.4927 0.7715 0.0166 0.9824 0.0573 -0.0038 -0.0002 -0.0044 0.2322\n 152 0.5177 0.9735 0.4895 0.7726 0.0137 0.9794 0.0564 -0.0017 0.0006 -0.0031 0.2335\n 153 0.5216 0.9711 0.4915 0.7718 0.0163 0.9836 0.0547 -0.0005 0.0004 -0.0005 0.2348\n 154 0.5215 0.9710 0.4913 0.7756 0.0169 0.9843 0.0566 0.0035 -0.0044 0.0007 0.2346\n 155 0.5169 0.9751 0.4893 0.7757 0.0146 0.9823 0.0575 0.0046 -0.0064 0.0005 0.2333\n 156 0.5152 0.9722 0.4888 0.7798 0.0136 0.9842 0.0600 0.0044 -0.0058 0.0018 0.2352\n 157 0.5177 0.9746 0.4913 0.7838 0.0133 0.9871 0.0616 0.0045 -0.0037 0.0005 0.2394\n 158 0.5177 0.9796 0.4886 0.7842 0.0107 0.9895 0.0621 0.0049 -0.0025 0.0010 0.2461\n 159 0.5121 0.9746 0.4868 0.7861 0.0122 0.9896 0.0635 0.0033 -0.0022 0.0014 0.2524\n 0 0.8011 -0.0020 -0.0005 -0.0025 -0.0006 -0.0005 0.7999 0.8025 -0.0011 0.0004 -0.0023\n 1 0.7254 -0.0040 0.0018 -0.0014 0.0009 -0.0037 0.8206 0.7303 0.0774 0.0073 0.0925\n 2 0.6563 -0.0001 0.0010 -0.0015 0.0012 -0.0051 0.8369 0.6562 0.1091 0.0186 0.1556\n 3 0.5972 0.0033 -0.0034 -0.0019 0.0058 -0.0040 0.8547 0.5993 0.1197 0.0248 0.1866\n 4 0.5387 0.0009 -0.0055 -0.0043 0.0070 -0.0026 0.8688 0.5474 0.1274 0.0289 0.2038\n 5 0.4913 -0.0005 -0.0087 -0.0068 0.0032 -0.0028 0.8842 0.4983 0.1326 0.0303 0.2164\n 6 0.4448 0.0068 -0.0113 -0.0058 0.0014 -0.0074 0.8962 0.4568 0.1390 0.0327 0.2277\n 7 0.4046 0.0077 -0.0125 -0.0061 0.0016 -0.0087 0.9053 0.4188 0.1407 0.0365 0.2330\n 8 0.3729 0.0101 -0.0114 -0.0016 -0.0019 -0.0089 0.9145 0.3851 0.1477 0.0360 0.2315\n 9 0.3390 0.0158 -0.0080 0.0004 0.0004 -0.0087 0.9220 0.3573 0.1494 0.0354 0.2327\n 10 0.3094 0.0170 -0.0073 -0.0005 0.0023 -0.0084 0.9328 0.3339 0.1502 0.0366 0.2313\n 11 0.2857 0.0235 -0.0061 0.0026 0.0028 -0.0105 0.9386 0.3113 0.1486 0.0383 0.2252\n 12 0.2658 0.0286 -0.0042 -0.0024 0.0022 -0.0090 0.9442 0.2947 0.1539 0.0371 0.2226\n 13 0.2496 0.0353 -0.0101 -0.0030 0.0044 -0.0030 0.9510 0.2759 0.1558 0.0437 0.2225\n 14 0.2329 0.0379 -0.0061 -0.0000 0.0035 -0.0011 0.9579 0.2657 0.1509 0.0434 0.2163\n 15 0.2161 0.0457 -0.0048 0.0002 0.0042 0.0031 0.9606 0.2531 0.1480 0.0467 0.2124\n 16 0.2053 0.0471 -0.0025 0.0014 0.0062 0.0016 0.9666 0.2394 0.1459 0.0494 0.2069\n 17 0.1936 0.0497 -0.0036 -0.0002 0.0088 0.0066 0.9692 0.2358 0.1440 0.0522 0.1984\n 18 0.1842 0.0547 -0.0031 0.0023 0.0051 0.0115 0.9741 0.2265 0.1497 0.0572 0.1891\n 19 0.1788 0.0579 -0.0036 0.0007 0.0036 0.0164 0.9756 0.2217 0.1537 0.0592 0.1796\n 20 0.1775 0.0596 -0.0037 0.0037 0.0061 0.0181 0.9745 0.2183 0.1601 0.0639 0.1682\n 21 0.1706 0.0646 -0.0070 0.0078 0.0064 0.0223 0.9785 0.2118 0.1681 0.0657 0.1579\n 22 0.1650 0.0673 -0.0081 0.0080 0.0055 0.0245 0.9810 0.2081 0.1738 0.0720 0.1508\n 23 0.1597 0.0743 -0.0079 0.0093 0.0086 0.0299 0.9832 0.2023 0.1820 0.0757 0.1418\n 24 0.1548 0.0776 -0.0043 0.0112 0.0107 0.0332 0.9863 0.1975 0.1899 0.0829 0.1312\n 25 0.1544 0.0805 0.0033 0.0116 0.0133 0.0383 0.9858 0.1931 0.1956 0.0886 0.1259\n 26 0.1507 0.0833 0.0069 0.0102 0.0140 0.0443 0.9919 0.1916 0.2080 0.0951 0.1203\n 27 0.1465 0.0834 0.0088 0.0136 0.0128 0.0465 0.9910 0.1897 0.2156 0.1039 0.1122\n 28 0.1417 0.0823 0.0072 0.0154 0.0157 0.0489 0.9936 0.1866 0.2312 0.1144 0.1092\n 29 0.1432 0.0863 0.0065 0.0133 0.0189 0.0510 0.9940 0.1873 0.2468 0.1232 0.1000\n 30 0.1409 0.0904 0.0065 0.0144 0.0202 0.0546 0.9975 0.1846 0.2601 0.1344 0.0916\n 31 0.1404 0.0933 0.0092 0.0141 0.0191 0.0588 0.9977 0.1864 0.2787 0.1504 0.0829\n 32 0.1336 0.0973 0.0129 0.0148 0.0191 0.0609 0.9984 0.1857 0.2982 0.1677 0.0751\n 33 0.1338 0.0959 0.0133 0.0136 0.0249 0.0649 0.9971 0.1835 0.3159 0.1889 0.0702\n 34 0.1319 0.0984 0.0178 0.0173 0.0295 0.0680 0.9936 0.1844 0.3342 0.2119 0.0644\n 35 0.1310 0.1022 0.0176 0.0187 0.0306 0.0736 0.9944 0.1818 0.3574 0.2297 0.0612\n 36 0.1287 0.1046 0.0183 0.0249 0.0328 0.0765 0.9916 0.1802 0.3742 0.2494 0.0563\n 37 0.1310 0.1101 0.0177 0.0258 0.0343 0.0820 0.9921 0.1784 0.3941 0.2735 0.0499\n 38 0.1330 0.1113 0.0163 0.0292 0.0341 0.0884 0.9929 0.1759 0.4161 0.2965 0.0487\n 39 0.1331 0.1108 0.0168 0.0301 0.0328 0.0966 0.9899 0.1746 0.4367 0.3187 0.0448\n 40 0.1325 0.1170 0.0188 0.0313 0.0335 0.1004 0.9907 0.1715 0.4538 0.3406 0.0392\n 41 0.1313 0.1186 0.0159 0.0325 0.0334 0.1083 0.9959 0.1721 0.4705 0.3639 0.0325\n 42 0.1327 0.1228 0.0169 0.0357 0.0351 0.1130 0.9935 0.1714 0.4889 0.3908 0.0292\n 43 0.1318 0.1231 0.0182 0.0402 0.0392 0.1134 0.9964 0.1687 0.5106 0.4135 0.0249\n 44 0.1324 0.1231 0.0154 0.0420 0.0405 0.1171 0.9979 0.1619 0.5261 0.4353 0.0234\n 45 0.1343 0.1249 0.0152 0.0387 0.0427 0.1220 0.9968 0.1609 0.5370 0.4564 0.0224\n 46 0.1300 0.1314 0.0128 0.0384 0.0446 0.1286 0.9946 0.1644 0.5527 0.4742 0.0189\n 47 0.1304 0.1380 0.0110 0.0411 0.0501 0.1323 0.9924 0.1626 0.5645 0.4891 0.0177\n 48 0.1319 0.1409 0.0112 0.0408 0.0522 0.1358 0.9959 0.1565 0.5794 0.5105 0.0158\n 49 0.1320 0.1423 0.0148 0.0405 0.0527 0.1439 0.9954 0.1541 0.5936 0.5271 0.0134\n 50 0.1337 0.1444 0.0149 0.0394 0.0538 0.1524 0.9932 0.1501 0.6048 0.5377 0.0165\n 51 0.1403 0.1446 0.0189 0.0433 0.0591 0.1615 0.9928 0.1409 0.6108 0.5469 0.0167\n 52 0.1458 0.1515 0.0225 0.0448 0.0618 0.1697 0.9947 0.1330 0.6093 0.5478 0.0163\n 53 0.1517 0.1589 0.0183 0.0453 0.0622 0.1766 0.9942 0.1302 0.6084 0.5494 0.0173\n 54 0.1658 0.1653 0.0184 0.0475 0.0625 0.1858 0.9959 0.1264 0.6064 0.5526 0.0146\n 55 0.1742 0.1778 0.0195 0.0495 0.0659 0.1936 0.9920 0.1210 0.6082 0.5535 0.0123\n 56 0.1822 0.1883 0.0212 0.0500 0.0709 0.2076 0.9921 0.1145 0.6020 0.5559 0.0104\n 57 0.1953 0.2046 0.0223 0.0519 0.0750 0.2202 0.9906 0.1086 0.6021 0.5551 0.0082\n 58 0.2072 0.2177 0.0236 0.0568 0.0770 0.2356 0.9866 0.1018 0.5924 0.5497 0.0087\n 59 0.2204 0.2341 0.0253 0.0591 0.0813 0.2546 0.9867 0.0959 0.5819 0.5469 0.0066\n 60 0.2384 0.2500 0.0309 0.0648 0.0905 0.2749 0.9860 0.0891 0.5676 0.5356 0.0107\n 61 0.2590 0.2732 0.0331 0.0696 0.0950 0.2949 0.9830 0.0800 0.5518 0.5183 0.0120\n 62 0.2855 0.2967 0.0355 0.0749 0.0987 0.3245 0.9796 0.0707 0.5339 0.5023 0.0119\n 63 0.3114 0.3265 0.0413 0.0799 0.1034 0.3546 0.9674 0.0636 0.5160 0.4791 0.0158\n 64 0.3361 0.3510 0.0468 0.0843 0.1122 0.3824 0.9585 0.0582 0.4911 0.4562 0.0133\n 65 0.3642 0.3795 0.0541 0.0863 0.1179 0.4124 0.9469 0.0555 0.4691 0.4329 0.0085\n 66 0.3915 0.4060 0.0620 0.0934 0.1285 0.4424 0.9331 0.0504 0.4429 0.4113 0.0060\n 67 0.4212 0.4380 0.0669 0.0992 0.1376 0.4741 0.9155 0.0448 0.4192 0.3838 0.0060\n 68 0.4505 0.4717 0.0783 0.1032 0.1436 0.5062 0.8945 0.0403 0.3923 0.3633 0.0053\n 69 0.4778 0.5056 0.0902 0.1048 0.1540 0.5419 0.8721 0.0370 0.3690 0.3413 0.0060\n 70 0.5013 0.5348 0.1019 0.1075 0.1638 0.5713 0.8471 0.0393 0.3464 0.3200 0.0068\n 71 0.5229 0.5656 0.1118 0.1113 0.1708 0.5998 0.8157 0.0334 0.3250 0.3020 0.0077\n 72 0.5374 0.5936 0.1254 0.1195 0.1780 0.6251 0.7827 0.0300 0.3022 0.2822 0.0120\n 73 0.5561 0.6225 0.1379 0.1237 0.1819 0.6522 0.7452 0.0282 0.2821 0.2621 0.0086\n 74 0.5745 0.6476 0.1486 0.1276 0.1891 0.6749 0.7064 0.0290 0.2654 0.2415 0.0102\n 75 0.5860 0.6673 0.1637 0.1308 0.1967 0.7012 0.6700 0.0286 0.2482 0.2281 0.0101\n 76 0.5977 0.6887 0.1796 0.1275 0.2039 0.7211 0.6309 0.0275 0.2277 0.2135 0.0094\n 77 0.6047 0.7132 0.1913 0.1240 0.2066 0.7383 0.5963 0.0271 0.2098 0.1965 0.0095\n 78 0.6062 0.7335 0.2059 0.1224 0.2078 0.7560 0.5624 0.0255 0.1964 0.1884 0.0105\n 79 0.6109 0.7530 0.2235 0.1244 0.2128 0.7755 0.5265 0.0242 0.1815 0.1737 0.0091\n 80 0.6146 0.7702 0.2400 0.1220 0.2214 0.7929 0.4944 0.0226 0.1658 0.1568 0.0106\n 81 0.6133 0.7925 0.2536 0.1253 0.2293 0.8087 0.4595 0.0204 0.1536 0.1447 0.0145\n 82 0.6132 0.8084 0.2681 0.1247 0.2353 0.8239 0.4301 0.0179 0.1474 0.1325 0.0138\n 83 0.6123 0.8224 0.2820 0.1242 0.2394 0.8347 0.4021 0.0184 0.1329 0.1249 0.0150\n 84 0.6145 0.8367 0.2948 0.1239 0.2436 0.8485 0.3738 0.0174 0.1194 0.1171 0.0159\n 85 0.6145 0.8518 0.3048 0.1233 0.2476 0.8616 0.3481 0.0153 0.1079 0.1057 0.0195\n 86 0.6105 0.8596 0.3164 0.1170 0.2511 0.8726 0.3232 0.0189 0.0982 0.0987 0.0252\n 87 0.6082 0.8679 0.3239 0.1148 0.2573 0.8837 0.3023 0.0140 0.0918 0.0916 0.0231\n 88 0.6029 0.8785 0.3343 0.1136 0.2647 0.8909 0.2793 0.0152 0.0855 0.0827 0.0240\n 89 0.5972 0.8892 0.3419 0.1132 0.2733 0.8962 0.2563 0.0143 0.0805 0.0739 0.0212\n 90 0.5930 0.8977 0.3520 0.1102 0.2835 0.9015 0.2454 0.0153 0.0723 0.0667 0.0227\n 91 0.5904 0.9027 0.3626 0.1064 0.2885 0.9040 0.2300 0.0175 0.0702 0.0563 0.0269\n 92 0.5854 0.9079 0.3738 0.1042 0.2973 0.9088 0.2161 0.0166 0.0677 0.0502 0.0335\n 93 0.5791 0.9123 0.3815 0.0991 0.3081 0.9134 0.2014 0.0150 0.0644 0.0448 0.0424\n 94 0.5734 0.9196 0.3842 0.0967 0.3144 0.9150 0.1886 0.0161 0.0571 0.0446 0.0510\n 95 0.5720 0.9242 0.3939 0.0955 0.3255 0.9216 0.1806 0.0166 0.0550 0.0433 0.0613\n 96 0.5693 0.9266 0.3993 0.0931 0.3340 0.9285 0.1703 0.0191 0.0493 0.0434 0.0690\n 97 0.5637 0.9325 0.4048 0.0911 0.3437 0.9311 0.1634 0.0172 0.0440 0.0363 0.0776\n 98 0.5640 0.9374 0.4062 0.0882 0.3511 0.9336 0.1551 0.0163 0.0359 0.0336 0.0900\n 99 0.5573 0.9415 0.4106 0.0880 0.3570 0.9397 0.1488 0.0160 0.0337 0.0302 0.1021\n 100 0.5567 0.9418 0.4160 0.0857 0.3680 0.9424 0.1414 0.0140 0.0342 0.0283 0.1116\n 101 0.5563 0.9457 0.4248 0.0818 0.3785 0.9429 0.1325 0.0116 0.0287 0.0268 0.1233\n 102 0.5556 0.9516 0.4335 0.0794 0.3867 0.9433 0.1265 0.0125 0.0311 0.0262 0.1334\n 103 0.5563 0.9528 0.4388 0.0723 0.3959 0.9424 0.1186 0.0110 0.0295 0.0263 0.1412\n 104 0.5528 0.9601 0.4433 0.0700 0.4111 0.9493 0.1117 0.0067 0.0232 0.0254 0.1458\n 105 0.5508 0.9611 0.4462 0.0657 0.4208 0.9500 0.1056 0.0076 0.0220 0.0250 0.1501\n 106 0.5461 0.9659 0.4464 0.0589 0.4324 0.9558 0.1045 0.0097 0.0207 0.0235 0.1582\n 107 0.5407 0.9671 0.4543 0.0574 0.4467 0.9570 0.0989 0.0075 0.0153 0.0199 0.1668\n 108 0.5386 0.9722 0.4552 0.0545 0.4626 0.9565 0.0942 0.0052 0.0142 0.0222 0.1756\n 109 0.5342 0.9736 0.4578 0.0511 0.4796 0.9571 0.0923 0.0042 0.0114 0.0231 0.1771\n 110 0.5322 0.9778 0.4610 0.0522 0.4957 0.9609 0.0895 0.0053 0.0083 0.0188 0.1760\n 111 0.5300 0.9788 0.4647 0.0483 0.5048 0.9638 0.0876 0.0070 0.0057 0.0175 0.1814\n 112 0.5297 0.9799 0.4645 0.0448 0.5176 0.9651 0.0842 0.0090 0.0092 0.0147 0.1867\n 113 0.5276 0.9806 0.4622 0.0438 0.5283 0.9657 0.0863 0.0104 0.0090 0.0162 0.1956\n 114 0.5225 0.9840 0.4665 0.0443 0.5430 0.9695 0.0836 0.0095 0.0089 0.0148 0.1990\n 115 0.5211 0.9893 0.4701 0.0392 0.5553 0.9722 0.0823 0.0105 0.0054 0.0160 0.2040\n 116 0.5193 0.9865 0.4723 0.0388 0.5671 0.9713 0.0817 0.0090 0.0065 0.0148 0.2036\n 117 0.5193 0.9890 0.4766 0.0380 0.5799 0.9712 0.0767 0.0101 0.0086 0.0167 0.2095\n 118 0.5183 0.9896 0.4782 0.0351 0.5924 0.9741 0.0725 0.0099 0.0075 0.0155 0.2106\n 119 0.5161 0.9925 0.4793 0.0352 0.6065 0.9739 0.0703 0.0086 0.0039 0.0138 0.2103\n 120 0.5121 0.9907 0.4835 0.0379 0.6172 0.9718 0.0726 0.0059 0.0044 0.0106 0.2170\n 121 0.5095 0.9902 0.4881 0.0357 0.6224 0.9717 0.0693 0.0079 0.0015 0.0098 0.2182\n 122 0.5113 0.9913 0.4882 0.0269 0.6319 0.9723 0.0666 0.0065 0.0035 0.0069 0.2238\n 123 0.5080 0.9950 0.4878 0.0221 0.6414 0.9749 0.0679 0.0083 0.0038 0.0060 0.2287\n 124 0.5101 0.9940 0.4849 0.0224 0.6498 0.9791 0.0648 0.0056 0.0015 0.0086 0.2334\n 125 0.5097 0.9976 0.4885 0.0212 0.6592 0.9789 0.0648 0.0103 0.0005 0.0095 0.2318\n 126 0.5057 0.9954 0.4880 0.0220 0.6733 0.9736 0.0656 0.0104 -0.0017 0.0097 0.2330\n 127 0.5044 0.9930 0.4898 0.0208 0.6828 0.9782 0.0685 0.0081 -0.0008 0.0078 0.2322\n 128 0.5033 0.9910 0.4898 0.0212 0.6892 0.9773 0.0693 0.0104 -0.0011 0.0051 0.2389\n 129 0.5076 0.9891 0.4886 0.0177 0.6959 0.9761 0.0666 0.0102 -0.0013 0.0043 0.2472\n 130 0.5089 0.9918 0.4873 0.0151 0.7038 0.9751 0.0669 0.0107 -0.0000 0.0060 0.2537\n 131 0.5096 0.9946 0.4864 0.0158 0.7087 0.9759 0.0662 0.0090 0.0014 0.0074 0.2531\n 132 0.5150 0.9942 0.4861 0.0175 0.7135 0.9762 0.0645 0.0068 0.0024 0.0066 0.2531\n 133 0.5124 0.9952 0.4907 0.0158 0.7191 0.9779 0.0661 0.0073 0.0016 0.0050 0.2577\n 134 0.5115 0.9897 0.4923 0.0148 0.7236 0.9791 0.0627 0.0048 -0.0000 0.0086 0.2573\n 135 0.5152 0.9910 0.4919 0.0172 0.7278 0.9808 0.0647 0.0041 -0.0010 0.0081 0.2586\n 136 0.5129 0.9949 0.4913 0.0154 0.7304 0.9836 0.0635 0.0043 0.0001 0.0086 0.2590\n 137 0.5117 0.9940 0.4897 0.0154 0.7370 0.9834 0.0637 0.0065 -0.0014 0.0064 0.2578\n 138 0.5111 0.9947 0.4906 0.0105 0.7411 0.9822 0.0628 0.0071 -0.0002 0.0062 0.2585\n 139 0.5059 0.9909 0.4917 0.0096 0.7463 0.9849 0.0639 0.0074 0.0020 0.0049 0.2570\n 140 0.5058 0.9928 0.4974 0.0142 0.7496 0.9908 0.0642 0.0050 0.0001 0.0020 0.2554\n 141 0.5055 0.9967 0.4981 0.0137 0.7509 0.9943 0.0608 0.0054 0.0008 -0.0023 0.2575\n 142 0.5062 0.9998 0.4972 0.0122 0.7577 0.9924 0.0588 0.0051 -0.0021 -0.0018 0.2588\n 143 0.5055 0.9987 0.5011 0.0120 0.7635 0.9931 0.0608 0.0049 -0.0036 0.0009 0.2559\n 144 0.5046 0.9983 0.5047 0.0113 0.7656 0.9929 0.0602 0.0035 -0.0039 0.0005 0.2540\n 145 0.5054 0.9974 0.5035 0.0138 0.7677 0.9931 0.0571 0.0030 -0.0067 0.0031 0.2547\n 146 0.5028 0.9979 0.5043 0.0154 0.7718 0.9952 0.0581 0.0027 -0.0047 0.0012 0.2523\n 147 0.4982 0.9966 0.5033 0.0169 0.7733 0.9911 0.0558 0.0015 -0.0032 -0.0016 0.2493\n 148 0.4979 0.9980 0.5038 0.0172 0.7720 0.9973 0.0562 0.0014 -0.0000 0.0046 0.2453\n 149 0.4959 0.9982 0.5009 0.0138 0.7731 1.0009 0.0557 0.0022 -0.0017 0.0030 0.2397\n 150 0.4973 1.0001 0.5020 0.0150 0.7747 1.0013 0.0581 0.0062 0.0021 0.0013 0.2369\n 151 0.4971 1.0003 0.5028 0.0184 0.7751 1.0012 0.0570 0.0021 0.0018 0.0026 0.2399\n 152 0.4955 1.0007 0.5024 0.0157 0.7759 0.9991 0.0558 -0.0012 -0.0017 0.0022 0.2361\n 153 0.4950 1.0031 0.5029 0.0165 0.7745 1.0004 0.0555 -0.0029 -0.0040 0.0037 0.2348\n 154 0.4970 1.0024 0.5041 0.0162 0.7801 0.9984 0.0566 -0.0053 -0.0010 0.0050 0.2328\n 155 0.4966 1.0016 0.5048 0.0152 0.7782 0.9955 0.0560 -0.0054 -0.0012 0.0016 0.2293\n 156 0.4905 1.0031 0.5033 0.0147 0.7761 0.9929 0.0565 -0.0067 0.0007 -0.0002 0.2303\n 157 0.4941 0.9997 0.5060 0.0156 0.7775 0.9950 0.0571 -0.0037 0.0015 0.0028 0.2296\n 158 0.4966 1.0008 0.5079 0.0102 0.7777 0.9905 0.0555 -0.0021 0.0005 0.0008 0.2281\n 159 0.4995 1.0036 0.5071 0.0104 0.7802 0.9926 0.0551 -0.0024 -0.0018 0.0001 0.2324\n 0 0.7991 0.0030 0.0015 -0.0011 -0.0013 0.0001 0.7999 0.7997 0.0009 0.0005 -0.0028\n 1 0.7229 0.0061 0.0023 -0.0015 0.0024 0.0020 0.8221 0.7238 0.0784 0.0061 0.0974\n 2 0.6557 0.0089 0.0049 0.0003 0.0022 -0.0008 0.8427 0.6586 0.1064 0.0161 0.1634\n 3 0.5938 0.0120 0.0014 0.0018 0.0008 -0.0021 0.8570 0.5991 0.1222 0.0221 0.2027\n 4 0.5360 0.0112 -0.0008 -0.0026 0.0021 -0.0061 0.8665 0.5443 0.1258 0.0223 0.2254\n 5 0.4843 0.0130 -0.0076 -0.0001 0.0046 -0.0032 0.8797 0.4969 0.1297 0.0243 0.2431\n 6 0.4403 0.0182 -0.0042 0.0007 0.0029 -0.0001 0.8905 0.4516 0.1316 0.0266 0.2552\n 7 0.4012 0.0225 -0.0060 -0.0002 0.0048 -0.0006 0.8955 0.4120 0.1313 0.0289 0.2700\n 8 0.3662 0.0246 -0.0032 0.0014 0.0031 -0.0015 0.9050 0.3781 0.1262 0.0266 0.2734\n 9 0.3346 0.0276 -0.0015 0.0034 0.0043 0.0017 0.9119 0.3458 0.1230 0.0277 0.2782\n 10 0.3092 0.0298 0.0008 0.0041 0.0034 0.0071 0.9238 0.3190 0.1199 0.0296 0.2785\n 11 0.2840 0.0318 0.0012 0.0005 0.0041 0.0089 0.9317 0.2980 0.1211 0.0364 0.2791\n 12 0.2640 0.0345 0.0007 0.0007 0.0037 0.0114 0.9383 0.2782 0.1189 0.0347 0.2723\n 13 0.2441 0.0372 -0.0025 -0.0012 0.0053 0.0130 0.9438 0.2680 0.1190 0.0344 0.2707\n 14 0.2253 0.0436 -0.0024 0.0021 0.0091 0.0133 0.9463 0.2548 0.1152 0.0334 0.2700\n 15 0.2107 0.0468 -0.0024 0.0076 0.0081 0.0161 0.9484 0.2442 0.1101 0.0311 0.2690\n 16 0.1984 0.0538 -0.0006 0.0091 0.0087 0.0199 0.9568 0.2342 0.1098 0.0317 0.2727\n 17 0.1857 0.0587 -0.0003 0.0087 0.0084 0.0221 0.9613 0.2272 0.1058 0.0298 0.2760\n 18 0.1770 0.0645 0.0011 0.0090 0.0060 0.0222 0.9653 0.2208 0.1063 0.0291 0.2785\n 19 0.1664 0.0673 0.0020 0.0078 0.0094 0.0278 0.9689 0.2125 0.1041 0.0294 0.2794\n 20 0.1590 0.0738 0.0015 0.0092 0.0068 0.0289 0.9700 0.2054 0.1042 0.0299 0.2807\n 21 0.1509 0.0722 0.0020 0.0078 0.0038 0.0353 0.9718 0.2007 0.0995 0.0284 0.2862\n 22 0.1492 0.0714 0.0025 0.0140 0.0066 0.0381 0.9720 0.1993 0.0989 0.0286 0.2911\n 23 0.1445 0.0706 0.0049 0.0148 0.0107 0.0411 0.9767 0.1999 0.0957 0.0306 0.2930\n 24 0.1402 0.0730 0.0063 0.0143 0.0122 0.0407 0.9761 0.2024 0.0999 0.0315 0.2912\n 25 0.1389 0.0725 0.0069 0.0163 0.0140 0.0406 0.9781 0.2060 0.0972 0.0301 0.2917\n 26 0.1364 0.0761 0.0068 0.0130 0.0147 0.0441 0.9750 0.2033 0.0948 0.0285 0.2932\n 27 0.1363 0.0786 0.0082 0.0151 0.0153 0.0463 0.9812 0.2048 0.0921 0.0291 0.2963\n 28 0.1339 0.0796 0.0069 0.0145 0.0171 0.0539 0.9840 0.2069 0.0884 0.0310 0.2958\n 29 0.1305 0.0847 0.0085 0.0156 0.0150 0.0565 0.9853 0.2108 0.0883 0.0348 0.2913\n 30 0.1276 0.0846 0.0087 0.0156 0.0124 0.0569 0.9906 0.2103 0.0888 0.0282 0.2906\n 31 0.1238 0.0827 0.0080 0.0176 0.0158 0.0603 0.9893 0.2130 0.0886 0.0267 0.2899\n 32 0.1210 0.0770 0.0100 0.0174 0.0171 0.0605 0.9918 0.2194 0.0885 0.0290 0.2966\n 33 0.1187 0.0773 0.0107 0.0185 0.0164 0.0598 0.9913 0.2219 0.0878 0.0299 0.2996\n 34 0.1158 0.0758 0.0095 0.0171 0.0140 0.0573 0.9951 0.2252 0.0877 0.0324 0.3007\n 35 0.1131 0.0785 0.0121 0.0212 0.0153 0.0587 0.9946 0.2275 0.0886 0.0303 0.3019\n 36 0.1122 0.0777 0.0109 0.0218 0.0127 0.0596 0.9941 0.2355 0.0893 0.0283 0.3032\n 37 0.1072 0.0748 0.0119 0.0230 0.0139 0.0585 0.9989 0.2410 0.0910 0.0281 0.3086\n 38 0.1028 0.0746 0.0094 0.0226 0.0179 0.0600 1.0018 0.2489 0.0894 0.0280 0.3093\n 39 0.0990 0.0747 0.0085 0.0216 0.0202 0.0572 1.0025 0.2566 0.0881 0.0277 0.3119\n 40 0.0972 0.0720 0.0094 0.0201 0.0221 0.0575 1.0000 0.2602 0.0866 0.0225 0.3188\n 41 0.0910 0.0694 0.0116 0.0167 0.0212 0.0590 0.9977 0.2646 0.0859 0.0206 0.3350\n 42 0.0881 0.0635 0.0127 0.0144 0.0243 0.0575 0.9993 0.2765 0.0842 0.0192 0.3470\n 43 0.0777 0.0648 0.0107 0.0142 0.0227 0.0554 1.0009 0.2916 0.0783 0.0182 0.3636\n 44 0.0751 0.0614 0.0109 0.0140 0.0234 0.0511 0.9961 0.3049 0.0730 0.0199 0.3763\n 45 0.0672 0.0573 0.0062 0.0126 0.0243 0.0472 0.9971 0.3239 0.0715 0.0206 0.3870\n 46 0.0605 0.0557 0.0099 0.0120 0.0209 0.0449 0.9990 0.3487 0.0693 0.0166 0.3983\n 47 0.0558 0.0487 0.0070 0.0078 0.0213 0.0421 1.0022 0.3729 0.0685 0.0143 0.4193\n 48 0.0541 0.0413 0.0061 0.0081 0.0214 0.0373 1.0017 0.3980 0.0627 0.0150 0.4442\n 49 0.0473 0.0383 0.0069 0.0082 0.0187 0.0349 1.0021 0.4264 0.0628 0.0124 0.4633\n 50 0.0406 0.0333 0.0086 0.0041 0.0166 0.0321 1.0035 0.4560 0.0572 0.0091 0.4903\n 51 0.0375 0.0293 0.0067 0.0051 0.0111 0.0277 1.0031 0.4869 0.0538 0.0077 0.5204\n 52 0.0345 0.0260 0.0056 0.0032 0.0092 0.0229 1.0014 0.5207 0.0548 0.0092 0.5525\n 53 0.0355 0.0241 0.0018 0.0041 0.0093 0.0204 0.9994 0.5490 0.0520 0.0077 0.5806\n 54 0.0338 0.0206 0.0042 0.0010 0.0103 0.0208 0.9973 0.5815 0.0465 0.0034 0.6118\n 55 0.0295 0.0149 0.0014 0.0002 0.0086 0.0173 0.9992 0.6120 0.0456 0.0037 0.6484\n 56 0.0271 0.0161 -0.0009 0.0042 0.0087 0.0183 0.9966 0.6404 0.0462 0.0017 0.6776\n 57 0.0234 0.0168 -0.0031 0.0043 0.0105 0.0146 0.9972 0.6661 0.0422 0.0024 0.7058\n 58 0.0197 0.0159 -0.0013 0.0039 0.0117 0.0134 0.9977 0.6918 0.0429 0.0049 0.7283\n 59 0.0190 0.0129 -0.0024 0.0057 0.0121 0.0116 0.9980 0.7171 0.0406 0.0068 0.7472\n 60 0.0188 0.0127 -0.0019 0.0055 0.0136 0.0106 0.9989 0.7410 0.0410 0.0035 0.7613\n 61 0.0136 0.0091 -0.0000 0.0055 0.0110 0.0114 1.0007 0.7634 0.0377 0.0030 0.7768\n 62 0.0130 0.0047 0.0062 0.0049 0.0092 0.0106 1.0040 0.7838 0.0360 0.0054 0.7941\n 63 0.0121 0.0046 0.0084 0.0015 0.0085 0.0119 1.0038 0.8002 0.0354 0.0076 0.8068\n 64 0.0067 0.0030 0.0063 0.0018 0.0057 0.0093 1.0053 0.8150 0.0358 0.0093 0.8085\n 65 0.0036 0.0040 0.0098 0.0048 0.0029 0.0103 1.0043 0.8335 0.0347 0.0092 0.8087\n 66 0.0022 0.0014 0.0081 0.0054 0.0016 0.0070 1.0051 0.8450 0.0368 0.0106 0.8054\n 67 0.0035 0.0008 0.0072 0.0045 -0.0017 0.0118 1.0106 0.8645 0.0355 0.0123 0.7970\n 68 0.0019 -0.0007 0.0076 0.0025 -0.0015 0.0157 1.0082 0.8786 0.0353 0.0115 0.7900\n 69 -0.0005 0.0007 0.0056 0.0028 -0.0028 0.0182 1.0054 0.8891 0.0327 0.0156 0.7795\n 70 0.0001 0.0043 0.0040 0.0038 -0.0027 0.0153 1.0046 0.9017 0.0336 0.0166 0.7638\n 71 0.0010 0.0058 0.0050 0.0017 0.0019 0.0136 1.0057 0.9121 0.0304 0.0145 0.7487\n 72 -0.0005 0.0014 0.0029 0.0017 0.0003 0.0109 1.0029 0.9236 0.0279 0.0121 0.7384\n 73 -0.0007 0.0043 -0.0017 0.0010 0.0004 0.0126 1.0038 0.9318 0.0295 0.0122 0.7401\n 74 0.0017 0.0049 -0.0038 0.0020 -0.0004 0.0123 1.0060 0.9391 0.0310 0.0104 0.7420\n 75 0.0020 0.0058 -0.0027 0.0012 -0.0029 0.0137 1.0065 0.9440 0.0288 0.0112 0.7460\n 76 0.0056 0.0060 -0.0008 -0.0000 -0.0048 0.0128 1.0063 0.9488 0.0278 0.0064 0.7508\n 77 0.0010 0.0034 0.0003 -0.0009 -0.0041 0.0157 1.0050 0.9543 0.0263 0.0042 0.7670\n 78 -0.0035 0.0037 -0.0005 -0.0013 -0.0014 0.0136 1.0037 0.9554 0.0244 0.0030 0.7852\n 79 -0.0036 0.0011 -0.0017 0.0015 -0.0033 0.0083 1.0048 0.9596 0.0232 0.0023 0.8049\n 80 -0.0045 0.0013 -0.0018 0.0058 -0.0019 0.0051 1.0035 0.9609 0.0182 0.0031 0.8223\n 81 -0.0054 0.0006 0.0006 0.0042 0.0025 0.0081 1.0022 0.9671 0.0173 0.0013 0.8407\n 82 -0.0044 -0.0017 0.0009 0.0039 0.0020 0.0053 1.0022 0.9687 0.0191 0.0033 0.8540\n 83 -0.0062 -0.0005 0.0028 0.0042 -0.0001 0.0053 1.0035 0.9725 0.0186 0.0011 0.8643\n 84 -0.0032 -0.0016 0.0028 0.0041 -0.0030 0.0062 1.0058 0.9703 0.0221 0.0036 0.8738\n 85 -0.0008 -0.0045 0.0032 0.0015 -0.0052 0.0076 1.0098 0.9704 0.0200 0.0038 0.8789\n 86 -0.0009 -0.0052 0.0032 0.0030 -0.0049 0.0053 1.0059 0.9697 0.0174 0.0017 0.8827\n 87 -0.0004 -0.0044 0.0010 0.0014 -0.0041 0.0082 1.0084 0.9742 0.0192 0.0018 0.8915\n 88 0.0004 -0.0036 -0.0013 -0.0013 -0.0069 0.0088 1.0077 0.9741 0.0203 -0.0020 0.9007\n 89 0.0016 -0.0057 -0.0028 -0.0019 -0.0092 0.0088 1.0056 0.9772 0.0227 -0.0029 0.9079\n 90 0.0043 -0.0044 -0.0029 -0.0042 -0.0110 0.0089 1.0077 0.9774 0.0187 -0.0027 0.9122\n 91 0.0029 -0.0048 -0.0007 -0.0074 -0.0067 0.0103 1.0058 0.9793 0.0164 -0.0001 0.9190\n 92 0.0046 -0.0024 0.0024 -0.0066 -0.0074 0.0084 1.0077 0.9780 0.0156 0.0033 0.9253\n 93 0.0057 0.0025 0.0060 -0.0083 -0.0069 0.0096 1.0043 0.9815 0.0139 -0.0000 0.9315\n 94 0.0058 -0.0008 0.0053 -0.0048 -0.0079 0.0051 1.0027 0.9819 0.0128 -0.0021 0.9381\n 95 0.0049 0.0001 0.0076 -0.0053 -0.0099 0.0030 1.0040 0.9834 0.0174 0.0006 0.9442\n 96 0.0026 0.0016 0.0087 -0.0013 -0.0097 -0.0002 1.0010 0.9825 0.0152 -0.0037 0.9445\n 97 0.0010 -0.0015 0.0051 -0.0056 -0.0109 -0.0001 0.9991 0.9858 0.0147 0.0001 0.9469\n 98 0.0017 -0.0009 0.0059 -0.0041 -0.0127 0.0027 1.0012 0.9888 0.0157 0.0004 0.9482\n 99 0.0033 0.0025 0.0038 -0.0023 -0.0113 0.0005 0.9991 0.9893 0.0158 -0.0009 0.9526\n 100 0.0022 -0.0025 0.0025 -0.0013 -0.0115 -0.0025 1.0013 0.9851 0.0158 -0.0003 0.9539\n 101 0.0027 -0.0007 0.0033 -0.0021 -0.0107 -0.0073 0.9994 0.9876 0.0167 0.0040 0.9537\n 102 0.0010 -0.0020 0.0027 -0.0006 -0.0094 -0.0082 0.9983 0.9895 0.0161 0.0061 0.9528\n 103 -0.0018 0.0012 0.0018 0.0010 -0.0121 -0.0074 0.9983 0.9941 0.0151 0.0051 0.9486\n 104 -0.0016 -0.0011 0.0026 -0.0014 -0.0084 -0.0067 1.0020 0.9956 0.0121 0.0063 0.9434\n 105 -0.0007 -0.0027 -0.0022 -0.0006 -0.0084 -0.0050 1.0045 1.0011 0.0128 0.0055 0.9396\n 106 -0.0002 -0.0030 -0.0008 -0.0023 -0.0017 -0.0051 1.0037 1.0037 0.0119 0.0047 0.9403\n 107 -0.0000 -0.0031 -0.0037 -0.0025 -0.0061 -0.0018 1.0033 1.0054 0.0141 0.0054 0.9383\n 108 0.0010 -0.0060 -0.0048 -0.0004 -0.0048 -0.0024 1.0029 1.0067 0.0164 0.0070 0.9398\n 109 0.0010 -0.0050 -0.0075 -0.0036 -0.0073 0.0003 1.0010 1.0062 0.0178 0.0050 0.9357\n 110 0.0013 -0.0048 -0.0074 -0.0052 -0.0027 -0.0005 0.9988 1.0052 0.0160 0.0050 0.9303\n 111 0.0031 -0.0044 -0.0053 0.0004 -0.0020 0.0006 0.9990 1.0037 0.0170 0.0060 0.9304\n 112 0.0040 -0.0049 -0.0044 0.0020 -0.0009 -0.0012 1.0010 1.0015 0.0203 0.0063 0.9264\n 113 0.0024 -0.0035 -0.0028 0.0016 -0.0004 -0.0013 1.0027 1.0006 0.0214 0.0027 0.9278\n 114 0.0036 -0.0006 -0.0018 0.0002 -0.0010 0.0011 1.0038 1.0032 0.0185 0.0014 0.9326\n 115 0.0036 -0.0017 -0.0026 0.0049 -0.0051 0.0024 1.0048 1.0039 0.0179 -0.0011 0.9367\n 116 0.0036 -0.0032 -0.0034 0.0032 -0.0023 0.0036 1.0062 1.0058 0.0195 -0.0015 0.9401\n 117 0.0018 -0.0049 0.0001 0.0010 0.0000 0.0044 1.0066 1.0055 0.0204 -0.0021 0.9432\n 118 -0.0000 -0.0055 -0.0014 -0.0013 -0.0025 0.0020 1.0021 1.0051 0.0189 -0.0044 0.9453\n 119 -0.0035 -0.0040 -0.0032 -0.0007 -0.0018 -0.0000 1.0024 1.0047 0.0187 -0.0066 0.9389\n 120 -0.0071 -0.0056 -0.0025 0.0002 -0.0020 -0.0040 1.0032 1.0047 0.0155 -0.0102 0.9278\n 121 -0.0053 -0.0073 -0.0031 0.0033 0.0006 -0.0011 1.0012 1.0025 0.0137 -0.0045 0.9215\n 122 -0.0010 -0.0068 -0.0031 0.0045 -0.0015 -0.0032 0.9998 1.0044 0.0113 -0.0052 0.9236\n 123 -0.0017 -0.0023 -0.0046 0.0052 0.0033 -0.0023 0.9987 1.0078 0.0105 -0.0070 0.9217\n 124 -0.0025 0.0006 -0.0001 0.0069 0.0048 -0.0008 0.9951 1.0050 0.0144 -0.0072 0.9174\n 125 -0.0014 0.0029 -0.0026 0.0083 0.0030 0.0000 0.9967 1.0052 0.0130 -0.0051 0.9144\n 126 0.0001 0.0038 -0.0015 0.0065 0.0026 0.0015 0.9945 1.0036 0.0144 -0.0030 0.9207\n 127 0.0019 0.0020 0.0014 0.0059 0.0048 0.0008 0.9966 1.0070 0.0151 -0.0008 0.9236\n 128 0.0013 0.0018 -0.0016 0.0077 0.0045 -0.0000 0.9966 1.0074 0.0135 -0.0022 0.9318\n 129 0.0014 -0.0000 -0.0017 0.0060 0.0053 0.0002 0.9890 1.0061 0.0140 -0.0034 0.9351\n 130 0.0012 0.0023 0.0008 0.0046 0.0053 0.0032 0.9867 1.0054 0.0131 -0.0067 0.9380\n 131 0.0045 0.0011 0.0021 0.0024 0.0070 0.0006 0.9872 1.0040 0.0125 -0.0076 0.9325\n 132 0.0077 0.0017 0.0034 0.0055 0.0049 -0.0010 0.9914 1.0053 0.0113 -0.0099 0.9200\n 133 0.0061 -0.0006 0.0036 0.0096 0.0063 -0.0023 0.9914 1.0067 0.0126 -0.0062 0.9187\n 134 0.0046 -0.0029 0.0047 0.0083 0.0047 -0.0058 0.9915 1.0034 0.0144 -0.0047 0.9198\n 135 0.0025 -0.0020 0.0026 0.0064 0.0047 -0.0078 0.9926 1.0030 0.0153 -0.0054 0.9198\n 136 -0.0023 -0.0027 0.0013 0.0059 0.0014 -0.0077 0.9938 1.0012 0.0143 -0.0046 0.9246\n 137 0.0004 -0.0045 0.0011 0.0046 0.0035 -0.0082 0.9927 1.0007 0.0133 -0.0049 0.9222\n 138 -0.0006 -0.0044 0.0010 0.0071 0.0049 -0.0112 0.9953 1.0034 0.0123 -0.0061 0.9246\n 139 -0.0025 -0.0051 -0.0003 0.0076 0.0030 -0.0067 0.9932 1.0029 0.0140 -0.0055 0.9267\n 140 -0.0038 -0.0068 0.0007 0.0066 -0.0003 -0.0066 0.9921 1.0051 0.0130 -0.0046 0.9243\n 141 -0.0023 -0.0119 0.0046 0.0063 0.0037 -0.0059 0.9913 1.0040 0.0146 -0.0022 0.9326\n 142 -0.0034 -0.0117 0.0020 0.0028 0.0030 -0.0026 0.9923 1.0021 0.0158 -0.0021 0.9364\n 143 -0.0047 -0.0134 0.0031 0.0018 0.0046 -0.0006 0.9908 0.9996 0.0136 -0.0015 0.9376\n 144 -0.0035 -0.0163 0.0017 0.0016 0.0035 -0.0008 0.9938 0.9976 0.0130 0.0009 0.9446\n 145 -0.0020 -0.0149 -0.0009 0.0021 0.0013 -0.0018 0.9938 0.9941 0.0150 -0.0055 0.9497\n 146 -0.0036 -0.0151 0.0024 -0.0013 0.0012 0.0004 0.9976 0.9917 0.0123 -0.0032 0.9441\n 147 -0.0032 -0.0116 0.0010 0.0008 -0.0024 0.0006 1.0010 0.9934 0.0108 -0.0065 0.9492\n 148 -0.0023 -0.0083 0.0027 0.0005 0.0006 -0.0014 1.0014 0.9927 0.0136 -0.0067 0.9400\n 149 -0.0010 -0.0060 0.0022 0.0034 -0.0011 -0.0018 0.9997 0.9933 0.0132 -0.0049 0.9404\n 150 -0.0024 -0.0073 -0.0011 0.0036 -0.0024 -0.0008 0.9975 0.9958 0.0165 -0.0064 0.9398\n 151 0.0007 -0.0051 0.0008 0.0036 -0.0030 -0.0012 1.0006 0.9932 0.0161 -0.0076 0.9372\n 152 -0.0008 -0.0076 0.0025 0.0028 -0.0048 -0.0003 1.0016 0.9920 0.0150 -0.0095 0.9325\n 153 -0.0029 -0.0045 0.0027 0.0051 -0.0046 -0.0039 0.9989 0.9913 0.0177 -0.0087 0.9232\n 154 0.0002 -0.0020 0.0061 0.0048 -0.0043 -0.0016 0.9989 0.9911 0.0152 -0.0072 0.9160\n 155 -0.0010 -0.0020 0.0015 0.0029 -0.0055 -0.0004 0.9990 0.9915 0.0107 -0.0076 0.9119\n 156 0.0028 -0.0030 0.0009 0.0049 -0.0073 0.0020 1.0000 0.9946 0.0127 -0.0082 0.9095\n 157 0.0030 -0.0035 0.0008 0.0052 -0.0066 0.0001 1.0014 0.9977 0.0125 -0.0046 0.9069\n 158 0.0063 -0.0023 -0.0005 0.0081 -0.0046 -0.0006 0.9983 0.9970 0.0109 -0.0030 0.9118\n 159 0.0049 0.0002 -0.0013 0.0054 -0.0036 0.0009 1.0011 0.9967 0.0066 -0.0033 0.9177\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\nfrom urllib.error import HTTPError\nfrom urllib.request import urlopen\nfrom zipfile import ZipFile\n\nimport anndata\nimport numpy as np\nimport pandas as pd\nfrom scipy import sparse\n\nfrom .. import logging as logg\nfrom .._settings import settings\nfrom .._utils._doctests import doctest_internet\nfrom ..readwrite import _download\nfrom ._utils import check_datasetdir_exists\n\nif TYPE_CHECKING:\n from typing import BinaryIO\n\n\ndef _filter_boring(dataframe: pd.DataFrame) -> pd.DataFrame:\n unique_vals = dataframe.apply(lambda x: len(x.unique()))\n is_boring = (unique_vals == 1) | (unique_vals == len(dataframe))\n return dataframe.loc[:, ~is_boring]\n\n\ndef sniff_url(accession: str):\n # Note that data is downloaded from gxa/sc/experiment, not experiments\n base_url = f\"https://www.ebi.ac.uk/gxa/sc/experiments/{accession}/\"\n try:\n with urlopen(base_url): # Check if server up/ dataset exists\n pass\n except HTTPError as e:\n e.msg = f\"{e.msg} ({base_url})\" # Report failed url\n raise\n\n\n@check_datasetdir_exists\ndef download_experiment(accession: str):\n sniff_url(accession)\n\n base_url = f\"https://www.ebi.ac.uk/gxa/sc/experiment/{accession}\"\n design_url = f\"{base_url}/download?accessKey=&fileType=\"\n mtx_url = f\"{base_url}/download/zip?accessKey=&fileType=\"\n\n experiment_dir = settings.datasetdir / accession\n experiment_dir.mkdir(parents=True, exist_ok=True)\n\n _download(\n design_url + \"experiment-design\",\n experiment_dir / \"experimental_design.tsv\",\n )\n _download(\n mtx_url + \"quantification-raw\",\n experiment_dir / \"expression_archive.zip\",\n )\n\n\ndef read_mtx_from_stream(stream: BinaryIO) -> sparse.csr_matrix:\n curline = stream.readline()\n while curline.startswith(b\"%\"):\n curline = stream.readline()\n n, m, _ = (int(x) for x in curline[:-1].split(b\" \"))\n\n max_int32 = np.iinfo(np.int32).max\n coord_dtype = np.int64 if n > max_int32 or m > max_int32 else np.int32\n\n data = pd.read_csv(\n stream,\n sep=r\"\\s+\",\n header=None,\n dtype={0: coord_dtype, 1: coord_dtype, 2: np.float32},\n )\n mtx = sparse.csr_matrix((data[2], (data[1] - 1, data[0] - 1)), shape=(m, n))\n return mtx\n\n\ndef read_expression_from_archive(archive: ZipFile) -> anndata.AnnData:\n info = archive.infolist()\n assert len(info) == 3\n mtx_data_info = next(i for i in info if i.filename.endswith(\".mtx\"))\n mtx_rows_info = next(i for i in info if i.filename.endswith(\".mtx_rows\"))\n mtx_cols_info = next(i for i in info if i.filename.endswith(\".mtx_cols\"))\n with archive.open(mtx_data_info, \"r\") as f:\n expr = read_mtx_from_stream(f)\n with archive.open(mtx_rows_info, \"r\") as f:\n # TODO: Check what other value could be\n varname = pd.read_csv(f, sep=\"\\t\", header=None)[1]\n with archive.open(mtx_cols_info, \"r\") as f:\n obsname = pd.read_csv(f, sep=\"\\t\", header=None).iloc[:, 0]\n adata = anndata.AnnData(expr)\n adata.var_names = varname\n adata.obs_names = obsname\n return adata\n\n\n@doctest_internet\ndef ebi_expression_atlas(\n accession: str, *, filter_boring: bool = False\n) -> anndata.AnnData:\n \"\"\"\\\n Load a dataset from the EBI Single Cell Expression Atlas.\n\n The atlas_ can be browsed online to find the ``accession`` you want.\n Downloaded datasets are saved in the directory specified by\n :attr:`~scanpy._settings.ScanpyConfig.datasetdir`.\n\n .. _atlas: https://www.ebi.ac.uk/gxa/sc/experiments\n\n Params\n ------\n accession\n Dataset accession. Like ``E-GEOD-98816`` or ``E-MTAB-4888``.\n This can be found in the url on the datasets page, for example E-GEOD-98816_.\n\n .. _E-GEOD-98816: https://www.ebi.ac.uk/gxa/sc/experiments/E-GEOD-98816/results/tsne\n filter_boring\n Whether boring labels in `.obs` should be automatically removed, such as\n labels with a single or :attr:`~anndata.AnnData.n_obs` distinct values.\n\n Returns\n -------\n Annotated data matrix.\n\n Example\n -------\n >>> import scanpy as sc\n >>> sc.datasets.ebi_expression_atlas(\"E-MTAB-4888\") # doctest: +ELLIPSIS\n AnnData object with n_obs × n_vars = 2261 × 23899\n obs: 'Sample Characteristic[organism]', 'Sample Characteristic Ontology Term[organism]', ..., 'Factor Value[cell type]', 'Factor Value Ontology Term[cell type]'\n \"\"\"\n experiment_dir = settings.datasetdir / accession\n dataset_path = experiment_dir / f\"{accession}.h5ad\"\n try:\n adata = anndata.read_h5ad(dataset_path)\n if filter_boring:\n adata.obs = _filter_boring(adata.obs)\n return adata\n except OSError:\n # Dataset couldn't be read for whatever reason\n pass\n\n download_experiment(accession)\n\n logg.info(f\"Downloaded {accession} to {experiment_dir.absolute()}\")\n\n with ZipFile(experiment_dir / \"expression_archive.zip\", \"r\") as f:\n adata = read_expression_from_archive(f)\n obs = pd.read_csv(experiment_dir / \"experimental_design.tsv\", sep=\"\\t\", index_col=0)\n\n adata.obs[obs.columns] = obs\n adata.write(dataset_path, compression=\"gzip\") # To be kind to disk space\n\n if filter_boring:\n adata.obs = _filter_boring(adata.obs)\n\n return adata\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom functools import wraps\nfrom typing import TYPE_CHECKING\n\nimport anndata as ad\nfrom packaging.version import Version\n\nfrom .._settings import settings\n\nif TYPE_CHECKING:\n from collections.abc import Callable\n from typing import ParamSpec, TypeVar\n\n P = ParamSpec(\"P\")\n R = TypeVar(\"R\")\n\n\ndef check_datasetdir_exists(f: Callable[P, R]) -> Callable[P, R]:\n @wraps(f)\n def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:\n settings.datasetdir.mkdir(exist_ok=True)\n return f(*args, **kwargs)\n\n return wrapper\n\n\ndef filter_oldformatwarning(f: Callable[P, R]) -> Callable[P, R]:\n \"\"\"\n Filters anndata.OldFormatWarning from being thrown by the wrapped function.\n \"\"\"\n\n @wraps(f)\n def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:\n with warnings.catch_warnings():\n if Version(ad.__version__).release >= (0, 8):\n warnings.filterwarnings(\n \"ignore\", category=ad.OldFormatWarning, module=\"anndata\"\n )\n return f(*args, **kwargs)\n\n return wrapper\n\n\n 0 0.8052 0.7880\n 1 0.7378 0.7237\n 2 0.6568 0.6589\n 3 0.5926 0.5886\n 4 0.5454 0.5314\n 5 0.4869 0.4905\n 6 0.4490 0.4493\n 7 0.4000 0.4066\n 8 0.3722 0.3757\n 9 0.3355 0.3014\n 10 0.3004 0.3247\n 11 0.2922 0.2865\n 12 0.2728 0.2553\n 13 0.2533 0.2504\n 14 0.2341 0.2504\n 15 0.2275 0.2192\n 16 0.2160 0.2120\n 17 0.2164 0.2017\n 18 0.2030 0.1865\n 19 0.2145 0.1973\n 20 0.1901 0.1991\n 21 0.1857 0.1772\n 22 0.2140 0.1781\n 23 0.1823 0.1899\n 24 0.1820 0.1734\n 25 0.2038 0.1831\n 26 0.1878 0.1833\n 27 0.2071 0.1546\n 28 0.1868 0.1561\n 29 0.1931 0.1545\n 30 0.1971 0.1606\n 31 0.2048 0.1480\n 32 0.2007 0.1665\n 33 0.2005 0.1567\n 34 0.2335 0.1688\n 35 0.1923 0.1687\n 36 0.2037 0.1463\n 37 0.2114 0.1620\n 38 0.2258 0.1510\n 39 0.2349 0.1464\n 40 0.2294 0.1453\n 41 0.2369 0.1557\n 42 0.2425 0.1334\n 43 0.2240 0.1440\n 44 0.2479 0.1287\n 45 0.2506 0.1361\n 46 0.2693 0.1287\n 47 0.2716 0.1226\n 48 0.2859 0.1214\n 49 0.2877 0.1017\n 50 0.2892 0.1042\n 51 0.2916 0.0954\n 52 0.3211 0.0971\n 53 0.3365 0.0777\n 54 0.3389 0.0806\n 55 0.3828 0.0666\n 56 0.3880 0.0715\n 57 0.4092 0.0604\n 58 0.4341 0.0590\n 59 0.4671 0.0591\n 60 0.4888 0.0547\n 61 0.5179 0.0392\n 62 0.5429 0.0457\n 63 0.5542 0.0351\n 64 0.5775 0.0466\n 65 0.6189 0.0559\n 66 0.6583 0.0100\n 67 0.6781 0.0164\n 68 0.6738 0.0121\n 69 0.7266 0.0161\n 70 0.7365 0.0267\n 71 0.7676 0.0108\n 72 0.7802 0.0189\n 73 0.7919 0.0223\n 74 0.7964 0.0000\n 75 0.8168 0.0206\n 76 0.8357 -0.0017\n 77 0.8590 0.0064\n 78 0.8631 -0.0063\n 79 0.8778 0.0118\n 80 0.8959 0.0151\n 81 0.8889 0.0045\n 82 0.8988 -0.0017\n 83 0.9048 0.0015\n 84 0.9058 0.0184\n 85 0.9106 0.0078\n 86 0.9427 0.0061\n 87 0.9504 -0.0172\n 88 0.9399 -0.0146\n 89 0.9312 -0.0045\n 90 0.9287 0.0014\n 91 0.9580 -0.0219\n 92 0.9492 0.0117\n 93 0.9513 -0.0021\n 94 0.9775 -0.0108\n 95 0.9771 0.0123\n 96 0.9576 0.0039\n 97 0.9823 -0.0137\n 98 0.9726 0.0075\n 99 0.9791 -0.0249\n 0 0.7856 0.7909\n 1 0.7232 0.7316\n 2 0.6652 0.6568\n 3 0.5844 0.5871\n 4 0.5261 0.5367\n 5 0.4828 0.4746\n 6 0.4317 0.4340\n 7 0.3951 0.4049\n 8 0.3363 0.3738\n 9 0.3265 0.3413\n 10 0.3014 0.3120\n 11 0.2903 0.2915\n 12 0.2452 0.2735\n 13 0.2541 0.2485\n 14 0.2376 0.2455\n 15 0.2073 0.2364\n 16 0.2148 0.2256\n 17 0.2077 0.2291\n 18 0.2010 0.2009\n 19 0.1830 0.2009\n 20 0.1730 0.1904\n 21 0.1850 0.1854\n 22 0.1864 0.1781\n 23 0.1844 0.2072\n 24 0.1840 0.1829\n 25 0.1703 0.1929\n 26 0.1910 0.1755\n 27 0.1793 0.1873\n 28 0.1672 0.1985\n 29 0.1709 0.1870\n 30 0.1602 0.1806\n 31 0.1674 0.1905\n 32 0.1586 0.1792\n 33 0.1521 0.1885\n 34 0.1617 0.1938\n 35 0.1813 0.1820\n 36 0.1710 0.1927\n 37 0.1813 0.1973\n 38 0.1668 0.1812\n 39 0.1697 0.1911\n 40 0.1802 0.1937\n 41 0.1641 0.2023\n 42 0.1605 0.1742\n 43 0.1634 0.2003\n 44 0.1485 0.2187\n 45 0.1607 0.1984\n 46 0.1578 0.2125\n 47 0.1378 0.2091\n 48 0.1630 0.2080\n 49 0.1525 0.2239\n 50 0.1459 0.2286\n 51 0.1377 0.2348\n 52 0.1228 0.2197\n 53 0.1515 0.2485\n 54 0.1118 0.2391\n 55 0.1275 0.2648\n 56 0.1248 0.2458\n 57 0.1199 0.2800\n 58 0.1128 0.2795\n 59 0.0929 0.2965\n 60 0.1039 0.3032\n 61 0.0973 0.3214\n 62 0.0875 0.3213\n 63 0.0771 0.3611\n 64 0.0652 0.3762\n 65 0.0894 0.3851\n 66 0.0579 0.4128\n 67 0.0593 0.4368\n 68 0.0598 0.4614\n 69 0.0671 0.5108\n 70 0.0523 0.5170\n 71 0.0545 0.5370\n 72 0.0182 0.5819\n 73 0.0376 0.6036\n 74 0.0318 0.6148\n 75 0.0389 0.6596\n 76 0.0265 0.6772\n 77 0.0222 0.7121\n 78 0.0297 0.7199\n 79 0.0300 0.7359\n 80 0.0108 0.7798\n 81 0.0335 0.7786\n 82 0.0247 0.8034\n 83 0.0248 0.8071\n 84 -0.0007 0.8537\n 85 0.0087 0.8359\n 86 0.0065 0.8508\n 87 -0.0025 0.8516\n 88 0.0052 0.8746\n 89 0.0136 0.8971\n 90 0.0023 0.8802\n 91 0.0133 0.9154\n 92 0.0067 0.9180\n 93 0.0115 0.9287\n 94 0.0010 0.9201\n 95 0.0070 0.9266\n 96 0.0115 0.9389\n 97 0.0145 0.9367\n 98 0.0012 0.9513\n 99 0.0075 0.9499\n\n\n\"\"\"Builtin Datasets.\"\"\"\n\nfrom __future__ import annotations\n\nfrom ._datasets import (\n blobs,\n burczynski06,\n krumsiek11,\n moignard15,\n paul15,\n pbmc3k,\n pbmc3k_processed,\n pbmc68k_reduced,\n toggleswitch,\n visium_sge,\n)\nfrom ._ebi_expression_atlas import ebi_expression_atlas\n\n__all__ = [\n \"blobs\",\n \"burczynski06\",\n \"krumsiek11\",\n \"moignard15\",\n \"paul15\",\n \"pbmc3k\",\n \"pbmc3k_processed\",\n \"pbmc68k_reduced\",\n \"toggleswitch\",\n \"visium_sge\",\n \"ebi_expression_atlas\",\n]\n\n\n\"\"\"This module contains helper functions for accessing data.\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom anndata import AnnData\nfrom packaging.version import Version\nfrom scipy.sparse import spmatrix\n\nif TYPE_CHECKING:\n from collections.abc import Iterable\n from typing import Any, Literal\n\n from anndata._core.sparse_dataset import BaseCompressedSparseDataset\n from anndata._core.views import ArrayView\n from numpy.typing import NDArray\n\n# --------------------------------------------------------------------------------\n# Plotting data helpers\n# --------------------------------------------------------------------------------\n\n\n# TODO: implement diffxpy method, make singledispatch\ndef rank_genes_groups_df(\n adata: AnnData,\n group: str | Iterable[str] | None,\n *,\n key: str = \"rank_genes_groups\",\n pval_cutoff: float | None = None,\n log2fc_min: float | None = None,\n log2fc_max: float | None = None,\n gene_symbols: str | None = None,\n) -> pd.DataFrame:\n \"\"\"\\\n :func:`scanpy.tl.rank_genes_groups` results in the form of a\n :class:`~pandas.DataFrame`.\n\n Params\n ------\n adata\n Object to get results from.\n group\n Which group (as in :func:`scanpy.tl.rank_genes_groups`'s `groupby`\n argument) to return results from. Can be a list. All groups are\n returned if groups is `None`.\n key\n Key differential expression groups were stored under.\n pval_cutoff\n Return only adjusted p-values below the cutoff.\n log2fc_min\n Minimum logfc to return.\n log2fc_max\n Maximum logfc to return.\n gene_symbols\n Column name in `.var` DataFrame that stores gene symbols. Specifying\n this will add that column to the returned dataframe.\n\n Example\n -------\n >>> import scanpy as sc\n >>> pbmc = sc.datasets.pbmc68k_reduced()\n >>> sc.tl.rank_genes_groups(pbmc, groupby=\"louvain\", use_raw=True)\n >>> dedf = sc.get.rank_genes_groups_df(pbmc, group=\"0\")\n \"\"\"\n if isinstance(group, str):\n group = [group]\n if group is None:\n group = list(adata.uns[key][\"names\"].dtype.names)\n method = adata.uns[key][\"params\"][\"method\"]\n if method == \"logreg\":\n colnames = [\"names\", \"scores\"]\n else:\n colnames = [\"names\", \"scores\", \"logfoldchanges\", \"pvals\", \"pvals_adj\"]\n\n d = [pd.DataFrame(adata.uns[key][c])[group] for c in colnames]\n d = pd.concat(d, axis=1, names=[None, \"group\"], keys=colnames)\n if Version(pd.__version__) >= Version(\"2.1\"):\n d = d.stack(level=1, future_stack=True).reset_index()\n else:\n d = d.stack(level=1).reset_index()\n d[\"group\"] = pd.Categorical(d[\"group\"], categories=group)\n d = d.sort_values([\"group\", \"level_0\"]).drop(columns=\"level_0\")\n\n if method != \"logreg\":\n if pval_cutoff is not None:\n d = d[d[\"pvals_adj\"] < pval_cutoff]\n if log2fc_min is not None:\n d = d[d[\"logfoldchanges\"] > log2fc_min]\n if log2fc_max is not None:\n d = d[d[\"logfoldchanges\"] < log2fc_max]\n if gene_symbols is not None:\n d = d.join(adata.var[gene_symbols], on=\"names\")\n\n for pts, name in {\"pts\": \"pct_nz_group\", \"pts_rest\": \"pct_nz_reference\"}.items():\n if pts in adata.uns[key]:\n pts_df = (\n adata.uns[key][pts][group]\n .rename_axis(index=\"names\")\n .reset_index()\n .melt(id_vars=\"names\", var_name=\"group\", value_name=name)\n )\n d = d.merge(pts_df)\n\n # remove group column for backward compat if len(group) == 1\n if len(group) == 1:\n d.drop(columns=\"group\", inplace=True)\n\n return d.reset_index(drop=True)\n\n\ndef _check_indices(\n dim_df: pd.DataFrame,\n alt_index: pd.Index,\n *,\n dim: Literal[\"obs\", \"var\"],\n keys: list[str],\n alias_index: pd.Index | None = None,\n use_raw: bool = False,\n) -> tuple[list[str], list[str], list[str]]:\n \"\"\"Common logic for checking indices for obs_df and var_df.\"\"\"\n alt_repr = \"adata.raw\" if use_raw else \"adata\"\n\n alt_dim = (\"obs\", \"var\")[dim == \"obs\"]\n\n alias_name = None\n if alias_index is not None:\n alt_names = pd.Series(alt_index, index=alias_index)\n alias_name = alias_index.name\n alt_search_repr = f\"{alt_dim}['{alias_name}']\"\n else:\n alt_names = pd.Series(alt_index, index=alt_index)\n alt_search_repr = f\"{alt_dim}_names\"\n\n col_keys = []\n index_keys = []\n index_aliases = []\n not_found = []\n\n # check that adata.obs does not contain duplicated columns\n # if duplicated columns names are present, they will\n # be further duplicated when selecting them.\n if not dim_df.columns.is_unique:\n dup_cols = dim_df.columns[dim_df.columns.duplicated()].tolist()\n raise ValueError(\n f\"adata.{dim} contains duplicated columns. Please rename or remove \"\n \"these columns first.\\n`\"\n f\"Duplicated columns {dup_cols}\"\n )\n\n if not alt_index.is_unique:\n raise ValueError(\n f\"{alt_repr}.{alt_dim}_names contains duplicated items\\n\"\n f\"Please rename these {alt_dim} names first for example using \"\n f\"`adata.{alt_dim}_names_make_unique()`\"\n )\n\n # use only unique keys, otherwise duplicated keys will\n # further duplicate when reordering the keys later in the function\n for key in np.unique(keys):\n if key in dim_df.columns:\n col_keys.append(key)\n if key in alt_names.index:\n raise KeyError(\n f\"The key '{key}' is found in both adata.{dim} and {alt_repr}.{alt_search_repr}.\"\n )\n elif key in alt_names.index:\n val = alt_names[key]\n if isinstance(val, pd.Series):\n # while var_names must be unique, adata.var[gene_symbols] does not\n # It's still ambiguous to refer to a duplicated entry though.\n assert alias_index is not None\n raise KeyError(\n f\"Found duplicate entries for '{key}' in {alt_repr}.{alt_search_repr}.\"\n )\n index_keys.append(val)\n index_aliases.append(key)\n else:\n not_found.append(key)\n if len(not_found) > 0:\n raise KeyError(\n f\"Could not find keys '{not_found}' in columns of `adata.{dim}` or in\"\n f\" {alt_repr}.{alt_search_repr}.\"\n )\n\n return col_keys, index_keys, index_aliases\n\n\ndef _get_array_values(\n X,\n dim_names: pd.Index,\n keys: list[str],\n *,\n axis: Literal[0, 1],\n backed: bool,\n):\n # TODO: This should be made easier on the anndata side\n mutable_idxer = [slice(None), slice(None)]\n idx = dim_names.get_indexer(keys)\n\n # for backed AnnData is important that the indices are ordered\n if backed:\n idx_order = np.argsort(idx)\n rev_idxer = mutable_idxer.copy()\n mutable_idxer[axis] = idx[idx_order]\n rev_idxer[axis] = np.argsort(idx_order)\n matrix = X[tuple(mutable_idxer)][tuple(rev_idxer)]\n else:\n mutable_idxer[axis] = idx\n matrix = X[tuple(mutable_idxer)]\n\n from scipy.sparse import issparse\n\n if issparse(matrix):\n matrix = matrix.toarray()\n\n return matrix\n\n\ndef obs_df(\n adata: AnnData,\n keys: Iterable[str] = (),\n obsm_keys: Iterable[tuple[str, int]] = (),\n *,\n layer: str | None = None,\n gene_symbols: str | None = None,\n use_raw: bool = False,\n) -> pd.DataFrame:\n \"\"\"\\\n Return values for observations in adata.\n\n Params\n ------\n adata\n AnnData object to get values from.\n keys\n Keys from either `.var_names`, `.var[gene_symbols]`, or `.obs.columns`.\n obsm_keys\n Tuple of `(key from obsm, column index of obsm[key])`.\n layer\n Layer of `adata` to use as expression values.\n gene_symbols\n Column of `adata.var` to search for `keys` in.\n use_raw\n Whether to get expression values from `adata.raw`.\n\n Returns\n -------\n A dataframe with `adata.obs_names` as index, and values specified by `keys`\n and `obsm_keys`.\n\n Examples\n --------\n Getting value for plotting:\n\n >>> import scanpy as sc\n >>> pbmc = sc.datasets.pbmc68k_reduced()\n >>> plotdf = sc.get.obs_df(\n ... pbmc,\n ... keys=[\"CD8B\", \"n_genes\"],\n ... obsm_keys=[(\"X_umap\", 0), (\"X_umap\", 1)]\n ... )\n >>> plotdf.columns\n Index(['CD8B', 'n_genes', 'X_umap-0', 'X_umap-1'], dtype='object')\n >>> plotdf.plot.scatter(\"X_umap-0\", \"X_umap-1\", c=\"CD8B\") # doctest: +SKIP\n \n\n Calculating mean expression for marker genes by cluster:\n\n >>> pbmc = sc.datasets.pbmc68k_reduced()\n >>> marker_genes = ['CD79A', 'MS4A1', 'CD8A', 'CD8B', 'LYZ']\n >>> genedf = sc.get.obs_df(\n ... pbmc,\n ... keys=[\"louvain\", *marker_genes]\n ... )\n >>> grouped = genedf.groupby(\"louvain\", observed=True)\n >>> mean, var = grouped.mean(), grouped.var()\n \"\"\"\n if use_raw:\n assert (\n layer is None\n ), \"Cannot specify use_raw=True and a layer at the same time.\"\n var = adata.raw.var\n else:\n var = adata.var\n alias_index = pd.Index(var[gene_symbols]) if gene_symbols is not None else None\n\n obs_cols, var_idx_keys, var_symbols = _check_indices(\n adata.obs,\n var.index,\n dim=\"obs\",\n keys=keys,\n alias_index=alias_index,\n use_raw=use_raw,\n )\n\n # Make df\n df = pd.DataFrame(index=adata.obs_names)\n\n # add var values\n if len(var_idx_keys) > 0:\n matrix = _get_array_values(\n _get_obs_rep(adata, layer=layer, use_raw=use_raw),\n var.index,\n var_idx_keys,\n axis=1,\n backed=adata.isbacked,\n )\n df = pd.concat(\n [df, pd.DataFrame(matrix, columns=var_symbols, index=adata.obs_names)],\n axis=1,\n )\n\n # add obs values\n if len(obs_cols) > 0:\n df = pd.concat([df, adata.obs[obs_cols]], axis=1)\n\n # reorder columns to given order (including duplicates keys if present)\n if keys:\n df = df[keys]\n\n for k, idx in obsm_keys:\n added_k = f\"{k}-{idx}\"\n val = adata.obsm[k]\n if isinstance(val, np.ndarray):\n df[added_k] = np.ravel(val[:, idx])\n elif isinstance(val, spmatrix):\n df[added_k] = np.ravel(val[:, idx].toarray())\n elif isinstance(val, pd.DataFrame):\n df[added_k] = val.loc[:, idx]\n\n return df\n\n\ndef var_df(\n adata: AnnData,\n keys: Iterable[str] = (),\n varm_keys: Iterable[tuple[str, int]] = (),\n *,\n layer: str | None = None,\n) -> pd.DataFrame:\n \"\"\"\\\n Return values for observations in adata.\n\n Params\n ------\n adata\n AnnData object to get values from.\n keys\n Keys from either `.obs_names`, or `.var.columns`.\n varm_keys\n Tuple of `(key from varm, column index of varm[key])`.\n layer\n Layer of `adata` to use as expression values.\n\n Returns\n -------\n A dataframe with `adata.var_names` as index, and values specified by `keys`\n and `varm_keys`.\n \"\"\"\n # Argument handling\n var_cols, obs_idx_keys, _ = _check_indices(\n adata.var, adata.obs_names, dim=\"var\", keys=keys\n )\n\n # initialize df\n df = pd.DataFrame(index=adata.var.index)\n\n if len(obs_idx_keys) > 0:\n matrix = _get_array_values(\n _get_obs_rep(adata, layer=layer),\n adata.obs_names,\n obs_idx_keys,\n axis=0,\n backed=adata.isbacked,\n ).T\n df = pd.concat(\n [df, pd.DataFrame(matrix, columns=obs_idx_keys, index=adata.var_names)],\n axis=1,\n )\n\n # add obs values\n if len(var_cols) > 0:\n df = pd.concat([df, adata.var[var_cols]], axis=1)\n\n # reorder columns to given order\n if keys:\n df = df[keys]\n\n for k, idx in varm_keys:\n added_k = f\"{k}-{idx}\"\n val = adata.varm[k]\n if isinstance(val, np.ndarray):\n df[added_k] = np.ravel(val[:, idx])\n elif isinstance(val, spmatrix):\n df[added_k] = np.ravel(val[:, idx].toarray())\n elif isinstance(val, pd.DataFrame):\n df[added_k] = val.loc[:, idx]\n return df\n\n\ndef _get_obs_rep(\n adata: AnnData,\n *,\n use_raw: bool = False,\n layer: str | None = None,\n obsm: str | None = None,\n obsp: str | None = None,\n) -> (\n np.ndarray\n | spmatrix\n | pd.DataFrame\n | ArrayView\n | BaseCompressedSparseDataset\n | None\n):\n \"\"\"\n Choose array aligned with obs annotation.\n \"\"\"\n # https://github.com/scverse/scanpy/issues/1546\n if not isinstance(use_raw, bool):\n raise TypeError(f\"use_raw expected to be bool, was {type(use_raw)}.\")\n\n is_layer = layer is not None\n is_raw = use_raw is not False\n is_obsm = obsm is not None\n is_obsp = obsp is not None\n choices_made = sum((is_layer, is_raw, is_obsm, is_obsp))\n assert choices_made in {0, 1}\n if choices_made == 0:\n return adata.X\n if is_layer:\n return adata.layers[layer]\n if use_raw:\n return adata.raw.X\n if is_obsm:\n return adata.obsm[obsm]\n if is_obsp:\n return adata.obsp[obsp]\n raise AssertionError(\n \"That was unexpected. Please report this bug at:\\n\\n\\t\"\n \"https://github.com/scverse/scanpy/issues\"\n )\n\n\ndef _set_obs_rep(\n adata: AnnData,\n val: Any,\n *,\n use_raw: bool = False,\n layer: str | None = None,\n obsm: str | None = None,\n obsp: str | None = None,\n):\n \"\"\"\n Set value for observation rep.\n \"\"\"\n is_layer = layer is not None\n is_raw = use_raw is not False\n is_obsm = obsm is not None\n is_obsp = obsp is not None\n choices_made = sum((is_layer, is_raw, is_obsm, is_obsp))\n assert choices_made <= 1\n if choices_made == 0:\n adata.X = val\n elif is_layer:\n adata.layers[layer] = val\n elif use_raw:\n adata.raw.X = val\n elif is_obsm:\n adata.obsm[obsm] = val\n elif is_obsp:\n adata.obsp[obsp] = val\n else:\n msg = (\n \"That was unexpected. Please report this bug at:\\n\\n\"\n \"\\thttps://github.com/scverse/scanpy/issues\"\n )\n raise AssertionError(msg)\n\n\ndef _check_mask(\n data: AnnData | np.ndarray,\n mask: NDArray[np.bool_] | str,\n dim: Literal[\"obs\", \"var\"],\n) -> NDArray[np.bool_]: # Could also be a series, but should be one or the other\n \"\"\"\n Validate mask argument\n Params\n ------\n data\n Annotated data matrix or numpy array.\n mask\n The mask. Either an appropriatley sized boolean array, or name of a column which will be used to mask.\n dim\n The dimension being masked.\n \"\"\"\n if isinstance(mask, str):\n if not isinstance(data, AnnData):\n msg = \"Cannot refer to mask with string without providing anndata object as argument\"\n raise ValueError(msg)\n\n annot: pd.DataFrame = getattr(data, dim)\n if mask not in annot.columns:\n msg = (\n f\"Did not find `adata.{dim}[{mask!r}]`. \"\n f\"Either add the mask first to `adata.{dim}`\"\n \"or consider using the mask argument with a boolean array.\"\n )\n raise ValueError(msg)\n mask_array = annot[mask].to_numpy()\n else:\n if len(mask) != data.shape[0 if dim == \"obs\" else 1]:\n raise ValueError(\"The shape of the mask do not match the data.\")\n mask_array = mask\n\n if not pd.api.types.is_bool_dtype(mask_array.dtype):\n raise ValueError(\"Mask array must be boolean.\")\n\n return mask_array\n\n\nfrom __future__ import annotations\n\nfrom functools import singledispatch\nfrom typing import TYPE_CHECKING, Literal, get_args\n\nimport numpy as np\nimport pandas as pd\nfrom anndata import AnnData, utils\nfrom scipy import sparse\nfrom sklearn.utils.sparsefuncs import csc_median_axis_0\n\nfrom .._utils import _resolve_axis\nfrom .get import _check_mask\n\nif TYPE_CHECKING:\n from collections.abc import Collection, Iterable\n from typing import Union\n\n from numpy.typing import NDArray\n\n Array = Union[np.ndarray, sparse.csc_matrix, sparse.csr_matrix]\n\n# Used with get_args\nAggType = Literal[\"count_nonzero\", \"mean\", \"sum\", \"var\", \"median\"]\n\n\nclass Aggregate:\n \"\"\"\\\n Functionality for generic grouping and aggregating.\n\n There is currently support for count_nonzero, sum, mean, and variance.\n\n **Implementation**\n\n Moments are computed using weighted sum aggregation of data by some feature\n via multiplication by a sparse coordinate matrix A.\n\n Runtime is effectively computation of the product `A @ X`, i.e. the count of (non-zero)\n entries in X with multiplicity the number of group memberships for that entry.\n This is `O(data)` for partitions (each observation belonging to exactly one group),\n independent of the number of groups.\n\n Params\n ------\n groupby\n :class:`~pandas.Categorical` containing values for grouping by.\n data\n Data matrix for aggregation.\n mask\n Mask to be used for aggregation.\n \"\"\"\n\n def __init__(\n self,\n groupby: pd.Categorical,\n data: Array,\n *,\n mask: NDArray[np.bool_] | None = None,\n ) -> None:\n self.groupby = groupby\n self.indicator_matrix = sparse_indicator(groupby, mask=mask)\n self.data = data\n\n groupby: pd.Categorical\n indicator_matrix: sparse.coo_matrix\n data: Array\n\n def count_nonzero(self) -> NDArray[np.integer]:\n \"\"\"\\\n Count the number of observations in each group.\n\n Returns\n -------\n Array of counts.\n \"\"\"\n # pattern = self.data._with_data(np.broadcast_to(1, len(self.data.data)))\n # return self.indicator_matrix @ pattern\n return utils.asarray(self.indicator_matrix @ (self.data != 0))\n\n def sum(self) -> Array:\n \"\"\"\\\n Compute the sum per feature per group of observations.\n\n Returns\n -------\n Array of sum.\n \"\"\"\n return utils.asarray(self.indicator_matrix @ self.data)\n\n def mean(self) -> Array:\n \"\"\"\\\n Compute the mean per feature per group of observations.\n\n Returns\n -------\n Array of mean.\n \"\"\"\n return (\n utils.asarray(self.indicator_matrix @ self.data)\n / np.bincount(self.groupby.codes)[:, None]\n )\n\n def mean_var(self, dof: int = 1) -> tuple[np.ndarray, np.ndarray]:\n \"\"\"\\\n Compute the count, as well as mean and variance per feature, per group of observations.\n\n The formula `Var(X) = E(X^2) - E(X)^2` suffers loss of precision when the variance is a\n very small fraction of the squared mean. In particular, when X is constant, the formula may\n nonetheless be non-zero. By default, our implementation resets the variance to exactly zero\n when the computed variance, relative to the squared mean, nears limit of precision of the\n floating-point significand.\n\n Params\n ------\n dof\n Degrees of freedom for variance.\n\n Returns\n -------\n Object with `count`, `mean`, and `var` attributes.\n \"\"\"\n assert dof >= 0\n\n group_counts = np.bincount(self.groupby.codes)\n mean_ = self.mean()\n # sparse matrices do not support ** for elementwise power.\n mean_sq = (\n utils.asarray(self.indicator_matrix @ _power(self.data, 2))\n / group_counts[:, None]\n )\n sq_mean = mean_**2\n var_ = mean_sq - sq_mean\n # TODO: Why these values exactly? Because they are high relative to the datatype?\n # (unchanged from original code: https://github.com/scverse/anndata/pull/564)\n precision = 2 << (42 if self.data.dtype == np.float64 else 20)\n # detects loss of precision in mean_sq - sq_mean, which suggests variance is 0\n var_[precision * var_ < sq_mean] = 0\n if dof != 0:\n var_ *= (group_counts / (group_counts - dof))[:, np.newaxis]\n return mean_, var_\n\n def median(self) -> Array:\n \"\"\"\\\n Compute the median per feature per group of observations.\n\n Returns\n -------\n Array of median.\n \"\"\"\n\n medians = []\n for group in np.unique(self.groupby.codes):\n group_mask = self.groupby.codes == group\n group_data = self.data[group_mask]\n if sparse.issparse(group_data):\n if group_data.format != \"csc\":\n group_data = group_data.tocsc()\n medians.append(csc_median_axis_0(group_data))\n else:\n medians.append(np.median(group_data, axis=0))\n return np.array(medians)\n\n\ndef _power(X: Array, power: float | int) -> Array:\n \"\"\"\\\n Generate elementwise power of a matrix.\n\n Needed for non-square sparse matrices because they do not support `**` so the `.power` function is used.\n\n Params\n ------\n X\n Matrix whose power is to be raised.\n power\n Integer power value\n\n Returns\n -------\n Matrix whose power has been raised.\n \"\"\"\n return X**power if isinstance(X, np.ndarray) else X.power(power)\n\n\ndef aggregate(\n adata: AnnData,\n by: str | Collection[str],\n func: AggType | Iterable[AggType],\n *,\n axis: Literal[\"obs\", 0, \"var\", 1] | None = None,\n mask: NDArray[np.bool_] | str | None = None,\n dof: int = 1,\n layer: str | None = None,\n obsm: str | None = None,\n varm: str | None = None,\n) -> AnnData:\n \"\"\"\\\n Aggregate data matrix based on some categorical grouping.\n\n This function is useful for pseudobulking as well as plotting.\n\n Aggregation to perform is specified by `func`, which can be a single metric or a\n list of metrics. Each metric is computed over the group and results in a new layer\n in the output `AnnData` object.\n\n If none of `layer`, `obsm`, or `varm` are passed in, `X` will be used for aggregation data.\n\n Params\n ------\n adata\n :class:`~anndata.AnnData` to be aggregated.\n by\n Key of the column to be grouped-by.\n func\n How to aggregate.\n axis\n Axis on which to find group by column.\n mask\n Boolean mask (or key to column containing mask) to apply along the axis.\n dof\n Degrees of freedom for variance. Defaults to 1.\n layer\n If not None, key for aggregation data.\n obsm\n If not None, key for aggregation data.\n varm\n If not None, key for aggregation data.\n\n Returns\n -------\n Aggregated :class:`~anndata.AnnData`.\n\n Examples\n --------\n\n Calculating mean expression and number of nonzero entries per cluster:\n\n >>> import scanpy as sc, pandas as pd\n >>> pbmc = sc.datasets.pbmc3k_processed().raw.to_adata()\n >>> pbmc.shape\n (2638, 13714)\n >>> aggregated = sc.get.aggregate(pbmc, by=\"louvain\", func=[\"mean\", \"count_nonzero\"])\n >>> aggregated\n AnnData object with n_obs × n_vars = 8 × 13714\n obs: 'louvain'\n var: 'n_cells'\n layers: 'mean', 'count_nonzero'\n\n We can group over multiple columns:\n\n >>> pbmc.obs[\"percent_mito_binned\"] = pd.cut(pbmc.obs[\"percent_mito\"], bins=5)\n >>> sc.get.aggregate(pbmc, by=[\"louvain\", \"percent_mito_binned\"], func=[\"mean\", \"count_nonzero\"])\n AnnData object with n_obs × n_vars = 40 × 13714\n obs: 'louvain', 'percent_mito_binned'\n var: 'n_cells'\n layers: 'mean', 'count_nonzero'\n\n Note that this filters out any combination of groups that wasn't present in the original data.\n \"\"\"\n if not isinstance(adata, AnnData):\n raise NotImplementedError(\n \"sc.get.aggregate is currently only implemented for AnnData input, \"\n f\"was passed {type(adata)}.\"\n )\n if axis is None:\n axis = 1 if varm else 0\n axis, axis_name = _resolve_axis(axis)\n if mask is not None:\n mask = _check_mask(adata, mask, axis_name)\n data = adata.X\n if sum(p is not None for p in [varm, obsm, layer]) > 1:\n raise TypeError(\"Please only provide one (or none) of varm, obsm, or layer\")\n\n if varm is not None:\n if axis != 1:\n raise ValueError(\"varm can only be used when axis is 1\")\n data = adata.varm[varm]\n elif obsm is not None:\n if axis != 0:\n raise ValueError(\"obsm can only be used when axis is 0\")\n data = adata.obsm[obsm]\n elif layer is not None:\n data = adata.layers[layer]\n if axis == 1:\n data = data.T\n elif axis == 1:\n # i.e., all of `varm`, `obsm`, `layers` are None so we use `X` which must be transposed\n data = data.T\n\n dim_df = getattr(adata, axis_name)\n categorical, new_label_df = _combine_categories(dim_df, by)\n # Actual computation\n layers = _aggregate(\n data,\n by=categorical,\n func=func,\n mask=mask,\n dof=dof,\n )\n\n # Define new var dataframe\n if obsm or varm:\n if isinstance(data, pd.DataFrame):\n # Check if there could be labels\n var = pd.DataFrame(index=data.columns)\n else:\n # Create them otherwise\n var = pd.DataFrame(index=pd.RangeIndex(data.shape[1]).astype(str))\n else:\n var = getattr(adata, \"var\" if axis == 0 else \"obs\")\n\n # It's all coming together\n result = AnnData(layers=layers, obs=new_label_df, var=var)\n\n if axis == 1:\n return result.T\n else:\n return result\n\n\n@singledispatch\ndef _aggregate(\n data,\n by: pd.Categorical,\n func: AggType | Iterable[AggType],\n *,\n mask: NDArray[np.bool_] | None = None,\n dof: int = 1,\n):\n raise NotImplementedError(f\"Data type {type(data)} not supported for aggregation\")\n\n\n@_aggregate.register(pd.DataFrame)\ndef aggregate_df(data, by, func, *, mask=None, dof=1):\n return _aggregate(data.values, by, func, mask=mask, dof=dof)\n\n\n@_aggregate.register(np.ndarray)\n@_aggregate.register(sparse.spmatrix)\ndef aggregate_array(\n data,\n by: pd.Categorical,\n func: AggType | Iterable[AggType],\n *,\n mask: NDArray[np.bool_] | None = None,\n dof: int = 1,\n) -> dict[AggType, np.ndarray]:\n groupby = Aggregate(groupby=by, data=data, mask=mask)\n result = {}\n\n funcs = set([func] if isinstance(func, str) else func)\n if unknown := funcs - set(get_args(AggType)):\n raise ValueError(f\"func {unknown} is not one of {get_args(AggType)}\")\n\n if \"sum\" in funcs: # sum is calculated separately from the rest\n agg = groupby.sum()\n result[\"sum\"] = agg\n # here and below for count, if var is present, these can be calculate alongside var\n if \"mean\" in funcs and \"var\" not in funcs:\n agg = groupby.mean()\n result[\"mean\"] = agg\n if \"count_nonzero\" in funcs:\n result[\"count_nonzero\"] = groupby.count_nonzero()\n if \"var\" in funcs:\n mean_, var_ = groupby.mean_var(dof)\n result[\"var\"] = var_\n if \"mean\" in funcs:\n result[\"mean\"] = mean_\n if \"median\" in funcs:\n agg = groupby.median()\n result[\"median\"] = agg\n return result\n\n\ndef _combine_categories(\n label_df: pd.DataFrame, cols: Collection[str] | str\n) -> tuple[pd.Categorical, pd.DataFrame]:\n \"\"\"\n Returns both the result categories and a dataframe labelling each row\n \"\"\"\n from itertools import product\n\n if isinstance(cols, str):\n cols = [cols]\n\n df = pd.DataFrame(\n {c: pd.Categorical(label_df[c]).remove_unused_categories() for c in cols},\n )\n n_categories = [len(df[c].cat.categories) for c in cols]\n\n # It's like np.concatenate([x for x in product(*[range(n) for n in n_categories])])\n code_combinations = np.indices(n_categories).reshape(len(n_categories), -1)\n result_categories = pd.Index(\n [\"_\".join(map(str, x)) for x in product(*[df[c].cat.categories for c in cols])]\n )\n\n # Dataframe with unique combination of categories for each row\n new_label_df = pd.DataFrame(\n {\n c: pd.Categorical.from_codes(code_combinations[i], df[c].cat.categories)\n for i, c in enumerate(cols)\n },\n index=result_categories,\n )\n\n # Calculating result codes\n factors = np.ones(len(cols) + 1, dtype=np.int32) # First factor needs to be 1\n np.cumprod(n_categories[::-1], out=factors[1:])\n factors = factors[:-1][::-1]\n\n code_array = np.zeros((len(cols), df.shape[0]), dtype=np.int32)\n for i, c in enumerate(cols):\n code_array[i] = df[c].cat.codes\n code_array *= factors[:, None]\n\n result_categorical = pd.Categorical.from_codes(\n code_array.sum(axis=0), categories=result_categories\n )\n\n # Filter unused categories\n result_categorical = result_categorical.remove_unused_categories()\n new_label_df = new_label_df.loc[result_categorical.categories]\n\n return result_categorical, new_label_df\n\n\ndef sparse_indicator(\n categorical: pd.Categorical,\n *,\n mask: NDArray[np.bool_] | None = None,\n weight: NDArray[np.floating] | None = None,\n) -> sparse.coo_matrix:\n if mask is not None and weight is None:\n weight = mask.astype(np.float32)\n elif mask is not None and weight is not None:\n weight = mask * weight\n elif mask is None and weight is None:\n weight = np.broadcast_to(1.0, len(categorical))\n A = sparse.coo_matrix(\n (weight, (categorical.codes, np.arange(len(categorical)))),\n shape=(len(categorical.categories), len(categorical)),\n )\n return A\n\n\nfrom __future__ import annotations\n\nfrom ._aggregated import aggregate\nfrom .get import (\n _check_mask,\n _get_obs_rep,\n _set_obs_rep,\n obs_df,\n rank_genes_groups_df,\n var_df,\n)\n\n__all__ = [\n \"_check_mask\",\n \"_get_obs_rep\",\n \"_set_obs_rep\",\n \"aggregate\",\n \"obs_df\",\n \"rank_genes_groups_df\",\n \"var_df\",\n]\n\n\n\"\"\"\\\nShared docstrings for plotting function parameters.\n\"\"\"\n\nfrom __future__ import annotations\n\ndoc_adata_color_etc = \"\"\"\\\nadata\n Annotated data matrix.\ncolor\n Keys for annotations of observations/cells or variables/genes, e.g.,\n `'ann1'` or `['ann1', 'ann2']`.\ngene_symbols\n Column name in `.var` DataFrame that stores gene symbols. By default `var_names`\n refer to the index column of the `.var` DataFrame. Setting this option allows\n alternative names to be used.\nuse_raw\n Use `.raw` attribute of `adata` for coloring with gene expression. If `None`,\n defaults to `True` if `layer` isn't provided and `adata.raw` is present.\nlayer\n Name of the AnnData object layer that wants to be plotted. By default\n adata.raw.X is plotted. If `use_raw=False` is set, then `adata.X` is plotted.\n If `layer` is set to a valid layer name, then the layer is plotted. `layer`\n takes precedence over `use_raw`.\\\n\"\"\"\n\ndoc_edges_arrows = \"\"\"\\\nedges\n Show edges.\nedges_width\n Width of edges.\nedges_color\n Color of edges. See :func:`~networkx.drawing.nx_pylab.draw_networkx_edges`.\nneighbors_key\n Where to look for neighbors connectivities.\n If not specified, this looks .obsp['connectivities'] for connectivities\n (default storage place for pp.neighbors).\n If specified, this looks\n .obsp[.uns[neighbors_key]['connectivities_key']] for connectivities.\narrows\n Show arrows (deprecated in favour of `scvelo.pl.velocity_embedding`).\narrows_kwds\n Passed to :meth:`~matplotlib.axes.Axes.quiver`\\\n\"\"\"\n\ndoc_cm_palette = \"\"\"\\\ncolor_map\n Color map to use for continous variables. Can be a name or a\n :class:`~matplotlib.colors.Colormap` instance (e.g. `\"magma`\", `\"viridis\"`\n or `mpl.cm.cividis`), see :func:`~matplotlib.pyplot.get_cmap`.\n If `None`, the value of `mpl.rcParams[\"image.cmap\"]` is used.\n The default `color_map` can be set using :func:`~scanpy.set_figure_params`.\npalette\n Colors to use for plotting categorical annotation groups.\n The palette can be a valid :class:`~matplotlib.colors.ListedColormap` name\n (`'Set2'`, `'tab20'`, …), a :class:`~cycler.Cycler` object, a dict mapping\n categories to colors, or a sequence of colors. Colors must be valid to\n matplotlib. (see :func:`~matplotlib.colors.is_color_like`).\n If `None`, `mpl.rcParams[\"axes.prop_cycle\"]` is used unless the categorical\n variable already has colors stored in `adata.uns[\"{var}_colors\"]`.\n If provided, values of `adata.uns[\"{var}_colors\"]` will be set.\\\n\"\"\"\n\n# Docs for pl.scatter\ndoc_scatter_basic = f\"\"\"\\\nsort_order\n For continuous annotations used as color parameter, plot data points\n with higher values on top of others.\ngroups\n Restrict to a few categories in categorical observation annotation.\n The default is not to restrict to any groups.\ndimensions\n 0-indexed dimensions of the embedding to plot as integers. E.g. [(0, 1), (1, 2)].\n Unlike `components`, this argument is used in the same way as `colors`, e.g. is\n used to specify a single plot at a time. Will eventually replace the components\n argument.\ncomponents\n For instance, `['1,2', '2,3']`. To plot all available components use\n `components='all'`.\nprojection\n Projection of plot (default: `'2d'`).\nlegend_loc\n Location of legend, either `'on data'`, `'right margin'`, `None`,\n or a valid keyword for the `loc` parameter of :class:`~matplotlib.legend.Legend`.\nlegend_fontsize\n Numeric size in pt or string describing the size.\n See :meth:`~matplotlib.text.Text.set_fontsize`.\nlegend_fontweight\n Legend font weight. A numeric value in range 0-1000 or a string.\n Defaults to `'bold'` if `legend_loc == 'on data'`, otherwise to `'normal'`.\n See :meth:`~matplotlib.text.Text.set_fontweight`.\nlegend_fontoutline\n Line width of the legend font outline in pt. Draws a white outline using\n the path effect :class:`~matplotlib.patheffects.withStroke`.\ncolorbar_loc\n Where to place the colorbar for continous variables. If `None`, no colorbar\n is added.\nsize\n Point size. If `None`, is automatically computed as 120000 / n_cells.\n Can be a sequence containing the size for each cell. The order should be\n the same as in adata.obs.\n{doc_cm_palette}\nna_color\n Color to use for null or masked values. Can be anything matplotlib accepts as a\n color. Used for all points if `color=None`.\nna_in_legend\n If there are missing values, whether they get an entry in the legend. Currently\n only implemented for categorical legends.\nframeon\n Draw a frame around the scatter plot. Defaults to value set in\n :func:`~scanpy.set_figure_params`, defaults to `True`.\ntitle\n Provide title for panels either as string or list of strings,\n e.g. `['title1', 'title2', ...]`.\n\"\"\"\n\ndoc_vbound_percentile = \"\"\"\\\nvmin\n The value representing the lower limit of the color scale. Values smaller than vmin are plotted\n with the same color as vmin. vmin can be a number, a string, a function or `None`. If\n vmin is a string and has the format `pN`, this is interpreted as a vmin=percentile(N).\n For example vmin='p1.5' is interpreted as the 1.5 percentile. If vmin is function, then\n vmin is interpreted as the return value of the function over the list of values to plot.\n For example to set vmin tp the mean of the values to plot, `def my_vmin(values): return\n np.mean(values)` and then set `vmin=my_vmin`. If vmin is None (default) an automatic\n minimum value is used as defined by matplotlib `scatter` function. When making multiple\n plots, vmin can be a list of values, one for each plot. For example `vmin=[0.1, 'p1', None, my_vmin]`\nvmax\n The value representing the upper limit of the color scale. The format is the same as for `vmin`.\nvcenter\n The value representing the center of the color scale. Useful for diverging colormaps.\n The format is the same as for `vmin`.\n Example: sc.pl.umap(adata, color='TREM2', vcenter='p50', cmap='RdBu_r')\\\n\"\"\"\n\ndoc_vboundnorm = \"\"\"\\\nvmin\n The value representing the lower limit of the color scale. Values smaller than vmin are plotted\n with the same color as vmin.\nvmax\n The value representing the upper limit of the color scale. Values larger than vmax are plotted\n with the same color as vmax.\nvcenter\n The value representing the center of the color scale. Useful for diverging colormaps.\nnorm\n Custom color normalization object from matplotlib. See\n `https://matplotlib.org/stable/tutorials/colors/colormapnorms.html` for details.\\\n\"\"\"\n\ndoc_outline = \"\"\"\\\nadd_outline\n If set to True, this will add a thin border around groups of dots. In some situations\n this can enhance the aesthetics of the resulting image\noutline_color\n Tuple with two valid color names used to adjust the add_outline. The first color is the\n border color (default: black), while the second color is a gap color between the\n border color and the scatter dot (default: white).\noutline_width\n Tuple with two width numbers used to adjust the outline. The first value is the width\n of the border color as a fraction of the scatter dot size (default: 0.3). The second value is\n width of the gap color (default: 0.05).\\\n\"\"\"\n\ndoc_panels = \"\"\"\\\nncols\n Number of panels per row.\nwspace\n Adjust the width of the space between multiple panels.\nhspace\n Adjust the height of the space between multiple panels.\nreturn_fig\n Return the matplotlib figure.\\\n\"\"\"\n\n# Docs for pl.pca, pl.tsne, … (everything in _tools.scatterplots)\ndoc_scatter_embedding = f\"\"\"\\\n{doc_scatter_basic}\n{doc_vbound_percentile}\n{doc_outline}\n{doc_panels}\nkwargs\n Arguments to pass to :func:`matplotlib.pyplot.scatter`,\n for instance: the maximum and minimum values (e.g. `vmin=-2, vmax=5`).\\\n\"\"\"\n\ndoc_show_save = \"\"\"\\\nshow\n Show the plot, do not return axis.\nsave\n If `True` or a `str`, save the figure.\n A string is appended to the default filename.\n Infer the filetype if ending on {`'.pdf'`, `'.png'`, `'.svg'`}.\\\n\"\"\"\n\ndoc_show_save_ax = f\"\"\"\\\n{doc_show_save}\nax\n A matplotlib axes object. Only works if plotting a single component.\\\n\"\"\"\n\ndoc_common_plot_args = \"\"\"\\\nadata\n Annotated data matrix.\nvar_names\n `var_names` should be a valid subset of `adata.var_names`.\n If `var_names` is a mapping, then the key is used as label\n to group the values (see `var_group_labels`). The mapping values\n should be sequences of valid `adata.var_names`. In this\n case either coloring or 'brackets' are used for the grouping\n of var names depending on the plot. When `var_names` is a mapping,\n then the `var_group_labels` and `var_group_positions` are set.\ngroupby\n The key of the observation grouping to consider.\nuse_raw\n Use `raw` attribute of `adata` if present.\nlog\n Plot on logarithmic axis.\nnum_categories\n Only used if groupby observation is not categorical. This value\n determines the number of groups into which the groupby observation\n should be subdivided.\ncategories_order\n Order in which to show the categories. Note: add_dendrogram or add_totals\n can change the categories order.\nfigsize\n Figure size when `multi_panel=True`.\n Otherwise the `rcParam['figure.figsize]` value is used.\n Format is (width, height)\ndendrogram\n If True or a valid dendrogram key, a dendrogram based on the hierarchical\n clustering between the `groupby` categories is added.\n The dendrogram information is computed using :func:`scanpy.tl.dendrogram`.\n If `tl.dendrogram` has not been called previously the function is called\n with default parameters.\ngene_symbols\n Column name in `.var` DataFrame that stores gene symbols.\n By default `var_names` refer to the index column of the `.var` DataFrame.\n Setting this option allows alternative names to be used.\nvar_group_positions\n Use this parameter to highlight groups of `var_names`.\n This will draw a 'bracket' or a color block between the given start and end\n positions. If the parameter `var_group_labels` is set, the corresponding\n labels are added on top/left. E.g. `var_group_positions=[(4,10)]`\n will add a bracket between the fourth `var_name` and the tenth `var_name`.\n By giving more positions, more brackets/color blocks are drawn.\nvar_group_labels\n Labels for each of the `var_group_positions` that want to be highlighted.\nvar_group_rotation\n Label rotation degrees.\n By default, labels larger than 4 characters are rotated 90 degrees.\nlayer\n Name of the AnnData object layer that wants to be plotted. By default adata.raw.X is plotted.\n If `use_raw=False` is set, then `adata.X` is plotted. If `layer` is set to a valid layer name,\n then the layer is plotted. `layer` takes precedence over `use_raw`.\\\n\"\"\"\n\ndoc_rank_genes_groups_plot_args = \"\"\"\\\nadata\n Annotated data matrix.\ngroups\n The groups for which to show the gene ranking.\nn_genes\n Number of genes to show. This can be a negative number to show for\n example the down regulated genes. eg: num_genes=-10. Is ignored if\n `gene_names` is passed.\ngene_symbols\n Column name in `.var` DataFrame that stores gene symbols. By default `var_names`\n refer to the index column of the `.var` DataFrame. Setting this option allows\n alternative names to be used.\ngroupby\n The key of the observation grouping to consider. By default,\n the groupby is chosen from the rank genes groups parameter but\n other groupby options can be used. It is expected that\n groupby is a categorical. If groupby is not a categorical observation,\n it would be subdivided into `num_categories` (see :func:`~scanpy.pl.dotplot`).\nmin_logfoldchange\n Value to filter genes in groups if their logfoldchange is less than the\n min_logfoldchange\nkey\n Key used to store the ranking results in `adata.uns`.\\\n\"\"\"\n\ndoc_rank_genes_groups_values_to_plot = \"\"\"\\\nvalues_to_plot\n Instead of the mean gene value, plot the values computed by `sc.rank_genes_groups`.\n The options are: ['scores', 'logfoldchanges', 'pvals', 'pvals_adj',\n 'log10_pvals', 'log10_pvals_adj']. When plotting logfoldchanges a divergent\n colormap is recommended. See examples below.\nvar_names\n Genes to plot. Sometimes is useful to pass a specific list of var names (e.g. genes)\n to check their fold changes or p-values, instead of the top/bottom genes. The\n var_names could be a dictionary or a list as in :func:`~scanpy.pl.dotplot` or\n :func:`~scanpy.pl.matrixplot`. See examples below.\\\n\"\"\"\n\ndoc_scatter_spatial = \"\"\"\\\nlibrary_id\n library_id for Visium data, e.g. key in `adata.uns[\"spatial\"]`.\nimg_key\n Key for image data, used to get `img` and `scale_factor` from `\"images\"`\n and `\"scalefactors\"` entires for this library. To use spatial coordinates,\n but not plot an image, pass `img_key=None`.\nimg\n image data to plot, overrides `img_key`.\nscale_factor\n Scaling factor used to map from coordinate space to pixel space.\n Found by default if `library_id` and `img_key` can be resolved.\n Otherwise defaults to `1.`.\nspot_size\n Diameter of spot (in coordinate space) for each point. Diameter\n in pixels of the spots will be `size * spot_size * scale_factor`.\n This argument is required if it cannot be resolved from library info.\ncrop_coord\n Coordinates to use for cropping the image (left, right, top, bottom).\n These coordinates are expected to be in pixel space (same as `basis`)\n and will be transformed by `scale_factor`.\n If not provided, image is automatically cropped to bounds of `basis`,\n plus a border.\nalpha_img\n Alpha value for image.\nbw\n Plot image data in gray scale.\\\n\"\"\"\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom matplotlib import pyplot as plt\nfrom matplotlib.colors import is_color_like\nfrom packaging.version import Version\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._settings import settings\nfrom .._utils import _doc_params, _empty\nfrom ._baseplot_class import BasePlot, doc_common_groupby_plot_args\nfrom ._docs import doc_common_plot_args, doc_show_save_ax, doc_vboundnorm\nfrom ._utils import (\n _deprecated_scale,\n _dk,\n check_colornorm,\n make_grid_spec,\n savefig_or_show,\n)\n\nif TYPE_CHECKING:\n from collections.abc import Mapping, Sequence\n from typing import Literal, Self\n\n from anndata import AnnData\n from matplotlib.axes import Axes\n from matplotlib.colors import Colormap, Normalize\n\n from .._utils import Empty\n from ._baseplot_class import _VarNames\n from ._utils import DensityNorm, _AxesSubplot\n\n\n@_doc_params(common_plot_args=doc_common_plot_args)\nclass StackedViolin(BasePlot):\n \"\"\"\\\n Stacked violin plots.\n\n Makes a compact image composed of individual violin plots\n (from :func:`~seaborn.violinplot`) stacked on top of each other.\n Useful to visualize gene expression per cluster.\n\n Wraps :func:`seaborn.violinplot` for :class:`~anndata.AnnData`.\n\n Parameters\n ----------\n {common_plot_args}\n title\n Title for the figure\n stripplot\n Add a stripplot on top of the violin plot.\n See :func:`~seaborn.stripplot`.\n jitter\n Add jitter to the stripplot (only when stripplot is True)\n See :func:`~seaborn.stripplot`.\n size\n Size of the jitter points.\n order\n Order in which to show the categories. Note: if `dendrogram=True`\n the categories order will be given by the dendrogram and `order`\n will be ignored.\n density_norm\n The method used to scale the width of each violin.\n If 'width' (the default), each violin will have the same width.\n If 'area', each violin will have the same area.\n If 'count', a violin’s width corresponds to the number of observations.\n row_palette\n The row palette determines the colors to use for the stacked violins.\n The value should be a valid seaborn or matplotlib palette name\n (see :func:`~seaborn.color_palette`).\n Alternatively, a single color name or hex value can be passed,\n e.g. `'red'` or `'#cc33ff'`.\n standard_scale\n Whether or not to standardize a dimension between 0 and 1,\n meaning for each variable or observation,\n subtract the minimum and divide each by its maximum.\n swap_axes\n By default, the x axis contains `var_names` (e.g. genes) and the y axis\n the `groupby` categories. By setting `swap_axes` then x are the `groupby`\n categories and y the `var_names`. When swapping\n axes var_group_positions are no longer used\n kwds\n Are passed to :func:`~seaborn.violinplot`.\n\n\n See also\n --------\n :func:`~scanpy.pl.stacked_violin`: simpler way to call StackedViolin but with less\n options.\n :func:`~scanpy.pl.violin` and :func:`~scanpy.pl.rank_genes_groups_stacked_violin`:\n to plot marker genes identified using :func:`~scanpy.tl.rank_genes_groups`\n\n Examples\n -------\n\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n >>> sc.pl.StackedViolin(adata, markers, groupby='bulk_labels', dendrogram=True) # doctest: +ELLIPSIS\n \n\n Using var_names as dict:\n\n >>> markers = {{'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}}\n >>> sc.pl.StackedViolin(adata, markers, groupby='bulk_labels', dendrogram=True) # doctest: +ELLIPSIS\n \n \"\"\"\n\n DEFAULT_SAVE_PREFIX = \"stacked_violin_\"\n DEFAULT_COLOR_LEGEND_TITLE = \"Median expression\\nin group\"\n\n DEFAULT_COLORMAP = \"Blues\"\n DEFAULT_STRIPPLOT = False\n DEFAULT_JITTER = False\n DEFAULT_JITTER_SIZE = 1\n DEFAULT_LINE_WIDTH = 0.2\n DEFAULT_ROW_PALETTE = None\n DEFAULT_DENSITY_NORM: DensityNorm = \"width\"\n DEFAULT_PLOT_YTICKLABELS = False\n DEFAULT_YLIM = None\n DEFAULT_PLOT_X_PADDING = 0.5 # a unit is the distance between two x-axis ticks\n DEFAULT_PLOT_Y_PADDING = 0.5 # a unit is the distance between two y-axis ticks\n\n # set by default the violin plot cut=0 to limit the extend\n # of the violin plot as this produces better plots that wont extend\n # to negative values for example. From seaborn.violin documentation:\n #\n # cut: Distance, in units of bandwidth size, to extend the density past\n # the extreme datapoints. Set to 0 to limit the violin range within\n # the range of the observed data (i.e., to have the same effect as\n # trim=True in ggplot.\n DEFAULT_CUT = 0\n\n # inner{“box”, “quartile”, “point”, “stick”, None} (Default seaborn: box)\n # Representation of the datapoints in the violin interior. If box, draw a\n # miniature boxplot. If quartiles, draw the quartiles of the distribution.\n # If point or stick, show each underlying datapoint. Using\n # None will draw unadorned violins.\n DEFAULT_INNER = None\n\n def __getattribute__(self, name: str) -> object:\n \"\"\"Called unconditionally when accessing an instance attribute\"\"\"\n # If the user has set the deprecated version on the class,\n # and our code accesses the new version from the instance,\n # return the user-specified version instead and warn.\n # This is done because class properties are hard to do.\n if name == \"DEFAULT_DENSITY_NORM\" and hasattr(self, \"DEFAULT_SCALE\"):\n msg = \"Don’t set DEFAULT_SCALE, use DEFAULT_DENSITY_NORM instead\"\n warnings.warn(msg, FutureWarning)\n return object.__getattribute__(self, \"DEFAULT_SCALE\")\n return object.__getattribute__(self, name)\n\n @old_positionals(\n \"use_raw\",\n \"log\",\n \"num_categories\",\n \"categories_order\",\n \"title\",\n \"figsize\",\n \"gene_symbols\",\n \"var_group_positions\",\n \"var_group_labels\",\n \"var_group_rotation\",\n \"layer\",\n \"standard_scale\",\n \"ax\",\n \"vmin\",\n \"vmax\",\n \"vcenter\",\n \"norm\",\n )\n def __init__(\n self,\n adata: AnnData,\n var_names: _VarNames | Mapping[str, _VarNames],\n groupby: str | Sequence[str],\n *,\n use_raw: bool | None = None,\n log: bool = False,\n num_categories: int = 7,\n categories_order: Sequence[str] | None = None,\n title: str | None = None,\n figsize: tuple[float, float] | None = None,\n gene_symbols: str | None = None,\n var_group_positions: Sequence[tuple[int, int]] | None = None,\n var_group_labels: Sequence[str] | None = None,\n var_group_rotation: float | None = None,\n layer: str | None = None,\n standard_scale: Literal[\"var\", \"group\"] | None = None,\n ax: _AxesSubplot | None = None,\n vmin: float | None = None,\n vmax: float | None = None,\n vcenter: float | None = None,\n norm: Normalize | None = None,\n **kwds,\n ):\n BasePlot.__init__(\n self,\n adata,\n var_names,\n groupby,\n use_raw=use_raw,\n log=log,\n num_categories=num_categories,\n categories_order=categories_order,\n title=title,\n figsize=figsize,\n gene_symbols=gene_symbols,\n var_group_positions=var_group_positions,\n var_group_labels=var_group_labels,\n var_group_rotation=var_group_rotation,\n layer=layer,\n ax=ax,\n vmin=vmin,\n vmax=vmax,\n vcenter=vcenter,\n norm=norm,\n **kwds,\n )\n\n if standard_scale == \"obs\":\n standard_scale = \"group\"\n msg = \"`standard_scale='obs'` is deprecated, use `standard_scale='group'` instead\"\n warnings.warn(msg, FutureWarning)\n if standard_scale == \"group\":\n self.obs_tidy = self.obs_tidy.sub(self.obs_tidy.min(1), axis=0)\n self.obs_tidy = self.obs_tidy.div(self.obs_tidy.max(1), axis=0).fillna(0)\n elif standard_scale == \"var\":\n self.obs_tidy -= self.obs_tidy.min(0)\n self.obs_tidy = (self.obs_tidy / self.obs_tidy.max(0)).fillna(0)\n elif standard_scale is None:\n pass\n else:\n logg.warning(\"Unknown type for standard_scale, ignored\")\n\n # Set default style parameters\n self.cmap = self.DEFAULT_COLORMAP\n self.row_palette = self.DEFAULT_ROW_PALETTE\n self.stripplot = self.DEFAULT_STRIPPLOT\n self.jitter = self.DEFAULT_JITTER\n self.jitter_size = self.DEFAULT_JITTER_SIZE\n self.plot_yticklabels = self.DEFAULT_PLOT_YTICKLABELS\n self.ylim = self.DEFAULT_YLIM\n self.plot_x_padding = self.DEFAULT_PLOT_X_PADDING\n self.plot_y_padding = self.DEFAULT_PLOT_Y_PADDING\n\n self.kwds.setdefault(\"cut\", self.DEFAULT_CUT)\n self.kwds.setdefault(\"inner\", self.DEFAULT_INNER)\n self.kwds.setdefault(\"linewidth\", self.DEFAULT_LINE_WIDTH)\n self.kwds.setdefault(\"density_norm\", self.DEFAULT_DENSITY_NORM)\n\n @old_positionals(\n \"cmap\",\n \"stripplot\",\n \"jitter\",\n \"jitter_size\",\n \"linewidth\",\n \"row_palette\",\n \"density_norm\",\n \"yticklabels\",\n \"ylim\",\n \"x_padding\",\n \"y_padding\",\n )\n def style(\n self,\n *,\n cmap: Colormap | str | None | Empty = _empty,\n stripplot: bool | Empty = _empty,\n jitter: float | bool | Empty = _empty,\n jitter_size: int | float | Empty = _empty,\n linewidth: float | None | Empty = _empty,\n row_palette: str | None | Empty = _empty,\n density_norm: DensityNorm | Empty = _empty,\n yticklabels: bool | Empty = _empty,\n ylim: tuple[float, float] | None | Empty = _empty,\n x_padding: float | Empty = _empty,\n y_padding: float | Empty = _empty,\n # deprecated\n scale: DensityNorm | Empty = _empty,\n ) -> Self:\n r\"\"\"\\\n Modifies plot visual parameters\n\n Parameters\n ----------\n cmap\n Matplotlib color map, specified by name or directly.\n If ``None``, use :obj:`matplotlib.rcParams`\\ ``[\"image.cmap\"]``\n stripplot\n Add a stripplot on top of the violin plot.\n See :func:`~seaborn.stripplot`.\n jitter\n Add jitter to the stripplot (only when stripplot is True)\n See :func:`~seaborn.stripplot`.\n jitter_size\n Size of the jitter points.\n linewidth\n line width for the violin plots.\n If None, use :obj:`matplotlib.rcParams`\\ ``[\"lines.linewidth\"]``\n row_palette\n The row palette determines the colors to use for the stacked violins.\n If ``None``, use :obj:`matplotlib.rcParams`\\ ``[\"axes.prop_cycle\"]``\n The value should be a valid seaborn or matplotlib palette name\n (see :func:`~seaborn.color_palette`).\n Alternatively, a single color name or hex value can be passed,\n e.g. `'red'` or `'#cc33ff'`.\n density_norm\n The method used to scale the width of each violin.\n If 'width' (the default), each violin will have the same width.\n If 'area', each violin will have the same area.\n If 'count', a violin’s width corresponds to the number of observations.\n yticklabels\n Set to true to view the y tick labels.\n ylim\n minimum and maximum values for the y-axis.\n If not ``None``, all rows will have the same y-axis range.\n Example: ``ylim=(0, 5)``\n x_padding\n Space between the plot left/right borders and the violins. A unit\n is the distance between the x ticks.\n y_padding\n Space between the plot top/bottom borders and the violins. A unit is\n the distance between the y ticks.\n\n Returns\n -------\n :class:`~scanpy.pl.StackedViolin`\n\n Examples\n -------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n\n Change color map and turn off edges\n\n >>> sc.pl.StackedViolin(adata, markers, groupby='bulk_labels') \\\n ... .style(row_palette='Blues', linewidth=0).show()\n \"\"\"\n super().style(cmap=cmap)\n\n if row_palette is not _empty:\n self.row_palette = row_palette\n self.kwds[\"color\"] = self.row_palette\n if stripplot is not _empty:\n self.stripplot = stripplot\n if jitter is not _empty:\n self.jitter = jitter\n if jitter_size is not _empty:\n self.jitter_size = jitter_size\n if yticklabels is not _empty:\n self.plot_yticklabels = yticklabels\n if self.plot_yticklabels:\n # space needs to be added to avoid overlapping\n # of labels and legend or dendrogram/totals.\n self.wspace = 0.3\n else:\n self.wspace = StackedViolin.DEFAULT_WSPACE\n if ylim is not _empty:\n self.ylim = ylim\n if x_padding is not _empty:\n self.plot_x_padding = x_padding\n if y_padding is not _empty:\n self.plot_y_padding = y_padding\n if linewidth is not _empty:\n self.kwds[\"linewidth\"] = linewidth\n if (density_norm := _deprecated_scale(density_norm, scale)) is not _empty:\n self.kwds[\"density_norm\"] = density_norm\n\n return self\n\n def _mainplot(self, ax: Axes):\n # to make the stacked violin plots, the\n # `ax` is subdivided horizontally and in each horizontal sub ax\n # a seaborn violin plot is added.\n\n # work on a copy of the dataframes. This is to avoid changes\n # on the original data frames after repetitive calls to the\n # StackedViolin object, for example once with swap_axes and other without\n _matrix = self.obs_tidy.copy()\n\n if self.var_names_idx_order is not None:\n _matrix = _matrix.iloc[:, self.var_names_idx_order]\n\n # get mean values for color and transform to color values\n # using colormap\n _color_df = (\n _matrix.groupby(level=0, observed=True)\n .median()\n .loc[\n self.categories_order\n if self.categories_order is not None\n else self.categories\n ]\n )\n if self.are_axes_swapped:\n _color_df = _color_df.T\n\n cmap = plt.get_cmap(self.kwds.pop(\"cmap\", self.cmap))\n normalize = check_colornorm(\n self.vboundnorm.vmin,\n self.vboundnorm.vmax,\n self.vboundnorm.vcenter,\n self.vboundnorm.norm,\n )\n colormap_array = cmap(normalize(_color_df.values))\n x_spacer_size = self.plot_x_padding\n y_spacer_size = self.plot_y_padding\n\n # All columns should have a unique name, yet, frequently\n # gene names are repeated in self.var_names, otherwise the\n # violin plot will not distinguish those genes\n _matrix.columns = [f\"{x}_{idx}\" for idx, x in enumerate(_matrix.columns)]\n\n # Ensure the categories axis is always ordered identically.\n # If the axes are not swapped, the above _matrix.columns is used in the actual violin plot (i.e., unique names).\n # If they are swapped, then use the same as the labels used below.\n # Without this, `_make_rows_of_violinplots` does not know about the order of the categories in labels.\n labels = _color_df.columns\n x_axis_order = labels if self.are_axes_swapped else _matrix.columns\n\n self._make_rows_of_violinplots(\n ax,\n _matrix,\n colormap_array,\n _color_df,\n x_spacer_size,\n y_spacer_size,\n x_axis_order,\n )\n\n # turn on axis for `ax` as this is turned off\n # by make_grid_spec when the axis is subdivided earlier.\n ax.set_frame_on(True)\n ax.axis(\"on\")\n ax.patch.set_alpha(0.0)\n\n # add tick labels\n ax.set_ylim(_color_df.shape[0] + y_spacer_size, -y_spacer_size)\n ax.set_xlim(-x_spacer_size, _color_df.shape[1] + x_spacer_size)\n\n # 0.5 to position the ticks on the center of the violins\n y_ticks = np.arange(_color_df.shape[0]) + 0.5\n ax.set_yticks(y_ticks)\n ax.set_yticklabels(\n [_color_df.index[idx] for idx, _ in enumerate(y_ticks)], minor=False\n )\n\n # 0.5 to position the ticks on the center of the violins\n x_ticks = np.arange(_color_df.shape[1]) + 0.5\n ax.set_xticks(x_ticks)\n ax.set_xticklabels(labels, minor=False, ha=\"center\")\n # rotate x tick labels if they are longer than 2 characters\n if max([len(x) for x in labels]) > 2:\n ax.tick_params(axis=\"x\", labelrotation=90)\n ax.tick_params(axis=\"both\", labelsize=\"small\")\n ax.grid(visible=False)\n\n return normalize\n\n def _make_rows_of_violinplots(\n self,\n ax,\n _matrix,\n colormap_array,\n _color_df,\n x_spacer_size: float | int,\n y_spacer_size: float | int,\n x_axis_order,\n ):\n import seaborn as sns # Slow import, only import if called\n\n row_palette = self.kwds.pop(\"color\", self.row_palette)\n if row_palette is not None:\n if is_color_like(row_palette):\n row_colors = [row_palette] * _color_df.shape[0]\n else:\n row_colors = sns.color_palette(row_palette, n_colors=_color_df.shape[0])\n # when row_palette is used, there is no need for a legend\n self.legends_width = 0.0\n else:\n row_colors = [None] * _color_df.shape[0]\n\n # transform the dataframe into a dataframe having three columns:\n # the categories name (from groupby),\n # the gene name\n # the expression value\n # This format is convenient to aggregate per gene or per category\n # while making the violin plots.\n if Version(pd.__version__) >= Version(\"2.1\"):\n stack_kwargs = {\"future_stack\": True}\n else:\n stack_kwargs = {\"dropna\": False}\n\n df = (\n pd.DataFrame(_matrix.stack(**stack_kwargs))\n .reset_index()\n .rename(\n columns={\n \"level_1\": \"genes\",\n _matrix.index.name: \"categories\",\n 0: \"values\",\n }\n )\n )\n df[\"genes\"] = (\n df[\"genes\"].astype(\"category\").cat.reorder_categories(_matrix.columns)\n )\n df[\"categories\"] = (\n df[\"categories\"]\n .astype(\"category\")\n .cat.reorder_categories(_matrix.index.categories)\n )\n\n # the ax need to be subdivided\n # define a layout of nrows = len(categories) rows\n # each row is one violin plot.\n num_rows, num_cols = _color_df.shape\n height_ratios = [y_spacer_size] + [1] * num_rows + [y_spacer_size]\n width_ratios = [x_spacer_size] + [1] * num_cols + [x_spacer_size]\n\n fig, gs = make_grid_spec(\n ax,\n nrows=num_rows + 2,\n ncols=num_cols + 2,\n hspace=0.2 if self.plot_yticklabels else 0,\n wspace=0,\n height_ratios=height_ratios,\n width_ratios=width_ratios,\n )\n axs_list = []\n for idx, row_label in enumerate(_color_df.index):\n row_ax = fig.add_subplot(gs[idx + 1, 1:-1])\n axs_list.append(row_ax)\n\n palette_colors = (\n list(colormap_array[idx, :]) if row_colors[idx] is None else None\n )\n\n if not self.are_axes_swapped:\n x = \"genes\"\n _df = df[df.categories == row_label]\n else:\n x = \"categories\"\n # because of the renamed matrix columns here\n # we need to use this instead of the 'row_label'\n # (in _color_df the values are not renamed as those\n # values will be used to label the ticks)\n _df = df[df.genes == _matrix.columns[idx]]\n\n row_ax = sns.violinplot(\n x=x,\n y=\"values\",\n data=_df,\n orient=\"vertical\",\n ax=row_ax,\n # use a single `color`` if row_colors[idx] is defined\n # else use the palette\n hue=None if palette_colors is None else x,\n palette=palette_colors,\n color=row_colors[idx],\n order=x_axis_order,\n hue_order=x_axis_order,\n **self.kwds,\n )\n if self.stripplot:\n row_ax = sns.stripplot(\n x=x,\n y=\"values\",\n data=_df,\n jitter=self.jitter,\n color=\"black\",\n size=self.jitter_size,\n ax=row_ax,\n )\n\n self._setup_violin_axes_ticks(row_ax, num_cols)\n\n def _setup_violin_axes_ticks(self, row_ax: Axes, num_cols: int):\n \"\"\"\n Configures each of the violin plot axes ticks like remove or add labels etc.\n\n \"\"\"\n # remove the default seaborn grids because in such a compact\n # plot are unnecessary\n\n row_ax.grid(visible=False)\n if self.ylim is not None:\n row_ax.set_ylim(self.ylim)\n if self.log:\n row_ax.set_yscale(\"log\")\n\n if self.plot_yticklabels:\n for spine in [\"top\", \"bottom\", \"left\"]:\n row_ax.spines[spine].set_visible(False)\n\n # make line a bit ticker to see the extend of the yaxis in the\n # final plot\n row_ax.spines[\"right\"].set_linewidth(1.5)\n row_ax.spines[\"right\"].set_position((\"data\", num_cols))\n\n row_ax.tick_params(\n axis=\"y\",\n left=False,\n right=True,\n labelright=True,\n labelleft=False,\n labelsize=\"x-small\",\n )\n # use only the smallest and the largest y ticks\n # and align the firts label on top of the tick and\n # the second below the tick. This avoid overlapping\n # of nearby ticks\n import matplotlib.ticker as ticker\n\n # use MaxNLocator to set 2 ticks\n row_ax.yaxis.set_major_locator(\n ticker.MaxNLocator(nbins=2, steps=[1, 1.2, 10])\n )\n yticks = row_ax.get_yticks()\n row_ax.set_yticks([yticks[0], yticks[-1]])\n ticklabels = row_ax.get_yticklabels()\n ticklabels[0].set_va(\"bottom\")\n ticklabels[-1].set_va(\"top\")\n else:\n row_ax.axis(\"off\")\n # remove labels\n row_ax.set_yticklabels([])\n row_ax.tick_params(axis=\"y\", left=False, right=False)\n\n row_ax.set_ylabel(\"\")\n\n row_ax.set_xlabel(\"\")\n\n row_ax.set_xticklabels([])\n row_ax.tick_params(\n axis=\"x\", bottom=False, top=False, labeltop=False, labelbottom=False\n )\n\n\n@old_positionals(\n \"log\",\n \"use_raw\",\n \"num_categories\",\n \"title\",\n \"colorbar_title\",\n \"figsize\",\n \"dendrogram\",\n \"gene_symbols\",\n \"var_group_positions\",\n \"var_group_labels\",\n \"standard_scale\",\n \"var_group_rotation\",\n \"layer\",\n \"stripplot\",\n # 17 positionals are enough for backwards compatibility\n)\n@_doc_params(\n show_save_ax=doc_show_save_ax,\n common_plot_args=doc_common_plot_args,\n groupby_plots_args=doc_common_groupby_plot_args,\n vminmax=doc_vboundnorm,\n)\ndef stacked_violin(\n adata: AnnData,\n var_names: _VarNames | Mapping[str, _VarNames],\n groupby: str | Sequence[str],\n *,\n log: bool = False,\n use_raw: bool | None = None,\n num_categories: int = 7,\n title: str | None = None,\n colorbar_title: str | None = StackedViolin.DEFAULT_COLOR_LEGEND_TITLE,\n figsize: tuple[float, float] | None = None,\n dendrogram: bool | str = False,\n gene_symbols: str | None = None,\n var_group_positions: Sequence[tuple[int, int]] | None = None,\n var_group_labels: Sequence[str] | None = None,\n standard_scale: Literal[\"var\", \"group\"] | None = None,\n var_group_rotation: float | None = None,\n layer: str | None = None,\n categories_order: Sequence[str] | None = None,\n swap_axes: bool = False,\n show: bool | None = None,\n save: bool | str | None = None,\n return_fig: bool | None = False,\n ax: _AxesSubplot | None = None,\n vmin: float | None = None,\n vmax: float | None = None,\n vcenter: float | None = None,\n norm: Normalize | None = None,\n # Style options\n cmap: Colormap | str | None = StackedViolin.DEFAULT_COLORMAP,\n stripplot: bool = StackedViolin.DEFAULT_STRIPPLOT,\n jitter: float | bool = StackedViolin.DEFAULT_JITTER,\n size: int | float = StackedViolin.DEFAULT_JITTER_SIZE,\n row_palette: str | None = StackedViolin.DEFAULT_ROW_PALETTE,\n density_norm: DensityNorm | Empty = _empty,\n yticklabels: bool = StackedViolin.DEFAULT_PLOT_YTICKLABELS,\n # deprecated\n order: Sequence[str] | None | Empty = _empty,\n scale: DensityNorm | Empty = _empty,\n **kwds,\n) -> StackedViolin | dict | None:\n \"\"\"\\\n Stacked violin plots.\n\n Makes a compact image composed of individual violin plots\n (from :func:`~seaborn.violinplot`) stacked on top of each other.\n Useful to visualize gene expression per cluster.\n\n Wraps :func:`seaborn.violinplot` for :class:`~anndata.AnnData`.\n\n This function provides a convenient interface to the\n :class:`~scanpy.pl.StackedViolin` class. If you need more flexibility,\n you should use :class:`~scanpy.pl.StackedViolin` directly.\n\n Parameters\n ----------\n {common_plot_args}\n {groupby_plots_args}\n stripplot\n Add a stripplot on top of the violin plot.\n See :func:`~seaborn.stripplot`.\n jitter\n Add jitter to the stripplot (only when stripplot is True)\n See :func:`~seaborn.stripplot`.\n size\n Size of the jitter points.\n density_norm\n The method used to scale the width of each violin.\n If 'width' (the default), each violin will have the same width.\n If 'area', each violin will have the same area.\n If 'count', a violin’s width corresponds to the number of observations.\n yticklabels\n Set to true to view the y tick labels.\n row_palette\n Be default, median values are mapped to the violin color using a\n color map (see `cmap` argument). Alternatively, a 'row_palette` can\n be given to color each violin plot row using a different colors.\n The value should be a valid seaborn or matplotlib palette name\n (see :func:`~seaborn.color_palette`).\n Alternatively, a single color name or hex value can be passed,\n e.g. `'red'` or `'#cc33ff'`.\n {show_save_ax}\n {vminmax}\n kwds\n Are passed to :func:`~seaborn.violinplot`.\n\n Returns\n -------\n If `return_fig` is `True`, returns a :class:`~scanpy.pl.StackedViolin` object,\n else if `show` is false, return axes dict\n\n See also\n --------\n :class:`~scanpy.pl.StackedViolin`: The StackedViolin class can be used to to control\n several visual parameters not available in this function.\n :func:`~scanpy.pl.rank_genes_groups_stacked_violin` to plot marker genes identified\n using the :func:`~scanpy.tl.rank_genes_groups` function.\n\n Examples\n -------\n\n Visualization of violin plots of a few genes grouped by the category `bulk_labels`:\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n sc.pl.stacked_violin(adata, markers, groupby='bulk_labels', dendrogram=True)\n\n Same visualization but passing var_names as dict, which adds a grouping of\n the genes on top of the image:\n\n .. plot::\n :context: close-figs\n\n markers = {{'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}}\n sc.pl.stacked_violin(adata, markers, groupby='bulk_labels', dendrogram=True)\n\n Get StackedViolin object for fine tuning\n\n .. plot::\n :context: close-figs\n\n vp = sc.pl.stacked_violin(adata, markers, 'bulk_labels', return_fig=True)\n vp.add_totals().style(ylim=(0,5)).show()\n\n The axes used can be obtained using the get_axes() method:\n\n .. code-block:: python\n\n axes_dict = vp.get_axes()\n print(axes_dict)\n\n \"\"\"\n if order is not _empty:\n msg = (\n \"`order` is deprecated (and never worked for `stacked_violin`), \"\n \"use categories_order instead\"\n )\n warnings.warn(msg, FutureWarning)\n # no reason to set `categories_order` here, as `order` never worked.\n\n vp = StackedViolin(\n adata,\n var_names,\n groupby=groupby,\n use_raw=use_raw,\n log=log,\n num_categories=num_categories,\n categories_order=categories_order,\n standard_scale=standard_scale,\n title=title,\n figsize=figsize,\n gene_symbols=gene_symbols,\n var_group_positions=var_group_positions,\n var_group_labels=var_group_labels,\n var_group_rotation=var_group_rotation,\n layer=layer,\n ax=ax,\n vmin=vmin,\n vmax=vmax,\n vcenter=vcenter,\n norm=norm,\n **kwds,\n )\n\n if dendrogram:\n vp.add_dendrogram(dendrogram_key=_dk(dendrogram))\n if swap_axes:\n vp.swap_axes()\n vp = vp.style(\n cmap=cmap,\n stripplot=stripplot,\n jitter=jitter,\n jitter_size=size,\n row_palette=row_palette,\n density_norm=_deprecated_scale(density_norm, scale),\n yticklabels=yticklabels,\n linewidth=kwds.get(\"linewidth\", _empty),\n ).legend(title=colorbar_title)\n if return_fig:\n return vp\n vp.make_figure()\n savefig_or_show(StackedViolin.DEFAULT_SAVE_PREFIX, show=show, save=save)\n show = settings.autoshow if show is None else show\n if show:\n return None\n return vp.get_axes()\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\nfrom anndata import AnnData\nfrom matplotlib import pyplot as plt\nfrom matplotlib import rcParams\n\nfrom .._compat import old_positionals\nfrom .._settings import settings\nfrom . import _utils\n\n# --------------------------------------------------------------------------------\n# Plot result of preprocessing functions\n# --------------------------------------------------------------------------------\n\n\n@old_positionals(\"log\", \"show\", \"save\", \"highly_variable_genes\")\ndef highly_variable_genes(\n adata_or_result: AnnData | pd.DataFrame | np.recarray,\n *,\n log: bool = False,\n show: bool | None = None,\n save: bool | str | None = None,\n highly_variable_genes: bool = True,\n) -> None:\n \"\"\"Plot dispersions or normalized variance versus means for genes.\n\n Produces Supp. Fig. 5c of Zheng et al. (2017) and MeanVarPlot() and\n VariableFeaturePlot() of Seurat.\n\n Parameters\n ----------\n adata\n Result of :func:`~scanpy.pp.highly_variable_genes`.\n log\n Plot on logarithmic axes.\n show\n Show the plot, do not return axis.\n save\n If `True` or a `str`, save the figure.\n A string is appended to the default filename.\n Infer the filetype if ending on {{`'.pdf'`, `'.png'`, `'.svg'`}}.\n \"\"\"\n if isinstance(adata_or_result, AnnData):\n result = adata_or_result.var\n seurat_v3_flavor = adata_or_result.uns[\"hvg\"][\"flavor\"] == \"seurat_v3\"\n else:\n result = adata_or_result\n if isinstance(result, pd.DataFrame):\n seurat_v3_flavor = \"variances_norm\" in result.columns\n else:\n seurat_v3_flavor = False\n if highly_variable_genes:\n gene_subset = result.highly_variable\n else:\n gene_subset = result.gene_subset\n means = result.means\n\n if seurat_v3_flavor:\n var_or_disp = result.variances\n var_or_disp_norm = result.variances_norm\n else:\n var_or_disp = result.dispersions\n var_or_disp_norm = result.dispersions_norm\n size = rcParams[\"figure.figsize\"]\n plt.figure(figsize=(2 * size[0], size[1]))\n plt.subplots_adjust(wspace=0.3)\n for idx, d in enumerate([var_or_disp_norm, var_or_disp]):\n plt.subplot(1, 2, idx + 1)\n for label, color, mask in zip(\n [\"highly variable genes\", \"other genes\"],\n [\"black\", \"grey\"],\n [gene_subset, ~gene_subset],\n ):\n if False:\n means_, var_or_disps_ = np.log10(means[mask]), np.log10(d[mask])\n else:\n means_, var_or_disps_ = means[mask], d[mask]\n plt.scatter(means_, var_or_disps_, label=label, c=color, s=1)\n if log: # there's a bug in autoscale\n plt.xscale(\"log\")\n plt.yscale(\"log\")\n y_min = np.min(var_or_disp)\n y_min = 0.95 * y_min if y_min > 0 else 1e-1\n plt.xlim(0.95 * np.min(means), 1.05 * np.max(means))\n plt.ylim(y_min, 1.05 * np.max(var_or_disp))\n if idx == 0:\n plt.legend()\n plt.xlabel((\"$log_{10}$ \" if False else \"\") + \"mean expressions of genes\")\n data_type = \"dispersions\" if not seurat_v3_flavor else \"variances\"\n plt.ylabel(\n (\"$log_{10}$ \" if False else \"\")\n + f\"{data_type} of genes\"\n + (\" (normalized)\" if idx == 0 else \" (not normalized)\")\n )\n\n show = settings.autoshow if show is None else show\n _utils.savefig_or_show(\"filter_genes_dispersion\", show=show, save=save)\n if show:\n return None\n return plt.gca()\n\n\n# backwards compat\n@old_positionals(\"log\", \"show\", \"save\")\ndef filter_genes_dispersion(\n result: np.recarray,\n *,\n log: bool = False,\n show: bool | None = None,\n save: bool | str | None = None,\n) -> None:\n \"\"\"\\\n Plot dispersions versus means for genes.\n\n Produces Supp. Fig. 5c of Zheng et al. (2017) and MeanVarPlot() of Seurat.\n\n Parameters\n ----------\n result\n Result of :func:`~scanpy.pp.filter_genes_dispersion`.\n log\n Plot on logarithmic axes.\n show\n Show the plot, do not return axis.\n save\n If `True` or a `str`, save the figure.\n A string is appended to the default filename.\n Infer the filetype if ending on {{`'.pdf'`, `'.png'`, `'.svg'`}}.\n \"\"\"\n highly_variable_genes(\n result, log=log, show=show, save=save, highly_variable_genes=False\n )\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nfrom matplotlib import pyplot as plt\nfrom matplotlib import rcParams\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._settings import settings\nfrom .._utils import _doc_params, _empty\nfrom ._baseplot_class import BasePlot, doc_common_groupby_plot_args\nfrom ._docs import (\n doc_common_plot_args,\n doc_show_save_ax,\n doc_vboundnorm,\n)\nfrom ._utils import _dk, check_colornorm, fix_kwds, savefig_or_show\n\nif TYPE_CHECKING:\n from collections.abc import Mapping, Sequence\n from typing import Literal, Self\n\n import pandas as pd\n from anndata import AnnData\n from matplotlib.axes import Axes\n from matplotlib.colors import Colormap, Normalize\n\n from .._utils import Empty\n from ._baseplot_class import _VarNames\n from ._utils import ColorLike, _AxesSubplot\n\n\n@_doc_params(common_plot_args=doc_common_plot_args)\nclass MatrixPlot(BasePlot):\n \"\"\"\\\n Allows the visualization of values using a color map.\n\n Parameters\n ----------\n {common_plot_args}\n title\n Title for the figure.\n expression_cutoff\n Expression cutoff that is used for binarizing the gene expression and\n determining the fraction of cells expressing given genes. A gene is\n expressed only if the expression value is greater than this threshold.\n mean_only_expressed\n If True, gene expression is averaged only over the cells\n expressing the given genes.\n standard_scale\n Whether or not to standardize that dimension between 0 and 1,\n meaning for each variable or group,\n subtract the minimum and divide each by its maximum.\n values_df\n Optionally, a dataframe with the values to plot can be given. The\n index should be the grouby categories and the columns the genes names.\n\n kwds\n Are passed to :func:`matplotlib.pyplot.scatter`.\n\n See also\n --------\n :func:`~scanpy.pl.matrixplot`: Simpler way to call MatrixPlot but with less options.\n :func:`~scanpy.pl.rank_genes_groups_matrixplot`: to plot marker genes identified\n using the :func:`~scanpy.tl.rank_genes_groups` function.\n\n Examples\n --------\n\n Simple visualization of the average expression of a few genes grouped by\n the category 'bulk_labels'.\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n sc.pl.MatrixPlot(adata, markers, groupby='bulk_labels').show()\n\n Same visualization but passing var_names as dict, which adds a grouping of\n the genes on top of the image:\n\n .. plot::\n :context: close-figs\n\n markers = {{'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}}\n sc.pl.MatrixPlot(adata, markers, groupby='bulk_labels').show()\n \"\"\"\n\n DEFAULT_SAVE_PREFIX = \"matrixplot_\"\n DEFAULT_COLOR_LEGEND_TITLE = \"Mean expression\\nin group\"\n\n # default style parameters\n DEFAULT_COLORMAP = rcParams[\"image.cmap\"]\n DEFAULT_EDGE_COLOR = \"gray\"\n DEFAULT_EDGE_LW = 0.1\n\n @old_positionals(\n \"use_raw\",\n \"log\",\n \"num_categories\",\n \"categories_order\",\n \"title\",\n \"figsize\",\n \"gene_symbols\",\n \"var_group_positions\",\n \"var_group_labels\",\n \"var_group_rotation\",\n \"layer\",\n \"standard_scale\",\n \"ax\",\n \"values_df\",\n \"vmin\",\n \"vmax\",\n \"vcenter\",\n \"norm\",\n )\n def __init__(\n self,\n adata: AnnData,\n var_names: _VarNames | Mapping[str, _VarNames],\n groupby: str | Sequence[str],\n *,\n use_raw: bool | None = None,\n log: bool = False,\n num_categories: int = 7,\n categories_order: Sequence[str] | None = None,\n title: str | None = None,\n figsize: tuple[float, float] | None = None,\n gene_symbols: str | None = None,\n var_group_positions: Sequence[tuple[int, int]] | None = None,\n var_group_labels: Sequence[str] | None = None,\n var_group_rotation: float | None = None,\n layer: str | None = None,\n standard_scale: Literal[\"var\", \"group\"] | None = None,\n ax: _AxesSubplot | None = None,\n values_df: pd.DataFrame | None = None,\n vmin: float | None = None,\n vmax: float | None = None,\n vcenter: float | None = None,\n norm: Normalize | None = None,\n **kwds,\n ):\n BasePlot.__init__(\n self,\n adata,\n var_names,\n groupby,\n use_raw=use_raw,\n log=log,\n num_categories=num_categories,\n categories_order=categories_order,\n title=title,\n figsize=figsize,\n gene_symbols=gene_symbols,\n var_group_positions=var_group_positions,\n var_group_labels=var_group_labels,\n var_group_rotation=var_group_rotation,\n layer=layer,\n ax=ax,\n vmin=vmin,\n vmax=vmax,\n vcenter=vcenter,\n norm=norm,\n **kwds,\n )\n\n if values_df is None:\n # compute mean value\n values_df = (\n self.obs_tidy.groupby(level=0, observed=True)\n .mean()\n .loc[\n self.categories_order\n if self.categories_order is not None\n else self.categories\n ]\n )\n\n if standard_scale == \"group\":\n values_df = values_df.sub(values_df.min(1), axis=0)\n values_df = values_df.div(values_df.max(1), axis=0).fillna(0)\n elif standard_scale == \"var\":\n values_df -= values_df.min(0)\n values_df = (values_df / values_df.max(0)).fillna(0)\n elif standard_scale is None:\n pass\n else:\n logg.warning(\"Unknown type for standard_scale, ignored\")\n\n self.values_df = values_df\n\n self.cmap = self.DEFAULT_COLORMAP\n self.edge_color = self.DEFAULT_EDGE_COLOR\n self.edge_lw = self.DEFAULT_EDGE_LW\n\n def style(\n self,\n cmap: Colormap | str | None | Empty = _empty,\n edge_color: ColorLike | None | Empty = _empty,\n edge_lw: float | None | Empty = _empty,\n ) -> Self:\n \"\"\"\\\n Modifies plot visual parameters.\n\n Parameters\n ----------\n cmap\n Matplotlib color map, specified by name or directly.\n If ``None``, use :obj:`matplotlib.rcParams`\\\\ ``[\"image.cmap\"]``\n edge_color\n Edge color between the squares of matrix plot.\n If ``None``, use :obj:`matplotlib.rcParams`\\\\ ``[\"patch.edgecolor\"]``\n edge_lw\n Edge line width.\n If ``None``, use :obj:`matplotlib.rcParams`\\\\ ``[\"lines.linewidth\"]``\n\n Returns\n -------\n :class:`~scanpy.pl.MatrixPlot`\n\n Examples\n -------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n\n adata = sc.datasets.pbmc68k_reduced()\n markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n\n Change color map and turn off edges:\n\n\n .. plot::\n :context: close-figs\n\n (\n sc.pl.MatrixPlot(adata, markers, groupby='bulk_labels')\n .style(cmap='Blues', edge_color='none')\n .show()\n )\n\n \"\"\"\n super().style(cmap=cmap)\n\n if edge_color is not _empty:\n self.edge_color = edge_color\n if edge_lw is not _empty:\n self.edge_lw = edge_lw\n\n return self\n\n def _mainplot(self, ax: Axes):\n # work on a copy of the dataframes. This is to avoid changes\n # on the original data frames after repetitive calls to the\n # MatrixPlot object, for example once with swap_axes and other without\n\n _color_df = self.values_df.copy()\n if self.var_names_idx_order is not None:\n _color_df = _color_df.iloc[:, self.var_names_idx_order]\n\n if self.categories_order is not None:\n _color_df = _color_df.loc[self.categories_order, :]\n\n if self.are_axes_swapped:\n _color_df = _color_df.T\n cmap = plt.get_cmap(self.kwds.get(\"cmap\", self.cmap))\n if \"cmap\" in self.kwds:\n del self.kwds[\"cmap\"]\n normalize = check_colornorm(\n self.vboundnorm.vmin,\n self.vboundnorm.vmax,\n self.vboundnorm.vcenter,\n self.vboundnorm.norm,\n )\n\n for axis in [\"top\", \"bottom\", \"left\", \"right\"]:\n ax.spines[axis].set_linewidth(1.5)\n\n kwds = fix_kwds(\n self.kwds,\n cmap=cmap,\n edgecolor=self.edge_color,\n linewidth=self.edge_lw,\n norm=normalize,\n )\n _ = ax.pcolor(_color_df, **kwds)\n\n y_labels = _color_df.index\n x_labels = _color_df.columns\n\n y_ticks = np.arange(len(y_labels)) + 0.5\n ax.set_yticks(y_ticks)\n ax.set_yticklabels(y_labels)\n\n x_ticks = np.arange(len(x_labels)) + 0.5\n ax.set_xticks(x_ticks)\n ax.set_xticklabels(x_labels, rotation=90, ha=\"center\", minor=False)\n\n ax.tick_params(axis=\"both\", labelsize=\"small\")\n ax.grid(visible=False)\n\n # to be consistent with the heatmap plot, is better to\n # invert the order of the y-axis, such that the first group is on\n # top\n ax.set_ylim(len(y_labels), 0)\n ax.set_xlim(0, len(x_labels))\n\n return normalize\n\n\n@old_positionals(\n \"use_raw\",\n \"log\",\n \"num_categories\",\n \"figsize\",\n \"dendrogram\",\n \"title\",\n \"cmap\",\n \"colorbar_title\",\n \"gene_symbols\",\n \"var_group_positions\",\n \"var_group_labels\",\n \"var_group_rotation\",\n \"layer\",\n \"standard_scale\",\n # 17 positionals are enough for backwards compatibility\n)\n@_doc_params(\n show_save_ax=doc_show_save_ax,\n common_plot_args=doc_common_plot_args,\n groupby_plots_args=doc_common_groupby_plot_args,\n vminmax=doc_vboundnorm,\n)\ndef matrixplot(\n adata: AnnData,\n var_names: _VarNames | Mapping[str, _VarNames],\n groupby: str | Sequence[str],\n *,\n use_raw: bool | None = None,\n log: bool = False,\n num_categories: int = 7,\n categories_order: Sequence[str] | None = None,\n figsize: tuple[float, float] | None = None,\n dendrogram: bool | str = False,\n title: str | None = None,\n cmap: Colormap | str | None = MatrixPlot.DEFAULT_COLORMAP,\n colorbar_title: str | None = MatrixPlot.DEFAULT_COLOR_LEGEND_TITLE,\n gene_symbols: str | None = None,\n var_group_positions: Sequence[tuple[int, int]] | None = None,\n var_group_labels: Sequence[str] | None = None,\n var_group_rotation: float | None = None,\n layer: str | None = None,\n standard_scale: Literal[\"var\", \"group\"] | None = None,\n values_df: pd.DataFrame | None = None,\n swap_axes: bool = False,\n show: bool | None = None,\n save: str | bool | None = None,\n ax: _AxesSubplot | None = None,\n return_fig: bool | None = False,\n vmin: float | None = None,\n vmax: float | None = None,\n vcenter: float | None = None,\n norm: Normalize | None = None,\n **kwds,\n) -> MatrixPlot | dict[str, Axes] | None:\n \"\"\"\\\n Creates a heatmap of the mean expression values per group of each var_names.\n\n This function provides a convenient interface to the :class:`~scanpy.pl.MatrixPlot`\n class. If you need more flexibility, you should use :class:`~scanpy.pl.MatrixPlot`\n directly.\n\n Parameters\n ----------\n {common_plot_args}\n {groupby_plots_args}\n {show_save_ax}\n {vminmax}\n kwds\n Are passed to :func:`matplotlib.pyplot.pcolor`.\n\n Returns\n -------\n If `return_fig` is `True`, returns a :class:`~scanpy.pl.MatrixPlot` object,\n else if `show` is false, return axes dict\n\n See also\n --------\n :class:`~scanpy.pl.MatrixPlot`: The MatrixPlot class can be used to to control\n several visual parameters not available in this function.\n :func:`~scanpy.pl.rank_genes_groups_matrixplot`: to plot marker genes\n identified using the :func:`~scanpy.tl.rank_genes_groups` function.\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n sc.pl.matrixplot(adata, markers, groupby='bulk_labels', dendrogram=True)\n\n Using var_names as dict:\n\n .. plot::\n :context: close-figs\n\n markers = {{'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}}\n sc.pl.matrixplot(adata, markers, groupby='bulk_labels', dendrogram=True)\n\n Get Matrix object for fine tuning:\n\n .. plot::\n :context: close-figs\n\n mp = sc.pl.matrixplot(adata, markers, 'bulk_labels', return_fig=True)\n mp.add_totals().style(edge_color='black').show()\n\n The axes used can be obtained using the get_axes() method\n\n .. plot::\n :context: close-figs\n\n axes_dict = mp.get_axes()\n \"\"\"\n\n mp = MatrixPlot(\n adata,\n var_names,\n groupby=groupby,\n use_raw=use_raw,\n log=log,\n num_categories=num_categories,\n categories_order=categories_order,\n standard_scale=standard_scale,\n title=title,\n figsize=figsize,\n gene_symbols=gene_symbols,\n var_group_positions=var_group_positions,\n var_group_labels=var_group_labels,\n var_group_rotation=var_group_rotation,\n layer=layer,\n values_df=values_df,\n ax=ax,\n vmin=vmin,\n vmax=vmax,\n vcenter=vcenter,\n norm=norm,\n **kwds,\n )\n\n if dendrogram:\n mp.add_dendrogram(dendrogram_key=_dk(dendrogram))\n if swap_axes:\n mp.swap_axes()\n\n mp = mp.style(cmap=cmap).legend(title=colorbar_title)\n if return_fig:\n return mp\n mp.make_figure()\n savefig_or_show(MatrixPlot.DEFAULT_SAVE_PREFIX, show=show, save=save)\n show = settings.autoshow if show is None else show\n if show:\n return None\n return mp.get_axes()\n\n\n\"\"\"Set the default matplotlib.rcParams.\"\"\"\n\nfrom __future__ import annotations\n\nimport matplotlib as mpl\nfrom cycler import cycler\nfrom matplotlib import rcParams\n\nfrom . import palettes\n\n\ndef set_rcParams_scanpy(fontsize=14, color_map=None):\n \"\"\"Set matplotlib.rcParams to Scanpy defaults.\n\n Call this through `settings.set_figure_params`.\n \"\"\"\n\n # figure\n rcParams[\"figure.figsize\"] = (4, 4)\n rcParams[\"figure.subplot.left\"] = 0.18\n rcParams[\"figure.subplot.right\"] = 0.96\n rcParams[\"figure.subplot.bottom\"] = 0.15\n rcParams[\"figure.subplot.top\"] = 0.91\n\n rcParams[\"lines.linewidth\"] = 1.5 # the line width of the frame\n rcParams[\"lines.markersize\"] = 6\n rcParams[\"lines.markeredgewidth\"] = 1\n\n # font\n rcParams[\"font.sans-serif\"] = [\n \"Arial\",\n \"Helvetica\",\n \"DejaVu Sans\",\n \"Bitstream Vera Sans\",\n \"sans-serif\",\n ]\n fontsize = fontsize\n rcParams[\"font.size\"] = fontsize\n rcParams[\"legend.fontsize\"] = 0.92 * fontsize\n rcParams[\"axes.titlesize\"] = fontsize\n rcParams[\"axes.labelsize\"] = fontsize\n\n # legend\n rcParams[\"legend.numpoints\"] = 1\n rcParams[\"legend.scatterpoints\"] = 1\n rcParams[\"legend.handlelength\"] = 0.5\n rcParams[\"legend.handletextpad\"] = 0.4\n\n # color cycle\n rcParams[\"axes.prop_cycle\"] = cycler(color=palettes.default_20)\n\n # lines\n rcParams[\"axes.linewidth\"] = 0.8\n rcParams[\"axes.edgecolor\"] = \"black\"\n rcParams[\"axes.facecolor\"] = \"white\"\n\n # ticks\n rcParams[\"xtick.color\"] = \"k\"\n rcParams[\"ytick.color\"] = \"k\"\n rcParams[\"xtick.labelsize\"] = fontsize\n rcParams[\"ytick.labelsize\"] = fontsize\n\n # axes grid\n rcParams[\"axes.grid\"] = True\n rcParams[\"grid.color\"] = \".8\"\n\n # color map\n rcParams[\"image.cmap\"] = rcParams[\"image.cmap\"] if color_map is None else color_map\n\n\ndef set_rcParams_defaults():\n \"\"\"Reset `matplotlib.rcParams` to defaults.\"\"\"\n rcParams.update(mpl.rcParamsDefault)\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom matplotlib import pyplot as plt\n\nfrom .._compat import old_positionals\nfrom .._settings import settings\nfrom .._utils import _doc_params\nfrom ..preprocessing._normalization import normalize_total\nfrom . import _utils\nfrom ._docs import doc_show_save_ax\n\nif TYPE_CHECKING:\n from anndata import AnnData\n from matplotlib.axes import Axes\n\n\n@old_positionals(\"show\", \"save\", \"ax\", \"gene_symbols\", \"log\")\n@_doc_params(show_save_ax=doc_show_save_ax)\ndef highest_expr_genes(\n adata: AnnData,\n n_top: int = 30,\n *,\n show: bool | None = None,\n save: str | bool | None = None,\n ax: Axes | None = None,\n gene_symbols: str | None = None,\n log: bool = False,\n **kwds,\n):\n \"\"\"\\\n Fraction of counts assigned to each gene over all cells.\n\n Computes, for each gene, the fraction of counts assigned to that gene within\n a cell. The `n_top` genes with the highest mean fraction over all cells are\n plotted as boxplots.\n\n This plot is similar to the `scater` package function `plotHighestExprs(type\n = \"highest-expression\")`, see `here\n `__. Quoting\n from there:\n\n *We expect to see the “usual suspects”, i.e., mitochondrial genes, actin,\n ribosomal protein, MALAT1. A few spike-in transcripts may also be\n present here, though if all of the spike-ins are in the top 50, it\n suggests that too much spike-in RNA was added. A large number of\n pseudo-genes or predicted genes may indicate problems with alignment.*\n -- Davis McCarthy and Aaron Lun\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n n_top\n Number of top\n {show_save_ax}\n gene_symbols\n Key for field in .var that stores gene symbols if you do not want to use .var_names.\n log\n Plot x-axis in log scale\n **kwds\n Are passed to :func:`~seaborn.boxplot`.\n\n Returns\n -------\n If `show==False` a :class:`~matplotlib.axes.Axes`.\n \"\"\"\n import seaborn as sns # Slow import, only import if called\n from scipy.sparse import issparse\n\n # compute the percentage of each gene per cell\n norm_dict = normalize_total(adata, target_sum=100, inplace=False)\n\n # identify the genes with the highest mean\n if issparse(norm_dict[\"X\"]):\n mean_percent = norm_dict[\"X\"].mean(axis=0).A1\n top_idx = np.argsort(mean_percent)[::-1][:n_top]\n counts_top_genes = norm_dict[\"X\"][:, top_idx].toarray()\n else:\n mean_percent = norm_dict[\"X\"].mean(axis=0)\n top_idx = np.argsort(mean_percent)[::-1][:n_top]\n counts_top_genes = norm_dict[\"X\"][:, top_idx]\n columns = (\n adata.var_names[top_idx]\n if gene_symbols is None\n else adata.var[gene_symbols][top_idx]\n )\n counts_top_genes = pd.DataFrame(\n counts_top_genes, index=adata.obs_names, columns=columns\n )\n\n if not ax:\n # figsize is hardcoded to produce a tall image. To change the fig size,\n # a matplotlib.axes.Axes object needs to be passed.\n height = (n_top * 0.2) + 1.5\n fig, ax = plt.subplots(figsize=(5, height))\n sns.boxplot(data=counts_top_genes, orient=\"h\", ax=ax, fliersize=1, **kwds)\n ax.set_xlabel(\"% of total counts\")\n if log:\n ax.set_xscale(\"log\")\n show = settings.autoshow if show is None else show\n _utils.savefig_or_show(\"highest_expr_genes\", show=show, save=save)\n if show:\n return None\n return ax\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom collections.abc import Mapping, Sequence\nfrom typing import TYPE_CHECKING, Callable, Literal, TypedDict, Union, overload\n\nimport matplotlib as mpl\nimport numpy as np\nfrom cycler import Cycler, cycler\nfrom matplotlib import axes, gridspec, rcParams, ticker\nfrom matplotlib import pyplot as plt\nfrom matplotlib.axes import Axes\nfrom matplotlib.collections import PatchCollection\nfrom matplotlib.colors import is_color_like\nfrom matplotlib.figure import SubplotParams as sppars\nfrom matplotlib.patches import Circle\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._settings import settings\nfrom .._utils import NeighborsView, _empty\nfrom . import palettes\n\nif TYPE_CHECKING:\n from collections.abc import Collection\n\n from anndata import AnnData\n from matplotlib.colors import Colormap\n from matplotlib.figure import Figure\n from matplotlib.typing import MarkerType\n from numpy.typing import ArrayLike\n from PIL.Image import Image\n\n from .._utils import Empty\n\n # TODO: more\n DensityNorm = Literal[\"area\", \"count\", \"width\"]\n\n# These are needed by _wraps_plot_scatter\nVBound = Union[str, float, Callable[[Sequence[float]], float]]\n_FontWeight = Literal[\"light\", \"normal\", \"medium\", \"semibold\", \"bold\", \"heavy\", \"black\"]\n_FontSize = Literal[\n \"xx-small\", \"x-small\", \"small\", \"medium\", \"large\", \"x-large\", \"xx-large\"\n]\n_LegendLoc = Literal[\n \"none\",\n \"right margin\",\n \"on data\",\n \"on data export\",\n \"best\",\n \"upper right\",\n \"upper left\",\n \"lower left\",\n \"lower right\",\n \"right\",\n \"center left\",\n \"center right\",\n \"lower center\",\n \"upper center\",\n \"center\",\n]\nColorLike = Union[str, tuple[float, ...]]\n\n\nclass _AxesSubplot(Axes, axes.SubplotBase):\n \"\"\"Intersection between Axes and SubplotBase: Has methods of both\"\"\"\n\n\n# -------------------------------------------------------------------------------\n# Simple plotting functions\n# -------------------------------------------------------------------------------\n\n\n@old_positionals(\n \"xlabel\",\n \"ylabel\",\n \"xticks\",\n \"yticks\",\n \"title\",\n \"colorbar_shrink\",\n \"color_map\",\n \"show\",\n \"save\",\n \"ax\",\n)\ndef matrix(\n matrix: ArrayLike | Image,\n *,\n xlabel: str | None = None,\n ylabel: str | None = None,\n xticks: Collection[str] | None = None,\n yticks: Collection[str] | None = None,\n title: str | None = None,\n colorbar_shrink: float = 0.5,\n color_map: str | Colormap | None = None,\n show: bool | None = None,\n save: bool | str | None = None,\n ax: Axes | None = None,\n) -> None:\n \"\"\"Plot a matrix.\"\"\"\n if ax is None:\n ax = plt.gca()\n img = ax.imshow(matrix, cmap=color_map)\n if xlabel is not None:\n ax.set_xlabel(xlabel)\n if ylabel is not None:\n ax.set_ylabel(ylabel)\n if title is not None:\n ax.set_title(title)\n if xticks is not None:\n ax.set_xticks(range(len(xticks)), xticks, rotation=\"vertical\")\n if yticks is not None:\n ax.set_yticks(range(len(yticks)), yticks)\n plt.colorbar(\n img, shrink=colorbar_shrink, ax=ax\n ) # need a figure instance for colorbar\n savefig_or_show(\"matrix\", show=show, save=save)\n\n\ndef timeseries(X, **kwargs):\n \"\"\"Plot X. See timeseries_subplot.\"\"\"\n plt.figure(\n figsize=tuple(2 * s for s in rcParams[\"figure.figsize\"]),\n subplotpars=sppars(left=0.12, right=0.98, bottom=0.13),\n )\n timeseries_subplot(X, **kwargs)\n\n\ndef timeseries_subplot(\n X: np.ndarray,\n *,\n time=None,\n color=None,\n var_names=(),\n highlights_x=(),\n xlabel=\"\",\n ylabel=\"gene expression\",\n yticks=None,\n xlim=None,\n legend=True,\n palette: Sequence[str] | Cycler | None = None,\n color_map=\"viridis\",\n ax: Axes | None = None,\n marker: str | Sequence[str] = \".\",\n):\n \"\"\"\\\n Plot X.\n\n Parameters\n ----------\n X\n Call this with:\n X with one column, color categorical.\n X with one column, color continuous.\n X with n columns, color is of length n.\n \"\"\"\n\n if color is not None:\n use_color_map = isinstance(color[0], (float, np.floating))\n palette = default_palette(palette)\n x_range = np.arange(X.shape[0]) if time is None else time\n if X.ndim == 1:\n X = X[:, None]\n if X.shape[1] > 1:\n colors = palette[: X.shape[1]].by_key()[\"color\"]\n subsets = [(x_range, X[:, i]) for i in range(X.shape[1])]\n elif use_color_map:\n colors = [color]\n subsets = [(x_range, X[:, 0])]\n else:\n levels, _ = np.unique(color, return_inverse=True)\n colors = np.array(palette[: len(levels)].by_key()[\"color\"])\n subsets = [(x_range[color == level], X[color == level, :]) for level in levels]\n\n if isinstance(marker, str):\n marker = [marker]\n if len(marker) != len(subsets) and len(marker) == 1:\n marker = [marker[0] for _ in range(len(subsets))]\n\n if ax is None:\n ax = plt.subplot()\n for i, (x, y) in enumerate(subsets):\n ax.scatter(\n x,\n y,\n marker=marker[i],\n edgecolor=\"face\",\n s=rcParams[\"lines.markersize\"],\n c=colors[i],\n label=var_names[i] if len(var_names) > 0 else \"\",\n cmap=color_map,\n rasterized=settings._vector_friendly,\n )\n ylim = ax.get_ylim()\n for h in highlights_x:\n ax.plot([h, h], [ylim[0], ylim[1]], \"--\", color=\"black\")\n ax.set_ylim(ylim)\n if xlim is not None:\n ax.set_xlim(xlim)\n ax.set_xlabel(xlabel)\n ax.set_ylabel(ylabel)\n if yticks is not None:\n ax.set_yticks(yticks)\n if len(var_names) > 0 and legend:\n ax.legend(frameon=False)\n\n\ndef timeseries_as_heatmap(\n X: np.ndarray, *, var_names: Collection[str] = (), highlights_x=(), color_map=None\n):\n \"\"\"\\\n Plot timeseries as heatmap.\n\n Parameters\n ----------\n X\n Data array.\n var_names\n Array of strings naming variables stored in columns of X.\n \"\"\"\n if len(var_names) == 0:\n var_names = np.arange(X.shape[1])\n if var_names.ndim == 2:\n var_names = var_names[:, 0]\n\n # transpose X\n X = X.T\n min_x = np.min(X)\n\n # insert space into X\n if False:\n # generate new array with highlights_x\n space = 10 # integer\n x_new = np.zeros((X.shape[0], X.shape[1] + space * len(highlights_x)))\n hold = 0\n _hold = 0\n space_sum = 0\n for ih, h in enumerate(highlights_x):\n _h = h + space_sum\n x_new[:, _hold:_h] = X[:, hold:h]\n x_new[:, _h : _h + space] = min_x * np.ones((X.shape[0], space))\n # update variables\n space_sum += space\n _hold = _h + space\n hold = h\n x_new[:, _hold:] = X[:, hold:]\n\n _, ax = plt.subplots(figsize=(1.5 * 4, 2 * 4))\n img = ax.imshow(\n np.array(X, dtype=np.float64),\n aspect=\"auto\",\n interpolation=\"nearest\",\n cmap=color_map,\n )\n plt.colorbar(img, shrink=0.5)\n plt.yticks(range(X.shape[0]), var_names)\n for h in highlights_x:\n plt.plot([h, h], [0, X.shape[0]], \"--\", color=\"black\")\n plt.xlim([0, X.shape[1] - 1])\n plt.ylim([0, X.shape[0] - 1])\n\n\n# -------------------------------------------------------------------------------\n# Colors in addition to matplotlib's colors\n# -------------------------------------------------------------------------------\n\n\nadditional_colors = {\n \"gold2\": \"#eec900\",\n \"firebrick3\": \"#cd2626\",\n \"khaki2\": \"#eee685\",\n \"slategray3\": \"#9fb6cd\",\n \"palegreen3\": \"#7ccd7c\",\n \"tomato2\": \"#ee5c42\",\n \"grey80\": \"#cccccc\",\n \"grey90\": \"#e5e5e5\",\n \"wheat4\": \"#8b7e66\",\n \"grey65\": \"#a6a6a6\",\n \"grey10\": \"#1a1a1a\",\n \"grey20\": \"#333333\",\n \"grey50\": \"#7f7f7f\",\n \"grey30\": \"#4d4d4d\",\n \"grey40\": \"#666666\",\n \"antiquewhite2\": \"#eedfcc\",\n \"grey77\": \"#c4c4c4\",\n \"snow4\": \"#8b8989\",\n \"chartreuse3\": \"#66cd00\",\n \"yellow4\": \"#8b8b00\",\n \"darkolivegreen2\": \"#bcee68\",\n \"olivedrab3\": \"#9acd32\",\n \"azure3\": \"#c1cdcd\",\n \"violetred\": \"#d02090\",\n \"mediumpurple3\": \"#8968cd\",\n \"purple4\": \"#551a8b\",\n \"seagreen4\": \"#2e8b57\",\n \"lightblue3\": \"#9ac0cd\",\n \"orchid3\": \"#b452cd\",\n \"indianred 3\": \"#cd5555\",\n \"grey60\": \"#999999\",\n \"mediumorchid1\": \"#e066ff\",\n \"plum3\": \"#cd96cd\",\n \"palevioletred3\": \"#cd6889\",\n}\n\n# -------------------------------------------------------------------------------\n# Helper functions\n# -------------------------------------------------------------------------------\n\n\ndef savefig(writekey, dpi=None, ext=None):\n \"\"\"Save current figure to file.\n\n The `filename` is generated as follows:\n\n filename = settings.figdir / (writekey + settings.plot_suffix + '.' + settings.file_format_figs)\n \"\"\"\n if dpi is None:\n # we need this as in notebooks, the internal figures are also influenced by 'savefig.dpi' this...\n if (\n not isinstance(rcParams[\"savefig.dpi\"], str)\n and rcParams[\"savefig.dpi\"] < 150\n ):\n if settings._low_resolution_warning:\n logg.warning(\n \"You are using a low resolution (dpi<150) for saving figures.\\n\"\n \"Consider running `set_figure_params(dpi_save=...)`, which will \"\n \"adjust `matplotlib.rcParams['savefig.dpi']`\"\n )\n settings._low_resolution_warning = False\n else:\n dpi = rcParams[\"savefig.dpi\"]\n settings.figdir.mkdir(parents=True, exist_ok=True)\n if ext is None:\n ext = settings.file_format_figs\n filename = settings.figdir / f\"{writekey}{settings.plot_suffix}.{ext}\"\n # output the following msg at warning level; it's really important for the user\n logg.warning(f\"saving figure to file {filename}\")\n plt.savefig(filename, dpi=dpi, bbox_inches=\"tight\")\n\n\ndef savefig_or_show(\n writekey: str,\n show: bool | None = None,\n dpi: int | None = None,\n ext: str | None = None,\n save: bool | str | None = None,\n):\n if isinstance(save, str):\n # check whether `save` contains a figure extension\n if ext is None:\n for try_ext in [\".svg\", \".pdf\", \".png\"]:\n if save.endswith(try_ext):\n ext = try_ext[1:]\n save = save.replace(try_ext, \"\")\n break\n # append it\n writekey += save\n save = True\n save = settings.autosave if save is None else save\n show = settings.autoshow if show is None else show\n if save:\n savefig(writekey, dpi=dpi, ext=ext)\n if show:\n plt.show()\n if save:\n plt.close() # clear figure\n\n\ndef default_palette(\n palette: str | Sequence[str] | Cycler | None = None,\n) -> str | Cycler:\n if palette is None:\n return rcParams[\"axes.prop_cycle\"]\n elif not isinstance(palette, (str, Cycler)):\n return cycler(color=palette)\n else:\n return palette\n\n\ndef _validate_palette(adata: AnnData, key: str) -> None:\n \"\"\"\n checks if the list of colors in adata.uns[f'{key}_colors'] is valid\n and updates the color list in adata.uns[f'{key}_colors'] if needed.\n\n Not only valid matplotlib colors are checked but also if the color name\n is a valid R color name, in which case it will be translated to a valid name\n \"\"\"\n\n _palette = []\n color_key = f\"{key}_colors\"\n\n for color in adata.uns[color_key]:\n if not is_color_like(color):\n # check if the color is a valid R color and translate it\n # to a valid hex color value\n if color in additional_colors:\n color = additional_colors[color]\n else:\n logg.warning(\n f\"The following color value found in adata.uns['{key}_colors'] \"\n f\"is not valid: '{color}'. Default colors will be used instead.\"\n )\n _set_default_colors_for_categorical_obs(adata, key)\n _palette = None\n break\n _palette.append(color)\n # Don’t modify if nothing changed\n if _palette is None or np.array_equal(_palette, adata.uns[color_key]):\n return\n adata.uns[color_key] = _palette\n\n\ndef _set_colors_for_categorical_obs(\n adata, value_to_plot, palette: str | Sequence[str] | Cycler\n):\n \"\"\"\n Sets the adata.uns[value_to_plot + '_colors'] according to the given palette\n\n Parameters\n ----------\n adata\n annData object\n value_to_plot\n name of a valid categorical observation\n palette\n Palette should be either a valid :func:`~matplotlib.pyplot.colormaps` string,\n a sequence of colors (in a format that can be understood by matplotlib,\n eg. RGB, RGBS, hex, or a cycler object with key='color'\n\n Returns\n -------\n None\n \"\"\"\n from matplotlib.colors import to_hex\n\n if adata.obs[value_to_plot].dtype == bool:\n categories = (\n adata.obs[value_to_plot].astype(str).astype(\"category\").cat.categories\n )\n else:\n categories = adata.obs[value_to_plot].cat.categories\n # check is palette is a valid matplotlib colormap\n if isinstance(palette, str) and palette in plt.colormaps():\n # this creates a palette from a colormap. E.g. 'Accent, Dark2, tab20'\n cmap = plt.get_cmap(palette)\n colors_list = [to_hex(x) for x in cmap(np.linspace(0, 1, len(categories)))]\n elif isinstance(palette, Mapping):\n colors_list = [to_hex(palette[k], keep_alpha=True) for k in categories]\n else:\n # check if palette is a list and convert it to a cycler, thus\n # it doesnt matter if the list is shorter than the categories length:\n if isinstance(palette, Sequence):\n if len(palette) < len(categories):\n logg.warning(\n \"Length of palette colors is smaller than the number of \"\n f\"categories (palette length: {len(palette)}, \"\n f\"categories length: {len(categories)}. \"\n \"Some categories will have the same color.\"\n )\n # check that colors are valid\n _color_list = []\n for color in palette:\n if not is_color_like(color):\n # check if the color is a valid R color and translate it\n # to a valid hex color value\n if color in additional_colors:\n color = additional_colors[color]\n else:\n raise ValueError(\n \"The following color value of the given palette \"\n f\"is not valid: {color}\"\n )\n _color_list.append(color)\n\n palette = cycler(color=_color_list)\n if not isinstance(palette, Cycler):\n raise ValueError(\n \"Please check that the value of 'palette' is a valid \"\n \"matplotlib colormap string (eg. Set2), a list of color names \"\n \"or a cycler with a 'color' key.\"\n )\n if \"color\" not in palette.keys:\n raise ValueError(\"Please set the palette key 'color'.\")\n\n cc = palette()\n colors_list = [to_hex(next(cc)[\"color\"]) for x in range(len(categories))]\n\n adata.uns[value_to_plot + \"_colors\"] = colors_list\n\n\ndef _set_default_colors_for_categorical_obs(adata, value_to_plot):\n \"\"\"\n Sets the adata.uns[value_to_plot + '_colors'] using default color palettes\n\n Parameters\n ----------\n adata\n AnnData object\n value_to_plot\n Name of a valid categorical observation\n\n Returns\n -------\n None\n \"\"\"\n if adata.obs[value_to_plot].dtype == bool:\n categories = (\n adata.obs[value_to_plot].astype(str).astype(\"category\").cat.categories\n )\n else:\n categories = adata.obs[value_to_plot].cat.categories\n\n length = len(categories)\n\n # check if default matplotlib palette has enough colors\n if len(rcParams[\"axes.prop_cycle\"].by_key()[\"color\"]) >= length:\n cc = rcParams[\"axes.prop_cycle\"]()\n palette = [next(cc)[\"color\"] for _ in range(length)]\n\n else:\n if length <= 20:\n palette = palettes.default_20\n elif length <= 28:\n palette = palettes.default_28\n elif length <= len(palettes.default_102): # 103 colors\n palette = palettes.default_102\n else:\n palette = [\"grey\" for _ in range(length)]\n logg.info(\n f\"the obs value {value_to_plot!r} has more than 103 categories. Uniform \"\n \"'grey' color will be used for all categories.\"\n )\n\n _set_colors_for_categorical_obs(adata, value_to_plot, palette[:length])\n\n\ndef add_colors_for_categorical_sample_annotation(\n adata, key, *, palette=None, force_update_colors=False\n):\n color_key = f\"{key}_colors\"\n colors_needed = len(adata.obs[key].cat.categories)\n if palette and force_update_colors:\n _set_colors_for_categorical_obs(adata, key, palette)\n elif color_key in adata.uns and len(adata.uns[color_key]) <= colors_needed:\n _validate_palette(adata, key)\n else:\n _set_default_colors_for_categorical_obs(adata, key)\n\n\ndef plot_edges(axs, adata, basis, edges_width, edges_color, *, neighbors_key=None):\n import networkx as nx\n\n if not isinstance(axs, Sequence):\n axs = [axs]\n\n if neighbors_key is None:\n neighbors_key = \"neighbors\"\n if neighbors_key not in adata.uns:\n raise ValueError(\"`edges=True` requires `pp.neighbors` to be run before.\")\n neighbors = NeighborsView(adata, neighbors_key)\n g = nx.Graph(neighbors[\"connectivities\"])\n basis_key = _get_basis(adata, basis)\n\n with warnings.catch_warnings():\n warnings.simplefilter(\"ignore\")\n for ax in axs:\n edge_collection = nx.draw_networkx_edges(\n g,\n adata.obsm[basis_key],\n ax=ax,\n width=edges_width,\n edge_color=edges_color,\n )\n edge_collection.set_zorder(-2)\n edge_collection.set_rasterized(settings._vector_friendly)\n\n\ndef plot_arrows(axs, adata, basis, arrows_kwds=None):\n if not isinstance(axs, Sequence):\n axs = [axs]\n v_prefix = next(\n (p for p in [\"velocity\", \"Delta\"] if f\"{p}_{basis}\" in adata.obsm), None\n )\n if v_prefix is None:\n raise ValueError(\n \"`arrows=True` requires \"\n f\"`'velocity_{basis}'` from scvelo or \"\n f\"`'Delta_{basis}'` from velocyto.\"\n )\n if v_prefix == \"velocity\":\n logg.warning(\n \"The module `scvelo` has improved plotting facilities. \"\n \"Prefer using `scv.pl.velocity_embedding` to `arrows=True`.\"\n )\n\n basis_key = _get_basis(adata, basis)\n X = adata.obsm[basis_key]\n V = adata.obsm[f\"{v_prefix}_{basis}\"]\n for ax in axs:\n quiver_kwds = arrows_kwds if arrows_kwds is not None else {}\n ax.quiver(\n X[:, 0],\n X[:, 1],\n V[:, 0],\n V[:, 1],\n **quiver_kwds,\n rasterized=settings._vector_friendly,\n )\n\n\ndef scatter_group(\n ax: Axes,\n key: str,\n cat_code: int,\n adata: AnnData,\n Y: np.ndarray,\n *,\n projection: Literal[\"2d\", \"3d\"] = \"2d\",\n size: int = 3,\n alpha: float | None = None,\n marker: MarkerType = \".\",\n):\n \"\"\"Scatter of group using representation of data Y.\"\"\"\n mask_obs = adata.obs[key].cat.categories[cat_code] == adata.obs[key].values\n color = adata.uns[key + \"_colors\"][cat_code]\n if not isinstance(color[0], str):\n from matplotlib.colors import rgb2hex\n\n color = rgb2hex(adata.uns[key + \"_colors\"][cat_code])\n if not is_color_like(color):\n raise ValueError(f'\"{color}\" is not a valid matplotlib color.')\n data = [Y[mask_obs, 0], Y[mask_obs, 1]]\n if projection == \"3d\":\n data.append(Y[mask_obs, 2])\n ax.scatter(\n *data,\n marker=marker,\n alpha=alpha,\n c=color,\n edgecolors=\"none\",\n s=size,\n label=adata.obs[key].cat.categories[cat_code],\n rasterized=settings._vector_friendly,\n )\n return mask_obs\n\n\ndef setup_axes(\n ax: Axes | Sequence[Axes] | None = None,\n *,\n panels=\"blue\",\n colorbars=(False,),\n right_margin=None,\n left_margin=None,\n projection: Literal[\"2d\", \"3d\"] = \"2d\",\n show_ticks=False,\n):\n \"\"\"Grid of axes for plotting, legends and colorbars.\"\"\"\n check_projection(projection)\n if left_margin is not None:\n raise NotImplementedError(\"We currently don’t support `left_margin`.\")\n if np.any(colorbars) and right_margin is None:\n right_margin = 1 - rcParams[\"figure.subplot.right\"] + 0.21 # 0.25\n elif right_margin is None:\n right_margin = 1 - rcParams[\"figure.subplot.right\"] + 0.06 # 0.10\n # make a list of right margins for each panel\n if not isinstance(right_margin, list):\n right_margin_list = [right_margin for i in range(len(panels))]\n else:\n right_margin_list = right_margin\n\n # make a figure with len(panels) panels in a row side by side\n top_offset = 1 - rcParams[\"figure.subplot.top\"]\n bottom_offset = 0.15 if show_ticks else 0.08\n left_offset = 1 if show_ticks else 0.3 # in units of base_height\n base_height = rcParams[\"figure.figsize\"][1]\n height = base_height\n base_width = rcParams[\"figure.figsize\"][0]\n if show_ticks:\n base_width *= 1.1\n\n draw_region_width = (\n base_width - left_offset - top_offset - 0.5\n ) # this is kept constant throughout\n\n right_margin_factor = sum([1 + right_margin for right_margin in right_margin_list])\n width_without_offsets = (\n right_margin_factor * draw_region_width\n ) # this is the total width that keeps draw_region_width\n\n right_offset = (len(panels) - 1) * left_offset\n figure_width = width_without_offsets + left_offset + right_offset\n draw_region_width_frac = draw_region_width / figure_width\n left_offset_frac = left_offset / figure_width\n right_offset_frac = ( # noqa: F841 # TODO Does this need fixing?\n 1 - (len(panels) - 1) * left_offset_frac\n )\n\n if ax is None:\n plt.figure(\n figsize=(figure_width, height),\n subplotpars=sppars(left=0, right=1, bottom=bottom_offset),\n )\n left_positions = [left_offset_frac, left_offset_frac + draw_region_width_frac]\n for i in range(1, len(panels)):\n right_margin = right_margin_list[i - 1]\n left_positions.append(\n left_positions[-1] + right_margin * draw_region_width_frac\n )\n left_positions.append(left_positions[-1] + draw_region_width_frac)\n panel_pos = [[bottom_offset], [1 - top_offset], left_positions]\n\n axs = []\n if ax is None:\n for icolor, color in enumerate(panels):\n left = panel_pos[2][2 * icolor]\n bottom = panel_pos[0][0]\n width = draw_region_width / figure_width\n height = panel_pos[1][0] - bottom\n if projection == \"2d\":\n ax = plt.axes([left, bottom, width, height])\n elif projection == \"3d\":\n ax = plt.axes([left, bottom, width, height], projection=\"3d\")\n axs.append(ax)\n else:\n axs = ax if isinstance(ax, Sequence) else [ax]\n\n return axs, panel_pos, draw_region_width, figure_width\n\n\ndef scatter_base(\n Y: np.ndarray,\n *,\n colors: str | Sequence[ColorLike | np.ndarray] = \"blue\",\n sort_order=True,\n alpha=None,\n highlights=(),\n right_margin=None,\n left_margin=None,\n projection: Literal[\"2d\", \"3d\"] = \"2d\",\n title=None,\n component_name=\"DC\",\n component_indexnames=(1, 2, 3),\n axis_labels=None,\n colorbars=(False,),\n sizes=(1,),\n markers=\".\",\n color_map=\"viridis\",\n show_ticks=True,\n ax=None,\n) -> Axes | list[Axes]:\n \"\"\"Plot scatter plot of data.\n\n Parameters\n ----------\n Y\n Data array.\n projection\n\n Returns\n -------\n Depending on whether supplying a single array or a list of arrays,\n return a single axis or a list of axes.\n \"\"\"\n if isinstance(highlights, Mapping):\n highlights_indices = sorted(highlights)\n highlights_labels = [highlights[i] for i in highlights_indices]\n else:\n highlights_indices = highlights\n highlights_labels = []\n # if we have a single array, transform it into a list with a single array\n if isinstance(colors, str):\n colors = [colors]\n if isinstance(markers, str):\n markers = [markers]\n if len(sizes) != len(colors) and len(sizes) == 1:\n sizes = [sizes[0] for _ in range(len(colors))]\n if len(markers) != len(colors) and len(markers) == 1:\n markers = [markers[0] for _ in range(len(colors))]\n axs, panel_pos, draw_region_width, figure_width = setup_axes(\n ax,\n panels=colors,\n colorbars=colorbars,\n projection=projection,\n right_margin=right_margin,\n left_margin=left_margin,\n show_ticks=show_ticks,\n )\n for icolor, color in enumerate(colors):\n ax = axs[icolor]\n marker = markers[icolor]\n bottom = panel_pos[0][0]\n height = panel_pos[1][0] - bottom\n Y_sort = Y\n if not is_color_like(color) and sort_order:\n sort = np.argsort(color)\n color = color[sort]\n Y_sort = Y[sort]\n if projection == \"2d\":\n data = Y_sort[:, 0], Y_sort[:, 1]\n elif projection == \"3d\":\n data = Y_sort[:, 0], Y_sort[:, 1], Y_sort[:, 2]\n else:\n raise ValueError(f\"Unknown projection {projection!r} not in '2d', '3d'\")\n if not isinstance(color, str) or color != \"white\":\n sct = ax.scatter(\n *data,\n marker=marker,\n c=color,\n alpha=alpha,\n edgecolors=\"none\", # 'face',\n s=sizes[icolor],\n cmap=color_map,\n rasterized=settings._vector_friendly,\n )\n if colorbars[icolor]:\n width = 0.006 * draw_region_width / len(colors)\n left = (\n panel_pos[2][2 * icolor + 1]\n + (1.2 if projection == \"3d\" else 0.2) * width\n )\n rectangle = [left, bottom, width, height]\n fig = plt.gcf()\n ax_cb = fig.add_axes(rectangle)\n _ = plt.colorbar(\n sct, format=ticker.FuncFormatter(ticks_formatter), cax=ax_cb\n )\n # set the title\n if title is not None:\n ax.set_title(title[icolor])\n # output highlighted data points\n for iihighlight, ihighlight in enumerate(highlights_indices):\n ihighlight = ihighlight if isinstance(ihighlight, int) else int(ihighlight)\n data = [Y[ihighlight, 0]], [Y[ihighlight, 1]]\n if \"3d\" in projection:\n data = [Y[ihighlight, 0]], [Y[ihighlight, 1]], [Y[ihighlight, 2]]\n ax.scatter(\n *data,\n c=\"black\",\n facecolors=\"black\",\n edgecolors=\"black\",\n marker=\"x\",\n s=10,\n zorder=20,\n )\n highlight_text = (\n highlights_labels[iihighlight]\n if len(highlights_labels) > 0\n else str(ihighlight)\n )\n # the following is a Python 2 compatibility hack\n ax.text(\n *([d[0] for d in data] + [highlight_text]),\n zorder=20,\n fontsize=10,\n color=\"black\",\n )\n if not show_ticks:\n ax.set_xticks([])\n ax.set_yticks([])\n if \"3d\" in projection:\n ax.set_zticks([])\n # set default axis_labels\n if axis_labels is None:\n axis_labels = [\n [component_name + str(i) for i in component_indexnames]\n for _ in range(len(axs))\n ]\n else:\n axis_labels = [axis_labels for _ in range(len(axs))]\n for iax, ax in enumerate(axs):\n ax.set_xlabel(axis_labels[iax][0])\n ax.set_ylabel(axis_labels[iax][1])\n if \"3d\" in projection:\n # shift the label closer to the axis\n ax.set_zlabel(axis_labels[iax][2], labelpad=-7)\n for ax in axs:\n # scale limits to match data\n ax.autoscale_view()\n return axs\n\n\ndef scatter_single(ax: Axes, Y: np.ndarray, *args, **kwargs):\n \"\"\"Plot scatter plot of data.\n\n Parameters\n ----------\n ax\n Axis to plot on.\n Y\n Data array, data to be plotted needs to be in the first two columns.\n \"\"\"\n if \"s\" not in kwargs:\n kwargs[\"s\"] = 2 if Y.shape[0] > 500 else 10\n if \"edgecolors\" not in kwargs:\n kwargs[\"edgecolors\"] = \"face\"\n ax.scatter(Y[:, 0], Y[:, 1], **kwargs, rasterized=settings._vector_friendly)\n ax.set_xticks([])\n ax.set_yticks([])\n\n\ndef arrows_transitions(ax: Axes, X: np.ndarray, indices: Sequence[int], weight=None):\n \"\"\"\n Plot arrows of transitions in data matrix.\n\n Parameters\n ----------\n ax\n Axis object from matplotlib.\n X\n Data array, any representation wished (X, psi, phi, etc).\n indices\n Indices storing the transitions.\n \"\"\"\n step = 1\n width = axis_to_data(ax, 0.001)\n if X.shape[0] > 300:\n step = 5\n width = axis_to_data(ax, 0.0005)\n if X.shape[0] > 500:\n step = 30\n width = axis_to_data(ax, 0.0001)\n head_width = 10 * width\n for ix, x in enumerate(X):\n if ix % step != 0:\n continue\n X_step = X[indices[ix]] - x\n # don't plot arrow of length 0\n for itrans in range(X_step.shape[0]):\n alphai = 1\n widthi = width\n head_widthi = head_width\n if weight is not None:\n alphai *= weight[ix, itrans]\n widthi *= weight[ix, itrans]\n if not np.any(X_step[itrans, :1]):\n continue\n ax.arrow(\n x[0],\n x[1],\n X_step[itrans, 0],\n X_step[itrans, 1],\n length_includes_head=True,\n width=widthi,\n head_width=head_widthi,\n alpha=alphai,\n color=\"grey\",\n )\n\n\ndef ticks_formatter(x, pos):\n # pretty scientific notation\n if False:\n a, b = f\"{x:.2e}\".split(\"e\")\n b = int(b)\n return rf\"${a} \\times 10^{{{b}}}$\"\n else:\n return f\"{x:.3f}\".rstrip(\"0\").rstrip(\".\")\n\n\ndef pimp_axis(x_or_y_ax):\n \"\"\"Remove trailing zeros.\"\"\"\n x_or_y_ax.set_major_formatter(ticker.FuncFormatter(ticks_formatter))\n\n\ndef scale_to_zero_one(x):\n \"\"\"Take some 1d data and scale it so that min matches 0 and max 1.\"\"\"\n xscaled = x - np.min(x)\n xscaled /= np.max(xscaled)\n return xscaled\n\n\nclass _Level(TypedDict):\n total: int\n current: int\n\n\ndef hierarchy_pos(\n G, root: int, levels_: Mapping[int, int] | None = None, width=1.0, height=1.0\n) -> dict[int, tuple[float, float]]:\n \"\"\"Tree layout for networkx graph.\n\n See https://stackoverflow.com/questions/29586520/can-one-get-hierarchical-graphs-from-networkx-with-python-3\n answer by burubum.\n\n If there is a cycle that is reachable from root, then this will see\n infinite recursion.\n\n Parameters\n ----------\n G: the graph\n root: the root node\n levels: a dictionary\n key: level number (starting from 0)\n value: number of nodes in this level\n width: horizontal space allocated for drawing\n height: vertical space allocated for drawing\n \"\"\"\n\n def make_levels(\n levels: dict[int, _Level],\n node: int = root,\n current_level: int = 0,\n parent: int | None = None,\n ) -> dict[int, _Level]:\n \"\"\"Compute the number of nodes for each level\"\"\"\n if current_level not in levels:\n levels[current_level] = _Level(total=0, current=0)\n levels[current_level][\"total\"] += 1\n neighbors: list[int] = list(G.neighbors(node))\n if parent is not None:\n neighbors.remove(parent)\n for neighbor in neighbors:\n levels = make_levels(levels, neighbor, current_level + 1, node)\n return levels\n\n if levels_ is None:\n levels = make_levels({})\n else:\n levels = {k: _Level(total=0, current=0) for k, v in levels_.items()}\n\n def make_pos(\n pos: dict[int, tuple[float, float]],\n node: int = root,\n current_level: int = 0,\n parent: int | None = None,\n vert_loc: float = 0.0,\n ):\n dx = 1 / levels[current_level][\"total\"]\n left = dx / 2\n pos[node] = ((left + dx * levels[current_level][\"current\"]) * width, vert_loc)\n levels[current_level][\"current\"] += 1\n neighbors: list[int] = list(G.neighbors(node))\n if parent is not None:\n neighbors.remove(parent)\n for neighbor in neighbors:\n pos = make_pos(pos, neighbor, current_level + 1, node, vert_loc - vert_gap)\n return pos\n\n vert_gap = height / (max(levels.keys()) + 1)\n return make_pos({})\n\n\ndef hierarchy_sc(G, root, node_sets):\n import networkx as nx\n\n def make_sc_tree(sc_G, node=root, parent=None):\n sc_G.add_node(node)\n neighbors = G.neighbors(node)\n if parent is not None:\n sc_G.add_edge(parent, node)\n neighbors.remove(parent)\n old_node = node\n for n in node_sets[int(node)]:\n new_node = str(node) + \"_\" + str(n)\n sc_G.add_node(new_node)\n sc_G.add_edge(old_node, new_node)\n old_node = new_node\n for neighbor in neighbors:\n sc_G = make_sc_tree(sc_G, neighbor, node)\n return sc_G\n\n return make_sc_tree(nx.Graph())\n\n\ndef zoom(ax, xy=\"x\", factor=1):\n \"\"\"Zoom into axis.\n\n Parameters\n ----------\n \"\"\"\n limits = ax.get_xlim() if xy == \"x\" else ax.get_ylim()\n new_limits = 0.5 * (limits[0] + limits[1]) + 1.0 / factor * np.array(\n (-0.5, 0.5)\n ) * (limits[1] - limits[0])\n if xy == \"x\":\n ax.set_xlim(new_limits)\n else:\n ax.set_ylim(new_limits)\n\n\ndef get_ax_size(ax: Axes, fig: Figure):\n \"\"\"Get axis size\n\n Parameters\n ----------\n ax\n Axis object from matplotlib.\n fig\n Figure.\n \"\"\"\n bbox = ax.get_window_extent().transformed(fig.dpi_scale_trans.inverted())\n width, height = bbox.width, bbox.height\n width *= fig.dpi\n height *= fig.dpi\n\n\ndef axis_to_data(ax: Axes, width: float):\n \"\"\"For a width in axis coordinates, return the corresponding in data\n coordinates.\n\n Parameters\n ----------\n ax\n Axis object from matplotlib.\n width\n Width in xaxis coordinates.\n \"\"\"\n xlim = ax.get_xlim()\n widthx = width * (xlim[1] - xlim[0])\n ylim = ax.get_ylim()\n widthy = width * (ylim[1] - ylim[0])\n return 0.5 * (widthx + widthy)\n\n\ndef axis_to_data_points(ax: Axes, points_axis: np.ndarray):\n \"\"\"Map points in axis coordinates to data coordinates.\n\n Uses matplotlib.transform.\n\n Parameters\n ----------\n ax\n Axis object from matplotlib.\n points_axis\n Points in axis coordinates.\n \"\"\"\n axis_to_data = ax.transAxes + ax.transData.inverted()\n return axis_to_data.transform(points_axis)\n\n\ndef data_to_axis_points(ax: Axes, points_data: np.ndarray):\n \"\"\"Map points in data coordinates to axis coordinates.\n\n Uses matplotlib.transform.\n\n Parameters\n ----------\n ax\n Axis object from matplotlib.\n points_data\n Points in data coordinates.\n \"\"\"\n data_to_axis = axis_to_data.inverted()\n return data_to_axis(points_data)\n\n\ndef check_projection(projection):\n \"\"\"Validation for projection argument.\"\"\"\n if projection not in {\"2d\", \"3d\"}:\n raise ValueError(f\"Projection must be '2d' or '3d', was '{projection}'.\")\n if projection == \"3d\":\n from packaging.version import parse\n\n mpl_version = parse(mpl.__version__)\n if mpl_version < parse(\"3.3.3\"):\n raise ImportError(\n f\"3d plotting requires matplotlib > 3.3.3. Found {mpl.__version__}\"\n )\n\n\ndef circles(\n x, y, *, s, ax, marker=None, c=\"b\", vmin=None, vmax=None, scale_factor=1.0, **kwargs\n):\n \"\"\"\n Taken from here: https://gist.github.com/syrte/592a062c562cd2a98a83\n Make a scatter plot of circles.\n Similar to pl.scatter, but the size of circles are in data scale.\n Parameters\n ----------\n x, y : scalar or array_like, shape (n, )\n Input data\n s : scalar or array_like, shape (n, )\n Radius of circles.\n c : color or sequence of color, optional, default : 'b'\n `c` can be a single color format string, or a sequence of color\n specifications of length `N`, or a sequence of `N` numbers to be\n mapped to colors using the `cmap` and `norm` specified via kwargs.\n Note that `c` should not be a single numeric RGB or RGBA sequence\n because that is indistinguishable from an array of values\n to be colormapped. (If you insist, use `color` instead.)\n `c` can be a 2-D array in which the rows are RGB or RGBA, however.\n vmin, vmax : scalar, optional, default: None\n `vmin` and `vmax` are used in conjunction with `norm` to normalize\n luminance data. If either are `None`, the min and max of the\n color array is used.\n kwargs : `~matplotlib.collections.Collection` properties\n Eg. alpha, edgecolor(ec), facecolor(fc), linewidth(lw), linestyle(ls),\n norm, cmap, transform, etc.\n Returns\n -------\n paths : `~matplotlib.collections.PathCollection`\n Examples\n --------\n a = np.arange(11)\n circles(a, a, s=a*0.2, c=a, alpha=0.5, ec='none')\n pl.colorbar()\n License\n --------\n This code is under [The BSD 3-Clause License]\n (https://opensource.org/license/bsd-3-clause/)\n \"\"\"\n\n # You can set `facecolor` with an array for each patch,\n # while you can only set `facecolors` with a value for all.\n if scale_factor != 1.0:\n x = x * scale_factor\n y = y * scale_factor\n zipped = np.broadcast(x, y, s)\n patches = [Circle((x_, y_), s_) for x_, y_, s_ in zipped]\n collection = PatchCollection(patches, **kwargs)\n if isinstance(c, np.ndarray) and np.issubdtype(c.dtype, np.number):\n collection.set_array(np.ma.masked_invalid(c))\n collection.set_clim(vmin, vmax)\n else:\n collection.set_facecolor(c)\n\n ax.add_collection(collection)\n\n return collection\n\n\ndef make_grid_spec(\n ax_or_figsize: tuple[int, int] | _AxesSubplot,\n *,\n nrows: int,\n ncols: int,\n wspace: float | None = None,\n hspace: float | None = None,\n width_ratios: Sequence[float] | None = None,\n height_ratios: Sequence[float] | None = None,\n) -> tuple[Figure, gridspec.GridSpecBase]:\n kw = dict(\n wspace=wspace,\n hspace=hspace,\n width_ratios=width_ratios,\n height_ratios=height_ratios,\n )\n if isinstance(ax_or_figsize, tuple):\n fig = plt.figure(figsize=ax_or_figsize)\n return fig, gridspec.GridSpec(nrows, ncols, **kw)\n else:\n ax = ax_or_figsize\n ax.axis(\"off\")\n ax.set_frame_on(False)\n ax.set_xticks([])\n ax.set_yticks([])\n return ax.figure, ax.get_subplotspec().subgridspec(nrows, ncols, **kw)\n\n\ndef fix_kwds(kwds_dict, **kwargs):\n \"\"\"\n Given a dictionary of plot parameters (kwds_dict) and a dict of kwds,\n merge the parameters into a single consolidated dictionary to avoid\n argument duplication errors.\n\n If kwds_dict an kwargs have the same key, only the value in kwds_dict is kept.\n\n Parameters\n ----------\n kwds_dict kwds_dictionary\n kwargs\n\n Returns\n -------\n kwds_dict merged with kwargs\n\n Examples\n --------\n\n >>> def _example(**kwds):\n ... return fix_kwds(kwds, key1=\"value1\", key2=\"value2\")\n >>> _example(key1=\"value10\", key3=\"value3\")\n {'key1': 'value10', 'key2': 'value2', 'key3': 'value3'}\n \"\"\"\n\n kwargs.update(kwds_dict)\n\n return kwargs\n\n\ndef _get_basis(adata: AnnData, basis: str):\n if basis in adata.obsm:\n basis_key = basis\n\n elif f\"X_{basis}\" in adata.obsm:\n basis_key = f\"X_{basis}\"\n\n return basis_key\n\n\ndef check_colornorm(vmin=None, vmax=None, vcenter=None, norm=None):\n from matplotlib.colors import Normalize\n\n try:\n from matplotlib.colors import TwoSlopeNorm as DivNorm\n except ImportError:\n # matplotlib<3.2\n from matplotlib.colors import DivergingNorm as DivNorm\n\n if norm is not None:\n if (vmin is not None) or (vmax is not None) or (vcenter is not None):\n raise ValueError(\"Passing both norm and vmin/vmax/vcenter is not allowed.\")\n else:\n if vcenter is not None:\n norm = DivNorm(vmin=vmin, vmax=vmax, vcenter=vcenter)\n else:\n norm = Normalize(vmin=vmin, vmax=vmax)\n\n return norm\n\n\n@overload\ndef _deprecated_scale(\n density_norm: DensityNorm,\n scale: DensityNorm | Empty,\n *,\n default: DensityNorm,\n) -> DensityNorm: ...\n\n\n@overload\ndef _deprecated_scale(\n density_norm: DensityNorm | Empty,\n scale: DensityNorm | Empty,\n *,\n default: DensityNorm | Empty = _empty,\n) -> DensityNorm | Empty: ...\n\n\ndef _deprecated_scale(\n density_norm: DensityNorm | Empty,\n scale: DensityNorm | Empty,\n *,\n default: DensityNorm | Empty = _empty,\n) -> DensityNorm | Empty:\n if scale is _empty:\n return density_norm\n if density_norm != default:\n msg = \"can’t specify both `scale` and `density_norm`\"\n raise ValueError(msg)\n msg = \"`scale` is deprecated, use `density_norm` instead\"\n warnings.warn(msg, FutureWarning)\n return scale\n\n\ndef _dk(dendrogram: bool | str | None) -> str | None:\n \"\"\"Helper to convert the `dendrogram` parameter to a `dendrogram_key` parameter.\"\"\"\n return None if isinstance(dendrogram, bool) else dendrogram\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom matplotlib import pyplot as plt\n\nfrom .._compat import old_positionals\nfrom . import _utils\n\nif TYPE_CHECKING:\n from collections.abc import Sequence\n from typing import Literal, Union\n\n from anndata import AnnData\n from matplotlib.axes import Axes\n from matplotlib.figure import Figure\n\n Scale = Union[Literal[\"linear\", \"log\", \"symlog\", \"logit\"], str]\n\n\n@old_positionals(\n \"scale_hist_obs\", \"scale_hist_sim\", \"figsize\", \"return_fig\", \"show\", \"save\"\n)\ndef scrublet_score_distribution(\n adata: AnnData,\n *,\n scale_hist_obs: Scale = \"log\",\n scale_hist_sim: Scale = \"linear\",\n figsize: tuple[float | int, float | int] = (8, 3),\n return_fig: bool = False,\n show: bool = True,\n save: str | bool | None = None,\n) -> Figure | Sequence[tuple[Axes, Axes]] | tuple[Axes, Axes] | None:\n \"\"\"\\\n Plot histogram of doublet scores for observed transcriptomes and simulated doublets.\n\n The histogram for simulated doublets is useful for determining the correct doublet\n score threshold.\n\n Scrublet must have been run previously with the input object.\n\n Parameters\n ----------\n adata\n An AnnData object resulting from :func:`~scanpy.pp.scrublet`.\n scale_hist_obs\n Set y axis scale transformation in matplotlib for the plot of observed transcriptomes\n scale_hist_sim\n Set y axis scale transformation in matplotlib for the plot of simulated doublets\n figsize\n width, height\n show\n Show the plot, do not return axis.\n save\n If :data:`True` or a :class:`str`, save the figure.\n A string is appended to the default filename.\n Infer the filetype if ending on {`'.pdf'`, `'.png'`, `'.svg'`}.\n\n Returns\n -------\n If ``return_fig`` is True, a :class:`~matplotlib.figure.Figure`.\n If ``show==False`` a list of :class:`~matplotlib.axes.Axes`.\n\n See also\n --------\n :func:`~scanpy.pp.scrublet`: Main way of running Scrublet, runs\n preprocessing, doublet simulation and calling.\n :func:`~scanpy.pp.scrublet_simulate_doublets`: Run Scrublet's doublet\n simulation separately for advanced usage.\n \"\"\"\n\n if \"scrublet\" not in adata.uns:\n raise ValueError(\n \"Please run scrublet before trying to generate the scrublet plot.\"\n )\n\n # If batched_by is populated, then we know Scrublet was run over multiple batches\n\n if \"batched_by\" in adata.uns[\"scrublet\"]:\n batched_by = adata.uns[\"scrublet\"][\"batched_by\"]\n batches = adata.obs[batched_by].astype(\"category\", copy=False)\n n_batches = len(batches.cat.categories)\n figsize = (figsize[0], figsize[1] * n_batches)\n else:\n batches = pd.Series(\n np.broadcast_to(0, adata.n_obs), dtype=\"category\", index=adata.obs_names\n )\n n_batches = 1\n\n fig, axs = plt.subplots(n_batches, 2, figsize=figsize)\n\n for idx, (batch_key, sub_obs) in enumerate(\n adata.obs.groupby(batches, observed=True)\n ):\n obs_ax: Axes\n sim_ax: Axes\n # We'll need multiple rows if Scrublet was run in multiple batches\n if \"batched_by\" in adata.uns[\"scrublet\"]:\n threshold = adata.uns[\"scrublet\"][\"batches\"][batch_key].get(\n \"threshold\", None\n )\n doublet_scores_sim = adata.uns[\"scrublet\"][\"batches\"][batch_key][\n \"doublet_scores_sim\"\n ]\n axis_lab_suffix = f\" ({batch_key})\"\n obs_ax = axs[idx][0]\n sim_ax = axs[idx][1]\n\n else:\n threshold = adata.uns[\"scrublet\"].get(\"threshold\", None)\n doublet_scores_sim = adata.uns[\"scrublet\"][\"doublet_scores_sim\"]\n axis_lab_suffix = \"\"\n obs_ax = axs[0]\n sim_ax = axs[1]\n\n # Make the plots\n _plot_scores(\n obs_ax,\n sub_obs[\"doublet_score\"],\n scale=scale_hist_obs,\n title=f\"Observed transcriptomes {axis_lab_suffix}\",\n threshold=threshold,\n )\n _plot_scores(\n sim_ax,\n doublet_scores_sim,\n scale=scale_hist_sim,\n title=f\"Simulated doublets {axis_lab_suffix}\",\n threshold=threshold,\n )\n\n fig.tight_layout()\n\n _utils.savefig_or_show(\"scrublet_score_distribution\", show=show, save=save)\n if return_fig:\n return fig\n elif not show:\n return axs\n\n\ndef _plot_scores(\n ax: Axes,\n scores: np.ndarray,\n scale: Scale,\n title: str,\n threshold: float | None = None,\n) -> None:\n ax.hist(\n scores,\n np.linspace(0, 1, 50),\n color=\"gray\",\n linewidth=0,\n density=True,\n )\n ax.set_yscale(scale)\n yl = ax.get_ylim()\n ax.set_ylim(yl)\n\n if threshold is not None:\n ax.plot(threshold * np.ones(2), yl, c=\"black\", linewidth=1)\n\n ax.set_title(title)\n ax.set_xlabel(\"Doublet score\")\n ax.set_ylabel(\"Prob. density\")\n\n\n\"\"\"Plotting functions for AnnData.\"\"\"\n\nfrom __future__ import annotations\n\nfrom collections import OrderedDict\nfrom collections.abc import Collection, Mapping, Sequence\nfrom itertools import product\nfrom typing import TYPE_CHECKING, get_args\n\nimport matplotlib as mpl\nimport numpy as np\nimport pandas as pd\nfrom matplotlib import gridspec, patheffects, rcParams\nfrom matplotlib import pyplot as plt\nfrom matplotlib.colors import is_color_like\nfrom packaging.version import Version\nfrom pandas.api.types import CategoricalDtype, is_numeric_dtype\nfrom scipy.sparse import issparse\n\nfrom .. import get\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._settings import settings\nfrom .._utils import _check_use_raw, _doc_params, _empty, sanitize_anndata\nfrom . import _utils\nfrom ._docs import (\n doc_common_plot_args,\n doc_scatter_basic,\n doc_show_save_ax,\n doc_vboundnorm,\n)\nfrom ._utils import (\n _deprecated_scale,\n _dk,\n check_colornorm,\n scatter_base,\n scatter_group,\n setup_axes,\n)\n\nif TYPE_CHECKING:\n from collections.abc import Iterable\n from typing import Literal, Union\n\n from anndata import AnnData\n from cycler import Cycler\n from matplotlib.axes import Axes\n from matplotlib.colors import Colormap, ListedColormap, Normalize\n from seaborn import FacetGrid\n from seaborn.matrix import ClusterGrid\n\n from .._utils import Empty\n from ._utils import (\n ColorLike,\n DensityNorm,\n _FontSize,\n _FontWeight,\n _LegendLoc,\n )\n\n # TODO: is that all?\n _Basis = Literal[\"pca\", \"tsne\", \"umap\", \"diffmap\", \"draw_graph_fr\"]\n _VarNames = Union[str, Sequence[str]]\n\n\nVALID_LEGENDLOCS = frozenset(get_args(_utils._LegendLoc))\n\n\n@old_positionals(\n \"color\",\n \"use_raw\",\n \"layers\",\n \"sort_order\",\n \"alpha\",\n \"basis\",\n \"groups\",\n \"components\",\n \"projection\",\n \"legend_loc\",\n \"legend_fontsize\",\n \"legend_fontweight\",\n \"legend_fontoutline\",\n \"color_map\",\n # 17 positionals are enough for backwards compatibility\n)\n@_doc_params(scatter_temp=doc_scatter_basic, show_save_ax=doc_show_save_ax)\ndef scatter(\n adata: AnnData,\n x: str | None = None,\n y: str | None = None,\n *,\n color: str | Collection[str] | None = None,\n use_raw: bool | None = None,\n layers: str | Collection[str] | None = None,\n sort_order: bool = True,\n alpha: float | None = None,\n basis: _Basis | None = None,\n groups: str | Iterable[str] | None = None,\n components: str | Collection[str] | None = None,\n projection: Literal[\"2d\", \"3d\"] = \"2d\",\n legend_loc: _LegendLoc | None = \"right margin\",\n legend_fontsize: int | float | _FontSize | None = None,\n legend_fontweight: int | _FontWeight | None = None,\n legend_fontoutline: float | None = None,\n color_map: str | Colormap | None = None,\n palette: Cycler | ListedColormap | ColorLike | Sequence[ColorLike] | None = None,\n frameon: bool | None = None,\n right_margin: float | None = None,\n left_margin: float | None = None,\n size: int | float | None = None,\n marker: str | Sequence[str] = \".\",\n title: str | None = None,\n show: bool | None = None,\n save: str | bool | None = None,\n ax: Axes | None = None,\n) -> Axes | list[Axes] | None:\n \"\"\"\\\n Scatter plot along observations or variables axes.\n\n Color the plot using annotations of observations (`.obs`), variables\n (`.var`) or expression of genes (`.var_names`).\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n x\n x coordinate.\n y\n y coordinate.\n color\n Keys for annotations of observations/cells or variables/genes,\n or a hex color specification, e.g.,\n `'ann1'`, `'#fe57a1'`, or `['ann1', 'ann2']`.\n use_raw\n Whether to use `raw` attribute of `adata`. Defaults to `True` if `.raw` is present.\n layers\n Use the `layers` attribute of `adata` if present: specify the layer for\n `x`, `y` and `color`. If `layers` is a string, then it is expanded to\n `(layers, layers, layers)`.\n basis\n String that denotes a plotting tool that computed coordinates.\n {scatter_temp}\n {show_save_ax}\n\n Returns\n -------\n If `show==False` a :class:`~matplotlib.axes.Axes` or a list of it.\n \"\"\"\n args = locals()\n if _check_use_raw(adata, use_raw):\n var_index = adata.raw.var.index\n else:\n var_index = adata.var.index\n if basis is not None:\n return _scatter_obs(**args)\n if x is None or y is None:\n raise ValueError(\"Either provide a `basis` or `x` and `y`.\")\n if (\n (x in adata.obs.columns or x in var_index)\n and (y in adata.obs.columns or y in var_index)\n and (color is None or color in adata.obs.columns or color in var_index)\n ):\n return _scatter_obs(**args)\n if (\n (x in adata.var.columns or x in adata.obs.index)\n and (y in adata.var.columns or y in adata.obs.index)\n and (color is None or color in adata.var.columns or color in adata.obs.index)\n ):\n adata_T = adata.T\n axs = _scatter_obs(\n adata=adata_T,\n **{name: val for name, val in args.items() if name != \"adata\"},\n )\n # store .uns annotations that were added to the new adata object\n adata.uns = adata_T.uns\n return axs\n raise ValueError(\n \"`x`, `y`, and potential `color` inputs must all \"\n \"come from either `.obs` or `.var`\"\n )\n\n\ndef _scatter_obs(\n *,\n adata: AnnData,\n x=None,\n y=None,\n color=None,\n use_raw=None,\n layers=None,\n sort_order=True,\n alpha=None,\n basis=None,\n groups=None,\n components=None,\n projection: Literal[\"2d\", \"3d\"] = \"2d\",\n legend_loc: _LegendLoc | None = \"right margin\",\n legend_fontsize=None,\n legend_fontweight=None,\n legend_fontoutline=None,\n color_map=None,\n palette=None,\n frameon=None,\n right_margin=None,\n left_margin=None,\n size: int | float | None = None,\n marker=\".\",\n title=None,\n show=None,\n save=None,\n ax=None,\n) -> Axes | list[Axes] | None:\n \"\"\"See docstring of scatter.\"\"\"\n sanitize_anndata(adata)\n\n use_raw = _check_use_raw(adata, use_raw)\n\n # Process layers\n if layers in [\"X\", None] or (isinstance(layers, str) and layers in adata.layers):\n layers = (layers, layers, layers)\n elif isinstance(layers, Collection) and len(layers) == 3:\n layers = tuple(layers)\n for layer in layers:\n if layer not in adata.layers and layer not in [\"X\", None]:\n raise ValueError(\n \"`layers` should have elements that are \"\n \"either None or in adata.layers.keys().\"\n )\n else:\n raise ValueError(\n \"`layers` should be a string or a collection of strings \"\n f\"with length 3, had value '{layers}'\"\n )\n if use_raw and layers not in [(\"X\", \"X\", \"X\"), (None, None, None)]:\n ValueError(\"`use_raw` must be `False` if layers are used.\")\n\n if legend_loc not in VALID_LEGENDLOCS:\n raise ValueError(\n f\"Invalid `legend_loc`, need to be one of: {VALID_LEGENDLOCS}.\"\n )\n if components is None:\n components = \"1,2\" if \"2d\" in projection else \"1,2,3\"\n if isinstance(components, str):\n components = components.split(\",\")\n components = np.array(components).astype(int) - 1\n # color can be a obs column name or a matplotlib color specification\n keys = (\n [\"grey\"]\n if color is None\n else [color]\n if isinstance(color, str) or is_color_like(color)\n else color\n )\n if title is not None and isinstance(title, str):\n title = [title]\n highlights = adata.uns.get(\"highlights\", [])\n if basis is not None:\n try:\n # ignore the '0th' diffusion component\n if basis == \"diffmap\":\n components += 1\n Y = adata.obsm[\"X_\" + basis][:, components]\n # correct the component vector for use in labeling etc.\n if basis == \"diffmap\":\n components -= 1\n except KeyError:\n raise KeyError(\n f\"compute coordinates using visualization tool {basis} first\"\n )\n elif x is not None and y is not None:\n if use_raw:\n if x in adata.obs.columns:\n x_arr = adata.obs_vector(x)\n else:\n x_arr = adata.raw.obs_vector(x)\n if y in adata.obs.columns:\n y_arr = adata.obs_vector(y)\n else:\n y_arr = adata.raw.obs_vector(y)\n else:\n x_arr = adata.obs_vector(x, layer=layers[0])\n y_arr = adata.obs_vector(y, layer=layers[1])\n\n Y = np.c_[x_arr, y_arr]\n else:\n raise ValueError(\"Either provide a `basis` or `x` and `y`.\")\n\n if size is None:\n n = Y.shape[0]\n size = 120000 / n\n\n if legend_fontsize is None:\n legend_fontsize = rcParams[\"legend.fontsize\"]\n\n palette_was_none = False\n if palette is None:\n palette_was_none = True\n if isinstance(palette, Sequence) and not isinstance(palette, str):\n palettes = palette if not is_color_like(palette[0]) else [palette]\n else:\n palettes = [palette for _ in range(len(keys))]\n palettes = [_utils.default_palette(palette) for palette in palettes]\n\n if basis is not None:\n component_name = (\n \"DC\"\n if basis == \"diffmap\"\n else \"tSNE\"\n if basis == \"tsne\"\n else \"UMAP\"\n if basis == \"umap\"\n else \"PC\"\n if basis == \"pca\"\n else \"TriMap\"\n if basis == \"trimap\"\n else basis.replace(\"draw_graph_\", \"\").upper()\n if \"draw_graph\" in basis\n else basis\n )\n else:\n component_name = None\n axis_labels = (x, y) if component_name is None else None\n show_ticks = component_name is None\n\n # generate the colors\n color_ids: list[np.ndarray | ColorLike] = []\n categoricals = []\n colorbars = []\n for ikey, key in enumerate(keys):\n c = \"white\"\n categorical = False # by default, assume continuous or flat color\n colorbar = None\n # test whether we have categorial or continuous annotation\n if key in adata.obs_keys():\n if isinstance(adata.obs[key].dtype, CategoricalDtype):\n categorical = True\n else:\n c = adata.obs[key].to_numpy()\n # coloring according to gene expression\n elif use_raw and adata.raw is not None and key in adata.raw.var_names:\n c = adata.raw.obs_vector(key)\n elif key in adata.var_names:\n c = adata.obs_vector(key, layer=layers[2])\n elif is_color_like(key): # a flat color\n c = key\n colorbar = False\n else:\n raise ValueError(\n f\"key {key!r} is invalid! pass valid observation annotation, \"\n f\"one of {adata.obs_keys()} or a gene name {adata.var_names}\"\n )\n if colorbar is None:\n colorbar = not categorical\n colorbars.append(colorbar)\n if categorical:\n categoricals.append(ikey)\n color_ids.append(c)\n\n if right_margin is None and len(categoricals) > 0 and legend_loc == \"right margin\":\n right_margin = 0.5\n if title is None and keys[0] is not None:\n title = [\n key.replace(\"_\", \" \") if not is_color_like(key) else \"\" for key in keys\n ]\n\n axs: list[Axes] = scatter_base(\n Y,\n title=title,\n alpha=alpha,\n component_name=component_name,\n axis_labels=axis_labels,\n component_indexnames=components + 1,\n projection=projection,\n colors=color_ids,\n highlights=highlights,\n colorbars=colorbars,\n right_margin=right_margin,\n left_margin=left_margin,\n sizes=[size for _ in keys],\n markers=marker,\n color_map=color_map,\n show_ticks=show_ticks,\n ax=ax,\n )\n\n def add_centroid(centroids, name, Y, mask):\n Y_mask = Y[mask]\n if Y_mask.shape[0] == 0:\n return\n median = np.median(Y_mask, axis=0)\n i = np.argmin(np.sum(np.abs(Y_mask - median), axis=1))\n centroids[name] = Y_mask[i]\n\n # loop over all categorical annotation and plot it\n for ikey, palette in zip(categoricals, palettes):\n key = keys[ikey]\n _utils.add_colors_for_categorical_sample_annotation(\n adata, key, palette=palette, force_update_colors=not palette_was_none\n )\n # actually plot the groups\n mask_remaining = np.ones(Y.shape[0], dtype=bool)\n centroids = {}\n if groups is None:\n for iname, name in enumerate(adata.obs[key].cat.categories):\n if name not in settings.categories_to_ignore:\n mask = scatter_group(\n axs[ikey],\n key,\n iname,\n adata,\n Y,\n projection=projection,\n size=size,\n alpha=alpha,\n marker=marker,\n )\n mask_remaining[mask] = False\n if legend_loc.startswith(\"on data\"):\n add_centroid(centroids, name, Y, mask)\n else:\n groups = [groups] if isinstance(groups, str) else groups\n for name in groups:\n if name not in set(adata.obs[key].cat.categories):\n raise ValueError(\n f\"{name!r} is invalid! specify valid name, \"\n f\"one of {adata.obs[key].cat.categories}\"\n )\n else:\n iname = np.flatnonzero(\n adata.obs[key].cat.categories.values == name\n )[0]\n mask = scatter_group(\n axs[ikey],\n key,\n iname,\n adata,\n Y,\n projection=projection,\n size=size,\n alpha=alpha,\n marker=marker,\n )\n if legend_loc.startswith(\"on data\"):\n add_centroid(centroids, name, Y, mask)\n mask_remaining[mask] = False\n if mask_remaining.sum() > 0:\n data = [Y[mask_remaining, 0], Y[mask_remaining, 1]]\n if projection == \"3d\":\n data.append(Y[mask_remaining, 2])\n axs[ikey].scatter(\n *data,\n marker=marker,\n c=\"lightgrey\",\n s=size,\n edgecolors=\"none\",\n zorder=-1,\n )\n legend = None\n if legend_loc.startswith(\"on data\"):\n if legend_fontweight is None:\n legend_fontweight = \"bold\"\n if legend_fontoutline is not None:\n path_effect = [\n patheffects.withStroke(linewidth=legend_fontoutline, foreground=\"w\")\n ]\n else:\n path_effect = None\n for name, pos in centroids.items():\n axs[ikey].text(\n pos[0],\n pos[1],\n name,\n weight=legend_fontweight,\n verticalalignment=\"center\",\n horizontalalignment=\"center\",\n fontsize=legend_fontsize,\n path_effects=path_effect,\n )\n\n all_pos = np.zeros((len(adata.obs[key].cat.categories), 2))\n for iname, name in enumerate(adata.obs[key].cat.categories):\n all_pos[iname] = centroids.get(name, [np.nan, np.nan])\n if legend_loc == \"on data export\":\n filename = settings.writedir / \"pos.csv\"\n logg.warning(f\"exporting label positions to {filename}\")\n settings.writedir.mkdir(parents=True, exist_ok=True)\n np.savetxt(filename, all_pos, delimiter=\",\")\n elif legend_loc == \"right margin\":\n legend = axs[ikey].legend(\n frameon=False,\n loc=\"center left\",\n bbox_to_anchor=(1, 0.5),\n ncol=(\n 1\n if len(adata.obs[key].cat.categories) <= 14\n else 2\n if len(adata.obs[key].cat.categories) <= 30\n else 3\n ),\n fontsize=legend_fontsize,\n )\n elif legend_loc != \"none\":\n legend = axs[ikey].legend(\n frameon=False, loc=legend_loc, fontsize=legend_fontsize\n )\n if legend is not None:\n if Version(mpl.__version__) < Version(\"3.7\"):\n _attr = \"legendHandles\"\n else:\n _attr = \"legend_handles\"\n for handle in getattr(legend, _attr):\n handle.set_sizes([300.0])\n\n # draw a frame around the scatter\n frameon = settings._frameon if frameon is None else frameon\n if not frameon and x is None and y is None:\n for ax in axs:\n ax.set_xlabel(\"\")\n ax.set_ylabel(\"\")\n ax.set_frame_on(False)\n\n show = settings.autoshow if show is None else show\n _utils.savefig_or_show(\"scatter\" if basis is None else basis, show=show, save=save)\n if show:\n return None\n if len(keys) > 1:\n return axs\n return axs[0]\n\n\n@old_positionals(\n \"dictionary\",\n \"indices\",\n \"labels\",\n \"color\",\n \"n_points\",\n \"log\",\n \"include_lowest\",\n \"show\",\n)\ndef ranking(\n adata: AnnData,\n attr: Literal[\"var\", \"obs\", \"uns\", \"varm\", \"obsm\"],\n keys: str | Sequence[str],\n *,\n dictionary: str | None = None,\n indices: Sequence[int] | None = None,\n labels: str | Sequence[str] | None = None,\n color: ColorLike = \"black\",\n n_points: int = 30,\n log: bool = False,\n include_lowest: bool = False,\n show: bool | None = None,\n) -> gridspec.GridSpec | None:\n \"\"\"\\\n Plot rankings.\n\n See, for example, how this is used in pl.pca_loadings.\n\n Parameters\n ----------\n adata\n The data.\n attr\n The attribute of AnnData that contains the score.\n keys\n The scores to look up an array from the attribute of adata.\n\n Returns\n -------\n Returns matplotlib gridspec with access to the axes.\n \"\"\"\n if isinstance(keys, str) and indices is not None:\n scores = getattr(adata, attr)[keys][:, indices]\n keys = [f\"{keys[:-1]}{i + 1}\" for i in indices]\n else:\n if dictionary is None:\n scores = getattr(adata, attr)[keys]\n else:\n scores = getattr(adata, attr)[dictionary][keys]\n n_panels = len(keys) if isinstance(keys, list) else 1\n if n_panels == 1:\n scores, keys = scores[:, None], [keys]\n if log:\n scores = np.log(scores)\n if labels is None:\n labels = (\n adata.var_names\n if attr in {\"var\", \"varm\"}\n else np.arange(scores.shape[0]).astype(str)\n )\n if isinstance(labels, str):\n labels = [labels + str(i + 1) for i in range(scores.shape[0])]\n if n_panels <= 5:\n n_rows, n_cols = 1, n_panels\n else:\n n_rows, n_cols = 2, int(n_panels / 2 + 0.5)\n _ = plt.figure(\n figsize=(\n n_cols * rcParams[\"figure.figsize\"][0],\n n_rows * rcParams[\"figure.figsize\"][1],\n )\n )\n left, bottom = 0.2 / n_cols, 0.13 / n_rows\n gs = gridspec.GridSpec(\n wspace=0.2,\n nrows=n_rows,\n ncols=n_cols,\n left=left,\n bottom=bottom,\n right=1 - (n_cols - 1) * left - 0.01 / n_cols,\n top=1 - (n_rows - 1) * bottom - 0.1 / n_rows,\n )\n for iscore, score in enumerate(scores.T):\n plt.subplot(gs[iscore])\n order_scores = np.argsort(score)[::-1]\n if not include_lowest:\n indices = order_scores[: n_points + 1]\n else:\n indices = order_scores[: n_points // 2]\n neg_indices = order_scores[-(n_points - (n_points // 2)) :]\n txt_args = dict(\n color=color,\n rotation=\"vertical\",\n verticalalignment=\"bottom\",\n horizontalalignment=\"center\",\n fontsize=8,\n )\n for ig, g in enumerate(indices):\n plt.text(ig, score[g], labels[g], **txt_args)\n if include_lowest:\n score_mid = (score[g] + score[neg_indices[0]]) / 2\n if (len(indices) + len(neg_indices)) < len(order_scores):\n plt.text(len(indices), score_mid, \"⋮\", **txt_args)\n for ig, g in enumerate(neg_indices):\n plt.text(ig + len(indices) + 2, score[g], labels[g], **txt_args)\n else:\n for ig, g in enumerate(neg_indices):\n plt.text(ig + len(indices), score[g], labels[g], **txt_args)\n plt.xticks([])\n plt.title(keys[iscore].replace(\"_\", \" \"))\n if n_panels <= 5 or iscore > n_cols:\n plt.xlabel(\"ranking\")\n plt.xlim(-0.9, n_points + 0.9 + (1 if include_lowest else 0))\n score_min, score_max = (\n np.min(score[neg_indices if include_lowest else indices]),\n np.max(score[indices]),\n )\n plt.ylim(\n (0.95 if score_min > 0 else 1.05) * score_min,\n (1.05 if score_max > 0 else 0.95) * score_max,\n )\n show = settings.autoshow if show is None else show\n if show:\n return None\n return gs\n\n\n@old_positionals(\n \"log\",\n \"use_raw\",\n \"stripplot\",\n \"jitter\",\n \"size\",\n \"layer\",\n \"scale\",\n \"order\",\n \"multi_panel\",\n \"xlabel\",\n \"ylabel\",\n \"rotation\",\n \"show\",\n \"save\",\n \"ax\",\n)\n@_doc_params(show_save_ax=doc_show_save_ax)\ndef violin(\n adata: AnnData,\n keys: str | Sequence[str],\n groupby: str | None = None,\n *,\n log: bool = False,\n use_raw: bool | None = None,\n stripplot: bool = True,\n jitter: float | bool = True,\n size: int = 1,\n layer: str | None = None,\n density_norm: DensityNorm = \"width\",\n order: Sequence[str] | None = None,\n multi_panel: bool | None = None,\n xlabel: str = \"\",\n ylabel: str | Sequence[str] | None = None,\n rotation: float | None = None,\n show: bool | None = None,\n save: bool | str | None = None,\n ax: Axes | None = None,\n # deprecatd\n scale: DensityNorm | Empty = _empty,\n **kwds,\n) -> Axes | FacetGrid | None:\n \"\"\"\\\n Violin plot.\n\n Wraps :func:`seaborn.violinplot` for :class:`~anndata.AnnData`.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n keys\n Keys for accessing variables of `.var_names` or fields of `.obs`.\n groupby\n The key of the observation grouping to consider.\n log\n Plot on logarithmic axis.\n use_raw\n Whether to use `raw` attribute of `adata`. Defaults to `True` if `.raw` is present.\n stripplot\n Add a stripplot on top of the violin plot.\n See :func:`~seaborn.stripplot`.\n jitter\n Add jitter to the stripplot (only when stripplot is True)\n See :func:`~seaborn.stripplot`.\n size\n Size of the jitter points.\n layer\n Name of the AnnData object layer that wants to be plotted. By\n default adata.raw.X is plotted. If `use_raw=False` is set,\n then `adata.X` is plotted. If `layer` is set to a valid layer name,\n then the layer is plotted. `layer` takes precedence over `use_raw`.\n density_norm\n The method used to scale the width of each violin.\n If 'width' (the default), each violin will have the same width.\n If 'area', each violin will have the same area.\n If 'count', a violin’s width corresponds to the number of observations.\n order\n Order in which to show the categories.\n multi_panel\n Display keys in multiple panels also when `groupby is not None`.\n xlabel\n Label of the x axis. Defaults to `groupby` if `rotation` is `None`,\n otherwise, no label is shown.\n ylabel\n Label of the y axis. If `None` and `groupby` is `None`, defaults\n to `'value'`. If `None` and `groubpy` is not `None`, defaults to `keys`.\n rotation\n Rotation of xtick labels.\n {show_save_ax}\n **kwds\n Are passed to :func:`~seaborn.violinplot`.\n\n Returns\n -------\n A :class:`~matplotlib.axes.Axes` object if `ax` is `None` else `None`.\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.pl.violin(adata, keys='S_score')\n\n Plot by category. Rotate x-axis labels so that they do not overlap.\n\n .. plot::\n :context: close-figs\n\n sc.pl.violin(adata, keys='S_score', groupby='bulk_labels', rotation=90)\n\n Set order of categories to be plotted or select specific categories to be plotted.\n\n .. plot::\n :context: close-figs\n\n groupby_order = ['CD34+', 'CD19+ B']\n sc.pl.violin(adata, keys='S_score', groupby='bulk_labels', rotation=90,\n order=groupby_order)\n\n Plot multiple keys.\n\n .. plot::\n :context: close-figs\n\n sc.pl.violin(adata, keys=['S_score', 'G2M_score'], groupby='bulk_labels',\n rotation=90)\n\n For large datasets consider omitting the overlaid scatter plot.\n\n .. plot::\n :context: close-figs\n\n sc.pl.violin(adata, keys='S_score', stripplot=False)\n\n .. currentmodule:: scanpy\n\n See also\n --------\n pl.stacked_violin\n \"\"\"\n import seaborn as sns # Slow import, only import if called\n\n sanitize_anndata(adata)\n use_raw = _check_use_raw(adata, use_raw)\n if isinstance(keys, str):\n keys = [keys]\n keys = list(OrderedDict.fromkeys(keys)) # remove duplicates, preserving the order\n density_norm = _deprecated_scale(density_norm, scale, default=\"width\")\n del scale\n\n if isinstance(ylabel, (str, type(None))):\n ylabel = [ylabel] * (1 if groupby is None else len(keys))\n if groupby is None:\n if len(ylabel) != 1:\n raise ValueError(\n f\"Expected number of y-labels to be `1`, found `{len(ylabel)}`.\"\n )\n elif len(ylabel) != len(keys):\n raise ValueError(\n f\"Expected number of y-labels to be `{len(keys)}`, \"\n f\"found `{len(ylabel)}`.\"\n )\n\n if groupby is not None:\n obs_df = get.obs_df(adata, keys=[groupby] + keys, layer=layer, use_raw=use_raw)\n if kwds.get(\"palette\", None) is None:\n if not isinstance(adata.obs[groupby].dtype, CategoricalDtype):\n raise ValueError(\n f\"The column `adata.obs[{groupby!r}]` needs to be categorical, \"\n f\"but is of dtype {adata.obs[groupby].dtype}.\"\n )\n _utils.add_colors_for_categorical_sample_annotation(adata, groupby)\n kwds[\"hue\"] = groupby\n kwds[\"palette\"] = dict(\n zip(obs_df[groupby].cat.categories, adata.uns[f\"{groupby}_colors\"])\n )\n else:\n obs_df = get.obs_df(adata, keys=keys, layer=layer, use_raw=use_raw)\n if groupby is None:\n obs_tidy = pd.melt(obs_df, value_vars=keys)\n x = \"variable\"\n ys = [\"value\"]\n else:\n obs_tidy = obs_df\n x = groupby\n ys = keys\n\n if multi_panel and groupby is None and len(ys) == 1:\n # This is a quick and dirty way for adapting scales across several\n # keys if groupby is None.\n y = ys[0]\n\n g: sns.axisgrid.FacetGrid = sns.catplot(\n y=y,\n data=obs_tidy,\n kind=\"violin\",\n density_norm=density_norm,\n col=x,\n col_order=keys,\n sharey=False,\n cut=0,\n inner=None,\n **kwds,\n )\n\n if stripplot:\n grouped_df = obs_tidy.groupby(x, observed=True)\n for ax_id, key in zip(range(g.axes.shape[1]), keys):\n sns.stripplot(\n y=y,\n data=grouped_df.get_group(key),\n jitter=jitter,\n size=size,\n color=\"black\",\n ax=g.axes[0, ax_id],\n )\n if log:\n g.set(yscale=\"log\")\n g.set_titles(col_template=\"{col_name}\").set_xlabels(\"\")\n if rotation is not None:\n for ax in g.axes[0]:\n ax.tick_params(axis=\"x\", labelrotation=rotation)\n else:\n # set by default the violin plot cut=0 to limit the extend\n # of the violin plot (see stacked_violin code) for more info.\n kwds.setdefault(\"cut\", 0)\n kwds.setdefault(\"inner\")\n\n if ax is None:\n axs, _, _, _ = setup_axes(\n ax,\n panels=[\"x\"] if groupby is None else keys,\n show_ticks=True,\n right_margin=0.3,\n )\n else:\n axs = [ax]\n for ax, y, ylab in zip(axs, ys, ylabel):\n ax = sns.violinplot(\n x=x,\n y=y,\n data=obs_tidy,\n order=order,\n orient=\"vertical\",\n density_norm=density_norm,\n ax=ax,\n **kwds,\n )\n if stripplot:\n ax = sns.stripplot(\n x=x,\n y=y,\n data=obs_tidy,\n order=order,\n jitter=jitter,\n color=\"black\",\n size=size,\n ax=ax,\n )\n if xlabel == \"\" and groupby is not None and rotation is None:\n xlabel = groupby.replace(\"_\", \" \")\n ax.set_xlabel(xlabel)\n if ylab is not None:\n ax.set_ylabel(ylab)\n\n if log:\n ax.set_yscale(\"log\")\n if rotation is not None:\n ax.tick_params(axis=\"x\", labelrotation=rotation)\n show = settings.autoshow if show is None else show\n _utils.savefig_or_show(\"violin\", show=show, save=save)\n if show:\n return None\n if multi_panel and groupby is None and len(ys) == 1:\n return g\n if len(axs) == 1:\n return axs[0]\n return axs\n\n\n@old_positionals(\"use_raw\", \"show\", \"save\")\n@_doc_params(show_save_ax=doc_show_save_ax)\ndef clustermap(\n adata: AnnData,\n obs_keys: str | None = None,\n *,\n use_raw: bool | None = None,\n show: bool | None = None,\n save: bool | str | None = None,\n **kwds,\n) -> ClusterGrid | None:\n \"\"\"\\\n Hierarchically-clustered heatmap.\n\n Wraps :func:`seaborn.clustermap` for :class:`~anndata.AnnData`.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n obs_keys\n Categorical annotation to plot with a different color map.\n Currently, only a single key is supported.\n use_raw\n Whether to use `raw` attribute of `adata`. Defaults to `True` if `.raw` is present.\n {show_save_ax}\n **kwds\n Keyword arguments passed to :func:`~seaborn.clustermap`.\n\n Returns\n -------\n If `show` is `False`, a :class:`~seaborn.matrix.ClusterGrid` object\n (see :func:`~seaborn.clustermap`).\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.krumsiek11()\n sc.pl.clustermap(adata)\n\n .. plot::\n :context: close-figs\n\n sc.pl.clustermap(adata, obs_keys='cell_type')\n \"\"\"\n import seaborn as sns # Slow import, only import if called\n\n if not isinstance(obs_keys, (str, type(None))):\n raise ValueError(\"Currently, only a single key is supported.\")\n sanitize_anndata(adata)\n use_raw = _check_use_raw(adata, use_raw)\n X = adata.raw.X if use_raw else adata.X\n if issparse(X):\n X = X.toarray()\n df = pd.DataFrame(X, index=adata.obs_names, columns=adata.var_names)\n if obs_keys is not None:\n row_colors = adata.obs[obs_keys]\n _utils.add_colors_for_categorical_sample_annotation(adata, obs_keys)\n # do this more efficiently... just a quick solution\n lut = dict(zip(row_colors.cat.categories, adata.uns[obs_keys + \"_colors\"]))\n row_colors = adata.obs[obs_keys].map(lut)\n g = sns.clustermap(df, row_colors=row_colors.values, **kwds)\n else:\n g = sns.clustermap(df, **kwds)\n show = settings.autoshow if show is None else show\n _utils.savefig_or_show(\"clustermap\", show=show, save=save)\n if show:\n plt.show()\n return None\n return g\n\n\n@old_positionals(\n \"use_raw\",\n \"log\",\n \"num_categories\",\n \"dendrogram\",\n \"gene_symbols\",\n \"var_group_positions\",\n \"var_group_labels\",\n \"var_group_rotation\",\n \"layer\",\n \"standard_scale\",\n \"swap_axes\",\n \"show_gene_labels\",\n \"show\",\n \"save\",\n \"figsize\",\n \"vmin\",\n \"vmax\",\n \"vcenter\",\n \"norm\",\n)\n@_doc_params(\n vminmax=doc_vboundnorm,\n show_save_ax=doc_show_save_ax,\n common_plot_args=doc_common_plot_args,\n)\ndef heatmap(\n adata: AnnData,\n var_names: _VarNames | Mapping[str, _VarNames],\n groupby: str | Sequence[str],\n *,\n use_raw: bool | None = None,\n log: bool = False,\n num_categories: int = 7,\n dendrogram: bool | str = False,\n gene_symbols: str | None = None,\n var_group_positions: Sequence[tuple[int, int]] | None = None,\n var_group_labels: Sequence[str] | None = None,\n var_group_rotation: float | None = None,\n layer: str | None = None,\n standard_scale: Literal[\"var\", \"obs\"] | None = None,\n swap_axes: bool = False,\n show_gene_labels: bool | None = None,\n show: bool | None = None,\n save: str | bool | None = None,\n figsize: tuple[float, float] | None = None,\n vmin: float | None = None,\n vmax: float | None = None,\n vcenter: float | None = None,\n norm: Normalize | None = None,\n **kwds,\n) -> dict[str, Axes] | None:\n \"\"\"\\\n Heatmap of the expression values of genes.\n\n If `groupby` is given, the heatmap is ordered by the respective group. For\n example, a list of marker genes can be plotted, ordered by clustering. If\n the `groupby` observation annotation is not categorical the observation\n annotation is turned into a categorical by binning the data into the number\n specified in `num_categories`.\n\n Parameters\n ----------\n {common_plot_args}\n standard_scale\n Whether or not to standardize that dimension between 0 and 1, meaning for each variable or observation,\n subtract the minimum and divide each by its maximum.\n swap_axes\n By default, the x axis contains `var_names` (e.g. genes) and the y axis the `groupby`\n categories (if any). By setting `swap_axes` then x are the `groupby` categories and y the `var_names`.\n show_gene_labels\n By default gene labels are shown when there are 50 or less genes. Otherwise the labels are removed.\n {show_save_ax}\n {vminmax}\n **kwds\n Are passed to :func:`matplotlib.pyplot.imshow`.\n\n Returns\n -------\n Dict of :class:`~matplotlib.axes.Axes`\n\n Examples\n -------\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n sc.pl.heatmap(adata, markers, groupby='bulk_labels', swap_axes=True)\n\n .. currentmodule:: scanpy\n\n See also\n --------\n pl.rank_genes_groups_heatmap\n tl.rank_genes_groups\n \"\"\"\n var_names, var_group_labels, var_group_positions = _check_var_names_type(\n var_names, var_group_labels, var_group_positions\n )\n\n categories, obs_tidy = _prepare_dataframe(\n adata,\n var_names,\n groupby,\n use_raw=use_raw,\n log=log,\n num_categories=num_categories,\n gene_symbols=gene_symbols,\n layer=layer,\n )\n\n # check if var_group_labels are a subset of categories:\n if var_group_labels is not None:\n if set(var_group_labels).issubset(categories):\n var_groups_subset_of_groupby = True\n else:\n var_groups_subset_of_groupby = False\n\n if standard_scale == \"obs\":\n obs_tidy = obs_tidy.sub(obs_tidy.min(1), axis=0)\n obs_tidy = obs_tidy.div(obs_tidy.max(1), axis=0).fillna(0)\n elif standard_scale == \"var\":\n obs_tidy -= obs_tidy.min(0)\n obs_tidy = (obs_tidy / obs_tidy.max(0)).fillna(0)\n elif standard_scale is None:\n pass\n else:\n logg.warning(\"Unknown type for standard_scale, ignored\")\n\n if groupby is None or len(categories) <= 1:\n categorical = False\n # dendrogram can only be computed between groupby categories\n dendrogram = False\n else:\n categorical = True\n # get categories colors\n if isinstance(groupby, str) and isinstance(\n adata.obs[groupby].dtype, CategoricalDtype\n ):\n # saved category colors only work when groupby is valid adata.obs\n # categorical column. When groupby is a numerical column\n # or when groupby is a list of columns the colors are assigned on the fly,\n # which may create inconsistencies in multiple runs that require sorting\n # of the categories (eg. when dendrogram is plotted).\n if groupby + \"_colors\" not in adata.uns:\n # if colors are not found, assign a new palette\n # and save it using the same code for embeddings\n from ._tools.scatterplots import _get_palette\n\n _get_palette(adata, groupby)\n groupby_colors = adata.uns[groupby + \"_colors\"]\n else:\n # this case happen when adata.obs[groupby] is numeric\n # the values are converted into a category on the fly\n groupby_colors = None\n\n if dendrogram:\n dendro_data = _reorder_categories_after_dendrogram(\n adata,\n groupby,\n dendrogram_key=_dk(dendrogram),\n var_names=var_names,\n var_group_labels=var_group_labels,\n var_group_positions=var_group_positions,\n categories=categories,\n )\n\n var_group_labels = dendro_data[\"var_group_labels\"]\n var_group_positions = dendro_data[\"var_group_positions\"]\n\n # reorder obs_tidy\n if dendro_data[\"var_names_idx_ordered\"] is not None:\n obs_tidy = obs_tidy.iloc[:, dendro_data[\"var_names_idx_ordered\"]]\n var_names = [var_names[x] for x in dendro_data[\"var_names_idx_ordered\"]]\n\n obs_tidy.index = obs_tidy.index.reorder_categories(\n [categories[x] for x in dendro_data[\"categories_idx_ordered\"]],\n ordered=True,\n )\n\n # reorder groupby colors\n if groupby_colors is not None:\n groupby_colors = [\n groupby_colors[x] for x in dendro_data[\"categories_idx_ordered\"]\n ]\n\n if show_gene_labels is None:\n if len(var_names) <= 50:\n show_gene_labels = True\n else:\n show_gene_labels = False\n logg.warning(\n \"Gene labels are not shown when more than 50 genes are visualized. \"\n \"To show gene labels set `show_gene_labels=True`\"\n )\n if categorical:\n obs_tidy = obs_tidy.sort_index()\n\n colorbar_width = 0.2\n norm = check_colornorm(vmin, vmax, vcenter, norm)\n\n if not swap_axes:\n # define a layout of 2 rows x 4 columns\n # first row is for 'brackets' (if no brackets needed, the height of this row\n # is zero) second row is for main content. This second row is divided into\n # three axes:\n # first ax is for the categories defined by `groupby`\n # second ax is for the heatmap\n # third ax is for the dendrogram\n # fourth ax is for colorbar\n\n dendro_width = 1 if dendrogram else 0\n groupby_width = 0.2 if categorical else 0\n if figsize is None:\n height = 6\n heatmap_width = len(var_names) * 0.3 if show_gene_labels else 8\n width = heatmap_width + dendro_width + groupby_width\n else:\n width, height = figsize\n heatmap_width = width - (dendro_width + groupby_width)\n\n if var_group_positions is not None and len(var_group_positions) > 0:\n # add some space in case 'brackets' want to be plotted on top of the image\n height_ratios = [0.15, height]\n else:\n height_ratios = [0, height]\n\n width_ratios = [\n groupby_width,\n heatmap_width,\n dendro_width,\n colorbar_width,\n ]\n fig = plt.figure(figsize=(width, height))\n\n axs = gridspec.GridSpec(\n nrows=2,\n ncols=4,\n width_ratios=width_ratios,\n wspace=0.15 / width,\n hspace=0.13 / height,\n height_ratios=height_ratios,\n )\n\n heatmap_ax = fig.add_subplot(axs[1, 1])\n kwds.setdefault(\"interpolation\", \"nearest\")\n im = heatmap_ax.imshow(obs_tidy.values, aspect=\"auto\", norm=norm, **kwds)\n\n heatmap_ax.set_ylim(obs_tidy.shape[0] - 0.5, -0.5)\n heatmap_ax.set_xlim(-0.5, obs_tidy.shape[1] - 0.5)\n heatmap_ax.tick_params(axis=\"y\", left=False, labelleft=False)\n heatmap_ax.set_ylabel(\"\")\n heatmap_ax.grid(visible=False)\n\n if show_gene_labels:\n heatmap_ax.tick_params(axis=\"x\", labelsize=\"small\")\n heatmap_ax.set_xticks(np.arange(len(var_names)))\n heatmap_ax.set_xticklabels(var_names, rotation=90)\n else:\n heatmap_ax.tick_params(axis=\"x\", labelbottom=False, bottom=False)\n # plot colorbar\n _plot_colorbar(im, fig, axs[1, 3])\n\n if categorical:\n groupby_ax = fig.add_subplot(axs[1, 0])\n (\n label2code,\n ticks,\n labels,\n groupby_cmap,\n norm,\n ) = _plot_categories_as_colorblocks(\n groupby_ax, obs_tidy, colors=groupby_colors, orientation=\"left\"\n )\n\n # add lines to main heatmap\n line_positions = (\n np.cumsum(obs_tidy.index.value_counts(sort=False))[:-1] - 0.5\n )\n heatmap_ax.hlines(\n line_positions,\n -0.5,\n len(var_names) - 0.5,\n lw=1,\n color=\"black\",\n zorder=10,\n clip_on=False,\n )\n\n if dendrogram:\n dendro_ax = fig.add_subplot(axs[1, 2], sharey=heatmap_ax)\n _plot_dendrogram(\n dendro_ax, adata, groupby, dendrogram_key=_dk(dendrogram), ticks=ticks\n )\n\n # plot group legends on top of heatmap_ax (if given)\n if var_group_positions is not None and len(var_group_positions) > 0:\n gene_groups_ax = fig.add_subplot(axs[0, 1], sharex=heatmap_ax)\n _plot_gene_groups_brackets(\n gene_groups_ax,\n group_positions=var_group_positions,\n group_labels=var_group_labels,\n rotation=var_group_rotation,\n left_adjustment=-0.3,\n right_adjustment=0.3,\n )\n\n # swap axes case\n else:\n # define a layout of 3 rows x 3 columns\n # The first row is for the dendrogram (if not dendrogram height is zero)\n # second row is for main content. This col is divided into three axes:\n # first ax is for the heatmap\n # second ax is for 'brackets' if any (othwerise width is zero)\n # third ax is for colorbar\n\n dendro_height = 0.8 if dendrogram else 0\n groupby_height = 0.13 if categorical else 0\n if figsize is None:\n heatmap_height = len(var_names) * 0.18 if show_gene_labels else 4\n width = 10\n height = heatmap_height + dendro_height + groupby_height\n else:\n width, height = figsize\n heatmap_height = height - (dendro_height + groupby_height)\n\n height_ratios = [dendro_height, heatmap_height, groupby_height]\n\n if var_group_positions is not None and len(var_group_positions) > 0:\n # add some space in case 'brackets' want to be plotted on top of the image\n width_ratios = [width, 0.14, colorbar_width]\n else:\n width_ratios = [width, 0, colorbar_width]\n\n fig = plt.figure(figsize=(width, height))\n axs = gridspec.GridSpec(\n nrows=3,\n ncols=3,\n wspace=0.25 / width,\n hspace=0.3 / height,\n width_ratios=width_ratios,\n height_ratios=height_ratios,\n )\n\n # plot heatmap\n heatmap_ax = fig.add_subplot(axs[1, 0])\n\n kwds.setdefault(\"interpolation\", \"nearest\")\n im = heatmap_ax.imshow(obs_tidy.T.values, aspect=\"auto\", norm=norm, **kwds)\n heatmap_ax.set_xlim(0 - 0.5, obs_tidy.shape[0] - 0.5)\n heatmap_ax.set_ylim(obs_tidy.shape[1] - 0.5, -0.5)\n heatmap_ax.tick_params(axis=\"x\", bottom=False, labelbottom=False)\n heatmap_ax.set_xlabel(\"\")\n heatmap_ax.grid(visible=False)\n if show_gene_labels:\n heatmap_ax.tick_params(axis=\"y\", labelsize=\"small\", length=1)\n heatmap_ax.set_yticks(np.arange(len(var_names)))\n heatmap_ax.set_yticklabels(var_names, rotation=0)\n else:\n heatmap_ax.tick_params(axis=\"y\", labelleft=False, left=False)\n\n if categorical:\n groupby_ax = fig.add_subplot(axs[2, 0])\n (\n label2code,\n ticks,\n labels,\n groupby_cmap,\n norm,\n ) = _plot_categories_as_colorblocks(\n groupby_ax, obs_tidy, colors=groupby_colors, orientation=\"bottom\"\n )\n # add lines to main heatmap\n line_positions = (\n np.cumsum(obs_tidy.index.value_counts(sort=False))[:-1] - 0.5\n )\n heatmap_ax.vlines(\n line_positions,\n -0.5,\n len(var_names) - 0.5,\n lw=1,\n color=\"black\",\n zorder=10,\n clip_on=False,\n )\n\n if dendrogram:\n dendro_ax = fig.add_subplot(axs[0, 0], sharex=heatmap_ax)\n _plot_dendrogram(\n dendro_ax,\n adata,\n groupby,\n dendrogram_key=_dk(dendrogram),\n ticks=ticks,\n orientation=\"top\",\n )\n\n # plot group legends next to the heatmap_ax (if given)\n if var_group_positions is not None and len(var_group_positions) > 0:\n gene_groups_ax = fig.add_subplot(axs[1, 1])\n arr = []\n for idx, (label, pos) in enumerate(\n zip(var_group_labels, var_group_positions)\n ):\n label_code = label2code[label] if var_groups_subset_of_groupby else idx\n arr += [label_code] * (pos[1] + 1 - pos[0])\n gene_groups_ax.imshow(\n np.array([arr]).T, aspect=\"auto\", cmap=groupby_cmap, norm=norm\n )\n gene_groups_ax.axis(\"off\")\n\n # plot colorbar\n _plot_colorbar(im, fig, axs[1, 2])\n\n return_ax_dict = {\"heatmap_ax\": heatmap_ax}\n if categorical:\n return_ax_dict[\"groupby_ax\"] = groupby_ax\n if dendrogram:\n return_ax_dict[\"dendrogram_ax\"] = dendro_ax\n if var_group_positions is not None and len(var_group_positions) > 0:\n return_ax_dict[\"gene_groups_ax\"] = gene_groups_ax\n\n _utils.savefig_or_show(\"heatmap\", show=show, save=save)\n show = settings.autoshow if show is None else show\n if show:\n return None\n return return_ax_dict\n\n\n@old_positionals(\n \"use_raw\",\n \"log\",\n \"dendrogram\",\n \"gene_symbols\",\n \"var_group_positions\",\n \"var_group_labels\",\n \"layer\",\n \"show\",\n \"save\",\n \"figsize\",\n)\n@_doc_params(show_save_ax=doc_show_save_ax, common_plot_args=doc_common_plot_args)\ndef tracksplot(\n adata: AnnData,\n var_names: _VarNames | Mapping[str, _VarNames],\n groupby: str | Sequence[str],\n *,\n use_raw: bool | None = None,\n log: bool = False,\n dendrogram: bool | str = False,\n gene_symbols: str | None = None,\n var_group_positions: Sequence[tuple[int, int]] | None = None,\n var_group_labels: Sequence[str] | None = None,\n layer: str | None = None,\n show: bool | None = None,\n save: str | bool | None = None,\n figsize: tuple[float, float] | None = None,\n **kwds,\n) -> dict[str, Axes] | None:\n \"\"\"\\\n In this type of plot each var_name is plotted as a filled line plot where the\n y values correspond to the var_name values and x is each of the cells. Best results\n are obtained when using raw counts that are not log.\n\n `groupby` is required to sort and order the values using the respective group\n and should be a categorical value.\n\n Parameters\n ----------\n {common_plot_args}\n {show_save_ax}\n **kwds\n Are passed to :func:`~seaborn.heatmap`.\n\n Returns\n -------\n A list of :class:`~matplotlib.axes.Axes`.\n\n Examples\n --------\n\n Using var_names as list:\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n sc.pl.tracksplot(adata, markers, groupby='bulk_labels', dendrogram=True)\n\n Using var_names as dict:\n\n .. plot::\n :context: close-figs\n\n markers = {{'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}}\n sc.pl.tracksplot(adata, markers, groupby='bulk_labels', dendrogram=True)\n\n .. currentmodule:: scanpy\n\n See also\n --------\n pl.rank_genes_groups_tracksplot: to plot marker genes identified using the :func:`~scanpy.tl.rank_genes_groups` function.\n \"\"\"\n\n if groupby not in adata.obs_keys() or adata.obs[groupby].dtype.name != \"category\":\n raise ValueError(\n \"groupby has to be a valid categorical observation. \"\n f\"Given value: {groupby}, valid categorical observations: \"\n f'{[x for x in adata.obs_keys() if adata.obs[x].dtype.name == \"category\"]}'\n )\n\n var_names, var_group_labels, var_group_positions = _check_var_names_type(\n var_names, var_group_labels, var_group_positions\n )\n\n categories, obs_tidy = _prepare_dataframe(\n adata,\n var_names,\n groupby,\n use_raw=use_raw,\n log=log,\n num_categories=None, # TODO: fix this line\n gene_symbols=gene_symbols,\n layer=layer,\n )\n\n # get categories colors:\n if groupby + \"_colors\" not in adata.uns:\n from ._utils import _set_default_colors_for_categorical_obs\n\n _set_default_colors_for_categorical_obs(adata, groupby)\n groupby_colors = adata.uns[groupby + \"_colors\"]\n\n if dendrogram:\n # compute dendrogram if needed and reorder\n # rows and columns to match leaves order.\n dendro_data = _reorder_categories_after_dendrogram(\n adata,\n groupby,\n dendrogram_key=_dk(dendrogram),\n var_names=var_names,\n var_group_labels=var_group_labels,\n var_group_positions=var_group_positions,\n categories=categories,\n )\n # reorder obs_tidy\n if dendro_data[\"var_names_idx_ordered\"] is not None:\n obs_tidy = obs_tidy.iloc[:, dendro_data[\"var_names_idx_ordered\"]]\n var_names = [var_names[x] for x in dendro_data[\"var_names_idx_ordered\"]]\n\n obs_tidy.index = obs_tidy.index.reorder_categories(\n [categories[x] for x in dendro_data[\"categories_idx_ordered\"]],\n ordered=True,\n )\n categories = [categories[x] for x in dendro_data[\"categories_idx_ordered\"]]\n\n groupby_colors = [\n groupby_colors[x] for x in dendro_data[\"categories_idx_ordered\"]\n ]\n\n obs_tidy = obs_tidy.sort_index()\n\n # obtain the start and end of each category and make\n # a list of ranges that will be used to plot a different\n # color\n cumsum = [0] + list(np.cumsum(obs_tidy.index.value_counts(sort=False)))\n x_values = [(x, y) for x, y in zip(cumsum[:-1], cumsum[1:])]\n\n dendro_height = 1 if dendrogram else 0\n\n groupby_height = 0.24\n # +2 because of dendrogram on top and categories at bottom\n num_rows = len(var_names) + 2\n if figsize is None:\n width = 12\n track_height = 0.25\n else:\n width, height = figsize\n track_height = (height - (dendro_height + groupby_height)) / len(var_names)\n\n height_ratios = [dendro_height] + [track_height] * len(var_names) + [groupby_height]\n height = sum(height_ratios)\n\n obs_tidy = obs_tidy.T\n\n fig = plt.figure(figsize=(width, height))\n axs = gridspec.GridSpec(\n ncols=2,\n nrows=num_rows,\n wspace=1.0 / width,\n hspace=0,\n height_ratios=height_ratios,\n width_ratios=[width, 0.14],\n )\n axs_list = []\n first_ax = None\n for idx, var in enumerate(var_names):\n ax_idx = idx + 1 # this is because of the dendrogram\n if first_ax is None:\n ax = fig.add_subplot(axs[ax_idx, 0])\n first_ax = ax\n else:\n ax = fig.add_subplot(axs[ax_idx, 0], sharex=first_ax)\n axs_list.append(ax)\n for cat_idx, category in enumerate(categories):\n x_start, x_end = x_values[cat_idx]\n ax.fill_between(\n range(x_start, x_end),\n 0,\n obs_tidy.iloc[idx, x_start:x_end],\n lw=0.1,\n color=groupby_colors[cat_idx],\n )\n\n # remove the xticks labels except for the last processed plot.\n # Because the plots share the x axis it is redundant and less compact\n # to plot the axis for each plot\n if idx < len(var_names) - 1:\n ax.tick_params(labelbottom=False, labeltop=False, bottom=False, top=False)\n ax.set_xlabel(\"\")\n if log:\n ax.set_yscale(\"log\")\n ax.spines[\"left\"].set_visible(False)\n ax.spines[\"top\"].set_visible(False)\n ax.spines[\"bottom\"].set_visible(False)\n ax.grid(visible=False)\n ymin, ymax = ax.get_ylim()\n ymax = int(ymax)\n ax.set_yticks([ymax])\n ax.set_yticklabels([str(ymax)], ha=\"left\", va=\"top\")\n ax.spines[\"right\"].set_position((\"axes\", 1.01))\n ax.tick_params(\n axis=\"y\",\n labelsize=\"x-small\",\n right=True,\n left=False,\n length=2,\n which=\"both\",\n labelright=True,\n labelleft=False,\n direction=\"in\",\n )\n ax.set_ylabel(var, rotation=0, fontsize=\"small\", ha=\"right\", va=\"bottom\")\n ax.yaxis.set_label_coords(-0.005, 0.1)\n ax.set_xlim(0, x_end)\n ax.tick_params(axis=\"x\", bottom=False, labelbottom=False)\n\n # the ax to plot the groupby categories is split to add a small space\n # between the rest of the plot and the categories\n axs2 = gridspec.GridSpecFromSubplotSpec(\n 2, 1, subplot_spec=axs[num_rows - 1, 0], height_ratios=[1, 1]\n )\n\n groupby_ax = fig.add_subplot(axs2[1])\n\n label2code, ticks, labels, groupby_cmap, norm = _plot_categories_as_colorblocks(\n groupby_ax, obs_tidy.T, colors=groupby_colors, orientation=\"bottom\"\n )\n # add lines to plot\n overlay_ax = fig.add_subplot(axs[1:-1, 0], sharex=first_ax)\n line_positions = np.cumsum(obs_tidy.T.index.value_counts(sort=False))[:-1]\n overlay_ax.vlines(line_positions, 0, 1, lw=0.5, linestyle=\"--\")\n overlay_ax.axis(\"off\")\n overlay_ax.set_ylim(0, 1)\n\n if dendrogram:\n dendro_ax = fig.add_subplot(axs[0], sharex=first_ax)\n _plot_dendrogram(\n dendro_ax,\n adata,\n groupby,\n dendrogram_key=_dk(dendrogram),\n orientation=\"top\",\n ticks=ticks,\n )\n\n if var_group_positions is not None and len(var_group_positions) > 0:\n gene_groups_ax = fig.add_subplot(axs[1:-1, 1])\n arr = []\n for idx, pos in enumerate(var_group_positions):\n arr += [idx] * (pos[1] + 1 - pos[0])\n\n gene_groups_ax.imshow(\n np.array([arr]).T, aspect=\"auto\", cmap=groupby_cmap, norm=norm\n )\n gene_groups_ax.axis(\"off\")\n\n return_ax_dict = {\"track_axes\": axs_list, \"groupby_ax\": groupby_ax}\n if dendrogram:\n return_ax_dict[\"dendrogram_ax\"] = dendro_ax\n if var_group_positions is not None and len(var_group_positions) > 0:\n return_ax_dict[\"gene_groups_ax\"] = gene_groups_ax\n\n _utils.savefig_or_show(\"tracksplot\", show=show, save=save)\n show = settings.autoshow if show is None else show\n if show:\n return None\n return return_ax_dict\n\n\n@_doc_params(show_save_ax=doc_show_save_ax)\ndef dendrogram(\n adata: AnnData,\n groupby: str,\n *,\n dendrogram_key: str | None = None,\n orientation: Literal[\"top\", \"bottom\", \"left\", \"right\"] = \"top\",\n remove_labels: bool = False,\n show: bool | None = None,\n save: str | bool | None = None,\n ax: Axes | None = None,\n) -> Axes:\n \"\"\"\\\n Plots a dendrogram of the categories defined in `groupby`.\n\n See :func:`~scanpy.tl.dendrogram`.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n groupby\n Categorical data column used to create the dendrogram\n dendrogram_key\n Key under with the dendrogram information was stored.\n By default the dendrogram information is stored under\n `.uns[f'dendrogram_{{groupby}}']`.\n orientation\n Origin of the tree. Will grow into the opposite direction.\n remove_labels\n Don’t draw labels. Used e.g. by :func:`scanpy.pl.matrixplot`\n to annotate matrix columns/rows.\n {show_save_ax}\n\n Returns\n -------\n :class:`matplotlib.axes.Axes`\n\n Examples\n --------\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.tl.dendrogram(adata, 'bulk_labels')\n sc.pl.dendrogram(adata, 'bulk_labels')\n\n .. currentmodule:: scanpy\n\n \"\"\"\n if ax is None:\n _, ax = plt.subplots()\n _plot_dendrogram(\n ax,\n adata,\n groupby,\n dendrogram_key=dendrogram_key,\n remove_labels=remove_labels,\n orientation=orientation,\n )\n _utils.savefig_or_show(\"dendrogram\", show=show, save=save)\n return ax\n\n\n@old_positionals(\n \"show_correlation_numbers\",\n \"dendrogram\",\n \"figsize\",\n \"show\",\n \"save\",\n \"ax\",\n \"vmin\",\n \"vmax\",\n \"vcenter\",\n \"norm\",\n)\n@_doc_params(show_save_ax=doc_show_save_ax, vminmax=doc_vboundnorm)\ndef correlation_matrix(\n adata: AnnData,\n groupby: str,\n *,\n show_correlation_numbers: bool = False,\n dendrogram: bool | str | None = None,\n figsize: tuple[float, float] | None = None,\n show: bool | None = None,\n save: str | bool | None = None,\n ax: Axes | None = None,\n vmin: float | None = None,\n vmax: float | None = None,\n vcenter: float | None = None,\n norm: Normalize | None = None,\n **kwds,\n) -> list[Axes] | None:\n \"\"\"\\\n Plots the correlation matrix computed as part of `sc.tl.dendrogram`.\n\n Parameters\n ----------\n adata\n groupby\n Categorical data column used to create the dendrogram\n show_correlation_numbers\n If `show_correlation=True`, plot the correlation on top of each cell.\n dendrogram\n If True or a valid dendrogram key, a dendrogram based on the\n hierarchical clustering between the `groupby` categories is added.\n The dendrogram is computed using :func:`scanpy.tl.dendrogram`.\n If `tl.dendrogram` has not been called previously,\n the function is called with default parameters.\n figsize\n By default a figure size that aims to produce a squared correlation\n matrix plot is used. Format is (width, height)\n {show_save_ax}\n {vminmax}\n **kwds\n Only if `show_correlation` is True:\n Are passed to :func:`matplotlib.pyplot.pcolormesh` when plotting the\n correlation heatmap. `cmap` can be used to change the color palette.\n\n Returns\n -------\n If `show=False`, returns a list of :class:`matplotlib.axes.Axes` objects.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> sc.tl.dendrogram(adata, 'bulk_labels')\n >>> sc.pl.correlation_matrix(adata, 'bulk_labels')\n \"\"\"\n\n dendrogram_key = _get_dendrogram_key(adata, _dk(dendrogram), groupby)\n\n index = adata.uns[dendrogram_key][\"categories_idx_ordered\"]\n corr_matrix = adata.uns[dendrogram_key][\"correlation_matrix\"]\n # reorder matrix columns according to the dendrogram\n if dendrogram is None:\n dendrogram = ax is None\n if dendrogram:\n if ax is not None:\n raise ValueError(\"Can only plot dendrogram when not plotting to an axis\")\n assert (len(index)) == corr_matrix.shape[0]\n corr_matrix = corr_matrix[index, :]\n corr_matrix = corr_matrix[:, index]\n labels = list(adata.obs[groupby].cat.categories)\n labels = np.array(labels).astype(\"str\")[index]\n else:\n labels = adata.obs[groupby].cat.categories\n num_rows = corr_matrix.shape[0]\n colorbar_height = 0.2\n dendrogram_width = 1.8 if dendrogram else 0\n if figsize is None:\n corr_matrix_height = num_rows * 0.6\n height = corr_matrix_height + colorbar_height\n width = corr_matrix_height + dendrogram_width\n else:\n width, height = figsize\n corr_matrix_height = height - colorbar_height\n\n fig = plt.figure(figsize=(width, height)) if ax is None else None\n # layout with 2 rows and 2 columns:\n # row 1: dendrogram + correlation matrix\n # row 2: nothing + colormap bar (horizontal)\n gs = gridspec.GridSpec(\n nrows=2,\n ncols=2,\n width_ratios=[dendrogram_width, corr_matrix_height],\n height_ratios=[corr_matrix_height, colorbar_height],\n wspace=0.01,\n hspace=0.05,\n )\n\n axs = []\n corr_matrix_ax = fig.add_subplot(gs[1]) if ax is None else ax\n if dendrogram:\n dendro_ax = fig.add_subplot(gs[0], sharey=corr_matrix_ax)\n _plot_dendrogram(\n dendro_ax,\n adata,\n groupby,\n dendrogram_key=dendrogram_key,\n remove_labels=True,\n orientation=\"left\",\n ticks=np.arange(corr_matrix.shape[0]) + 0.5,\n )\n axs.append(dendro_ax)\n # define some default pcolormesh parameters\n if \"edgecolors\" not in kwds:\n if corr_matrix.shape[0] > 30:\n # when there are too many rows it is better to remove\n # the black lines surrounding the boxes in the heatmap\n kwds[\"edgecolors\"] = \"none\"\n else:\n kwds[\"edgecolors\"] = \"black\"\n kwds.setdefault(\"linewidth\", 0.01)\n if vmax is None and vmin is None and norm is None:\n vmax = 1\n vmin = -1\n norm = check_colornorm(vmin, vmax, vcenter, norm)\n if \"cmap\" not in kwds:\n # by default use a divergent color map\n kwds[\"cmap\"] = \"bwr\"\n\n img_mat = corr_matrix_ax.pcolormesh(corr_matrix, norm=norm, **kwds)\n corr_matrix_ax.set_xlim(0, num_rows)\n corr_matrix_ax.set_ylim(0, num_rows)\n\n corr_matrix_ax.yaxis.tick_right()\n corr_matrix_ax.set_yticks(np.arange(corr_matrix.shape[0]) + 0.5)\n corr_matrix_ax.set_yticklabels(labels)\n\n corr_matrix_ax.xaxis.set_tick_params(labeltop=True)\n corr_matrix_ax.xaxis.set_tick_params(labelbottom=False)\n corr_matrix_ax.set_xticks(np.arange(corr_matrix.shape[0]) + 0.5)\n corr_matrix_ax.set_xticklabels(labels, rotation=45, ha=\"left\")\n\n for ax_name in \"xy\":\n corr_matrix_ax.tick_params(axis=ax_name, which=\"both\", bottom=False, top=False)\n\n if show_correlation_numbers:\n for row, col in product(range(num_rows), repeat=2):\n corr_matrix_ax.text(\n row + 0.5,\n col + 0.5,\n f\"{corr_matrix[row, col]:.2f}\",\n ha=\"center\",\n va=\"center\",\n )\n\n axs.append(corr_matrix_ax)\n\n if ax is None: # Plot colorbar\n colormap_ax = fig.add_subplot(gs[3])\n cobar = plt.colorbar(img_mat, cax=colormap_ax, orientation=\"horizontal\")\n cobar.solids.set_edgecolor(\"face\")\n axs.append(colormap_ax)\n\n show = settings.autoshow if show is None else show\n _utils.savefig_or_show(\"correlation_matrix\", show=show, save=save)\n if ax is not None or show:\n return None\n return axs\n\n\ndef _prepare_dataframe(\n adata: AnnData,\n var_names: _VarNames | Mapping[str, _VarNames],\n groupby: str | Sequence[str] | None = None,\n *,\n use_raw: bool | None = None,\n log: bool = False,\n num_categories: int = 7,\n layer: str | None = None,\n gene_symbols: str | None = None,\n) -> tuple[Sequence[str], pd.DataFrame]:\n \"\"\"\n Given the anndata object, prepares a data frame in which the row index are the categories\n defined by group by and the columns correspond to var_names.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n var_names\n `var_names` should be a valid subset of `adata.var_names`.\n groupby\n The key of the observation grouping to consider. It is expected that\n groupby is a categorical. If groupby is not a categorical observation,\n it would be subdivided into `num_categories`.\n use_raw\n Whether to use `raw` attribute of `adata`. Defaults to `True` if `.raw` is present.\n log\n Use the log of the values.\n layer\n AnnData layer to use. Takes precedence over `use_raw`\n num_categories\n Only used if groupby observation is not categorical. This value\n determines the number of groups into which the groupby observation\n should be subdivided.\n gene_symbols\n Key for field in .var that stores gene symbols.\n\n Returns\n -------\n Tuple of `pandas.DataFrame` and list of categories.\n \"\"\"\n\n sanitize_anndata(adata)\n use_raw = _check_use_raw(adata, use_raw, layer=layer)\n if isinstance(var_names, str):\n var_names = [var_names]\n\n groupby_index = None\n if groupby is not None:\n if isinstance(groupby, str):\n # if not a list, turn into a list\n groupby = [groupby]\n for group in groupby:\n if group not in list(adata.obs_keys()) + [adata.obs.index.name]:\n if adata.obs.index.name is not None:\n msg = f' or index name \"{adata.obs.index.name}\"'\n else:\n msg = \"\"\n raise ValueError(\n \"groupby has to be a valid observation. \"\n f\"Given {group}, is not in observations: {adata.obs_keys()}\" + msg\n )\n if group in adata.obs.columns and group == adata.obs.index.name:\n raise ValueError(\n f\"Given group {group} is both and index and a column level, \"\n \"which is ambiguous.\"\n )\n if group == adata.obs.index.name:\n groupby_index = group\n if groupby_index is not None:\n # obs_tidy contains adata.obs.index\n # and does not need to be given\n groupby = groupby.copy() # copy to not modify user passed parameter\n groupby.remove(groupby_index)\n keys = list(groupby) + list(np.unique(var_names))\n obs_tidy = get.obs_df(\n adata, keys=keys, layer=layer, use_raw=use_raw, gene_symbols=gene_symbols\n )\n assert np.all(np.array(keys) == np.array(obs_tidy.columns))\n\n if groupby_index is not None:\n # reset index to treat all columns the same way.\n obs_tidy.reset_index(inplace=True)\n groupby.append(groupby_index)\n\n if groupby is None:\n categorical = pd.Series(np.repeat(\"\", len(obs_tidy))).astype(\"category\")\n elif len(groupby) == 1 and is_numeric_dtype(obs_tidy[groupby[0]]):\n # if the groupby column is not categorical, turn it into one\n # by subdividing into `num_categories` categories\n categorical = pd.cut(obs_tidy[groupby[0]], num_categories)\n elif len(groupby) == 1:\n categorical = obs_tidy[groupby[0]].astype(\"category\")\n categorical.name = groupby[0]\n else:\n # join the groupby values using \"_\" to make a new 'category'\n categorical = obs_tidy[groupby].apply(\"_\".join, axis=1).astype(\"category\")\n categorical.name = \"_\".join(groupby)\n\n # preserve category order\n from itertools import product\n\n order = {\n \"_\".join(k): idx\n for idx, k in enumerate(\n product(*(obs_tidy[g].cat.categories for g in groupby))\n )\n }\n categorical = categorical.cat.reorder_categories(\n sorted(categorical.cat.categories, key=lambda x: order[x])\n )\n obs_tidy = obs_tidy[var_names].set_index(categorical)\n categories = obs_tidy.index.categories\n\n if log:\n obs_tidy = np.log1p(obs_tidy)\n\n return categories, obs_tidy\n\n\ndef _plot_gene_groups_brackets(\n gene_groups_ax: Axes,\n *,\n group_positions: Iterable[tuple[int, int]],\n group_labels: Sequence[str],\n left_adjustment: float = -0.3,\n right_adjustment: float = 0.3,\n rotation: float | None = None,\n orientation: Literal[\"top\", \"right\"] = \"top\",\n):\n \"\"\"\\\n Draws brackets that represent groups of genes on the give axis.\n For best results, this axis is located on top of an image whose\n x axis contains gene names.\n\n The gene_groups_ax should share the x axis with the main ax.\n\n Eg: gene_groups_ax = fig.add_subplot(axs[0, 0], sharex=dot_ax)\n\n This function is used by dotplot, heatmap etc.\n\n Parameters\n ----------\n gene_groups_ax\n In this axis the gene marks are drawn\n group_positions\n Each item in the list, should contain the start and end position that the\n bracket should cover.\n Eg. [(0, 4), (5, 8)] means that there are two brackets, one for the var_names (eg genes)\n in positions 0-4 and other for positions 5-8\n group_labels\n List of group labels\n left_adjustment\n adjustment to plot the bracket start slightly before or after the first gene position.\n If the value is negative the start is moved before.\n right_adjustment\n adjustment to plot the bracket end slightly before or after the last gene position\n If the value is negative the start is moved before.\n rotation\n rotation degrees for the labels. If not given, small labels (<4 characters) are not\n rotated, otherwise, they are rotated 90 degrees\n orientation\n location of the brackets. Either `top` or `right`\n Returns\n -------\n None\n \"\"\"\n import matplotlib.patches as patches\n from matplotlib.path import Path\n\n # get the 'brackets' coordinates as lists of start and end positions\n\n left = [x[0] + left_adjustment for x in group_positions]\n right = [x[1] + right_adjustment for x in group_positions]\n\n # verts and codes are used by PathPatch to make the brackets\n verts = []\n codes = []\n if orientation == \"top\":\n # rotate labels if any of them is longer than 4 characters\n if rotation is None and group_labels:\n rotation = 90 if max([len(x) for x in group_labels]) > 4 else 0\n for idx in range(len(left)):\n verts.append((left[idx], 0)) # lower-left\n verts.append((left[idx], 0.6)) # upper-left\n verts.append((right[idx], 0.6)) # upper-right\n verts.append((right[idx], 0)) # lower-right\n\n codes.append(Path.MOVETO)\n codes.append(Path.LINETO)\n codes.append(Path.LINETO)\n codes.append(Path.LINETO)\n\n try:\n group_x_center = left[idx] + float(right[idx] - left[idx]) / 2\n gene_groups_ax.text(\n group_x_center,\n 1.1,\n group_labels[idx],\n ha=\"center\",\n va=\"bottom\",\n rotation=rotation,\n )\n except Exception: # TODO catch the correct exception\n pass\n else:\n top = left\n bottom = right\n for idx in range(len(top)):\n verts.append((0, top[idx])) # upper-left\n verts.append((0.15, top[idx])) # upper-right\n verts.append((0.15, bottom[idx])) # lower-right\n verts.append((0, bottom[idx])) # lower-left\n\n codes.append(Path.MOVETO)\n codes.append(Path.LINETO)\n codes.append(Path.LINETO)\n codes.append(Path.LINETO)\n\n try:\n diff = bottom[idx] - top[idx]\n group_y_center = top[idx] + float(diff) / 2\n if diff * 2 < len(group_labels[idx]):\n # cut label to fit available space\n group_labels[idx] = group_labels[idx][: int(diff * 2)] + \".\"\n gene_groups_ax.text(\n 0.6,\n group_y_center,\n group_labels[idx],\n ha=\"right\",\n va=\"center\",\n rotation=270,\n fontsize=\"small\",\n )\n except Exception as e:\n print(f\"problems {e}\")\n pass\n\n path = Path(verts, codes)\n\n patch = patches.PathPatch(path, facecolor=\"none\", lw=1.5)\n\n gene_groups_ax.add_patch(patch)\n gene_groups_ax.grid(visible=False)\n gene_groups_ax.axis(\"off\")\n # remove y ticks\n gene_groups_ax.tick_params(axis=\"y\", left=False, labelleft=False)\n # remove x ticks and labels\n gene_groups_ax.tick_params(\n axis=\"x\", bottom=False, labelbottom=False, labeltop=False\n )\n\n\ndef _reorder_categories_after_dendrogram(\n adata: AnnData,\n groupby: str | Sequence[str],\n *,\n dendrogram_key: str | None,\n var_names: Sequence[str],\n var_group_labels: Sequence[str] | None,\n var_group_positions: Sequence[tuple[int, int]] | None,\n categories: Sequence[str],\n):\n \"\"\"\\\n Function used by plotting functions that need to reorder the the groupby\n observations based on the dendrogram results.\n\n The function checks if a dendrogram has already been precomputed.\n If not, `sc.tl.dendrogram` is run with default parameters.\n\n The results found in `.uns[dendrogram_key]` are used to reorder\n `var_group_labels` and `var_group_positions`.\n\n\n Returns\n -------\n dictionary with keys:\n 'categories_idx_ordered', 'var_group_names_idx_ordered',\n 'var_group_labels', and 'var_group_positions'\n \"\"\"\n\n dendrogram_key = _get_dendrogram_key(adata, dendrogram_key, groupby)\n\n if isinstance(groupby, str):\n groupby = [groupby]\n\n dendro_info = adata.uns[dendrogram_key]\n if groupby != dendro_info[\"groupby\"]:\n raise ValueError(\n \"Incompatible observations. The precomputed dendrogram contains \"\n f\"information for the observation: '{groupby}' while the plot is \"\n f\"made for the observation: '{dendro_info['groupby']}. \"\n \"Please run `sc.tl.dendrogram` using the right observation.'\"\n )\n\n if categories is None:\n categories = adata.obs[dendro_info[\"groupby\"]].cat.categories\n\n # order of groupby categories\n categories_idx_ordered = dendro_info[\"categories_idx_ordered\"]\n categories_ordered = dendro_info[\"categories_ordered\"]\n\n if len(categories) != len(categories_idx_ordered):\n raise ValueError(\n \"Incompatible observations. Dendrogram data has \"\n f\"{len(categories_idx_ordered)} categories but current groupby \"\n f\"observation {groupby!r} contains {len(categories)} categories. \"\n \"Most likely the underlying groupby observation changed after the \"\n \"initial computation of `sc.tl.dendrogram`. \"\n \"Please run `sc.tl.dendrogram` again.'\"\n )\n\n # reorder var_groups (if any)\n if var_group_positions is None or var_group_labels is None:\n assert var_group_positions is None\n assert var_group_labels is None\n var_names_idx_ordered = None\n elif set(var_group_labels) == set(categories):\n positions_ordered = []\n labels_ordered = []\n position_start = 0\n var_names_idx_ordered = []\n for cat_name in categories_ordered:\n idx = var_group_labels.index(cat_name)\n position = var_group_positions[idx]\n _var_names = var_names[position[0] : position[1] + 1]\n var_names_idx_ordered.extend(range(position[0], position[1] + 1))\n positions_ordered.append(\n (position_start, position_start + len(_var_names) - 1)\n )\n position_start += len(_var_names)\n labels_ordered.append(var_group_labels[idx])\n var_group_labels = labels_ordered\n var_group_positions = positions_ordered\n else:\n logg.warning(\n \"Groups are not reordered because the `groupby` categories \"\n \"and the `var_group_labels` are different.\\n\"\n f\"categories: {_format_first_three_categories(categories)}\\n\"\n f\"var_group_labels: {_format_first_three_categories(var_group_labels)}\"\n )\n var_names_idx_ordered = list(range(len(var_names)))\n\n if var_names_idx_ordered is not None:\n var_names_ordered = [var_names[x] for x in var_names_idx_ordered]\n else:\n var_names_ordered = None\n\n return dict(\n categories_idx_ordered=categories_idx_ordered,\n categories_ordered=dendro_info[\"categories_ordered\"],\n var_names_idx_ordered=var_names_idx_ordered,\n var_names_ordered=var_names_ordered,\n var_group_labels=var_group_labels,\n var_group_positions=var_group_positions,\n )\n\n\ndef _format_first_three_categories(categories):\n categories = list(categories)\n if len(categories) > 3:\n categories = categories[:3] + [\"etc.\"]\n return \", \".join(categories)\n\n\ndef _get_dendrogram_key(\n adata: AnnData, dendrogram_key: str | None, groupby: str | Sequence[str]\n) -> str:\n # the `dendrogram_key` can be a bool an NoneType or the name of the\n # dendrogram key. By default the name of the dendrogram key is 'dendrogram'\n if dendrogram_key is None:\n if isinstance(groupby, str):\n dendrogram_key = f\"dendrogram_{groupby}\"\n elif isinstance(groupby, Sequence):\n dendrogram_key = f'dendrogram_{\"_\".join(groupby)}'\n else:\n msg = f\"groupby has wrong type: {type(groupby).__name__}.\"\n raise AssertionError(msg)\n\n if dendrogram_key not in adata.uns:\n from ..tools._dendrogram import dendrogram\n\n logg.warning(\n f\"dendrogram data not found (using key={dendrogram_key}). \"\n \"Running `sc.tl.dendrogram` with default parameters. For fine \"\n \"tuning it is recommended to run `sc.tl.dendrogram` independently.\"\n )\n dendrogram(adata, groupby, key_added=dendrogram_key)\n\n if \"dendrogram_info\" not in adata.uns[dendrogram_key]:\n raise ValueError(\n f\"The given dendrogram key ({dendrogram_key!r}) does not contain \"\n \"valid dendrogram information.\"\n )\n\n return dendrogram_key\n\n\ndef _plot_dendrogram(\n dendro_ax: Axes,\n adata: AnnData,\n groupby: str | Sequence[str],\n *,\n dendrogram_key: str | None = None,\n orientation: Literal[\"top\", \"bottom\", \"left\", \"right\"] = \"right\",\n remove_labels: bool = True,\n ticks: Collection[float] | None = None,\n):\n \"\"\"\\\n Plots a dendrogram on the given ax using the precomputed dendrogram\n information stored in `.uns[dendrogram_key]`\n \"\"\"\n\n dendrogram_key = _get_dendrogram_key(adata, dendrogram_key, groupby)\n\n def translate_pos(pos_list, new_ticks, old_ticks):\n \"\"\"\\\n transforms the dendrogram coordinates to a given new position.\n The xlabel_pos and orig_ticks should be of the same\n length.\n\n This is mostly done for the heatmap case, where the position of the\n dendrogram leaves needs to be adjusted depending on the category size.\n\n Parameters\n ----------\n pos_list\n list of dendrogram positions that should be translated\n new_ticks\n sorted list of goal tick positions (e.g. [0,1,2,3] )\n old_ticks\n sorted list of original tick positions (e.g. [5, 15, 25, 35]),\n This list is usually the default position used by\n `scipy.cluster.hierarchy.dendrogram`.\n\n Returns\n -------\n translated list of positions\n\n Examples\n --------\n >>> translate_pos(\n ... [5, 15, 20, 21],\n ... [0, 1, 2, 3 ],\n ... [5, 15, 25, 35],\n ... )\n [0, 1, 1.5, 1.6]\n \"\"\"\n # of given coordinates.\n\n if not isinstance(old_ticks, list):\n # assume that the list is a numpy array\n old_ticks = old_ticks.tolist()\n new_xs = []\n for x_val in pos_list:\n if x_val in old_ticks:\n new_x_val = new_ticks[old_ticks.index(x_val)]\n else:\n # find smaller and bigger indices\n idx_next = np.searchsorted(old_ticks, x_val, side=\"left\")\n idx_prev = idx_next - 1\n old_min = old_ticks[idx_prev]\n old_max = old_ticks[idx_next]\n new_min = new_ticks[idx_prev]\n new_max = new_ticks[idx_next]\n new_x_val = ((x_val - old_min) / (old_max - old_min)) * (\n new_max - new_min\n ) + new_min\n new_xs.append(new_x_val)\n return new_xs\n\n dendro_info = adata.uns[dendrogram_key][\"dendrogram_info\"]\n leaves = dendro_info[\"ivl\"]\n icoord = np.array(dendro_info[\"icoord\"])\n dcoord = np.array(dendro_info[\"dcoord\"])\n\n orig_ticks = np.arange(5, len(leaves) * 10 + 5, 10).astype(float)\n # check that ticks has the same length as orig_ticks\n if ticks is not None and len(orig_ticks) != len(ticks):\n logg.warning(\n \"ticks argument does not have the same size as orig_ticks. \"\n \"The argument will be ignored\"\n )\n ticks = None\n\n for xs, ys in zip(icoord, dcoord):\n if ticks is not None:\n xs = translate_pos(xs, ticks, orig_ticks)\n if orientation in [\"right\", \"left\"]:\n xs, ys = ys, xs\n dendro_ax.plot(xs, ys, color=\"#555555\")\n\n dendro_ax.tick_params(bottom=False, top=False, left=False, right=False)\n ticks = ticks if ticks is not None else orig_ticks\n if orientation in [\"right\", \"left\"]:\n dendro_ax.set_yticks(ticks)\n dendro_ax.set_yticklabels(leaves, fontsize=\"small\", rotation=0)\n dendro_ax.tick_params(labelbottom=False, labeltop=False)\n if orientation == \"left\":\n xmin, xmax = dendro_ax.get_xlim()\n dendro_ax.set_xlim(xmax, xmin)\n dendro_ax.tick_params(labelleft=False, labelright=True)\n else:\n dendro_ax.set_xticks(ticks)\n dendro_ax.set_xticklabels(leaves, fontsize=\"small\", rotation=90)\n dendro_ax.tick_params(labelleft=False, labelright=False)\n if orientation == \"bottom\":\n ymin, ymax = dendro_ax.get_ylim()\n dendro_ax.set_ylim(ymax, ymin)\n dendro_ax.tick_params(labeltop=True, labelbottom=False)\n\n if remove_labels:\n dendro_ax.tick_params(\n labelbottom=False, labeltop=False, labelleft=False, labelright=False\n )\n\n dendro_ax.grid(visible=False)\n\n dendro_ax.spines[\"right\"].set_visible(False)\n dendro_ax.spines[\"top\"].set_visible(False)\n dendro_ax.spines[\"left\"].set_visible(False)\n dendro_ax.spines[\"bottom\"].set_visible(False)\n\n\ndef _plot_categories_as_colorblocks(\n groupby_ax: Axes,\n obs_tidy: pd.DataFrame,\n colors=None,\n orientation: Literal[\"top\", \"bottom\", \"left\", \"right\"] = \"left\",\n cmap_name: str = \"tab20\",\n):\n \"\"\"\\\n Plots categories as colored blocks. If orientation is 'left', the categories\n are plotted vertically, otherwise they are plotted horizontally.\n\n Parameters\n ----------\n groupby_ax\n obs_tidy\n colors\n Sequence of valid color names to use for each category.\n orientation\n cmap_name\n Name of colormap to use, in case colors is None\n\n Returns\n -------\n ticks position, labels, colormap\n \"\"\"\n\n groupby = obs_tidy.index.name\n from matplotlib.colors import BoundaryNorm, ListedColormap\n\n if colors is None:\n groupby_cmap = plt.get_cmap(cmap_name)\n else:\n groupby_cmap = ListedColormap(colors, groupby + \"_cmap\")\n norm = BoundaryNorm(np.arange(groupby_cmap.N + 1) - 0.5, groupby_cmap.N)\n\n # determine groupby label positions such that they appear\n # centered next/below to the color code rectangle assigned to the category\n value_sum = 0\n ticks = [] # list of centered position of the labels\n labels = []\n label2code = {} # dictionary of numerical values asigned to each label\n for code, (label, value) in enumerate(\n obs_tidy.index.value_counts(sort=False).items()\n ):\n ticks.append(value_sum + (value / 2))\n labels.append(label)\n value_sum += value\n label2code[label] = code\n\n groupby_ax.grid(visible=False)\n\n if orientation == \"left\":\n groupby_ax.imshow(\n np.array([[label2code[lab] for lab in obs_tidy.index]]).T,\n aspect=\"auto\",\n cmap=groupby_cmap,\n norm=norm,\n )\n if len(labels) > 1:\n groupby_ax.set_yticks(ticks)\n groupby_ax.set_yticklabels(labels)\n\n # remove y ticks\n groupby_ax.tick_params(axis=\"y\", left=False, labelsize=\"small\")\n # remove x ticks and labels\n groupby_ax.tick_params(axis=\"x\", bottom=False, labelbottom=False)\n\n # remove surrounding lines\n groupby_ax.spines[\"right\"].set_visible(False)\n groupby_ax.spines[\"top\"].set_visible(False)\n groupby_ax.spines[\"left\"].set_visible(False)\n groupby_ax.spines[\"bottom\"].set_visible(False)\n\n groupby_ax.set_ylabel(groupby)\n else:\n groupby_ax.imshow(\n np.array([[label2code[lab] for lab in obs_tidy.index]]),\n aspect=\"auto\",\n cmap=groupby_cmap,\n norm=norm,\n )\n if len(labels) > 1:\n groupby_ax.set_xticks(ticks)\n # if the labels are small do not rotate them\n rotation = 0 if max(len(str(x)) for x in labels) < 3 else 90\n groupby_ax.set_xticklabels(labels, rotation=rotation)\n\n # remove x ticks\n groupby_ax.tick_params(axis=\"x\", bottom=False, labelsize=\"small\")\n # remove y ticks and labels\n groupby_ax.tick_params(axis=\"y\", left=False, labelleft=False)\n\n # remove surrounding lines\n groupby_ax.spines[\"right\"].set_visible(False)\n groupby_ax.spines[\"top\"].set_visible(False)\n groupby_ax.spines[\"left\"].set_visible(False)\n groupby_ax.spines[\"bottom\"].set_visible(False)\n\n groupby_ax.set_xlabel(groupby)\n\n return label2code, ticks, labels, groupby_cmap, norm\n\n\ndef _plot_colorbar(mappable, fig, subplot_spec, max_cbar_height: float = 4.0):\n \"\"\"\n Plots a vertical color bar based on mappable.\n The height of the colorbar is min(figure-height, max_cmap_height)\n\n Parameters\n ----------\n mappable\n The image to which the colorbar applies.\n fig\n The figure object\n subplot_spec\n The gridspec subplot. Eg. axs[1,2]\n max_cbar_height\n The maximum colorbar height\n\n Returns\n -------\n color bar ax\n \"\"\"\n width, height = fig.get_size_inches()\n if height > max_cbar_height:\n # to make the colorbar shorter, the\n # ax is split and the lower portion is used.\n axs2 = gridspec.GridSpecFromSubplotSpec(\n 2,\n 1,\n subplot_spec=subplot_spec,\n height_ratios=[height - max_cbar_height, max_cbar_height],\n )\n heatmap_cbar_ax = fig.add_subplot(axs2[1])\n else:\n heatmap_cbar_ax = fig.add_subplot(subplot_spec)\n plt.colorbar(mappable, cax=heatmap_cbar_ax)\n return heatmap_cbar_ax\n\n\ndef _check_var_names_type(var_names, var_group_labels, var_group_positions):\n \"\"\"\n checks if var_names is a dict. Is this is the cases, then set the\n correct values for var_group_labels and var_group_positions\n\n Returns\n -------\n var_names, var_group_labels, var_group_positions\n\n \"\"\"\n if isinstance(var_names, Mapping):\n if var_group_labels is not None or var_group_positions is not None:\n logg.warning(\n \"`var_names` is a dictionary. This will reset the current \"\n \"value of `var_group_labels` and `var_group_positions`.\"\n )\n var_group_labels = []\n _var_names = []\n var_group_positions = []\n start = 0\n for label, vars_list in var_names.items():\n if isinstance(vars_list, str):\n vars_list = [vars_list]\n # use list() in case var_list is a numpy array or pandas series\n _var_names.extend(list(vars_list))\n var_group_labels.append(label)\n var_group_positions.append((start, start + len(vars_list) - 1))\n start += len(vars_list)\n var_names = _var_names\n\n elif isinstance(var_names, str):\n var_names = [var_names]\n\n return var_names, var_group_labels, var_group_positions\n\n\nfrom __future__ import annotations\n\nfrom . import palettes\nfrom ._anndata import (\n clustermap,\n correlation_matrix,\n dendrogram,\n heatmap,\n ranking,\n scatter,\n tracksplot,\n violin,\n)\nfrom ._dotplot import DotPlot, dotplot\nfrom ._matrixplot import MatrixPlot, matrixplot\nfrom ._preprocessing import filter_genes_dispersion, highly_variable_genes\nfrom ._qc import highest_expr_genes\nfrom ._rcmod import set_rcParams_defaults, set_rcParams_scanpy\nfrom ._scrublet import scrublet_score_distribution\nfrom ._stacked_violin import StackedViolin, stacked_violin\nfrom ._tools import (\n dpt_groups_pseudotime,\n dpt_timeseries,\n embedding_density,\n pca_loadings,\n pca_overview,\n pca_scatter,\n pca_variance_ratio,\n rank_genes_groups,\n rank_genes_groups_dotplot,\n rank_genes_groups_heatmap,\n rank_genes_groups_matrixplot,\n rank_genes_groups_stacked_violin,\n rank_genes_groups_tracksplot,\n rank_genes_groups_violin,\n sim,\n)\nfrom ._tools.paga import (\n paga,\n paga_adjacency, # noqa: F401\n paga_compare,\n paga_path,\n)\nfrom ._tools.scatterplots import (\n diffmap,\n draw_graph,\n embedding,\n pca,\n spatial,\n tsne,\n umap,\n)\nfrom ._utils import matrix, timeseries, timeseries_as_heatmap, timeseries_subplot\n\n__all__ = [\n \"palettes\",\n \"clustermap\",\n \"correlation_matrix\",\n \"dendrogram\",\n \"heatmap\",\n \"ranking\",\n \"scatter\",\n \"tracksplot\",\n \"violin\",\n \"DotPlot\",\n \"dotplot\",\n \"MatrixPlot\",\n \"matrixplot\",\n \"filter_genes_dispersion\",\n \"highly_variable_genes\",\n \"highest_expr_genes\",\n \"set_rcParams_defaults\",\n \"set_rcParams_scanpy\",\n \"StackedViolin\",\n \"stacked_violin\",\n \"scrublet_score_distribution\",\n \"dpt_groups_pseudotime\",\n \"dpt_timeseries\",\n \"embedding_density\",\n \"pca_loadings\",\n \"pca_overview\",\n \"pca_scatter\",\n \"pca_variance_ratio\",\n \"rank_genes_groups\",\n \"rank_genes_groups_dotplot\",\n \"rank_genes_groups_heatmap\",\n \"rank_genes_groups_matrixplot\",\n \"rank_genes_groups_stacked_violin\",\n \"rank_genes_groups_tracksplot\",\n \"rank_genes_groups_violin\",\n \"sim\",\n \"paga\",\n \"paga_compare\",\n \"paga_path\",\n \"diffmap\",\n \"draw_graph\",\n \"embedding\",\n \"pca\",\n \"spatial\",\n \"tsne\",\n \"umap\",\n \"matrix\",\n \"timeseries\",\n \"timeseries_as_heatmap\",\n \"timeseries_subplot\",\n]\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nfrom matplotlib import pyplot as plt\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._settings import settings\nfrom .._utils import _doc_params, _empty\nfrom ._baseplot_class import BasePlot, doc_common_groupby_plot_args\nfrom ._docs import doc_common_plot_args, doc_show_save_ax, doc_vboundnorm\nfrom ._utils import (\n _dk,\n check_colornorm,\n fix_kwds,\n make_grid_spec,\n savefig_or_show,\n)\n\nif TYPE_CHECKING:\n from collections.abc import Mapping, Sequence\n from typing import Literal, Self\n\n import pandas as pd\n from anndata import AnnData\n from matplotlib.axes import Axes\n from matplotlib.colors import Colormap, Normalize\n\n from .._utils import Empty\n from ._baseplot_class import _VarNames\n from ._utils import ColorLike, _AxesSubplot\n\n\n@_doc_params(common_plot_args=doc_common_plot_args)\nclass DotPlot(BasePlot):\n \"\"\"\\\n Allows the visualization of two values that are encoded as\n dot size and color. The size usually represents the fraction\n of cells (obs) that have a non-zero value for genes (var).\n\n For each var_name and each `groupby` category a dot is plotted.\n Each dot represents two values: mean expression within each category\n (visualized by color) and fraction of cells expressing the `var_name` in the\n category (visualized by the size of the dot). If `groupby` is not given,\n the dotplot assumes that all data belongs to a single category.\n\n .. note::\n A gene is considered expressed if the expression value in the `adata` (or\n `adata.raw`) is above the specified threshold which is zero by default.\n\n An example of dotplot usage is to visualize, for multiple marker genes,\n the mean value and the percentage of cells expressing the gene\n across multiple clusters.\n\n Parameters\n ----------\n {common_plot_args}\n title\n Title for the figure\n expression_cutoff\n Expression cutoff that is used for binarizing the gene expression and\n determining the fraction of cells expressing given genes. A gene is\n expressed only if the expression value is greater than this threshold.\n mean_only_expressed\n If True, gene expression is averaged only over the cells\n expressing the given genes.\n standard_scale\n Whether or not to standardize that dimension between 0 and 1,\n meaning for each variable or group,\n subtract the minimum and divide each by its maximum.\n kwds\n Are passed to :func:`matplotlib.pyplot.scatter`.\n\n See also\n --------\n :func:`~scanpy.pl.dotplot`: Simpler way to call DotPlot but with less options.\n :func:`~scanpy.pl.rank_genes_groups_dotplot`: to plot marker\n genes identified using the :func:`~scanpy.tl.rank_genes_groups` function.\n\n Examples\n --------\n\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n >>> sc.pl.DotPlot(adata, markers, groupby='bulk_labels').show()\n\n Using var_names as dict:\n\n >>> markers = {{'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}}\n >>> sc.pl.DotPlot(adata, markers, groupby='bulk_labels').show()\n\n \"\"\"\n\n DEFAULT_SAVE_PREFIX = \"dotplot_\"\n # default style parameters\n DEFAULT_COLORMAP = \"Reds\"\n DEFAULT_COLOR_ON = \"dot\"\n DEFAULT_DOT_MAX = None\n DEFAULT_DOT_MIN = None\n DEFAULT_SMALLEST_DOT = 0.0\n DEFAULT_LARGEST_DOT = 200.0\n DEFAULT_DOT_EDGECOLOR = \"black\"\n DEFAULT_DOT_EDGELW = 0.2\n DEFAULT_SIZE_EXPONENT = 1.5\n\n # default legend parameters\n DEFAULT_SIZE_LEGEND_TITLE = \"Fraction of cells\\nin group (%)\"\n DEFAULT_COLOR_LEGEND_TITLE = \"Mean expression\\nin group\"\n DEFAULT_LEGENDS_WIDTH = 1.5 # inches\n DEFAULT_PLOT_X_PADDING = 0.8 # a unit is the distance between two x-axis ticks\n DEFAULT_PLOT_Y_PADDING = 1.0 # a unit is the distance between two y-axis ticks\n\n @old_positionals(\n \"use_raw\",\n \"log\",\n \"num_categories\",\n \"categories_order\",\n \"title\",\n \"figsize\",\n \"gene_symbols\",\n \"var_group_positions\",\n \"var_group_labels\",\n \"var_group_rotation\",\n \"layer\",\n \"expression_cutoff\",\n \"mean_only_expressed\",\n \"standard_scale\",\n \"dot_color_df\",\n \"dot_size_df\",\n \"ax\",\n \"vmin\",\n \"vmax\",\n \"vcenter\",\n \"norm\",\n )\n def __init__(\n self,\n adata: AnnData,\n var_names: _VarNames | Mapping[str, _VarNames],\n groupby: str | Sequence[str],\n *,\n use_raw: bool | None = None,\n log: bool = False,\n num_categories: int = 7,\n categories_order: Sequence[str] | None = None,\n title: str | None = None,\n figsize: tuple[float, float] | None = None,\n gene_symbols: str | None = None,\n var_group_positions: Sequence[tuple[int, int]] | None = None,\n var_group_labels: Sequence[str] | None = None,\n var_group_rotation: float | None = None,\n layer: str | None = None,\n expression_cutoff: float = 0.0,\n mean_only_expressed: bool = False,\n standard_scale: Literal[\"var\", \"group\"] | None = None,\n dot_color_df: pd.DataFrame | None = None,\n dot_size_df: pd.DataFrame | None = None,\n ax: _AxesSubplot | None = None,\n vmin: float | None = None,\n vmax: float | None = None,\n vcenter: float | None = None,\n norm: Normalize | None = None,\n **kwds,\n ) -> None:\n BasePlot.__init__(\n self,\n adata,\n var_names,\n groupby,\n use_raw=use_raw,\n log=log,\n num_categories=num_categories,\n categories_order=categories_order,\n title=title,\n figsize=figsize,\n gene_symbols=gene_symbols,\n var_group_positions=var_group_positions,\n var_group_labels=var_group_labels,\n var_group_rotation=var_group_rotation,\n layer=layer,\n ax=ax,\n vmin=vmin,\n vmax=vmax,\n vcenter=vcenter,\n norm=norm,\n **kwds,\n )\n\n # for if category defined by groupby (if any) compute for each var_name\n # 1. the fraction of cells in the category having a value >expression_cutoff\n # 2. the mean value over the category\n\n # 1. compute fraction of cells having value > expression_cutoff\n # transform obs_tidy into boolean matrix using the expression_cutoff\n obs_bool = self.obs_tidy > expression_cutoff\n\n # compute the sum per group which in the boolean matrix this is the number\n # of values >expression_cutoff, and divide the result by the total number of\n # values in the group (given by `count()`)\n if dot_size_df is None:\n dot_size_df = (\n obs_bool.groupby(level=0, observed=True).sum()\n / obs_bool.groupby(level=0, observed=True).count()\n )\n\n if dot_color_df is None:\n # 2. compute mean expression value value\n if mean_only_expressed:\n dot_color_df = (\n self.obs_tidy.mask(~obs_bool)\n .groupby(level=0, observed=True)\n .mean()\n .fillna(0)\n )\n else:\n dot_color_df = self.obs_tidy.groupby(level=0, observed=True).mean()\n\n if standard_scale == \"group\":\n dot_color_df = dot_color_df.sub(dot_color_df.min(1), axis=0)\n dot_color_df = dot_color_df.div(dot_color_df.max(1), axis=0).fillna(0)\n elif standard_scale == \"var\":\n dot_color_df -= dot_color_df.min(0)\n dot_color_df = (dot_color_df / dot_color_df.max(0)).fillna(0)\n elif standard_scale is None:\n pass\n else:\n logg.warning(\"Unknown type for standard_scale, ignored\")\n else:\n # check that both matrices have the same shape\n if dot_color_df.shape != dot_size_df.shape:\n logg.error(\n \"the given dot_color_df data frame has a different shape than \"\n \"the data frame used for the dot size. Both data frames need \"\n \"to have the same index and columns\"\n )\n\n # Because genes (columns) can be duplicated (e.g. when the\n # same gene is reported as marker gene in two clusters)\n # they need to be removed first,\n # otherwise, the duplicated genes are further duplicated when reordering\n # Eg. A df with columns ['a', 'b', 'a'] after reordering columns\n # with df[['a', 'a', 'b']], results in a df with columns:\n # ['a', 'a', 'a', 'a', 'b']\n\n unique_var_names, unique_idx = np.unique(\n dot_color_df.columns, return_index=True\n )\n # remove duplicate columns\n if len(unique_var_names) != len(self.var_names):\n dot_color_df = dot_color_df.iloc[:, unique_idx]\n\n # get the same order for rows and columns in the dot_color_df\n # using the order from the doc_size_df\n dot_color_df = dot_color_df.loc[dot_size_df.index][dot_size_df.columns]\n\n self.dot_color_df, self.dot_size_df = (\n df.loc[\n categories_order if categories_order is not None else self.categories\n ]\n for df in (dot_color_df, dot_size_df)\n )\n self.standard_scale = standard_scale\n\n # Set default style parameters\n self.cmap = self.DEFAULT_COLORMAP\n self.dot_max = self.DEFAULT_DOT_MAX\n self.dot_min = self.DEFAULT_DOT_MIN\n self.smallest_dot = self.DEFAULT_SMALLEST_DOT\n self.largest_dot = self.DEFAULT_LARGEST_DOT\n self.color_on = self.DEFAULT_COLOR_ON\n self.size_exponent = self.DEFAULT_SIZE_EXPONENT\n self.grid = False\n self.plot_x_padding = self.DEFAULT_PLOT_X_PADDING\n self.plot_y_padding = self.DEFAULT_PLOT_Y_PADDING\n\n self.dot_edge_color = self.DEFAULT_DOT_EDGECOLOR\n self.dot_edge_lw = self.DEFAULT_DOT_EDGELW\n\n # set legend defaults\n self.color_legend_title = self.DEFAULT_COLOR_LEGEND_TITLE\n self.size_title = self.DEFAULT_SIZE_LEGEND_TITLE\n self.legends_width = self.DEFAULT_LEGENDS_WIDTH\n self.show_size_legend = True\n self.show_colorbar = True\n\n @old_positionals(\n \"cmap\",\n \"color_on\",\n \"dot_max\",\n \"dot_min\",\n \"smallest_dot\",\n \"largest_dot\",\n \"dot_edge_color\",\n \"dot_edge_lw\",\n \"size_exponent\",\n \"grid\",\n \"x_padding\",\n \"y_padding\",\n )\n def style(\n self,\n *,\n cmap: Colormap | str | None | Empty = _empty,\n color_on: Literal[\"dot\", \"square\"] | Empty = _empty,\n dot_max: float | None | Empty = _empty,\n dot_min: float | None | Empty = _empty,\n smallest_dot: float | Empty = _empty,\n largest_dot: float | Empty = _empty,\n dot_edge_color: ColorLike | None | Empty = _empty,\n dot_edge_lw: float | None | Empty = _empty,\n size_exponent: float | Empty = _empty,\n grid: bool | Empty = _empty,\n x_padding: float | Empty = _empty,\n y_padding: float | Empty = _empty,\n ) -> Self:\n r\"\"\"\\\n Modifies plot visual parameters\n\n Parameters\n ----------\n cmap\n String denoting matplotlib color map.\n color_on\n By default the color map is applied to the color of the ``\"dot\"``.\n Optionally, the colormap can be applied to a ``\"square\"`` behind the dot,\n in which case the dot is transparent and only the edge is shown.\n dot_max\n If ``None``, the maximum dot size is set to the maximum fraction value found (e.g. 0.6).\n If given, the value should be a number between 0 and 1.\n All fractions larger than dot_max are clipped to this value.\n dot_min\n If ``None``, the minimum dot size is set to 0.\n If given, the value should be a number between 0 and 1.\n All fractions smaller than dot_min are clipped to this value.\n smallest_dot\n All expression fractions with `dot_min` are plotted with this size.\n largest_dot\n All expression fractions with `dot_max` are plotted with this size.\n dot_edge_color\n Dot edge color.\n When `color_on='dot'`, ``None`` means no edge.\n When `color_on='square'`, ``None`` means that\n the edge color is white for darker colors and black for lighter background square colors.\n dot_edge_lw\n Dot edge line width.\n When `color_on='dot'`, ``None`` means no edge.\n When `color_on='square'`, ``None`` means a line width of 1.5.\n size_exponent\n Dot size is computed as:\n fraction ** size exponent and afterwards scaled to match the\n `smallest_dot` and `largest_dot` size parameters.\n Using a different size exponent changes the relative sizes of the dots\n to each other.\n grid\n Set to true to show grid lines. By default grid lines are not shown.\n Further configuration of the grid lines can be achieved directly on the\n returned ax.\n x_padding\n Space between the plot left/right borders and the dots center. A unit\n is the distance between the x ticks. Only applied when color_on = dot\n y_padding\n Space between the plot top/bottom borders and the dots center. A unit is\n the distance between the y ticks. Only applied when color_on = dot\n\n Returns\n -------\n :class:`~scanpy.pl.DotPlot`\n\n Examples\n -------\n\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n\n Change color map and apply it to the square behind the dot\n\n >>> sc.pl.DotPlot(adata, markers, groupby='bulk_labels') \\\n ... .style(cmap='RdBu_r', color_on='square').show()\n\n Add edge to dots and plot a grid\n\n >>> sc.pl.DotPlot(adata, markers, groupby='bulk_labels') \\\n ... .style(dot_edge_color='black', dot_edge_lw=1, grid=True) \\\n ... .show()\n \"\"\"\n super().style(cmap=cmap)\n\n if dot_max is not _empty:\n self.dot_max = dot_max\n if dot_min is not _empty:\n self.dot_min = dot_min\n if smallest_dot is not _empty:\n self.smallest_dot = smallest_dot\n if largest_dot is not _empty:\n self.largest_dot = largest_dot\n if color_on is not _empty:\n self.color_on = color_on\n if size_exponent is not _empty:\n self.size_exponent = size_exponent\n if dot_edge_color is not _empty:\n self.dot_edge_color = dot_edge_color\n if dot_edge_lw is not _empty:\n self.dot_edge_lw = dot_edge_lw\n if grid is not _empty:\n self.grid = grid\n if x_padding is not _empty:\n self.plot_x_padding = x_padding\n if y_padding is not _empty:\n self.plot_y_padding = y_padding\n\n return self\n\n @old_positionals(\n \"show\",\n \"show_size_legend\",\n \"show_colorbar\",\n \"size_title\",\n \"colorbar_title\",\n \"width\",\n )\n def legend(\n self,\n *,\n show: bool | None = True,\n show_size_legend: bool | None = True,\n show_colorbar: bool | None = True,\n size_title: str | None = DEFAULT_SIZE_LEGEND_TITLE,\n colorbar_title: str | None = DEFAULT_COLOR_LEGEND_TITLE,\n width: float | None = DEFAULT_LEGENDS_WIDTH,\n ) -> Self:\n \"\"\"\\\n Configures dot size and the colorbar legends\n\n Parameters\n ----------\n show\n Set to `False` to hide the default plot of the legends. This sets the\n legend width to zero, which will result in a wider main plot.\n show_size_legend\n Set to `False` to hide the dot size legend\n show_colorbar\n Set to `False` to hide the colorbar legend\n size_title\n Title for the dot size legend. Use '\\\\n' to add line breaks. Appears on top\n of dot sizes\n colorbar_title\n Title for the color bar. Use '\\\\n' to add line breaks. Appears on top of the\n color bar\n width\n Width of the legends area. The unit is the same as in matplotlib (inches).\n\n Returns\n -------\n :class:`~scanpy.pl.DotPlot`\n\n Examples\n --------\n\n Set color bar title:\n\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> markers = {'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}\n >>> dp = sc.pl.DotPlot(adata, markers, groupby='bulk_labels')\n >>> dp.legend(colorbar_title='log(UMI counts + 1)').show()\n \"\"\"\n\n if not show:\n # turn of legends by setting width to 0\n self.legends_width = 0\n else:\n self.color_legend_title = colorbar_title\n self.size_title = size_title\n self.legends_width = width\n self.show_size_legend = show_size_legend\n self.show_colorbar = show_colorbar\n\n return self\n\n def _plot_size_legend(self, size_legend_ax: Axes):\n # for the dot size legend, use step between dot_max and dot_min\n # based on how different they are.\n diff = self.dot_max - self.dot_min\n if 0.3 < diff <= 0.6:\n step = 0.1\n elif diff <= 0.3:\n step = 0.05\n else:\n step = 0.2\n # a descending range that is afterwards inverted is used\n # to guarantee that dot_max is in the legend.\n size_range = np.arange(self.dot_max, self.dot_min, step * -1)[::-1]\n if self.dot_min != 0 or self.dot_max != 1:\n dot_range = self.dot_max - self.dot_min\n size_values = (size_range - self.dot_min) / dot_range\n else:\n size_values = size_range\n\n size = size_values**self.size_exponent\n size = size * (self.largest_dot - self.smallest_dot) + self.smallest_dot\n\n # plot size bar\n size_legend_ax.scatter(\n np.arange(len(size)) + 0.5,\n np.repeat(0, len(size)),\n s=size,\n color=\"gray\",\n edgecolor=\"black\",\n linewidth=self.dot_edge_lw,\n zorder=100,\n )\n size_legend_ax.set_xticks(np.arange(len(size)) + 0.5)\n labels = [f\"{np.round((x * 100), decimals=0).astype(int)}\" for x in size_range]\n size_legend_ax.set_xticklabels(labels, fontsize=\"small\")\n\n # remove y ticks and labels\n size_legend_ax.tick_params(\n axis=\"y\", left=False, labelleft=False, labelright=False\n )\n\n # remove surrounding lines\n size_legend_ax.spines[\"right\"].set_visible(False)\n size_legend_ax.spines[\"top\"].set_visible(False)\n size_legend_ax.spines[\"left\"].set_visible(False)\n size_legend_ax.spines[\"bottom\"].set_visible(False)\n size_legend_ax.grid(visible=False)\n\n ymax = size_legend_ax.get_ylim()[1]\n size_legend_ax.set_ylim(-1.05 - self.largest_dot * 0.003, 4)\n size_legend_ax.set_title(self.size_title, y=ymax + 0.45, size=\"small\")\n\n xmin, xmax = size_legend_ax.get_xlim()\n size_legend_ax.set_xlim(xmin - 0.15, xmax + 0.5)\n\n def _plot_legend(self, legend_ax, return_ax_dict, normalize):\n # to maintain the fixed height size of the legends, a\n # spacer of variable height is added at the bottom.\n # The structure for the legends is:\n # first row: variable space to keep the other rows of\n # the same size (avoid stretching)\n # second row: legend for dot size\n # third row: spacer to avoid color and size legend titles to overlap\n # fourth row: colorbar\n\n cbar_legend_height = self.min_figure_height * 0.08\n size_legend_height = self.min_figure_height * 0.27\n spacer_height = self.min_figure_height * 0.3\n\n height_ratios = [\n self.height - size_legend_height - cbar_legend_height - spacer_height,\n size_legend_height,\n spacer_height,\n cbar_legend_height,\n ]\n fig, legend_gs = make_grid_spec(\n legend_ax, nrows=4, ncols=1, height_ratios=height_ratios\n )\n\n if self.show_size_legend:\n size_legend_ax = fig.add_subplot(legend_gs[1])\n self._plot_size_legend(size_legend_ax)\n return_ax_dict[\"size_legend_ax\"] = size_legend_ax\n\n if self.show_colorbar:\n color_legend_ax = fig.add_subplot(legend_gs[3])\n\n self._plot_colorbar(color_legend_ax, normalize)\n return_ax_dict[\"color_legend_ax\"] = color_legend_ax\n\n def _mainplot(self, ax: Axes):\n # work on a copy of the dataframes. This is to avoid changes\n # on the original data frames after repetitive calls to the\n # DotPlot object, for example once with swap_axes and other without\n\n _color_df = self.dot_color_df.copy()\n _size_df = self.dot_size_df.copy()\n if self.var_names_idx_order is not None:\n _color_df = _color_df.iloc[:, self.var_names_idx_order]\n _size_df = _size_df.iloc[:, self.var_names_idx_order]\n\n if self.categories_order is not None:\n _color_df = _color_df.loc[self.categories_order, :]\n _size_df = _size_df.loc[self.categories_order, :]\n\n if self.are_axes_swapped:\n _size_df = _size_df.T\n _color_df = _color_df.T\n self.cmap = self.kwds.pop(\"cmap\", self.cmap)\n\n normalize, dot_min, dot_max = self._dotplot(\n _size_df,\n _color_df,\n ax,\n cmap=self.cmap,\n color_on=self.color_on,\n dot_max=self.dot_max,\n dot_min=self.dot_min,\n standard_scale=self.standard_scale,\n edge_color=self.dot_edge_color,\n edge_lw=self.dot_edge_lw,\n smallest_dot=self.smallest_dot,\n largest_dot=self.largest_dot,\n size_exponent=self.size_exponent,\n grid=self.grid,\n x_padding=self.plot_x_padding,\n y_padding=self.plot_y_padding,\n vmin=self.vboundnorm.vmin,\n vmax=self.vboundnorm.vmax,\n vcenter=self.vboundnorm.vcenter,\n norm=self.vboundnorm.norm,\n **self.kwds,\n )\n\n self.dot_min, self.dot_max = dot_min, dot_max\n return normalize\n\n @staticmethod\n def _dotplot(\n dot_size: pd.DataFrame,\n dot_color: pd.DataFrame,\n dot_ax: Axes,\n *,\n cmap: Colormap | str | None,\n color_on: Literal[\"dot\", \"square\"],\n dot_max: float | None,\n dot_min: float | None,\n standard_scale: Literal[\"var\", \"group\"] | None,\n smallest_dot: float,\n largest_dot: float,\n size_exponent: float,\n edge_color: ColorLike | None,\n edge_lw: float | None,\n grid: bool,\n x_padding: float,\n y_padding: float,\n vmin: float | None,\n vmax: float | None,\n vcenter: float | None,\n norm: Normalize | None,\n **kwds,\n ):\n \"\"\"\\\n Makes a *dot plot* given two data frames, one containing\n the doc size and other containing the dot color. The indices and\n columns of the data frame are used to label the output image\n\n The dots are plotted using :func:`matplotlib.pyplot.scatter`. Thus, additional\n arguments can be passed.\n\n Parameters\n ----------\n dot_size\n Data frame containing the dot_size.\n dot_color\n Data frame containing the dot_color, should have the same,\n shape, columns and indices as dot_size.\n dot_ax\n matplotlib axis\n cmap\n color_on\n dot_max\n dot_min\n standard_scale\n smallest_dot\n edge_color\n edge_lw\n grid\n x_padding\n y_padding\n See `style`\n kwds\n Are passed to :func:`matplotlib.pyplot.scatter`.\n\n Returns\n -------\n matplotlib.colors.Normalize, dot_min, dot_max\n\n \"\"\"\n assert dot_size.shape == dot_color.shape, (\n \"please check that dot_size \" \"and dot_color dataframes have the same shape\"\n )\n\n assert list(dot_size.index) == list(dot_color.index), (\n \"please check that dot_size \" \"and dot_color dataframes have the same index\"\n )\n\n assert list(dot_size.columns) == list(dot_color.columns), (\n \"please check that the dot_size \"\n \"and dot_color dataframes have the same columns\"\n )\n\n if standard_scale == \"group\":\n dot_color = dot_color.sub(dot_color.min(1), axis=0)\n dot_color = dot_color.div(dot_color.max(1), axis=0).fillna(0)\n elif standard_scale == \"var\":\n dot_color -= dot_color.min(0)\n dot_color = (dot_color / dot_color.max(0)).fillna(0)\n elif standard_scale is None:\n pass\n\n # make scatter plot in which\n # x = var_names\n # y = groupby category\n # size = fraction\n # color = mean expression\n\n # +0.5 in y and x to set the dot center at 0.5 multiples\n # this facilitates dendrogram and totals alignment for\n # matrixplot, dotplot and stackec_violin using the same coordinates.\n y, x = np.indices(dot_color.shape)\n y = y.flatten() + 0.5\n x = x.flatten() + 0.5\n frac = dot_size.values.flatten()\n mean_flat = dot_color.values.flatten()\n cmap = plt.get_cmap(cmap)\n if dot_max is None:\n dot_max = np.ceil(max(frac) * 10) / 10\n else:\n if dot_max < 0 or dot_max > 1:\n raise ValueError(\"`dot_max` value has to be between 0 and 1\")\n if dot_min is None:\n dot_min = 0\n else:\n if dot_min < 0 or dot_min > 1:\n raise ValueError(\"`dot_min` value has to be between 0 and 1\")\n\n if dot_min != 0 or dot_max != 1:\n # clip frac between dot_min and dot_max\n frac = np.clip(frac, dot_min, dot_max)\n old_range = dot_max - dot_min\n # re-scale frac between 0 and 1\n frac = (frac - dot_min) / old_range\n\n size = frac**size_exponent\n # rescale size to match smallest_dot and largest_dot\n size = size * (largest_dot - smallest_dot) + smallest_dot\n normalize = check_colornorm(vmin, vmax, vcenter, norm)\n\n if color_on == \"square\":\n if edge_color is None:\n from seaborn.utils import relative_luminance\n\n # use either black or white for the edge color\n # depending on the luminance of the background\n # square color\n edge_color = []\n for color_value in cmap(normalize(mean_flat)):\n lum = relative_luminance(color_value)\n edge_color.append(\".15\" if lum > 0.408 else \"w\")\n\n edge_lw = 1.5 if edge_lw is None else edge_lw\n\n # first make a heatmap similar to `sc.pl.matrixplot`\n # (squares with the asigned colormap). Circles will be plotted\n # on top\n dot_ax.pcolor(dot_color.values, cmap=cmap, norm=normalize)\n for axis in [\"top\", \"bottom\", \"left\", \"right\"]:\n dot_ax.spines[axis].set_linewidth(1.5)\n kwds = fix_kwds(\n kwds,\n s=size,\n linewidth=edge_lw,\n facecolor=\"none\",\n edgecolor=edge_color,\n )\n dot_ax.scatter(x, y, **kwds)\n else:\n edge_color = \"none\" if edge_color is None else edge_color\n edge_lw = 0.0 if edge_lw is None else edge_lw\n\n color = cmap(normalize(mean_flat))\n kwds = fix_kwds(\n kwds,\n s=size,\n color=color,\n linewidth=edge_lw,\n edgecolor=edge_color,\n )\n dot_ax.scatter(x, y, **kwds)\n\n y_ticks = np.arange(dot_color.shape[0]) + 0.5\n dot_ax.set_yticks(y_ticks)\n dot_ax.set_yticklabels(\n [dot_color.index[idx] for idx, _ in enumerate(y_ticks)], minor=False\n )\n\n x_ticks = np.arange(dot_color.shape[1]) + 0.5\n dot_ax.set_xticks(x_ticks)\n dot_ax.set_xticklabels(\n [dot_color.columns[idx] for idx, _ in enumerate(x_ticks)],\n rotation=90,\n ha=\"center\",\n minor=False,\n )\n dot_ax.tick_params(axis=\"both\", labelsize=\"small\")\n dot_ax.grid(visible=False)\n\n # to be consistent with the heatmap plot, is better to\n # invert the order of the y-axis, such that the first group is on\n # top\n dot_ax.set_ylim(dot_color.shape[0], 0)\n dot_ax.set_xlim(0, dot_color.shape[1])\n\n if color_on == \"dot\":\n # add padding to the x and y lims when the color is not in the square\n # default y range goes from 0.5 to num cols + 0.5\n # and default x range goes from 0.5 to num rows + 0.5, thus\n # the padding needs to be corrected.\n x_padding = x_padding - 0.5\n y_padding = y_padding - 0.5\n dot_ax.set_ylim(dot_color.shape[0] + y_padding, -y_padding)\n\n dot_ax.set_xlim(-x_padding, dot_color.shape[1] + x_padding)\n\n if grid:\n dot_ax.grid(visible=True, color=\"gray\", linewidth=0.1)\n dot_ax.set_axisbelow(True)\n\n return normalize, dot_min, dot_max\n\n\n@old_positionals(\n \"use_raw\",\n \"log\",\n \"num_categories\",\n \"expression_cutoff\",\n \"mean_only_expressed\",\n \"cmap\",\n \"dot_max\",\n \"dot_min\",\n \"standard_scale\",\n \"smallest_dot\",\n \"title\",\n \"colorbar_title\",\n \"size_title\",\n # No need to have backwards compat for > 16 positional parameters\n)\n@_doc_params(\n show_save_ax=doc_show_save_ax,\n common_plot_args=doc_common_plot_args,\n groupby_plots_args=doc_common_groupby_plot_args,\n vminmax=doc_vboundnorm,\n)\ndef dotplot(\n adata: AnnData,\n var_names: _VarNames | Mapping[str, _VarNames],\n groupby: str | Sequence[str],\n *,\n use_raw: bool | None = None,\n log: bool = False,\n num_categories: int = 7,\n categories_order: Sequence[str] | None = None,\n expression_cutoff: float = 0.0,\n mean_only_expressed: bool = False,\n standard_scale: Literal[\"var\", \"group\"] | None = None,\n title: str | None = None,\n colorbar_title: str | None = DotPlot.DEFAULT_COLOR_LEGEND_TITLE,\n size_title: str | None = DotPlot.DEFAULT_SIZE_LEGEND_TITLE,\n figsize: tuple[float, float] | None = None,\n dendrogram: bool | str = False,\n gene_symbols: str | None = None,\n var_group_positions: Sequence[tuple[int, int]] | None = None,\n var_group_labels: Sequence[str] | None = None,\n var_group_rotation: float | None = None,\n layer: str | None = None,\n swap_axes: bool | None = False,\n dot_color_df: pd.DataFrame | None = None,\n show: bool | None = None,\n save: str | bool | None = None,\n ax: _AxesSubplot | None = None,\n return_fig: bool | None = False,\n vmin: float | None = None,\n vmax: float | None = None,\n vcenter: float | None = None,\n norm: Normalize | None = None,\n # Style parameters\n cmap: Colormap | str | None = DotPlot.DEFAULT_COLORMAP,\n dot_max: float | None = DotPlot.DEFAULT_DOT_MAX,\n dot_min: float | None = DotPlot.DEFAULT_DOT_MIN,\n smallest_dot: float = DotPlot.DEFAULT_SMALLEST_DOT,\n **kwds,\n) -> DotPlot | dict | None:\n \"\"\"\\\n Makes a *dot plot* of the expression values of `var_names`.\n\n For each var_name and each `groupby` category a dot is plotted.\n Each dot represents two values: mean expression within each category\n (visualized by color) and fraction of cells expressing the `var_name` in the\n category (visualized by the size of the dot). If `groupby` is not given,\n the dotplot assumes that all data belongs to a single category.\n\n .. note::\n A gene is considered expressed if the expression value in the `adata` (or\n `adata.raw`) is above the specified threshold which is zero by default.\n\n An example of dotplot usage is to visualize, for multiple marker genes,\n the mean value and the percentage of cells expressing the gene\n across multiple clusters.\n\n This function provides a convenient interface to the :class:`~scanpy.pl.DotPlot`\n class. If you need more flexibility, you should use :class:`~scanpy.pl.DotPlot`\n directly.\n\n Parameters\n ----------\n {common_plot_args}\n {groupby_plots_args}\n size_title\n Title for the size legend. New line character (\\\\n) can be used.\n expression_cutoff\n Expression cutoff that is used for binarizing the gene expression and\n determining the fraction of cells expressing given genes. A gene is\n expressed only if the expression value is greater than this threshold.\n mean_only_expressed\n If True, gene expression is averaged only over the cells\n expressing the given genes.\n dot_max\n If ``None``, the maximum dot size is set to the maximum fraction value found\n (e.g. 0.6). If given, the value should be a number between 0 and 1.\n All fractions larger than dot_max are clipped to this value.\n dot_min\n If ``None``, the minimum dot size is set to 0. If given,\n the value should be a number between 0 and 1.\n All fractions smaller than dot_min are clipped to this value.\n smallest_dot\n All expression levels with `dot_min` are plotted with this size.\n {show_save_ax}\n {vminmax}\n kwds\n Are passed to :func:`matplotlib.pyplot.scatter`.\n\n Returns\n -------\n If `return_fig` is `True`, returns a :class:`~scanpy.pl.DotPlot` object,\n else if `show` is false, return axes dict\n\n See also\n --------\n :class:`~scanpy.pl.DotPlot`: The DotPlot class can be used to to control\n several visual parameters not available in this function.\n :func:`~scanpy.pl.rank_genes_groups_dotplot`: to plot marker genes\n identified using the :func:`~scanpy.tl.rank_genes_groups` function.\n\n Examples\n --------\n\n Create a dot plot using the given markers and the PBMC example dataset grouped by\n the category 'bulk_labels'.\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n sc.pl.dotplot(adata, markers, groupby='bulk_labels', dendrogram=True)\n\n Using var_names as dict:\n\n .. plot::\n :context: close-figs\n\n markers = {{'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}}\n sc.pl.dotplot(adata, markers, groupby='bulk_labels', dendrogram=True)\n\n Get DotPlot object for fine tuning\n\n .. plot::\n :context: close-figs\n\n dp = sc.pl.dotplot(adata, markers, 'bulk_labels', return_fig=True)\n dp.add_totals().style(dot_edge_color='black', dot_edge_lw=0.5).show()\n\n The axes used can be obtained using the get_axes() method\n\n .. code-block:: python\n\n axes_dict = dp.get_axes()\n print(axes_dict)\n\n \"\"\"\n\n # backwards compatibility: previous version of dotplot used `color_map`\n # instead of `cmap`\n cmap = kwds.pop(\"color_map\", cmap)\n\n dp = DotPlot(\n adata,\n var_names,\n groupby,\n use_raw=use_raw,\n log=log,\n num_categories=num_categories,\n categories_order=categories_order,\n expression_cutoff=expression_cutoff,\n mean_only_expressed=mean_only_expressed,\n standard_scale=standard_scale,\n title=title,\n figsize=figsize,\n gene_symbols=gene_symbols,\n var_group_positions=var_group_positions,\n var_group_labels=var_group_labels,\n var_group_rotation=var_group_rotation,\n layer=layer,\n dot_color_df=dot_color_df,\n ax=ax,\n vmin=vmin,\n vmax=vmax,\n vcenter=vcenter,\n norm=norm,\n **kwds,\n )\n\n if dendrogram:\n dp.add_dendrogram(dendrogram_key=_dk(dendrogram))\n if swap_axes:\n dp.swap_axes()\n\n dp = dp.style(\n cmap=cmap,\n dot_max=dot_max,\n dot_min=dot_min,\n smallest_dot=smallest_dot,\n dot_edge_lw=kwds.pop(\"linewidth\", _empty),\n ).legend(colorbar_title=colorbar_title, size_title=size_title)\n\n if return_fig:\n return dp\n else:\n dp.make_figure()\n savefig_or_show(DotPlot.DEFAULT_SAVE_PREFIX, show=show, save=save)\n show = settings.autoshow if show is None else show\n if not show:\n return dp.get_axes()\n\n\n\"\"\"BasePlot for dotplot, matrixplot and stacked_violin\"\"\"\n\nfrom __future__ import annotations\n\nfrom collections.abc import Mapping\nfrom typing import TYPE_CHECKING, NamedTuple\nfrom warnings import warn\n\nimport numpy as np\nfrom matplotlib import gridspec\nfrom matplotlib import pyplot as plt\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._utils import _empty\nfrom ._anndata import _get_dendrogram_key, _plot_dendrogram, _prepare_dataframe\nfrom ._utils import check_colornorm, make_grid_spec\n\nif TYPE_CHECKING:\n from collections.abc import Iterable, Sequence\n from typing import Literal, Self, Union\n\n import pandas as pd\n from anndata import AnnData\n from matplotlib.axes import Axes\n from matplotlib.colors import Colormap, Normalize\n\n from .._utils import Empty\n from ._utils import ColorLike, _AxesSubplot\n\n _VarNames = Union[str, Sequence[str]]\n\n\nclass VBoundNorm(NamedTuple):\n vmin: float | None\n vmax: float | None\n vcenter: float | None\n norm: Normalize | None\n\n\ndoc_common_groupby_plot_args = \"\"\"\\\ntitle\n Title for the figure\ncolorbar_title\n Title for the color bar. New line character (\\\\n) can be used.\ncmap\n String denoting matplotlib color map.\nstandard_scale\n Whether or not to standardize the given dimension between 0 and 1, meaning for\n each variable or group, subtract the minimum and divide each by its maximum.\nswap_axes\n By default, the x axis contains `var_names` (e.g. genes) and the y axis\n the `groupby` categories. By setting `swap_axes` then x are the\n `groupby` categories and y the `var_names`.\nreturn_fig\n Returns :class:`DotPlot` object. Useful for fine-tuning\n the plot. Takes precedence over `show=False`.\n\"\"\"\n\n\nclass BasePlot:\n \"\"\"\\\n Generic class for the visualization of AnnData categories and\n selected `var` (features or genes).\n\n Takes care of the visual location of a main plot, additional plots\n in the margins (e.g. dendrogram, margin totals) and legends. Also\n understand how to adapt the visual parameter if the plot is rotated\n\n Classed based on BasePlot implement their own _mainplot() method.\n\n The BasePlot works by method chaining. For example:\n BasePlot(adata, ...).legend(title='legend').style(cmap='binary').show()\n \"\"\"\n\n DEFAULT_SAVE_PREFIX = \"baseplot_\"\n MIN_FIGURE_HEIGHT = 2.5\n DEFAULT_CATEGORY_HEIGHT = 0.35\n DEFAULT_CATEGORY_WIDTH = 0.37\n\n # gridspec parameter. Sets the space between mainplot, dendrogram and legend\n DEFAULT_WSPACE = 0\n\n DEFAULT_COLORMAP = \"winter\"\n DEFAULT_LEGENDS_WIDTH = 1.5\n DEFAULT_COLOR_LEGEND_TITLE = \"Expression\\nlevel in group\"\n\n MAX_NUM_CATEGORIES = 500 # maximum number of categories allowed to be plotted\n\n @old_positionals(\n \"use_raw\",\n \"log\",\n \"num_categories\",\n \"categories_order\",\n \"title\",\n \"figsize\",\n \"gene_symbols\",\n \"var_group_positions\",\n \"var_group_labels\",\n \"var_group_rotation\",\n \"layer\",\n \"ax\",\n \"vmin\",\n \"vmax\",\n \"vcenter\",\n \"norm\",\n )\n def __init__(\n self,\n adata: AnnData,\n var_names: _VarNames | Mapping[str, _VarNames],\n groupby: str | Sequence[str],\n *,\n use_raw: bool | None = None,\n log: bool = False,\n num_categories: int = 7,\n categories_order: Sequence[str] | None = None,\n title: str | None = None,\n figsize: tuple[float, float] | None = None,\n gene_symbols: str | None = None,\n var_group_positions: Sequence[tuple[int, int]] | None = None,\n var_group_labels: Sequence[str] | None = None,\n var_group_rotation: float | None = None,\n layer: str | None = None,\n ax: _AxesSubplot | None = None,\n vmin: float | None = None,\n vmax: float | None = None,\n vcenter: float | None = None,\n norm: Normalize | None = None,\n **kwds,\n ):\n self.var_names = var_names\n self.var_group_labels = var_group_labels\n self.var_group_positions = var_group_positions\n self.var_group_rotation = var_group_rotation\n self.width, self.height = figsize if figsize is not None else (None, None)\n\n self.has_var_groups = (\n var_group_positions is not None and len(var_group_positions) > 0\n )\n\n self._update_var_groups()\n\n self.categories, self.obs_tidy = _prepare_dataframe(\n adata,\n self.var_names,\n groupby,\n use_raw=use_raw,\n log=log,\n num_categories=num_categories,\n layer=layer,\n gene_symbols=gene_symbols,\n )\n if len(self.categories) > self.MAX_NUM_CATEGORIES:\n warn(\n f\"Over {self.MAX_NUM_CATEGORIES} categories found. \"\n \"Plot would be very large.\"\n )\n\n if categories_order is not None and (\n set(self.obs_tidy.index.categories) != set(categories_order)\n ):\n logg.error(\n \"Please check that the categories given by \"\n \"the `order` parameter match the categories that \"\n \"want to be reordered.\\n\\n\"\n \"Mismatch: \"\n f\"{set(self.obs_tidy.index.categories).difference(categories_order)}\\n\\n\"\n f\"Given order categories: {categories_order}\\n\\n\"\n f\"{groupby} categories: {list(self.obs_tidy.index.categories)}\\n\"\n )\n return\n\n self.adata = adata\n self.groupby = [groupby] if isinstance(groupby, str) else groupby\n self.log = log\n self.kwds = kwds\n\n self.vboundnorm = VBoundNorm(vmin=vmin, vmax=vmax, vcenter=vcenter, norm=norm)\n\n # set default values for legend\n self.color_legend_title = self.DEFAULT_COLOR_LEGEND_TITLE\n self.legends_width = self.DEFAULT_LEGENDS_WIDTH\n\n # set style defaults\n self.cmap = self.DEFAULT_COLORMAP\n\n # style default parameters\n self.are_axes_swapped = False\n self.categories_order = categories_order\n self.var_names_idx_order = None\n\n self.wspace = self.DEFAULT_WSPACE\n\n # minimum height required for legends to plot properly\n self.min_figure_height = self.MIN_FIGURE_HEIGHT\n\n self.fig_title = title\n\n self.group_extra_size = 0\n self.plot_group_extra = None\n # after .render() is called the fig value is assigned and ax_dict\n # contains a dictionary of the axes used in the plot\n self.fig = None\n self.ax_dict = None\n self.ax = ax\n\n @old_positionals(\"swap_axes\")\n def swap_axes(self, *, swap_axes: bool | None = True) -> Self:\n \"\"\"\n Plots a transposed image.\n\n By default, the x axis contains `var_names` (e.g. genes) and the y\n axis the `groupby` categories. By setting `swap_axes` then x are\n the `groupby` categories and y the `var_names`.\n\n Parameters\n ----------\n swap_axes\n Boolean to turn on (True) or off (False) 'swap_axes'. Default True\n\n\n Returns\n -------\n Returns `self` for method chaining.\n\n \"\"\"\n self.DEFAULT_CATEGORY_HEIGHT, self.DEFAULT_CATEGORY_WIDTH = (\n self.DEFAULT_CATEGORY_WIDTH,\n self.DEFAULT_CATEGORY_HEIGHT,\n )\n\n self.are_axes_swapped = swap_axes\n return self\n\n @old_positionals(\"show\", \"dendrogram_key\", \"size\")\n def add_dendrogram(\n self,\n *,\n show: bool | None = True,\n dendrogram_key: str | None = None,\n size: float | None = 0.8,\n ) -> Self:\n r\"\"\"\\\n Show dendrogram based on the hierarchical clustering between the `groupby`\n categories. Categories are reordered to match the dendrogram order.\n\n The dendrogram information is computed using :func:`scanpy.tl.dendrogram`.\n If `sc.tl.dendrogram` has not been called previously the function is called\n with default parameters.\n\n The dendrogram is by default shown on the right side of the plot or on top\n if the axes are swapped.\n\n `var_names` are reordered to produce a more pleasing output if:\n * The data contains `var_groups`\n * the `var_groups` match the categories.\n\n The previous conditions happen by default when using Plot\n to show the results from :func:`~scanpy.tl.rank_genes_groups` (aka gene markers), by\n calling `scanpy.tl.rank_genes_groups_(plot_name)`.\n\n\n Parameters\n ----------\n show\n Boolean to turn on (True) or off (False) 'add_dendrogram'\n dendrogram_key\n Needed if `sc.tl.dendrogram` saved the dendrogram using a key different\n than the default name.\n size\n size of the dendrogram. Corresponds to width when dendrogram shown on\n the right of the plot, or height when shown on top. The unit is the same\n as in matplotlib (inches).\n\n Returns\n -------\n Returns `self` for method chaining.\n\n\n Examples\n --------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> markers = {'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}\n >>> plot = sc.pl._baseplot_class.BasePlot(adata, markers, groupby='bulk_labels').add_dendrogram()\n >>> plot.plot_group_extra # doctest: +NORMALIZE_WHITESPACE\n {'kind': 'dendrogram',\n 'width': 0.8,\n 'dendrogram_key': None,\n 'dendrogram_ticks': array([0.5, 1.5, 2.5, 3.5, 4.5, 5.5, 6.5, 7.5, 8.5, 9.5])}\n \"\"\"\n\n if not show:\n self.plot_group_extra = None\n return self\n\n if self.groupby is None or len(self.categories) <= 2:\n # dendrogram can only be computed between groupby categories\n logg.warning(\n \"Dendrogram not added. Dendrogram is added only \"\n \"when the number of categories to plot > 2\"\n )\n return self\n\n self.group_extra_size = size\n\n # to correctly plot the dendrogram the categories need to be ordered\n # according to the dendrogram ordering.\n self._reorder_categories_after_dendrogram(dendrogram_key)\n\n dendro_ticks = np.arange(len(self.categories)) + 0.5\n\n self.group_extra_size = size\n self.plot_group_extra = {\n \"kind\": \"dendrogram\",\n \"width\": size,\n \"dendrogram_key\": dendrogram_key,\n \"dendrogram_ticks\": dendro_ticks,\n }\n return self\n\n @old_positionals(\"show\", \"sort\", \"size\", \"color\")\n def add_totals(\n self,\n *,\n show: bool | None = True,\n sort: Literal[\"ascending\", \"descending\"] | None = None,\n size: float | None = 0.8,\n color: ColorLike | Sequence[ColorLike] | None = None,\n ) -> Self:\n r\"\"\"\\\n Show barplot for the number of cells in in `groupby` category.\n\n The barplot is by default shown on the right side of the plot or on top\n if the axes are swapped.\n\n\n Parameters\n ----------\n show\n Boolean to turn on (True) or off (False) 'add_totals'\n sort\n Set to either 'ascending' or 'descending' to reorder the categories\n by cell number\n size\n size of the barplot. Corresponds to width when shown on\n the right of the plot, or height when shown on top. The unit is the same\n as in matplotlib (inches).\n color\n Color for the bar plots or list of colors for each of the bar plots.\n By default, each bar plot uses the colors assigned in\n `adata.uns[{groupby}_colors]`.\n\n\n Returns\n -------\n Returns `self` for method chaining.\n\n\n Examples\n --------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> markers = {'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}\n >>> plot = sc.pl._baseplot_class.BasePlot(adata, markers, groupby='bulk_labels').add_totals()\n >>> plot.plot_group_extra['counts_df'] # doctest: +SKIP\n bulk_labels\n CD4+/CD25 T Reg 68\n CD4+/CD45RA+/CD25- Naive T 8\n CD4+/CD45RO+ Memory 19\n CD8+ Cytotoxic T 54\n CD8+/CD45RA+ Naive Cytotoxic 43\n CD14+ Monocyte 129\n CD19+ B 95\n CD34+ 13\n CD56+ NK 31\n Dendritic 240\n Name: count, dtype: int64\n \"\"\"\n self.group_extra_size = size\n\n if not show:\n # hide totals\n self.plot_group_extra = None\n self.group_extra_size = 0\n return self\n\n _sort = sort is not None\n _ascending = sort == \"ascending\"\n counts_df = self.obs_tidy.index.value_counts(sort=_sort, ascending=_ascending)\n\n if _sort:\n self.categories_order = counts_df.index\n\n self.plot_group_extra = {\n \"kind\": \"group_totals\",\n \"width\": size,\n \"sort\": sort,\n \"counts_df\": counts_df,\n \"color\": color,\n }\n return self\n\n @old_positionals(\"cmap\")\n def style(self, *, cmap: Colormap | str | None | Empty = _empty) -> Self:\n \"\"\"\\\n Set visual style parameters\n\n Parameters\n ----------\n cmap\n Matplotlib color map, specified by name or directly.\n If ``None``, use :obj:`matplotlib.rcParams`\\\\ ``[\"image.cmap\"]``\n\n Returns\n -------\n Returns `self` for method chaining.\n \"\"\"\n\n if cmap is not _empty:\n self.cmap = cmap\n return self\n\n @old_positionals(\"show\", \"title\", \"width\")\n def legend(\n self,\n *,\n show: bool | None = True,\n title: str | None = DEFAULT_COLOR_LEGEND_TITLE,\n width: float | None = DEFAULT_LEGENDS_WIDTH,\n ) -> Self:\n r\"\"\"\\\n Configure legend parameters\n\n Parameters\n ----------\n show\n Set to 'False' to hide the default plot of the legend. This sets the\n legend width to zero which will result in a wider main plot.\n title\n Legend title. Appears on top of the color bar. Use '\\\\n' to add line breaks.\n width\n Width of the legend. The unit is the same as in matplotlib (inches)\n\n Returns\n -------\n Returns `self` for method chaining.\n\n\n Examples\n --------\n\n Set legend title:\n\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> markers = {'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}\n >>> dp = sc.pl._baseplot_class.BasePlot(adata, markers, groupby='bulk_labels') \\\n ... .legend(title='log(UMI counts + 1)')\n >>> dp.color_legend_title\n 'log(UMI counts + 1)'\n \"\"\"\n\n if not show:\n # turn of legends by setting width to 0\n self.legends_width = 0\n else:\n self.color_legend_title = title\n self.legends_width = width\n\n return self\n\n def get_axes(self) -> dict[str, Axes]:\n if self.ax_dict is None:\n self.make_figure()\n return self.ax_dict\n\n def _plot_totals(\n self, total_barplot_ax: Axes, orientation: Literal[\"top\", \"right\"]\n ):\n \"\"\"\n Makes the bar plot for totals\n \"\"\"\n params = self.plot_group_extra\n counts_df: pd.DataFrame = params[\"counts_df\"]\n if self.categories_order is not None:\n counts_df = counts_df.loc[self.categories_order]\n if params[\"color\"] is None:\n color = self.adata.uns.get(f\"{self.groupby}_colors\", \"salmon\")\n else:\n color = params[\"color\"]\n\n if orientation == \"top\":\n counts_df.plot(\n kind=\"bar\",\n color=color,\n position=0.5,\n ax=total_barplot_ax,\n edgecolor=\"black\",\n width=0.65,\n )\n # add numbers to the top of the bars\n max_y = max([p.get_height() for p in total_barplot_ax.patches])\n\n for p in total_barplot_ax.patches:\n p.set_x(p.get_x() + 0.5)\n if p.get_height() >= 1000:\n display_number = f\"{np.round(p.get_height() / 1000, decimals=1)}k\"\n else:\n display_number = np.round(p.get_height(), decimals=1)\n total_barplot_ax.annotate(\n display_number,\n (p.get_x() + p.get_width() / 2.0, (p.get_height() + max_y * 0.05)),\n ha=\"center\",\n va=\"top\",\n xytext=(0, 10),\n fontsize=\"x-small\",\n textcoords=\"offset points\",\n )\n # for k in total_barplot_ax.spines.keys():\n # total_barplot_ax.spines[k].set_visible(False)\n total_barplot_ax.set_ylim(0, max_y * 1.4)\n\n elif orientation == \"right\":\n counts_df.plot(\n kind=\"barh\",\n color=color,\n position=-0.3,\n ax=total_barplot_ax,\n edgecolor=\"black\",\n width=0.65,\n )\n\n # add numbers to the right of the bars\n max_x = max([p.get_width() for p in total_barplot_ax.patches])\n for p in total_barplot_ax.patches:\n if p.get_width() >= 1000:\n display_number = f\"{np.round(p.get_width() / 1000, decimals=1)}k\"\n else:\n display_number = np.round(p.get_width(), decimals=1)\n total_barplot_ax.annotate(\n display_number,\n ((p.get_width()), p.get_y() + p.get_height()),\n ha=\"center\",\n va=\"top\",\n xytext=(10, 10),\n fontsize=\"x-small\",\n textcoords=\"offset points\",\n )\n total_barplot_ax.set_xlim(0, max_x * 1.4)\n\n total_barplot_ax.grid(visible=False)\n total_barplot_ax.axis(\"off\")\n\n def _plot_colorbar(self, color_legend_ax: Axes, normalize) -> None:\n \"\"\"\n Plots a horizontal colorbar given the ax an normalize values\n\n Parameters\n ----------\n color_legend_ax\n normalize\n\n Returns\n -------\n `None`, updates color_legend_ax\n \"\"\"\n cmap = plt.get_cmap(self.cmap)\n\n import matplotlib.colorbar\n from matplotlib.cm import ScalarMappable\n\n mappable = ScalarMappable(norm=normalize, cmap=cmap)\n\n matplotlib.colorbar.Colorbar(\n color_legend_ax, mappable=mappable, orientation=\"horizontal\"\n )\n\n color_legend_ax.set_title(self.color_legend_title, fontsize=\"small\")\n\n color_legend_ax.xaxis.set_tick_params(labelsize=\"small\")\n\n def _plot_legend(self, legend_ax, return_ax_dict, normalize):\n # to maintain the fixed height size of the legends, a\n # spacer of variable height is added at top and bottom.\n # The structure for the legends is:\n # first row: variable space to keep the first rows of the same size\n # second row: size legend\n\n legend_height = self.min_figure_height * 0.08\n height_ratios = [\n self.height - legend_height,\n legend_height,\n ]\n fig, legend_gs = make_grid_spec(\n legend_ax, nrows=2, ncols=1, height_ratios=height_ratios\n )\n\n color_legend_ax = fig.add_subplot(legend_gs[1])\n\n self._plot_colorbar(color_legend_ax, normalize)\n return_ax_dict[\"color_legend_ax\"] = color_legend_ax\n\n def _mainplot(self, ax: Axes):\n y_labels = self.categories\n x_labels = self.var_names\n\n if self.var_names_idx_order is not None:\n x_labels = [x_labels[x] for x in self.var_names_idx_order]\n\n if self.categories_order is not None:\n y_labels = self.categories_order\n\n if self.are_axes_swapped:\n x_labels, y_labels = y_labels, x_labels\n ax.set_xlabel(self.groupby)\n else:\n ax.set_ylabel(self.groupby)\n\n y_ticks = np.arange(len(y_labels)) + 0.5\n ax.set_yticks(y_ticks)\n ax.set_yticklabels(y_labels)\n\n x_ticks = np.arange(len(x_labels)) + 0.5\n ax.set_xticks(x_ticks)\n ax.set_xticklabels(x_labels, rotation=90, ha=\"center\", minor=False)\n\n ax.tick_params(axis=\"both\", labelsize=\"small\")\n ax.grid(visible=False)\n\n # to be consistent with the heatmap plot, is better to\n # invert the order of the y-axis, such that the first group is on\n # top\n ax.set_ylim(len(y_labels), 0)\n ax.set_xlim(0, len(x_labels))\n\n return check_colornorm(\n self.vboundnorm.vmin,\n self.vboundnorm.vmax,\n self.vboundnorm.vcenter,\n self.vboundnorm.norm,\n )\n\n def make_figure(self):\n r\"\"\"\n Renders the image but does not call :func:`matplotlib.pyplot.show`. Useful\n when several plots are put together into one figure.\n\n See also\n --------\n `show()`: Renders and shows the plot.\n `savefig()`: Saves the plot.\n\n Examples\n --------\n\n >>> import scanpy as sc\n >>> import matplotlib.pyplot as plt\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']\n >>> fig, (ax0, ax1) = plt.subplots(1, 2)\n >>> sc.pl.MatrixPlot(adata, markers, groupby='bulk_labels', ax=ax0) \\\n ... .style(cmap='Blues', edge_color='none').make_figure()\n >>> sc.pl.DotPlot(adata, markers, groupby='bulk_labels', ax=ax1).make_figure()\n \"\"\"\n\n category_height = self.DEFAULT_CATEGORY_HEIGHT\n category_width = self.DEFAULT_CATEGORY_WIDTH\n\n if self.height is None:\n mainplot_height = len(self.categories) * category_height\n mainplot_width = (\n len(self.var_names) * category_width + self.group_extra_size\n )\n if self.are_axes_swapped:\n mainplot_height, mainplot_width = mainplot_width, mainplot_height\n\n height = mainplot_height + 1 # +1 for labels\n\n # if the number of categories is small use\n # a larger height, otherwise the legends do not fit\n self.height = max([self.min_figure_height, height])\n self.width = mainplot_width + self.legends_width\n else:\n self.min_figure_height = self.height\n mainplot_height = self.height\n\n mainplot_width = self.width - (self.legends_width + self.group_extra_size)\n\n return_ax_dict = {}\n # define a layout of 1 rows x 2 columns\n # first ax is for the main figure.\n # second ax is to plot legends\n legends_width_spacer = 0.7 / self.width\n\n self.fig, gs = make_grid_spec(\n self.ax or (self.width, self.height),\n nrows=1,\n ncols=2,\n wspace=legends_width_spacer,\n width_ratios=[mainplot_width + self.group_extra_size, self.legends_width],\n )\n\n if self.has_var_groups:\n # add some space in case 'brackets' want to be plotted on top of the image\n if self.are_axes_swapped:\n var_groups_height = category_height\n else:\n var_groups_height = category_height / 2\n\n else:\n var_groups_height = 0\n\n mainplot_width = mainplot_width - self.group_extra_size\n spacer_height = self.height - var_groups_height - mainplot_height\n if not self.are_axes_swapped:\n height_ratios = [spacer_height, var_groups_height, mainplot_height]\n width_ratios = [mainplot_width, self.group_extra_size]\n\n else:\n height_ratios = [spacer_height, self.group_extra_size, mainplot_height]\n width_ratios = [mainplot_width, var_groups_height]\n # gridspec is the same but rows and columns are swapped\n\n if self.fig_title is not None and self.fig_title.strip() != \"\":\n # for the figure title use the ax that contains\n # all the main graphical elements (main plot, dendrogram etc)\n # otherwise the title may overlay with the figure.\n # also, this puts the title centered on the main figure and not\n # centered between the main figure and the legends\n _ax = self.fig.add_subplot(gs[0, 0])\n _ax.axis(\"off\")\n _ax.set_title(self.fig_title)\n\n # the main plot is divided into three rows and two columns\n # first row is an spacer that is adjusted in case the\n # legends need more height than the main plot\n # second row is for brackets (if needed),\n # third row is for mainplot and dendrogram/totals (legend goes in gs[0,1]\n # defined earlier)\n mainplot_gs = gridspec.GridSpecFromSubplotSpec(\n nrows=3,\n ncols=2,\n wspace=self.wspace,\n hspace=0.0,\n subplot_spec=gs[0, 0],\n width_ratios=width_ratios,\n height_ratios=height_ratios,\n )\n main_ax = self.fig.add_subplot(mainplot_gs[2, 0])\n return_ax_dict[\"mainplot_ax\"] = main_ax\n if not self.are_axes_swapped:\n if self.plot_group_extra is not None:\n group_extra_ax = self.fig.add_subplot(mainplot_gs[2, 1], sharey=main_ax)\n group_extra_orientation = \"right\"\n if self.has_var_groups:\n gene_groups_ax = self.fig.add_subplot(mainplot_gs[1, 0], sharex=main_ax)\n var_group_orientation = \"top\"\n else:\n if self.plot_group_extra:\n group_extra_ax = self.fig.add_subplot(mainplot_gs[1, 0], sharex=main_ax)\n group_extra_orientation = \"top\"\n if self.has_var_groups:\n gene_groups_ax = self.fig.add_subplot(mainplot_gs[2, 1], sharey=main_ax)\n var_group_orientation = \"right\"\n\n if self.plot_group_extra is not None:\n if self.plot_group_extra[\"kind\"] == \"dendrogram\":\n _plot_dendrogram(\n group_extra_ax,\n self.adata,\n self.groupby,\n dendrogram_key=self.plot_group_extra[\"dendrogram_key\"],\n ticks=self.plot_group_extra[\"dendrogram_ticks\"],\n orientation=group_extra_orientation,\n )\n if self.plot_group_extra[\"kind\"] == \"group_totals\":\n self._plot_totals(group_extra_ax, group_extra_orientation)\n\n return_ax_dict[\"group_extra_ax\"] = group_extra_ax\n\n # plot group legends on top or left of main_ax (if given)\n if self.has_var_groups:\n self._plot_var_groups_brackets(\n gene_groups_ax,\n group_positions=self.var_group_positions,\n group_labels=self.var_group_labels,\n rotation=self.var_group_rotation,\n left_adjustment=0.2,\n right_adjustment=0.7,\n orientation=var_group_orientation,\n )\n return_ax_dict[\"gene_group_ax\"] = gene_groups_ax\n\n # plot the mainplot\n normalize = self._mainplot(main_ax)\n\n # code from pandas.plot in add_totals adds\n # minor ticks that need to be removed\n main_ax.yaxis.set_tick_params(which=\"minor\", left=False, right=False)\n main_ax.xaxis.set_tick_params(which=\"minor\", top=False, bottom=False, length=0)\n main_ax.set_zorder(100)\n if self.legends_width > 0:\n legend_ax = self.fig.add_subplot(gs[0, 1])\n self._plot_legend(legend_ax, return_ax_dict, normalize)\n\n self.ax_dict = return_ax_dict\n\n def show(self, return_axes: bool | None = None) -> dict[str, Axes] | None:\n \"\"\"\n Show the figure\n\n Parameters\n ----------\n return_axes\n If true return a dictionary with the figure axes. When return_axes is true\n then :func:`matplotlib.pyplot.show` is not called.\n\n Returns\n -------\n If `return_axes=True`: Dict of :class:`matplotlib.axes.Axes`. The dict key\n indicates the type of ax (eg. `mainplot_ax`)\n\n See also\n --------\n `render()`: Renders the plot but does not call :func:`matplotlib.pyplot.show`\n `savefig()`: Saves the plot.\n\n Examples\n -------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> markers = [\"C1QA\", \"PSAP\", \"CD79A\", \"CD79B\", \"CST3\", \"LYZ\"]\n >>> sc.pl._baseplot_class.BasePlot(adata, markers, groupby=\"bulk_labels\").show()\n \"\"\"\n\n self.make_figure()\n\n if return_axes:\n return self.ax_dict\n else:\n plt.show()\n\n def savefig(self, filename: str, bbox_inches: str | None = \"tight\", **kwargs):\n \"\"\"\n Save the current figure\n\n Parameters\n ----------\n filename\n Figure filename. Figure *format* is taken from the file ending unless\n the parameter `format` is given.\n bbox_inches\n By default is set to 'tight' to avoid cropping of the legends.\n kwargs\n Passed to :func:`matplotlib.pyplot.savefig`\n\n See also\n --------\n `render()`: Renders the plot but does not call :func:`matplotlib.pyplot.show`\n `show()`: Renders and shows the plot\n\n Examples\n -------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> markers = [\"C1QA\", \"PSAP\", \"CD79A\", \"CD79B\", \"CST3\", \"LYZ\"]\n >>> sc.pl._baseplot_class.BasePlot(\n ... adata, markers, groupby=\"bulk_labels\"\n ... ).savefig(\"plot.pdf\")\n \"\"\"\n self.make_figure()\n plt.savefig(filename, bbox_inches=bbox_inches, **kwargs)\n\n def _reorder_categories_after_dendrogram(self, dendrogram_key: str | None) -> None:\n \"\"\"\\\n Function used by plotting functions that need to reorder the the groupby\n observations based on the dendrogram results.\n\n The function checks if a dendrogram has already been precomputed.\n If not, `sc.tl.dendrogram` is run with default parameters.\n\n The results found in `.uns[dendrogram_key]` are used to reorder\n `var_group_labels` and `var_group_positions`.\n\n\n Returns\n -------\n `None`, internally updates\n 'categories_idx_ordered', 'var_group_names_idx_ordered',\n 'var_group_labels' and 'var_group_positions'\n \"\"\"\n\n def _format_first_three_categories(_categories):\n \"\"\"used to clean up warning message\"\"\"\n _categories = list(_categories)\n if len(_categories) > 3:\n _categories = _categories[:3] + [\"etc.\"]\n return \", \".join(_categories)\n\n key = _get_dendrogram_key(self.adata, dendrogram_key, self.groupby)\n\n dendro_info = self.adata.uns[key]\n if self.groupby != dendro_info[\"groupby\"]:\n raise ValueError(\n \"Incompatible observations. The precomputed dendrogram contains \"\n f\"information for the observation: '{self.groupby}' while the plot is \"\n f\"made for the observation: '{dendro_info['groupby']}. \"\n \"Please run `sc.tl.dendrogram` using the right observation.'\"\n )\n\n # order of groupby categories\n categories_idx_ordered = dendro_info[\"categories_idx_ordered\"]\n categories_ordered = dendro_info[\"categories_ordered\"]\n\n if len(self.categories) != len(categories_idx_ordered):\n raise ValueError(\n \"Incompatible observations. Dendrogram data has \"\n f\"{len(categories_idx_ordered)} categories but current groupby \"\n f\"observation {self.groupby!r} contains {len(self.categories)} categories. \"\n \"Most likely the underlying groupby observation changed after the \"\n \"initial computation of `sc.tl.dendrogram`. \"\n \"Please run `sc.tl.dendrogram` again.'\"\n )\n\n # reorder var_groups (if any)\n if self.var_names is not None:\n var_names_idx_ordered = list(range(len(self.var_names)))\n\n if self.has_var_groups:\n if set(self.var_group_labels) == set(self.categories):\n positions_ordered = []\n labels_ordered = []\n position_start = 0\n var_names_idx_ordered = []\n for cat_name in categories_ordered:\n idx = self.var_group_labels.index(cat_name)\n position = self.var_group_positions[idx]\n _var_names = self.var_names[position[0] : position[1] + 1]\n var_names_idx_ordered.extend(range(position[0], position[1] + 1))\n positions_ordered.append(\n (position_start, position_start + len(_var_names) - 1)\n )\n position_start += len(_var_names)\n labels_ordered.append(self.var_group_labels[idx])\n self.var_group_labels = labels_ordered\n self.var_group_positions = positions_ordered\n else:\n logg.warning(\n \"Groups are not reordered because the `groupby` categories \"\n \"and the `var_group_labels` are different.\\n\"\n f\"categories: {_format_first_three_categories(self.categories)}\\n\"\n \"var_group_labels: \"\n f\"{_format_first_three_categories(self.var_group_labels)}\"\n )\n\n if var_names_idx_ordered is not None:\n var_names_ordered = [self.var_names[x] for x in var_names_idx_ordered]\n else:\n var_names_ordered = None\n\n self.categories_idx_ordered = categories_idx_ordered\n self.categories_order = dendro_info[\"categories_ordered\"]\n self.var_names_idx_order = var_names_idx_ordered\n self.var_names_ordered = var_names_ordered\n\n @staticmethod\n def _plot_var_groups_brackets(\n gene_groups_ax: Axes,\n *,\n group_positions: Iterable[tuple[int, int]],\n group_labels: Sequence[str],\n left_adjustment: float = -0.3,\n right_adjustment: float = 0.3,\n rotation: float | None = None,\n orientation: Literal[\"top\", \"right\"] = \"top\",\n ) -> None:\n \"\"\"\\\n Draws brackets that represent groups of genes on the give axis.\n For best results, this axis is located on top of an image whose\n x axis contains gene names.\n\n The gene_groups_ax should share the x axis with the main ax.\n\n Eg: gene_groups_ax = fig.add_subplot(axs[0, 0], sharex=dot_ax)\n\n Parameters\n ----------\n gene_groups_ax\n In this axis the gene marks are drawn\n group_positions\n Each item in the list, should contain the start and end position that the\n bracket should cover.\n Eg. [(0, 4), (5, 8)] means that there are two brackets, one for the var_names (eg genes)\n in positions 0-4 and other for positions 5-8\n group_labels\n List of group labels\n left_adjustment\n adjustment to plot the bracket start slightly before or after the first gene position.\n If the value is negative the start is moved before.\n right_adjustment\n adjustment to plot the bracket end slightly before or after the last gene position\n If the value is negative the start is moved before.\n rotation\n rotation degrees for the labels. If not given, small labels (<4 characters) are not\n rotated, otherwise, they are rotated 90 degrees\n orientation\n location of the brackets. Either `top` or `right`\n \"\"\"\n import matplotlib.patches as patches\n from matplotlib.path import Path\n\n # get the 'brackets' coordinates as lists of start and end positions\n\n left = [x[0] + left_adjustment for x in group_positions]\n right = [x[1] + right_adjustment for x in group_positions]\n\n # verts and codes are used by PathPatch to make the brackets\n verts = []\n codes = []\n if orientation == \"top\":\n # rotate labels if any of them is longer than 4 characters\n if rotation is None and group_labels:\n rotation = 90 if max([len(x) for x in group_labels]) > 4 else 0\n for idx, (left_coor, right_coor) in enumerate(zip(left, right)):\n verts.append((left_coor, 0)) # lower-left\n verts.append((left_coor, 0.6)) # upper-left\n verts.append((right_coor, 0.6)) # upper-right\n verts.append((right_coor, 0)) # lower-right\n\n codes.append(Path.MOVETO)\n codes.append(Path.LINETO)\n codes.append(Path.LINETO)\n codes.append(Path.LINETO)\n\n group_x_center = left[idx] + float(right[idx] - left[idx]) / 2\n gene_groups_ax.text(\n group_x_center,\n 1.1,\n group_labels[idx],\n ha=\"center\",\n va=\"bottom\",\n rotation=rotation,\n )\n else:\n top = left\n bottom = right\n for idx, (top_coor, bottom_coor) in enumerate(zip(top, bottom)):\n verts.append((0, top_coor)) # upper-left\n verts.append((0.4, top_coor)) # upper-right\n verts.append((0.4, bottom_coor)) # lower-right\n verts.append((0, bottom_coor)) # lower-left\n\n codes.append(Path.MOVETO)\n codes.append(Path.LINETO)\n codes.append(Path.LINETO)\n codes.append(Path.LINETO)\n\n diff = bottom[idx] - top[idx]\n group_y_center = top[idx] + float(diff) / 2\n if diff * 2 < len(group_labels[idx]):\n # cut label to fit available space\n group_labels[idx] = group_labels[idx][: int(diff * 2)] + \".\"\n gene_groups_ax.text(\n 1.1,\n group_y_center,\n group_labels[idx],\n ha=\"right\",\n va=\"center\",\n rotation=270,\n fontsize=\"small\",\n )\n\n path = Path(verts, codes)\n\n patch = patches.PathPatch(path, facecolor=\"none\", lw=1.5)\n\n gene_groups_ax.add_patch(patch)\n gene_groups_ax.grid(visible=False)\n gene_groups_ax.axis(\"off\")\n # remove y ticks\n gene_groups_ax.tick_params(axis=\"y\", left=False, labelleft=False)\n # remove x ticks and labels\n gene_groups_ax.tick_params(\n axis=\"x\", bottom=False, labelbottom=False, labeltop=False\n )\n\n def _update_var_groups(self) -> None:\n \"\"\"\n checks if var_names is a dict. Is this is the cases, then set the\n correct values for var_group_labels and var_group_positions\n\n updates var_names, var_group_labels, var_group_positions\n \"\"\"\n if isinstance(self.var_names, Mapping):\n if self.has_var_groups:\n logg.warning(\n \"`var_names` is a dictionary. This will reset the current \"\n \"values of `var_group_labels` and `var_group_positions`.\"\n )\n var_group_labels = []\n _var_names = []\n var_group_positions = []\n start = 0\n for label, vars_list in self.var_names.items():\n if isinstance(vars_list, str):\n vars_list = [vars_list]\n # use list() in case var_list is a numpy array or pandas series\n _var_names.extend(list(vars_list))\n var_group_labels.append(label)\n var_group_positions.append((start, start + len(vars_list) - 1))\n start += len(vars_list)\n self.var_names = _var_names\n self.var_group_labels = var_group_labels\n self.var_group_positions = var_group_positions\n self.has_var_groups = True\n\n elif isinstance(self.var_names, str):\n self.var_names = [self.var_names]\n\n\n\"\"\"Color palettes in addition to matplotlib's palettes.\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nfrom matplotlib import cm, colors\n\nif TYPE_CHECKING:\n from collections.abc import Mapping, Sequence\n\n# Colorblindness adjusted vega_10\n# See https://github.com/scverse/scanpy/issues/387\nvega_10 = list(map(colors.to_hex, cm.tab10.colors))\nvega_10_scanpy = vega_10.copy()\nvega_10_scanpy[2] = \"#279e68\" # green\nvega_10_scanpy[4] = \"#aa40fc\" # purple\nvega_10_scanpy[8] = \"#b5bd61\" # kakhi\n\n# default matplotlib 2.0 palette\n# see 'category20' on https://github.com/vega/vega/wiki/Scales#scale-range-literals\nvega_20 = list(map(colors.to_hex, cm.tab20.colors))\n\n# reorderd, some removed, some added\nvega_20_scanpy = [\n # dark without grey:\n *vega_20[0:14:2],\n *vega_20[16::2],\n # light without grey:\n *vega_20[1:15:2],\n *vega_20[17::2],\n # manual additions:\n \"#ad494a\",\n \"#8c6d31\",\n]\nvega_20_scanpy[2] = vega_10_scanpy[2]\nvega_20_scanpy[4] = vega_10_scanpy[4]\nvega_20_scanpy[7] = vega_10_scanpy[8] # kakhi shifted by missing grey\n# TODO: also replace pale colors if necessary\n\ndefault_20 = vega_20_scanpy\n\n# https://graphicdesign.stackexchange.com/questions/3682/where-can-i-find-a-large-palette-set-of-contrasting-colors-for-coloring-many-d\n# update 1\n# orig reference https://research.wu.ac.at/en/publications/escaping-rgbland-selecting-colors-for-statistical-graphics-26\nzeileis_28 = [\n \"#023fa5\",\n \"#7d87b9\",\n \"#bec1d4\",\n \"#d6bcc0\",\n \"#bb7784\",\n \"#8e063b\",\n \"#4a6fe3\",\n \"#8595e1\",\n \"#b5bbe3\",\n \"#e6afb9\",\n \"#e07b91\",\n \"#d33f6a\",\n \"#11c638\",\n \"#8dd593\",\n \"#c6dec7\",\n \"#ead3c6\",\n \"#f0b98d\",\n \"#ef9708\",\n \"#0fcfc0\",\n \"#9cded6\",\n \"#d5eae7\",\n \"#f3e1eb\",\n \"#f6c4e1\",\n \"#f79cd4\",\n # these last ones were added:\n \"#7f7f7f\",\n \"#c7c7c7\",\n \"#1CE6FF\",\n \"#336600\",\n]\n\ndefault_28 = zeileis_28\n\n# from https://godsnotwheregodsnot.blogspot.com/2012/09/color-distribution-methodology.html\ngodsnot_102 = [\n # \"#000000\", # remove the black, as often, we have black colored annotation\n \"#FFFF00\",\n \"#1CE6FF\",\n \"#FF34FF\",\n \"#FF4A46\",\n \"#008941\",\n \"#006FA6\",\n \"#A30059\",\n \"#FFDBE5\",\n \"#7A4900\",\n \"#0000A6\",\n \"#63FFAC\",\n \"#B79762\",\n \"#004D43\",\n \"#8FB0FF\",\n \"#997D87\",\n \"#5A0007\",\n \"#809693\",\n \"#6A3A4C\",\n \"#1B4400\",\n \"#4FC601\",\n \"#3B5DFF\",\n \"#4A3B53\",\n \"#FF2F80\",\n \"#61615A\",\n \"#BA0900\",\n \"#6B7900\",\n \"#00C2A0\",\n \"#FFAA92\",\n \"#FF90C9\",\n \"#B903AA\",\n \"#D16100\",\n \"#DDEFFF\",\n \"#000035\",\n \"#7B4F4B\",\n \"#A1C299\",\n \"#300018\",\n \"#0AA6D8\",\n \"#013349\",\n \"#00846F\",\n \"#372101\",\n \"#FFB500\",\n \"#C2FFED\",\n \"#A079BF\",\n \"#CC0744\",\n \"#C0B9B2\",\n \"#C2FF99\",\n \"#001E09\",\n \"#00489C\",\n \"#6F0062\",\n \"#0CBD66\",\n \"#EEC3FF\",\n \"#456D75\",\n \"#B77B68\",\n \"#7A87A1\",\n \"#788D66\",\n \"#885578\",\n \"#FAD09F\",\n \"#FF8A9A\",\n \"#D157A0\",\n \"#BEC459\",\n \"#456648\",\n \"#0086ED\",\n \"#886F4C\",\n \"#34362D\",\n \"#B4A8BD\",\n \"#00A6AA\",\n \"#452C2C\",\n \"#636375\",\n \"#A3C8C9\",\n \"#FF913F\",\n \"#938A81\",\n \"#575329\",\n \"#00FECF\",\n \"#B05B6F\",\n \"#8CD0FF\",\n \"#3B9700\",\n \"#04F757\",\n \"#C8A1A1\",\n \"#1E6E00\",\n \"#7900D7\",\n \"#A77500\",\n \"#6367A9\",\n \"#A05837\",\n \"#6B002C\",\n \"#772600\",\n \"#D790FF\",\n \"#9B9700\",\n \"#549E79\",\n \"#FFF69F\",\n \"#201625\",\n \"#72418F\",\n \"#BC23FF\",\n \"#99ADC0\",\n \"#3A2465\",\n \"#922329\",\n \"#5B4534\",\n \"#FDE8DC\",\n \"#404E55\",\n \"#0089A3\",\n \"#CB7E98\",\n \"#A4E804\",\n \"#324E72\",\n]\n\ndefault_102 = godsnot_102\n\n\ndef _plot_color_cycle(clists: Mapping[str, Sequence[str]]):\n import matplotlib.pyplot as plt\n import numpy as np\n from matplotlib.colors import BoundaryNorm, ListedColormap\n\n fig, axes = plt.subplots(nrows=len(clists)) # type: plt.Figure, plt.Axes\n fig.subplots_adjust(top=0.95, bottom=0.01, left=0.3, right=0.99)\n axes[0].set_title(\"Color Maps/Cycles\", fontsize=14)\n\n for ax, (name, clist) in zip(axes, clists.items()):\n n = len(clist)\n ax.imshow(\n np.arange(n)[None, :].repeat(2, 0),\n aspect=\"auto\",\n cmap=ListedColormap(clist),\n norm=BoundaryNorm(np.arange(n + 1) - 0.5, n),\n )\n pos = list(ax.get_position().bounds)\n x_text = pos[0] - 0.01\n y_text = pos[1] + pos[3] / 2.0\n fig.text(x_text, y_text, name, va=\"center\", ha=\"right\", fontsize=10)\n\n # Turn off all ticks & spines\n for ax in axes:\n ax.set_axis_off()\n fig.show()\n\n\nif __name__ == \"__main__\":\n _plot_color_cycle(\n {name: colors for name, colors in globals().items() if isinstance(colors, list)}\n )\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom collections.abc import Collection, Mapping, Sequence\nfrom pathlib import Path\nfrom types import MappingProxyType\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nimport scipy\nfrom matplotlib import patheffects, rcParams, ticker\nfrom matplotlib import pyplot as plt\nfrom matplotlib.colors import is_color_like\nfrom pandas.api.types import CategoricalDtype\nfrom scipy.sparse import issparse\nfrom sklearn.utils import check_random_state\n\nfrom scanpy.tools._draw_graph import coerce_fa2_layout, fa2_positions\n\nfrom ... import _utils as _sc_utils\nfrom ... import logging as logg\nfrom ..._compat import old_positionals\nfrom ..._settings import settings\nfrom .. import _utils\nfrom .._utils import matrix\n\nif TYPE_CHECKING:\n from typing import Any, Literal, Union\n\n from anndata import AnnData\n from matplotlib.axes import Axes\n from matplotlib.colors import Colormap\n from scipy.sparse import spmatrix\n\n from ...tools._draw_graph import _Layout as _LayoutWithoutEqTree\n from .._utils import _FontSize, _FontWeight, _LegendLoc\n\n _Layout = Union[_LayoutWithoutEqTree, Literal[\"eq_tree\"]]\n\n\n@old_positionals(\n \"edges\",\n \"color\",\n \"alpha\",\n \"groups\",\n \"components\",\n \"projection\",\n \"legend_loc\",\n \"legend_fontsize\",\n \"legend_fontweight\",\n \"legend_fontoutline\",\n \"color_map\",\n \"palette\",\n \"frameon\",\n \"size\",\n \"title\",\n \"right_margin\",\n \"left_margin\",\n \"show\",\n \"save\",\n \"title_graph\",\n \"groups_graph\",\n)\ndef paga_compare(\n adata: AnnData,\n basis=None,\n *,\n edges=False,\n color=None,\n alpha=None,\n groups=None,\n components=None,\n projection: Literal[\"2d\", \"3d\"] = \"2d\",\n legend_loc: _LegendLoc | None = \"on data\",\n legend_fontsize: int | float | _FontSize | None = None,\n legend_fontweight: int | _FontWeight = \"bold\",\n legend_fontoutline=None,\n color_map=None,\n palette=None,\n frameon=False,\n size=None,\n title=None,\n right_margin=None,\n left_margin=0.05,\n show=None,\n save=None,\n title_graph=None,\n groups_graph=None,\n pos=None,\n **paga_graph_params,\n):\n \"\"\"\\\n Scatter and PAGA graph side-by-side.\n\n Consists in a scatter plot and the abstracted graph. See\n :func:`~scanpy.pl.paga` for all related parameters.\n\n See :func:`~scanpy.pl.paga_path` for visualizing gene changes along paths\n through the abstracted graph.\n\n Additional parameters are as follows.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n kwds_scatter\n Keywords for :func:`~scanpy.pl.scatter`.\n kwds_paga\n Keywords for :func:`~scanpy.pl.paga`.\n\n Returns\n -------\n A list of :class:`~matplotlib.axes.Axes` if `show` is `False`.\n \"\"\"\n axs, _, _, _ = _utils.setup_axes(panels=[0, 1], right_margin=right_margin)\n if color is None:\n color = adata.uns[\"paga\"][\"groups\"]\n suptitle = None # common title for entire figure\n if title_graph is None:\n suptitle = color if title is None else title\n title, title_graph = \"\", \"\"\n if basis is None:\n if \"X_draw_graph_fa\" in adata.obsm:\n basis = \"draw_graph_fa\"\n elif \"X_umap\" in adata.obsm:\n basis = \"umap\"\n elif \"X_tsne\" in adata.obsm:\n basis = \"tsne\"\n elif \"X_draw_graph_fr\" in adata.obsm:\n basis = \"draw_graph_fr\"\n else:\n basis = \"umap\"\n\n from .scatterplots import _components_to_dimensions, _get_basis, embedding\n\n embedding(\n adata,\n ax=axs[0],\n basis=basis,\n color=color,\n edges=edges,\n alpha=alpha,\n groups=groups,\n components=components,\n legend_loc=legend_loc,\n legend_fontsize=legend_fontsize,\n legend_fontweight=legend_fontweight,\n legend_fontoutline=legend_fontoutline,\n color_map=color_map,\n palette=palette,\n frameon=frameon,\n size=size,\n title=title,\n show=False,\n save=False,\n )\n\n if pos is None:\n if color == adata.uns[\"paga\"][\"groups\"]:\n # TODO: Use dimensions here\n _basis = _get_basis(adata, basis)\n dims = _components_to_dimensions(\n components=components, dimensions=None, total_dims=_basis.shape[1]\n )[0]\n coords = _basis[:, dims]\n pos = (\n pd.DataFrame(coords, columns=[\"x\", \"y\"], index=adata.obs_names)\n .groupby(adata.obs[color], observed=True)\n .median()\n .sort_index()\n ).to_numpy()\n else:\n pos = adata.uns[\"paga\"][\"pos\"]\n xlim, ylim = axs[0].get_xlim(), axs[0].get_ylim()\n axs[1].set_xlim(xlim)\n axs[1].set_ylim(ylim)\n if \"labels\" in paga_graph_params:\n labels = paga_graph_params.pop(\"labels\")\n else:\n labels = groups_graph\n if legend_fontsize is not None:\n paga_graph_params[\"fontsize\"] = legend_fontsize\n if legend_fontweight is not None:\n paga_graph_params[\"fontweight\"] = legend_fontweight\n if legend_fontoutline is not None:\n paga_graph_params[\"fontoutline\"] = legend_fontoutline\n paga(\n adata,\n ax=axs[1],\n show=False,\n save=False,\n title=title_graph,\n labels=labels,\n colors=color,\n frameon=frameon,\n pos=pos,\n **paga_graph_params,\n )\n if suptitle is not None:\n plt.suptitle(suptitle)\n _utils.savefig_or_show(\"paga_compare\", show=show, save=save)\n if show:\n return None\n return axs\n\n\ndef _compute_pos(\n adjacency_solid: spmatrix | np.ndarray,\n *,\n layout: _Layout | None = None,\n random_state: _sc_utils.AnyRandom = 0,\n init_pos: np.ndarray | None = None,\n adj_tree=None,\n root: int = 0,\n layout_kwds: Mapping[str, Any] = MappingProxyType({}),\n):\n import random\n\n import networkx as nx\n\n random_state = check_random_state(random_state)\n\n nx_g_solid = nx.Graph(adjacency_solid)\n if layout is None:\n layout = \"fr\"\n layout = coerce_fa2_layout(layout)\n if layout == \"fa\":\n # np.random.seed(random_state)\n if init_pos is None:\n init_coords = random_state.random_sample((adjacency_solid.shape[0], 2))\n else:\n init_coords = init_pos.copy()\n pos_list = fa2_positions(adjacency_solid, init_coords, **layout_kwds)\n pos = {n: (x, -y) for n, (x, y) in enumerate(pos_list)}\n elif layout == \"eq_tree\":\n nx_g_tree = nx.Graph(adj_tree)\n pos = _utils.hierarchy_pos(nx_g_tree, root)\n if len(pos) < adjacency_solid.shape[0]:\n raise ValueError(\n \"This is a forest and not a single tree. \"\n \"Try another `layout`, e.g., {'fr'}.\"\n )\n else:\n # igraph layouts\n random.seed(random_state.bytes(8))\n g = _sc_utils.get_igraph_from_adjacency(adjacency_solid)\n if \"rt\" in layout:\n g_tree = _sc_utils.get_igraph_from_adjacency(adj_tree)\n pos_list = g_tree.layout(\n layout, root=root if isinstance(root, list) else [root]\n ).coords\n elif layout == \"circle\":\n pos_list = g.layout(layout).coords\n else:\n # I don't know why this is necessary\n # np.random.seed(random_state)\n if init_pos is None:\n init_coords = random_state.random_sample(\n (adjacency_solid.shape[0], 2)\n ).tolist()\n else:\n init_pos = init_pos.copy()\n # this is a super-weird hack that is necessary as igraph’s\n # layout function seems to do some strange stuff here\n init_pos[:, 1] *= -1\n init_coords = init_pos.tolist()\n try:\n pos_list = g.layout(\n layout, seed=init_coords, weights=\"weight\", **layout_kwds\n ).coords\n except AttributeError: # hack for empty graphs...\n pos_list = g.layout(layout, seed=init_coords, **layout_kwds).coords\n pos = {n: (x, -y) for n, (x, y) in enumerate(pos_list)}\n if len(pos) == 1:\n pos[0] = (0.5, 0.5)\n pos_array = np.array([pos[n] for count, n in enumerate(nx_g_solid)])\n return pos_array\n\n\n@old_positionals(\n \"threshold\",\n \"color\",\n \"layout\",\n \"layout_kwds\",\n \"init_pos\",\n \"root\",\n \"labels\",\n \"single_component\",\n \"solid_edges\",\n \"dashed_edges\",\n \"transitions\",\n \"fontsize\",\n \"fontweight\",\n \"fontoutline\",\n \"text_kwds\",\n \"node_size_scale\",\n # 17 positionals are enough for backwards compat\n)\ndef paga(\n adata: AnnData,\n *,\n threshold: float | None = None,\n color: str | Mapping[str | int, Mapping[Any, float]] | None = None,\n layout: _Layout | None = None,\n layout_kwds: Mapping[str, Any] = MappingProxyType({}),\n init_pos: np.ndarray | None = None,\n root: int | str | Sequence[int] | None = 0,\n labels: str | Sequence[str] | Mapping[str, str] | None = None,\n single_component: bool = False,\n solid_edges: str = \"connectivities\",\n dashed_edges: str | None = None,\n transitions: str | None = None,\n fontsize: int | None = None,\n fontweight: str = \"bold\",\n fontoutline: int | None = None,\n text_kwds: Mapping[str, Any] = MappingProxyType({}),\n node_size_scale: float = 1.0,\n node_size_power: float = 0.5,\n edge_width_scale: float = 1.0,\n min_edge_width: float | None = None,\n max_edge_width: float | None = None,\n arrowsize: int = 30,\n title: str | None = None,\n left_margin: float = 0.01,\n random_state: int | None = 0,\n pos: np.ndarray | Path | str | None = None,\n normalize_to_color: bool = False,\n cmap: str | Colormap | None = None,\n cax: Axes | None = None,\n colorbar=None, # TODO: this seems to be unused\n cb_kwds: Mapping[str, Any] = MappingProxyType({}),\n frameon: bool | None = None,\n add_pos: bool = True,\n export_to_gexf: bool = False,\n use_raw: bool = True,\n colors=None, # backwards compat\n groups=None, # backwards compat\n plot: bool = True,\n show: bool | None = None,\n save: bool | str | None = None,\n ax: Axes | None = None,\n) -> Axes | list[Axes] | None:\n \"\"\"\\\n Plot the PAGA graph through thresholding low-connectivity edges.\n\n Compute a coarse-grained layout of the data. Reuse this by passing\n `init_pos='paga'` to :func:`~scanpy.tl.umap` or\n :func:`~scanpy.tl.draw_graph` and obtain embeddings with more meaningful\n global topology :cite:p:`Wolf2019`.\n\n This uses ForceAtlas2 or igraph's layout algorithms for most layouts :cite:p:`Csardi2006`.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n threshold\n Do not draw edges for weights below this threshold. Set to 0 if you want\n all edges. Discarding low-connectivity edges helps in getting a much\n clearer picture of the graph.\n color\n Gene name or `obs` annotation defining the node colors.\n Also plots the degree of the abstracted graph when\n passing {`'degree_dashed'`, `'degree_solid'`}.\n\n Can be also used to visualize pie chart at each node in the following form:\n `{: {: , ...}, ...}`. If the fractions\n do not sum to 1, a new category called `'rest'` colored grey will be created.\n labels\n The node labels. If `None`, this defaults to the group labels stored in\n the categorical for which :func:`~scanpy.tl.paga` has been computed.\n pos\n Two-column array-like storing the x and y coordinates for drawing.\n Otherwise, path to a `.gdf` file that has been exported from Gephi or\n a similar graph visualization software.\n layout\n Plotting layout that computes positions.\n `'fa'` stands for “ForceAtlas2”,\n `'fr'` stands for “Fruchterman-Reingold”,\n `'rt'` stands for “Reingold-Tilford”,\n `'eq_tree'` stands for “eqally spaced tree”.\n All but `'fa'` and `'eq_tree'` are igraph layouts.\n All other igraph layouts are also permitted.\n See also parameter `pos` and :func:`~scanpy.tl.draw_graph`.\n layout_kwds\n Keywords for the layout.\n init_pos\n Two-column array storing the x and y coordinates for initializing the\n layout.\n random_state\n For layouts with random initialization like `'fr'`, change this to use\n different intial states for the optimization. If `None`, the initial\n state is not reproducible.\n root\n If choosing a tree layout, this is the index of the root node or a list\n of root node indices. If this is a non-empty vector then the supplied\n node IDs are used as the roots of the trees (or a single tree if the\n graph is connected). If this is `None` or an empty list, the root\n vertices are automatically calculated based on topological sorting.\n transitions\n Key for `.uns['paga']` that specifies the matrix that stores the\n arrows, for instance `'transitions_confidence'`.\n solid_edges\n Key for `.uns['paga']` that specifies the matrix that stores the edges\n to be drawn solid black.\n dashed_edges\n Key for `.uns['paga']` that specifies the matrix that stores the edges\n to be drawn dashed grey. If `None`, no dashed edges are drawn.\n single_component\n Restrict to largest connected component.\n fontsize\n Font size for node labels.\n fontoutline\n Width of the white outline around fonts.\n text_kwds\n Keywords for :meth:`~matplotlib.axes.Axes.text`.\n node_size_scale\n Increase or decrease the size of the nodes.\n node_size_power\n The power with which groups sizes influence the radius of the nodes.\n edge_width_scale\n Edge with scale in units of `rcParams['lines.linewidth']`.\n min_edge_width\n Min width of solid edges.\n max_edge_width\n Max width of solid and dashed edges.\n arrowsize\n For directed graphs, choose the size of the arrow head head's length and\n width. See :py:class: `matplotlib.patches.FancyArrowPatch` for attribute\n `mutation_scale` for more info.\n export_to_gexf\n Export to gexf format to be read by graph visualization programs such as\n Gephi.\n normalize_to_color\n Whether to normalize categorical plots to `color` or the underlying\n grouping.\n cmap\n The color map.\n cax\n A matplotlib axes object for a potential colorbar.\n cb_kwds\n Keyword arguments for :class:`~matplotlib.colorbar.Colorbar`,\n for instance, `ticks`.\n add_pos\n Add the positions to `adata.uns['paga']`.\n title\n Provide a title.\n frameon\n Draw a frame around the PAGA graph.\n plot\n If `False`, do not create the figure, simply compute the layout.\n save\n If `True` or a `str`, save the figure.\n A string is appended to the default filename.\n Infer the filetype if ending on \\\\{`'.pdf'`, `'.png'`, `'.svg'`\\\\}.\n ax\n A matplotlib axes object.\n\n Returns\n -------\n If `show==False`, one or more :class:`~matplotlib.axes.Axes` objects.\n Adds `'pos'` to `adata.uns['paga']` if `add_pos` is `True`.\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc3k_processed()\n sc.tl.paga(adata, groups='louvain')\n sc.pl.paga(adata)\n\n You can increase node and edge sizes by specifying additional arguments.\n\n .. plot::\n :context: close-figs\n\n sc.pl.paga(adata, node_size_scale=10, edge_width_scale=2)\n\n Notes\n -----\n When initializing the positions, note that – for some reason – igraph\n mirrors coordinates along the x axis... that is, you should increase the\n `maxiter` parameter by 1 if the layout is flipped.\n\n .. currentmodule:: scanpy\n\n See also\n --------\n tl.paga\n pl.paga_compare\n pl.paga_path\n \"\"\"\n\n if groups is not None: # backwards compat\n labels = groups\n logg.warning(\"`groups` is deprecated in `pl.paga`: use `labels` instead\")\n if colors is None:\n colors = color\n\n groups_key = adata.uns[\"paga\"][\"groups\"]\n\n def is_flat(x):\n has_one_per_category = isinstance(x, Collection) and len(x) == len(\n adata.obs[groups_key].cat.categories\n )\n return has_one_per_category or x is None or isinstance(x, str)\n\n if isinstance(colors, Mapping) and isinstance(colors[next(iter(colors))], Mapping):\n # handle paga pie, remap string keys to integers\n names_to_ixs = {\n n: i for i, n in enumerate(adata.obs[groups_key].cat.categories)\n }\n colors = {names_to_ixs.get(n, n): v for n, v in colors.items()}\n if is_flat(colors):\n colors = [colors]\n\n if frameon is None:\n frameon = settings._frameon\n # labels is a list that contains no lists\n if is_flat(labels):\n labels = [labels for _ in range(len(colors))]\n\n if title is None and len(colors) > 1:\n title = [c for c in colors]\n elif isinstance(title, str):\n title = [title for c in colors]\n elif title is None:\n title = [None for c in colors]\n\n if colorbar is None:\n var_names = adata.var_names if adata.raw is None else adata.raw.var_names\n colorbars = [\n (\n (c in adata.obs_keys() and adata.obs[c].dtype.name != \"category\")\n or (c in var_names)\n )\n for c in colors\n ]\n else:\n colorbars = [False for _ in colors]\n\n if isinstance(root, str):\n if root not in labels:\n raise ValueError(\n \"If `root` is a string, \"\n f\"it needs to be one of {labels} not {root!r}.\"\n )\n root = list(labels).index(root)\n if isinstance(root, Sequence) and root[0] in labels:\n root = [list(labels).index(r) for r in root]\n\n # define the adjacency matrices\n adjacency_solid = adata.uns[\"paga\"][solid_edges].copy()\n adjacency_dashed = None\n if threshold is None:\n threshold = 0.01 # default threshold\n if threshold > 0:\n adjacency_solid.data[adjacency_solid.data < threshold] = 0\n adjacency_solid.eliminate_zeros()\n if dashed_edges is not None:\n adjacency_dashed = adata.uns[\"paga\"][dashed_edges].copy()\n if threshold > 0:\n adjacency_dashed.data[adjacency_dashed.data < threshold] = 0\n adjacency_dashed.eliminate_zeros()\n\n # compute positions\n if pos is None:\n adj_tree = None\n if layout in {\"rt\", \"rt_circular\", \"eq_tree\"}:\n adj_tree = adata.uns[\"paga\"][\"connectivities_tree\"]\n pos = _compute_pos(\n adjacency_solid,\n layout=layout,\n random_state=random_state,\n init_pos=init_pos,\n layout_kwds=layout_kwds,\n adj_tree=adj_tree,\n root=root,\n )\n\n if plot:\n axs, panel_pos, draw_region_width, figure_width = _utils.setup_axes(\n ax, panels=colors, colorbars=colorbars\n )\n\n if len(colors) == 1 and not isinstance(axs, list):\n axs = [axs]\n\n for icolor, c in enumerate(colors):\n if title[icolor] is not None:\n axs[icolor].set_title(title[icolor])\n sct = _paga_graph(\n adata,\n axs[icolor],\n colors=colors if isinstance(colors, Mapping) else c,\n solid_edges=solid_edges,\n dashed_edges=dashed_edges,\n transitions=transitions,\n threshold=threshold,\n adjacency_solid=adjacency_solid,\n adjacency_dashed=adjacency_dashed,\n root=root,\n labels=labels[icolor],\n fontsize=fontsize,\n fontweight=fontweight,\n fontoutline=fontoutline,\n text_kwds=text_kwds,\n node_size_scale=node_size_scale,\n node_size_power=node_size_power,\n edge_width_scale=edge_width_scale,\n min_edge_width=min_edge_width,\n max_edge_width=max_edge_width,\n normalize_to_color=normalize_to_color,\n frameon=frameon,\n cmap=cmap,\n colorbar=colorbars[icolor],\n cb_kwds=cb_kwds,\n use_raw=use_raw,\n title=title[icolor],\n export_to_gexf=export_to_gexf,\n single_component=single_component,\n arrowsize=arrowsize,\n pos=pos,\n )\n if colorbars[icolor]:\n if cax is None:\n bottom = panel_pos[0][0]\n height = panel_pos[1][0] - bottom\n width = 0.006 * draw_region_width / len(colors)\n left = panel_pos[2][2 * icolor + 1] + 0.2 * width\n rectangle = [left, bottom, width, height]\n fig = plt.gcf()\n ax_cb = fig.add_axes(rectangle)\n else:\n ax_cb = cax[icolor]\n\n _ = plt.colorbar(\n sct,\n format=ticker.FuncFormatter(_utils.ticks_formatter),\n cax=ax_cb,\n )\n if add_pos:\n adata.uns[\"paga\"][\"pos\"] = pos\n logg.hint(\"added 'pos', the PAGA positions (adata.uns['paga'])\")\n\n if not plot:\n return None\n _utils.savefig_or_show(\"paga\", show=show, save=save)\n if len(colors) == 1 and isinstance(axs, list):\n axs = axs[0]\n show = settings.autoshow if show is None else show\n if show:\n return None\n return axs\n\n\ndef _paga_graph(\n adata,\n ax,\n *,\n solid_edges=None,\n dashed_edges=None,\n adjacency_solid=None,\n adjacency_dashed=None,\n transitions=None,\n threshold=None,\n root=0,\n colors=None,\n labels=None,\n fontsize=None,\n fontweight=None,\n fontoutline=None,\n text_kwds: Mapping[str, Any] = MappingProxyType({}),\n node_size_scale=1.0,\n node_size_power=0.5,\n edge_width_scale=1.0,\n normalize_to_color=\"reference\",\n title=None,\n pos=None,\n cmap=None,\n frameon=True,\n min_edge_width=None,\n max_edge_width=None,\n export_to_gexf=False,\n colorbar=None,\n use_raw=True,\n cb_kwds: Mapping[str, Any] = MappingProxyType({}),\n single_component=False,\n arrowsize=30,\n):\n import networkx as nx\n\n node_labels = labels # rename for clarity\n if (\n node_labels is not None\n and isinstance(node_labels, str)\n and node_labels != adata.uns[\"paga\"][\"groups\"]\n ):\n raise ValueError(\n \"Provide a list of group labels for the PAGA groups {}, not {}.\".format(\n adata.uns[\"paga\"][\"groups\"], node_labels\n )\n )\n groups_key = adata.uns[\"paga\"][\"groups\"]\n if node_labels is None:\n node_labels = adata.obs[groups_key].cat.categories\n\n if (colors is None or colors == groups_key) and groups_key is not None:\n if groups_key + \"_colors\" not in adata.uns or len(\n adata.obs[groups_key].cat.categories\n ) != len(adata.uns[groups_key + \"_colors\"]):\n _utils.add_colors_for_categorical_sample_annotation(adata, groups_key)\n colors = adata.uns[groups_key + \"_colors\"]\n for iname, name in enumerate(adata.obs[groups_key].cat.categories):\n if name in settings.categories_to_ignore:\n colors[iname] = \"grey\"\n\n nx_g_solid = nx.Graph(adjacency_solid)\n if dashed_edges is not None:\n nx_g_dashed = nx.Graph(adjacency_dashed)\n\n # convert pos to array and dict\n if not isinstance(pos, (Path, str)):\n pos_array = pos\n else:\n pos = Path(pos)\n if pos.suffix != \".gdf\":\n raise ValueError(\n \"Currently only supporting reading positions from .gdf files. \"\n \"Consider generating them using, for instance, Gephi.\"\n )\n s = \"\" # read the node definition from the file\n with pos.open() as f:\n f.readline()\n for line in f:\n if line.startswith(\"edgedef>\"):\n break\n s += line\n from io import StringIO\n\n df = pd.read_csv(StringIO(s), header=-1)\n pos_array = df[[4, 5]].values\n\n # convert to dictionary\n pos = {n: [p[0], p[1]] for n, p in enumerate(pos_array)}\n\n # uniform color\n if isinstance(colors, str) and is_color_like(colors):\n colors = [colors for c in range(len(node_labels))]\n\n # color degree of the graph\n if isinstance(colors, str) and colors.startswith(\"degree\"):\n # see also tools.paga.paga_degrees\n if colors == \"degree_dashed\":\n colors = [d for _, d in nx_g_dashed.degree(weight=\"weight\")]\n elif colors == \"degree_solid\":\n colors = [d for _, d in nx_g_solid.degree(weight=\"weight\")]\n else:\n raise ValueError('`degree` either \"degree_dashed\" or \"degree_solid\".')\n colors = (np.array(colors) - np.min(colors)) / (np.max(colors) - np.min(colors))\n\n # plot gene expression\n var_names = adata.var_names if adata.raw is None else adata.raw.var_names\n if isinstance(colors, str) and colors in var_names:\n x_color = []\n cats = adata.obs[groups_key].cat.categories\n for icat, cat in enumerate(cats):\n subset = (cat == adata.obs[groups_key]).values\n if adata.raw is not None and use_raw:\n adata_gene = adata.raw[:, colors]\n else:\n adata_gene = adata[:, colors]\n x_color.append(np.mean(adata_gene.X[subset]))\n colors = x_color\n\n # plot continuous annotation\n if (\n isinstance(colors, str)\n and colors in adata.obs\n and not isinstance(adata.obs[colors].dtype, CategoricalDtype)\n ):\n x_color = []\n cats = adata.obs[groups_key].cat.categories\n for icat, cat in enumerate(cats):\n subset = (cat == adata.obs[groups_key]).values\n x_color.append(adata.obs.loc[subset, colors].mean())\n colors = x_color\n\n # plot categorical annotation\n if (\n isinstance(colors, str)\n and colors in adata.obs\n and isinstance(adata.obs[colors].dtype, CategoricalDtype)\n ):\n asso_names, asso_matrix = _sc_utils.compute_association_matrix_of_groups(\n adata,\n prediction=groups_key,\n reference=colors,\n normalization=\"reference\" if normalize_to_color else \"prediction\",\n )\n _utils.add_colors_for_categorical_sample_annotation(adata, colors)\n asso_colors = _sc_utils.get_associated_colors_of_groups(\n adata.uns[colors + \"_colors\"], asso_matrix\n )\n colors = asso_colors\n\n if len(colors) != len(node_labels):\n raise ValueError(\n f\"Expected `colors` to be of length `{len(node_labels)}`, \"\n f\"found `{len(colors)}`.\"\n )\n\n # count number of connected components\n n_components, labels = scipy.sparse.csgraph.connected_components(adjacency_solid)\n if n_components > 1 and not single_component:\n logg.debug(\n \"Graph has more than a single connected component. \"\n \"To restrict to this component, pass `single_component=True`.\"\n )\n if n_components > 1 and single_component:\n component_sizes = np.bincount(labels)\n largest_component = np.where(component_sizes == component_sizes.max())[0][0]\n adjacency_solid = adjacency_solid.tocsr()[labels == largest_component, :]\n adjacency_solid = adjacency_solid.tocsc()[:, labels == largest_component]\n colors = np.array(colors)[labels == largest_component]\n node_labels = np.array(node_labels)[labels == largest_component]\n cats_dropped = (\n adata.obs[groups_key].cat.categories[labels != largest_component].tolist()\n )\n logg.info(\n \"Restricting graph to largest connected component by dropping categories\\n\"\n f\"{cats_dropped}\"\n )\n nx_g_solid = nx.Graph(adjacency_solid)\n if dashed_edges is not None:\n raise ValueError(\"`single_component` only if `dashed_edges` is `None`.\")\n\n # edge widths\n base_edge_width = edge_width_scale * 5 * rcParams[\"lines.linewidth\"]\n\n # draw dashed edges\n if dashed_edges is not None:\n widths = [x[-1][\"weight\"] for x in nx_g_dashed.edges(data=True)]\n widths = base_edge_width * np.array(widths)\n if max_edge_width is not None:\n widths = np.clip(widths, None, max_edge_width)\n nx.draw_networkx_edges(\n nx_g_dashed,\n pos,\n ax=ax,\n width=widths,\n edge_color=\"grey\",\n style=\"dashed\",\n alpha=0.5,\n )\n\n # draw solid edges\n if transitions is None:\n widths = [x[-1][\"weight\"] for x in nx_g_solid.edges(data=True)]\n widths = base_edge_width * np.array(widths)\n if min_edge_width is not None or max_edge_width is not None:\n widths = np.clip(widths, min_edge_width, max_edge_width)\n with warnings.catch_warnings():\n warnings.simplefilter(\"ignore\")\n nx.draw_networkx_edges(\n nx_g_solid, pos, ax=ax, width=widths, edge_color=\"black\"\n )\n # draw directed edges\n else:\n adjacency_transitions = adata.uns[\"paga\"][transitions].copy()\n if threshold is None:\n threshold = 0.01\n adjacency_transitions.data[adjacency_transitions.data < threshold] = 0\n adjacency_transitions.eliminate_zeros()\n g_dir = nx.DiGraph(adjacency_transitions.T)\n widths = [x[-1][\"weight\"] for x in g_dir.edges(data=True)]\n widths = base_edge_width * np.array(widths)\n if min_edge_width is not None or max_edge_width is not None:\n widths = np.clip(widths, min_edge_width, max_edge_width)\n nx.draw_networkx_edges(\n g_dir, pos, ax=ax, width=widths, edge_color=\"black\", arrowsize=arrowsize\n )\n\n if export_to_gexf:\n if isinstance(colors[0], tuple):\n from matplotlib.colors import rgb2hex\n\n colors = [rgb2hex(c) for c in colors]\n for count, n in enumerate(nx_g_solid.nodes()):\n nx_g_solid.node[count][\"label\"] = str(node_labels[count])\n nx_g_solid.node[count][\"color\"] = str(colors[count])\n nx_g_solid.node[count][\"viz\"] = dict(\n position=dict(\n x=1000 * pos[count][0],\n y=1000 * pos[count][1],\n z=0,\n )\n )\n filename = settings.writedir / \"paga_graph.gexf\"\n logg.warning(f\"exporting to {filename}\")\n settings.writedir.mkdir(parents=True, exist_ok=True)\n nx.write_gexf(nx_g_solid, settings.writedir / \"paga_graph.gexf\")\n\n ax.set_frame_on(frameon)\n ax.set_xticks([])\n ax.set_yticks([])\n\n # groups sizes\n if groups_key is not None and groups_key + \"_sizes\" in adata.uns:\n groups_sizes = adata.uns[groups_key + \"_sizes\"]\n else:\n groups_sizes = np.ones(len(node_labels))\n base_scale_scatter = 2000\n base_pie_size = (\n base_scale_scatter / (np.sqrt(adjacency_solid.shape[0]) + 10) * node_size_scale\n )\n median_group_size = np.median(groups_sizes)\n groups_sizes = base_pie_size * np.power(\n groups_sizes / median_group_size, node_size_power\n )\n\n if fontsize is None:\n fontsize = rcParams[\"legend.fontsize\"]\n if fontoutline is not None:\n text_kwds = dict(text_kwds)\n text_kwds[\"path_effects\"] = [\n patheffects.withStroke(linewidth=fontoutline, foreground=\"w\")\n ]\n # usual scatter plot\n if not isinstance(colors[0], Mapping):\n n_groups = len(pos_array)\n sct = ax.scatter(\n pos_array[:, 0],\n pos_array[:, 1],\n c=colors[:n_groups],\n edgecolors=\"face\",\n s=groups_sizes,\n cmap=cmap,\n )\n for count, group in enumerate(node_labels):\n ax.text(\n pos_array[count, 0],\n pos_array[count, 1],\n group,\n verticalalignment=\"center\",\n horizontalalignment=\"center\",\n size=fontsize,\n fontweight=fontweight,\n **text_kwds,\n )\n # else pie chart plot\n else:\n for ix, (xx, yy) in enumerate(zip(pos_array[:, 0], pos_array[:, 1])):\n if not isinstance(colors[ix], Mapping):\n raise ValueError(\n f\"{colors[ix]} is neither a dict of valid \"\n \"matplotlib colors nor a valid matplotlib color.\"\n )\n color_single = colors[ix].keys()\n fracs = [colors[ix][c] for c in color_single]\n total = sum(fracs)\n\n if total < 1:\n color_single = list(color_single)\n color_single.append(\"grey\")\n fracs.append(1 - sum(fracs))\n elif not np.isclose(total, 1):\n raise ValueError(\n f\"Expected fractions for node `{ix}` to be \"\n f\"close to 1, found `{total}`.\"\n )\n\n cumsum = np.cumsum(fracs)\n cumsum = cumsum / cumsum[-1]\n cumsum = [0] + cumsum.tolist()\n\n for r1, r2, color in zip(cumsum[:-1], cumsum[1:], color_single):\n angles = np.linspace(2 * np.pi * r1, 2 * np.pi * r2, 20)\n x = [0] + np.cos(angles).tolist()\n y = [0] + np.sin(angles).tolist()\n\n xy = np.column_stack([x, y])\n s = np.abs(xy).max()\n\n sct = ax.scatter(\n [xx], [yy], marker=xy, s=s**2 * groups_sizes[ix], color=color\n )\n\n if node_labels is not None:\n ax.text(\n xx,\n yy,\n node_labels[ix],\n verticalalignment=\"center\",\n horizontalalignment=\"center\",\n size=fontsize,\n fontweight=fontweight,\n **text_kwds,\n )\n\n return sct\n\n\n@old_positionals(\n \"use_raw\",\n \"annotations\",\n \"color_map\",\n \"color_maps_annotations\",\n \"palette_groups\",\n \"n_avg\",\n \"groups_key\",\n \"xlim\",\n \"title\",\n \"left_margin\",\n \"ytick_fontsize\",\n \"title_fontsize\",\n \"show_node_names\",\n \"show_yticks\",\n \"show_colorbar\",\n \"legend_fontsize\",\n \"legend_fontweight\",\n \"normalize_to_zero_one\",\n \"as_heatmap\",\n \"return_data\",\n \"show\",\n \"save\",\n \"ax\",\n)\ndef paga_path(\n adata: AnnData,\n nodes: Sequence[str | int],\n keys: Sequence[str],\n *,\n use_raw: bool = True,\n annotations: Sequence[str] = (\"dpt_pseudotime\",),\n color_map: str | Colormap | None = None,\n color_maps_annotations: Mapping[str, str | Colormap] = MappingProxyType(\n dict(dpt_pseudotime=\"Greys\")\n ),\n palette_groups: Sequence[str] | None = None,\n n_avg: int = 1,\n groups_key: str | None = None,\n xlim: tuple[int | None, int | None] = (None, None),\n title: str | None = None,\n left_margin=None,\n ytick_fontsize: int | None = None,\n title_fontsize: int | None = None,\n show_node_names: bool = True,\n show_yticks: bool = True,\n show_colorbar: bool = True,\n legend_fontsize: int | float | _FontSize | None = None,\n legend_fontweight: int | _FontWeight | None = None,\n normalize_to_zero_one: bool = False,\n as_heatmap: bool = True,\n return_data: bool = False,\n show: bool | None = None,\n save: bool | str | None = None,\n ax: Axes | None = None,\n) -> tuple[Axes, pd.DataFrame] | Axes | pd.DataFrame | None:\n \"\"\"\\\n Gene expression and annotation changes along paths in the abstracted graph.\n\n Parameters\n ----------\n adata\n An annotated data matrix.\n nodes\n A path through nodes of the abstracted graph, that is, names or indices\n (within `.categories`) of groups that have been used to run PAGA.\n keys\n Either variables in `adata.var_names` or annotations in\n `adata.obs`. They are plotted using `color_map`.\n use_raw\n Use `adata.raw` for retrieving gene expressions if it has been set.\n annotations\n Plot these keys with `color_maps_annotations`. Need to be keys for\n `adata.obs`.\n color_map\n Matplotlib colormap.\n color_maps_annotations\n Color maps for plotting the annotations. Keys of the dictionary must\n appear in `annotations`.\n palette_groups\n Ususally, use the same `sc.pl.palettes...` as used for coloring the\n abstracted graph.\n n_avg\n Number of data points to include in computation of running average.\n groups_key\n Key of the grouping used to run PAGA. If `None`, defaults to\n `adata.uns['paga']['groups']`.\n as_heatmap\n Plot the timeseries as heatmap. If not plotting as heatmap,\n `annotations` have no effect.\n show_node_names\n Plot the node names on the nodes bar.\n show_colorbar\n Show the colorbar.\n show_yticks\n Show the y ticks.\n normalize_to_zero_one\n Shift and scale the running average to [0, 1] per gene.\n return_data\n Return the timeseries data in addition to the axes if `True`.\n show\n Show the plot, do not return axis.\n save\n If `True` or a `str`, save the figure.\n A string is appended to the default filename.\n Infer the filetype if ending on \\\\{`'.pdf'`, `'.png'`, `'.svg'`\\\\}.\n ax\n A matplotlib axes object.\n\n Returns\n -------\n A :class:`~matplotlib.axes.Axes` object, if `ax` is `None`, else `None`.\n If `return_data`, return the timeseries data in addition to an axes.\n \"\"\"\n ax_was_none = ax is None\n\n if groups_key is None:\n if \"groups\" not in adata.uns[\"paga\"]:\n raise KeyError(\n \"Pass the key of the grouping with which you ran PAGA, \"\n \"using the parameter `groups_key`.\"\n )\n groups_key = adata.uns[\"paga\"][\"groups\"]\n groups_names = adata.obs[groups_key].cat.categories\n\n if \"dpt_pseudotime\" not in adata.obs.columns:\n raise ValueError(\n \"`pl.paga_path` requires computation of a pseudotime `tl.dpt` \"\n \"for ordering at single-cell resolution\"\n )\n\n if palette_groups is None:\n _utils.add_colors_for_categorical_sample_annotation(adata, groups_key)\n palette_groups = adata.uns[f\"{groups_key}_colors\"]\n\n def moving_average(a):\n return _sc_utils.moving_average(a, n_avg)\n\n ax = plt.gca() if ax is None else ax\n\n X = []\n x_tick_locs = [0]\n x_tick_labels = []\n groups = []\n anno_dict = {anno: [] for anno in annotations}\n if isinstance(nodes[0], str):\n nodes_ints = []\n groups_names_set = set(groups_names)\n for node in nodes:\n if node not in groups_names_set:\n raise ValueError(\n f\"Each node/group needs to be in {groups_names.tolist()} \"\n f\"(`groups_key`={groups_key!r}) not {node!r}.\"\n )\n nodes_ints.append(groups_names.get_loc(node))\n nodes_strs = nodes\n else:\n nodes_ints = nodes\n nodes_strs = [groups_names[node] for node in nodes]\n\n adata_X = adata\n if use_raw and adata.raw is not None:\n adata_X = adata.raw\n\n for ikey, key in enumerate(keys):\n x = []\n for igroup, group in enumerate(nodes_ints):\n idcs = np.arange(adata.n_obs)[\n adata.obs[groups_key].values == nodes_strs[igroup]\n ]\n if len(idcs) == 0:\n raise ValueError(\n \"Did not find data points that match \"\n f\"`adata.obs[{groups_key!r}].values == {str(group)!r}`. \"\n f\"Check whether `adata.obs[{groups_key!r}]` \"\n \"actually contains what you expect.\"\n )\n idcs_group = np.argsort(\n adata.obs[\"dpt_pseudotime\"].values[\n adata.obs[groups_key].values == nodes_strs[igroup]\n ]\n )\n idcs = idcs[idcs_group]\n values = (\n adata.obs[key].values if key in adata.obs_keys() else adata_X[:, key].X\n )[idcs]\n x += (values.toarray() if issparse(values) else values).tolist()\n if ikey == 0:\n groups += [group] * len(idcs)\n x_tick_locs.append(len(x))\n for anno in annotations:\n series = adata.obs[anno]\n if isinstance(series.dtype, CategoricalDtype):\n series = series.cat.codes\n anno_dict[anno] += list(series.values[idcs])\n if n_avg > 1:\n x = moving_average(x)\n if ikey == 0:\n for key in annotations:\n if not isinstance(anno_dict[key][0], str):\n anno_dict[key] = moving_average(anno_dict[key])\n if normalize_to_zero_one:\n x -= np.min(x)\n x /= np.max(x)\n X.append(x)\n if not as_heatmap:\n ax.plot(x[xlim[0] : xlim[1]], label=key)\n if ikey == 0:\n for igroup, group in enumerate(nodes):\n if len(groups_names) > 0 and group not in groups_names:\n label = groups_names[group]\n else:\n label = group\n x_tick_labels.append(label)\n X = np.asarray(X).squeeze()\n if as_heatmap:\n img = ax.imshow(X, aspect=\"auto\", interpolation=\"nearest\", cmap=color_map)\n if show_yticks:\n ax.set_yticks(range(len(X)))\n ax.set_yticklabels(keys, fontsize=ytick_fontsize)\n else:\n ax.set_yticks([])\n ax.set_frame_on(False)\n ax.set_xticks([])\n ax.tick_params(axis=\"both\", which=\"both\", length=0)\n ax.grid(visible=False)\n if show_colorbar:\n plt.colorbar(img, ax=ax)\n left_margin = 0.2 if left_margin is None else left_margin\n plt.subplots_adjust(left=left_margin)\n else:\n left_margin = 0.4 if left_margin is None else left_margin\n if len(keys) > 1:\n plt.legend(\n frameon=False,\n loc=\"center left\",\n bbox_to_anchor=(-left_margin, 0.5),\n fontsize=legend_fontsize,\n )\n xlabel = groups_key\n if not as_heatmap:\n ax.set_xlabel(xlabel)\n plt.yticks([])\n if len(keys) == 1:\n plt.ylabel(keys[0] + \" (a.u.)\")\n else:\n import matplotlib.colors\n\n # groups bar\n ax_bounds = ax.get_position().bounds\n groups_axis = plt.axes(\n (\n ax_bounds[0],\n ax_bounds[1] - ax_bounds[3] / len(keys),\n ax_bounds[2],\n ax_bounds[3] / len(keys),\n )\n )\n groups = np.array(groups)[None, :]\n groups_axis.imshow(\n groups,\n aspect=\"auto\",\n interpolation=\"nearest\",\n cmap=matplotlib.colors.ListedColormap(\n # the following line doesn't work because of normalization\n # adata.uns['paga_groups_colors'])\n palette_groups[np.min(groups).astype(int) :],\n N=int(np.max(groups) + 1 - np.min(groups)),\n ),\n )\n if show_yticks:\n groups_axis.set_yticklabels([\"\", xlabel, \"\"], fontsize=ytick_fontsize)\n else:\n groups_axis.set_yticks([])\n groups_axis.set_frame_on(False)\n if show_node_names:\n ypos = (groups_axis.get_ylim()[1] + groups_axis.get_ylim()[0]) / 2\n x_tick_locs = _sc_utils.moving_average(x_tick_locs, n=2)\n for ilabel, label in enumerate(x_tick_labels):\n groups_axis.text(\n x_tick_locs[ilabel],\n ypos,\n x_tick_labels[ilabel],\n fontdict=dict(\n horizontalalignment=\"center\",\n verticalalignment=\"center\",\n ),\n )\n groups_axis.set_xticks([])\n groups_axis.grid(visible=False)\n groups_axis.tick_params(axis=\"both\", which=\"both\", length=0)\n # further annotations\n y_shift = ax_bounds[3] / len(keys)\n for ianno, anno in enumerate(annotations):\n if ianno > 0:\n y_shift = ax_bounds[3] / len(keys) / 2\n anno_axis = plt.axes(\n (\n ax_bounds[0],\n ax_bounds[1] - (ianno + 2) * y_shift,\n ax_bounds[2],\n y_shift,\n )\n )\n arr = np.array(anno_dict[anno])[None, :]\n if anno not in color_maps_annotations:\n color_map_anno = (\n \"Vega10\"\n if isinstance(adata.obs[anno].dtype, CategoricalDtype)\n else \"Greys\"\n )\n else:\n color_map_anno = color_maps_annotations[anno]\n img = anno_axis.imshow(\n arr,\n aspect=\"auto\",\n interpolation=\"nearest\",\n cmap=color_map_anno,\n )\n if show_yticks:\n anno_axis.set_yticklabels([\"\", anno, \"\"], fontsize=ytick_fontsize)\n anno_axis.tick_params(axis=\"both\", which=\"both\", length=0)\n else:\n anno_axis.set_yticks([])\n anno_axis.set_frame_on(False)\n anno_axis.set_xticks([])\n anno_axis.grid(visible=False)\n if title is not None:\n ax.set_title(title, fontsize=title_fontsize)\n if show is None and not ax_was_none:\n show = False\n else:\n show = settings.autoshow if show is None else show\n _utils.savefig_or_show(\"paga_path\", show=show, save=save)\n if return_data:\n df = pd.DataFrame(data=X.T, columns=keys)\n df[\"groups\"] = moving_average(groups) # groups is without moving average, yet\n if \"dpt_pseudotime\" in anno_dict:\n df[\"distance\"] = anno_dict[\"dpt_pseudotime\"].T\n if not ax_was_none or show:\n return df if return_data else None\n return (ax, df) if return_data else ax\n\n\ndef paga_adjacency(\n adata: AnnData,\n *,\n adjacency: str = \"connectivities\",\n adjacency_tree: str = \"connectivities_tree\",\n as_heatmap: bool = True,\n color_map: str | Colormap | None = None,\n show: bool | None = None,\n save: bool | str | None = None,\n) -> None:\n \"\"\"Connectivity of paga groups.\"\"\"\n connectivity = adata.uns[adjacency].toarray()\n connectivity_select = adata.uns[adjacency_tree]\n if as_heatmap:\n matrix(connectivity, color_map=color_map, show=False)\n for i in range(connectivity_select.shape[0]):\n neighbors = connectivity_select[i].nonzero()[1]\n plt.scatter([i for j in neighbors], neighbors, color=\"black\", s=1)\n # as a stripplot\n else:\n plt.figure()\n for i, cs in enumerate(connectivity):\n x = [i for j, d in enumerate(cs) if i != j]\n y = [c for j, c in enumerate(cs) if i != j]\n plt.scatter(x, y, color=\"gray\", s=1)\n neighbors = connectivity_select[i].nonzero()[1]\n plt.scatter([i for j in neighbors], cs[neighbors], color=\"black\", s=1)\n _utils.savefig_or_show(\"paga_connectivity\", show=show, save=save)\n\n\nfrom __future__ import annotations\n\nimport inspect\nimport sys\nfrom collections.abc import Mapping, Sequence # noqa: TCH003\nfrom copy import copy\nfrom functools import partial\nfrom itertools import combinations, product\nfrom numbers import Integral\nfrom typing import (\n TYPE_CHECKING,\n Any, # noqa: TCH003\n Literal, # noqa: TCH003\n)\n\nimport numpy as np\nimport pandas as pd\nfrom anndata import AnnData # noqa: TCH002\nfrom cycler import Cycler # noqa: TCH002\nfrom matplotlib import colormaps, colors, patheffects, rcParams\nfrom matplotlib import pyplot as plt\nfrom matplotlib.axes import Axes # noqa: TCH002\nfrom matplotlib.colors import (\n Colormap, # noqa: TCH002\n Normalize,\n)\nfrom matplotlib.figure import Figure # noqa: TCH002\nfrom numpy.typing import NDArray # noqa: TCH002\nfrom packaging.version import Version\n\nfrom ... import logging as logg\nfrom ..._settings import settings\nfrom ..._utils import (\n Empty, # noqa: TCH001\n _doc_params,\n _empty,\n sanitize_anndata,\n)\nfrom ...get import _check_mask\nfrom ...tools._draw_graph import _Layout # noqa: TCH001\nfrom .. import _utils\nfrom .._docs import (\n doc_adata_color_etc,\n doc_edges_arrows,\n doc_scatter_embedding,\n doc_scatter_spatial,\n doc_show_save_ax,\n)\nfrom .._utils import (\n ColorLike, # noqa: TCH001\n VBound, # noqa: TCH001\n _FontSize, # noqa: TCH001\n _FontWeight, # noqa: TCH001\n _LegendLoc, # noqa: TCH001\n check_colornorm,\n check_projection,\n circles,\n)\n\nif TYPE_CHECKING:\n from collections.abc import Collection\n\n\n@_doc_params(\n adata_color_etc=doc_adata_color_etc,\n edges_arrows=doc_edges_arrows,\n scatter_bulk=doc_scatter_embedding,\n show_save_ax=doc_show_save_ax,\n)\ndef embedding(\n adata: AnnData,\n basis: str,\n *,\n color: str | Sequence[str] | None = None,\n mask_obs: NDArray[np.bool_] | str | None = None,\n gene_symbols: str | None = None,\n use_raw: bool | None = None,\n sort_order: bool = True,\n edges: bool = False,\n edges_width: float = 0.1,\n edges_color: str | Sequence[float] | Sequence[str] = \"grey\",\n neighbors_key: str | None = None,\n arrows: bool = False,\n arrows_kwds: Mapping[str, Any] | None = None,\n groups: str | Sequence[str] | None = None,\n components: str | Sequence[str] | None = None,\n dimensions: tuple[int, int] | Sequence[tuple[int, int]] | None = None,\n layer: str | None = None,\n projection: Literal[\"2d\", \"3d\"] = \"2d\",\n scale_factor: float | None = None,\n color_map: Colormap | str | None = None,\n cmap: Colormap | str | None = None,\n palette: str | Sequence[str] | Cycler | None = None,\n na_color: ColorLike = \"lightgray\",\n na_in_legend: bool = True,\n size: float | Sequence[float] | None = None,\n frameon: bool | None = None,\n legend_fontsize: int | float | _FontSize | None = None,\n legend_fontweight: int | _FontWeight = \"bold\",\n legend_loc: _LegendLoc | None = \"right margin\",\n legend_fontoutline: int | None = None,\n colorbar_loc: str | None = \"right\",\n vmax: VBound | Sequence[VBound] | None = None,\n vmin: VBound | Sequence[VBound] | None = None,\n vcenter: VBound | Sequence[VBound] | None = None,\n norm: Normalize | Sequence[Normalize] | None = None,\n add_outline: bool | None = False,\n outline_width: tuple[float, float] = (0.3, 0.05),\n outline_color: tuple[str, str] = (\"black\", \"white\"),\n ncols: int = 4,\n hspace: float = 0.25,\n wspace: float | None = None,\n title: str | Sequence[str] | None = None,\n show: bool | None = None,\n save: bool | str | None = None,\n ax: Axes | None = None,\n return_fig: bool | None = None,\n marker: str | Sequence[str] = \".\",\n **kwargs,\n) -> Figure | Axes | list[Axes] | None:\n \"\"\"\\\n Scatter plot for user specified embedding basis (e.g. umap, pca, etc)\n\n Parameters\n ----------\n basis\n Name of the `obsm` basis to use.\n {adata_color_etc}\n {edges_arrows}\n {scatter_bulk}\n {show_save_ax}\n\n Returns\n -------\n If `show==False` a :class:`~matplotlib.axes.Axes` or a list of it.\n \"\"\"\n #####################\n # Argument handling #\n #####################\n\n check_projection(projection)\n sanitize_anndata(adata)\n\n basis_values = _get_basis(adata, basis)\n dimensions = _components_to_dimensions(\n components, dimensions, projection=projection, total_dims=basis_values.shape[1]\n )\n args_3d = dict(projection=\"3d\") if projection == \"3d\" else {}\n\n # Checking the mask format and if used together with groups\n if groups is not None and mask_obs is not None:\n raise ValueError(\"Groups and mask arguments are incompatible.\")\n if mask_obs is not None:\n mask_obs = _check_mask(adata, mask_obs, \"obs\")\n\n # Figure out if we're using raw\n if use_raw is None:\n # check if adata.raw is set\n use_raw = layer is None and adata.raw is not None\n if use_raw and layer is not None:\n raise ValueError(\n \"Cannot use both a layer and the raw representation. Was passed:\"\n f\"use_raw={use_raw}, layer={layer}.\"\n )\n if use_raw and adata.raw is None:\n raise ValueError(\n \"`use_raw` is set to True but AnnData object does not have raw. \"\n \"Please check.\"\n )\n\n if isinstance(groups, str):\n groups = [groups]\n\n # Color map\n if color_map is not None:\n if cmap is not None:\n raise ValueError(\"Cannot specify both `color_map` and `cmap`.\")\n else:\n cmap = color_map\n cmap = copy(colormaps.get_cmap(cmap))\n cmap.set_bad(na_color)\n # Prevents warnings during legend creation\n na_color = colors.to_hex(na_color, keep_alpha=True)\n\n # by default turn off edge color. Otherwise, for\n # very small sizes the edge will not reduce its size\n # (https://github.com/scverse/scanpy/issues/293)\n kwargs.setdefault(\"edgecolor\", \"none\")\n\n # Vectorized arguments\n\n # turn color into a python list\n color = [color] if isinstance(color, str) or color is None else list(color)\n\n # turn marker into a python list\n marker = [marker] if isinstance(marker, str) else list(marker)\n\n if title is not None:\n # turn title into a python list if not None\n title = [title] if isinstance(title, str) else list(title)\n\n # turn vmax and vmin into a sequence\n if isinstance(vmax, str) or not isinstance(vmax, Sequence):\n vmax = [vmax]\n if isinstance(vmin, str) or not isinstance(vmin, Sequence):\n vmin = [vmin]\n if isinstance(vcenter, str) or not isinstance(vcenter, Sequence):\n vcenter = [vcenter]\n if isinstance(norm, Normalize) or not isinstance(norm, Sequence):\n norm = [norm]\n\n # Size\n if \"s\" in kwargs and size is None:\n size = kwargs.pop(\"s\")\n if size is not None:\n # check if size is any type of sequence, and if so\n # set as ndarray\n if (\n size is not None\n and isinstance(size, (Sequence, pd.Series, np.ndarray))\n and len(size) == adata.shape[0]\n ):\n size = np.array(size, dtype=float)\n else:\n size = 120000 / adata.shape[0]\n\n ##########\n # Layout #\n ##########\n # Most of the code is for the case when multiple plots are required\n\n if wspace is None:\n # try to set a wspace that is not too large or too small given the\n # current figure size\n wspace = 0.75 / rcParams[\"figure.figsize\"][0] + 0.02\n\n if components is not None:\n color, dimensions = list(zip(*product(color, dimensions)))\n\n color, dimensions, marker = _broadcast_args(color, dimensions, marker)\n\n # 'color' is a list of names that want to be plotted.\n # Eg. ['Gene1', 'louvain', 'Gene2'].\n # component_list is a list of components [[0,1], [1,2]]\n if (\n not isinstance(color, str) and isinstance(color, Sequence) and len(color) > 1\n ) or len(dimensions) > 1:\n if ax is not None:\n raise ValueError(\n \"Cannot specify `ax` when plotting multiple panels \"\n \"(each for a given value of 'color').\"\n )\n\n # each plot needs to be its own panel\n fig, grid = _panel_grid(hspace, wspace, ncols, len(color))\n else:\n grid = None\n if ax is None:\n fig = plt.figure()\n ax = fig.add_subplot(111, **args_3d)\n\n ############\n # Plotting #\n ############\n axs = []\n\n # use itertools.product to make a plot for each color and for each component\n # For example if color=[gene1, gene2] and components=['1,2, '2,3'].\n # The plots are: [\n # color=gene1, components=[1,2], color=gene1, components=[2,3],\n # color=gene2, components = [1, 2], color=gene2, components=[2,3],\n # ]\n for count, (value_to_plot, dims) in enumerate(zip(color, dimensions)):\n kwargs_scatter = kwargs.copy() # is potentially mutated for each plot\n color_source_vector = _get_color_source_vector(\n adata,\n value_to_plot,\n layer=layer,\n mask_obs=mask_obs,\n use_raw=use_raw,\n gene_symbols=gene_symbols,\n groups=groups,\n )\n color_vector, color_type = _color_vector(\n adata,\n value_to_plot,\n values=color_source_vector,\n palette=palette,\n na_color=na_color,\n )\n\n # Order points\n order = slice(None)\n if sort_order and value_to_plot is not None and color_type == \"cont\":\n # Higher values plotted on top, null values on bottom\n order = np.argsort(-color_vector, kind=\"stable\")[::-1]\n elif sort_order and color_type == \"cat\":\n # Null points go on bottom\n order = np.argsort(~pd.isnull(color_source_vector), kind=\"stable\")\n # Set orders\n if isinstance(size, np.ndarray):\n size = np.array(size)[order]\n color_source_vector = color_source_vector[order]\n color_vector = color_vector[order]\n coords = basis_values[:, dims][order, :]\n\n # if plotting multiple panels, get the ax from the grid spec\n # else use the ax value (either user given or created previously)\n if grid:\n ax = plt.subplot(grid[count], **args_3d)\n axs.append(ax)\n if not (settings._frameon if frameon is None else frameon):\n ax.axis(\"off\")\n if title is None:\n if value_to_plot is not None:\n ax.set_title(value_to_plot)\n else:\n ax.set_title(\"\")\n else:\n try:\n ax.set_title(title[count])\n except IndexError:\n logg.warning(\n \"The title list is shorter than the number of panels. \"\n \"Using 'color' value instead for some plots.\"\n )\n ax.set_title(value_to_plot)\n\n if color_type == \"cont\":\n vmin_float, vmax_float, vcenter_float, norm_obj = _get_vboundnorm(\n vmin, vmax, vcenter, norm=norm, index=count, colors=color_vector\n )\n kwargs_scatter[\"norm\"] = check_colornorm(\n vmin_float,\n vmax_float,\n vcenter_float,\n norm_obj,\n )\n kwargs_scatter[\"cmap\"] = cmap\n\n # make the scatter plot\n if projection == \"3d\":\n cax = ax.scatter(\n coords[:, 0],\n coords[:, 1],\n coords[:, 2],\n c=color_vector,\n rasterized=settings._vector_friendly,\n marker=marker[count],\n **kwargs_scatter,\n )\n else:\n scatter = (\n partial(ax.scatter, s=size, plotnonfinite=True)\n if scale_factor is None\n else partial(\n circles, s=size, ax=ax, scale_factor=scale_factor\n ) # size in circles is radius\n )\n\n if add_outline:\n # the default outline is a black edge followed by a\n # thin white edged added around connected clusters.\n # To add an outline\n # three overlapping scatter plots are drawn:\n # First black dots with slightly larger size,\n # then, white dots a bit smaller, but still larger\n # than the final dots. Then the final dots are drawn\n # with some transparency.\n\n bg_width, gap_width = outline_width\n point = np.sqrt(size)\n gap_size = (point + (point * gap_width) * 2) ** 2\n bg_size = (np.sqrt(gap_size) + (point * bg_width) * 2) ** 2\n # the default black and white colors can be changes using\n # the contour_config parameter\n bg_color, gap_color = outline_color\n\n # remove edge from kwargs if present\n # because edge needs to be set to None\n kwargs_scatter[\"edgecolor\"] = \"none\"\n # For points, if user did not set alpha, set alpha to 0.7\n kwargs_scatter.setdefault(\"alpha\", 0.7)\n\n # remove alpha and color mapping for outline\n kwargs_outline = {\n k: v\n for k, v in kwargs.items()\n if k not in {\"alpha\", \"cmap\", \"norm\"}\n }\n\n for s, c in [(bg_size, bg_color), (gap_size, gap_color)]:\n ax.scatter(\n coords[:, 0],\n coords[:, 1],\n s=s,\n c=c,\n rasterized=settings._vector_friendly,\n marker=marker[count],\n **kwargs_outline,\n )\n\n cax = scatter(\n coords[:, 0],\n coords[:, 1],\n c=color_vector,\n rasterized=settings._vector_friendly,\n marker=marker[count],\n **kwargs_scatter,\n )\n\n # remove y and x ticks\n ax.set_yticks([])\n ax.set_xticks([])\n if projection == \"3d\":\n ax.set_zticks([])\n\n # set default axis_labels\n name = _basis2name(basis)\n axis_labels = [name + str(d + 1) for d in dims]\n\n ax.set_xlabel(axis_labels[0])\n ax.set_ylabel(axis_labels[1])\n if projection == \"3d\":\n # shift the label closer to the axis\n ax.set_zlabel(axis_labels[2], labelpad=-7)\n ax.autoscale_view()\n\n if edges:\n _utils.plot_edges(\n ax, adata, basis, edges_width, edges_color, neighbors_key=neighbors_key\n )\n if arrows:\n _utils.plot_arrows(ax, adata, basis, arrows_kwds)\n\n if value_to_plot is None:\n # if only dots were plotted without an associated value\n # there is not need to plot a legend or a colorbar\n continue\n\n if legend_fontoutline is not None:\n path_effect = [\n patheffects.withStroke(linewidth=legend_fontoutline, foreground=\"w\")\n ]\n else:\n path_effect = None\n\n # Adding legends\n if color_type == \"cat\":\n _add_categorical_legend(\n ax,\n color_source_vector,\n palette=_get_palette(adata, value_to_plot),\n scatter_array=coords,\n legend_loc=legend_loc,\n legend_fontweight=legend_fontweight,\n legend_fontsize=legend_fontsize,\n legend_fontoutline=path_effect,\n na_color=na_color,\n na_in_legend=na_in_legend,\n multi_panel=bool(grid),\n )\n elif colorbar_loc is not None:\n plt.colorbar(\n cax, ax=ax, pad=0.01, fraction=0.08, aspect=30, location=colorbar_loc\n )\n\n if return_fig is True:\n return fig\n axs = axs if grid else ax\n _utils.savefig_or_show(basis, show=show, save=save)\n show = settings.autoshow if show is None else show\n if show:\n return None\n return axs\n\n\ndef _panel_grid(hspace, wspace, ncols, num_panels):\n from matplotlib import gridspec\n\n n_panels_x = min(ncols, num_panels)\n n_panels_y = np.ceil(num_panels / n_panels_x).astype(int)\n # each panel will have the size of rcParams['figure.figsize']\n fig = plt.figure(\n figsize=(\n n_panels_x * rcParams[\"figure.figsize\"][0] * (1 + wspace),\n n_panels_y * rcParams[\"figure.figsize\"][1],\n ),\n )\n left = 0.2 / n_panels_x\n bottom = 0.13 / n_panels_y\n gs = gridspec.GridSpec(\n nrows=n_panels_y,\n ncols=n_panels_x,\n left=left,\n right=1 - (n_panels_x - 1) * left - 0.01 / n_panels_x,\n bottom=bottom,\n top=1 - (n_panels_y - 1) * bottom - 0.1 / n_panels_y,\n hspace=hspace,\n wspace=wspace,\n )\n return fig, gs\n\n\ndef _get_vboundnorm(\n vmin: Sequence[VBound],\n vmax: Sequence[VBound],\n vcenter: Sequence[VBound],\n *,\n norm: Sequence[Normalize],\n index: int,\n colors: Sequence[float],\n) -> tuple[float | None, float | None]:\n \"\"\"\n Evaluates the value of vmin, vmax and vcenter, which could be a\n str in which case is interpreted as a percentile and should\n be specified in the form 'pN' where N is the percentile.\n Eg. for a percentile of 85 the format would be 'p85'.\n Floats are accepted as p99.9\n\n Alternatively, vmin/vmax could be a function that is applied to\n the list of color values (`colors`). E.g.\n\n def my_vmax(colors): np.percentile(colors, p=80)\n\n\n Parameters\n ----------\n index\n This index of the plot\n colors\n Values for the plot\n\n Returns\n -------\n\n (vmin, vmax, vcenter, norm) containing None or float values for\n vmin, vmax, vcenter and matplotlib.colors.Normalize or None for norm.\n\n \"\"\"\n out = []\n for v_name, v in [(\"vmin\", vmin), (\"vmax\", vmax), (\"vcenter\", vcenter)]:\n if len(v) == 1:\n # this case usually happens when the user sets eg vmax=0.9, which\n # is internally converted into list of len=1, but is expected that this\n # value applies to all plots.\n v_value = v[0]\n else:\n try:\n v_value = v[index]\n except IndexError:\n logg.error(\n f\"The parameter {v_name} is not valid. If setting multiple {v_name} values,\"\n f\"check that the length of the {v_name} list is equal to the number \"\n \"of plots. \"\n )\n v_value = None\n\n if v_value is not None:\n if isinstance(v_value, str) and v_value.startswith(\"p\"):\n try:\n float(v_value[1:])\n except ValueError:\n logg.error(\n f\"The parameter {v_name}={v_value} for plot number {index + 1} is not valid. \"\n f\"Please check the correct format for percentiles.\"\n )\n # interpret value of vmin/vmax as quantile with the following syntax 'p99.9'\n v_value = np.nanpercentile(colors, q=float(v_value[1:]))\n elif callable(v_value):\n # interpret vmin/vmax as function\n v_value = v_value(colors)\n if not isinstance(v_value, float):\n logg.error(\n f\"The return of the function given for {v_name} is not valid. \"\n \"Please check that the function returns a number.\"\n )\n v_value = None\n else:\n try:\n float(v_value)\n except ValueError:\n logg.error(\n f\"The given {v_name}={v_value} for plot number {index + 1} is not valid. \"\n f\"Please check that the value given is a valid number, a string \"\n f\"starting with 'p' for percentiles or a valid function.\"\n )\n v_value = None\n out.append(v_value)\n out.append(norm[0] if len(norm) == 1 else norm[index])\n return tuple(out)\n\n\ndef _wraps_plot_scatter(wrapper):\n \"\"\"Update the wrapper function to use the correct signature.\"\"\"\n if sys.version_info < (3, 10):\n # Python 3.9 does not support `eval_str`, so we only support this in 3.10+\n return wrapper\n\n params = inspect.signature(embedding, eval_str=True).parameters.copy()\n wrapper_sig = inspect.signature(wrapper, eval_str=True)\n wrapper_params = wrapper_sig.parameters.copy()\n\n params.pop(\"basis\")\n params.pop(\"kwargs\")\n wrapper_params.pop(\"adata\")\n\n params.update(wrapper_params)\n annotations = {\n k: v.annotation\n for k, v in params.items()\n if v.annotation != inspect.Parameter.empty\n }\n if wrapper_sig.return_annotation is not inspect.Signature.empty:\n annotations[\"return\"] = wrapper_sig.return_annotation\n\n wrapper.__signature__ = inspect.Signature(\n list(params.values()), return_annotation=wrapper_sig.return_annotation\n )\n wrapper.__annotations__ = annotations\n\n return wrapper\n\n\n# API\n\n\n@_wraps_plot_scatter\n@_doc_params(\n adata_color_etc=doc_adata_color_etc,\n edges_arrows=doc_edges_arrows,\n scatter_bulk=doc_scatter_embedding,\n show_save_ax=doc_show_save_ax,\n)\ndef umap(adata: AnnData, **kwargs) -> Figure | Axes | list[Axes] | None:\n \"\"\"\\\n Scatter plot in UMAP basis.\n\n Parameters\n ----------\n {adata_color_etc}\n {edges_arrows}\n {scatter_bulk}\n {show_save_ax}\n\n Returns\n -------\n If `show==False` a :class:`~matplotlib.axes.Axes` or a list of it.\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.pl.umap(adata)\n\n Colour points by discrete variable (Louvain clusters).\n\n .. plot::\n :context: close-figs\n\n sc.pl.umap(adata, color=\"louvain\")\n\n Colour points by gene expression.\n\n .. plot::\n :context: close-figs\n\n sc.pl.umap(adata, color=\"HES4\")\n\n Plot muliple umaps for different gene expressions.\n\n .. plot::\n :context: close-figs\n\n sc.pl.umap(adata, color=[\"HES4\", \"TNFRSF4\"])\n\n .. currentmodule:: scanpy\n\n See also\n --------\n tl.umap\n \"\"\"\n return embedding(adata, \"umap\", **kwargs)\n\n\n@_wraps_plot_scatter\n@_doc_params(\n adata_color_etc=doc_adata_color_etc,\n edges_arrows=doc_edges_arrows,\n scatter_bulk=doc_scatter_embedding,\n show_save_ax=doc_show_save_ax,\n)\ndef tsne(adata: AnnData, **kwargs) -> Figure | Axes | list[Axes] | None:\n \"\"\"\\\n Scatter plot in tSNE basis.\n\n Parameters\n ----------\n {adata_color_etc}\n {edges_arrows}\n {scatter_bulk}\n {show_save_ax}\n\n Returns\n -------\n If `show==False` a :class:`~matplotlib.axes.Axes` or a list of it.\n\n Examples\n --------\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.tl.tsne(adata)\n sc.pl.tsne(adata, color='bulk_labels')\n\n .. currentmodule:: scanpy\n\n See also\n --------\n tl.tsne\n \"\"\"\n return embedding(adata, \"tsne\", **kwargs)\n\n\n@_wraps_plot_scatter\n@_doc_params(\n adata_color_etc=doc_adata_color_etc,\n scatter_bulk=doc_scatter_embedding,\n show_save_ax=doc_show_save_ax,\n)\ndef diffmap(adata: AnnData, **kwargs) -> Figure | Axes | list[Axes] | None:\n \"\"\"\\\n Scatter plot in Diffusion Map basis.\n\n Parameters\n ----------\n {adata_color_etc}\n {scatter_bulk}\n {show_save_ax}\n\n Returns\n -------\n If `show==False` a :class:`~matplotlib.axes.Axes` or a list of it.\n\n Examples\n --------\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.tl.diffmap(adata)\n sc.pl.diffmap(adata, color='bulk_labels')\n\n .. currentmodule:: scanpy\n\n See also\n --------\n tl.diffmap\n \"\"\"\n return embedding(adata, \"diffmap\", **kwargs)\n\n\n@_wraps_plot_scatter\n@_doc_params(\n adata_color_etc=doc_adata_color_etc,\n edges_arrows=doc_edges_arrows,\n scatter_bulk=doc_scatter_embedding,\n show_save_ax=doc_show_save_ax,\n)\ndef draw_graph(\n adata: AnnData, *, layout: _Layout | None = None, **kwargs\n) -> Figure | Axes | list[Axes] | None:\n \"\"\"\\\n Scatter plot in graph-drawing basis.\n\n Parameters\n ----------\n {adata_color_etc}\n layout\n One of the :func:`~scanpy.tl.draw_graph` layouts.\n By default, the last computed layout is used.\n {edges_arrows}\n {scatter_bulk}\n {show_save_ax}\n\n Returns\n -------\n If `show==False` a :class:`~matplotlib.axes.Axes` or a list of it.\n\n Examples\n --------\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.tl.draw_graph(adata)\n sc.pl.draw_graph(adata, color=['phase', 'bulk_labels'])\n\n .. currentmodule:: scanpy\n\n See also\n --------\n tl.draw_graph\n \"\"\"\n if layout is None:\n layout = str(adata.uns[\"draw_graph\"][\"params\"][\"layout\"])\n basis = f\"draw_graph_{layout}\"\n if f\"X_{basis}\" not in adata.obsm_keys():\n raise ValueError(\n f\"Did not find {basis} in adata.obs. Did you compute layout {layout}?\"\n )\n\n return embedding(adata, basis, **kwargs)\n\n\n@_wraps_plot_scatter\n@_doc_params(\n adata_color_etc=doc_adata_color_etc,\n scatter_bulk=doc_scatter_embedding,\n show_save_ax=doc_show_save_ax,\n)\ndef pca(\n adata: AnnData,\n *,\n annotate_var_explained: bool = False,\n show: bool | None = None,\n return_fig: bool | None = None,\n save: bool | str | None = None,\n **kwargs,\n) -> Figure | Axes | list[Axes] | None:\n \"\"\"\\\n Scatter plot in PCA coordinates.\n\n Use the parameter `annotate_var_explained` to annotate the explained variance.\n\n Parameters\n ----------\n {adata_color_etc}\n annotate_var_explained\n {scatter_bulk}\n {show_save_ax}\n\n Returns\n -------\n If `show==False` a :class:`~matplotlib.axes.Axes` or a list of it.\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc3k_processed()\n sc.pl.pca(adata)\n\n Colour points by discrete variable (Louvain clusters).\n\n .. plot::\n :context: close-figs\n\n sc.pl.pca(adata, color=\"louvain\")\n\n Colour points by gene expression.\n\n .. plot::\n :context: close-figs\n\n sc.pl.pca(adata, color=\"CST3\")\n\n .. currentmodule:: scanpy\n\n See also\n --------\n pp.pca\n \"\"\"\n if not annotate_var_explained:\n return embedding(\n adata, \"pca\", show=show, return_fig=return_fig, save=save, **kwargs\n )\n if \"pca\" not in adata.obsm and \"X_pca\" not in adata.obsm:\n raise KeyError(\n f\"Could not find entry in `obsm` for 'pca'.\\n\"\n f\"Available keys are: {list(adata.obsm.keys())}.\"\n )\n\n label_dict = {\n f\"PC{i + 1}\": f\"PC{i + 1} ({round(v * 100, 2)}%)\"\n for i, v in enumerate(adata.uns[\"pca\"][\"variance_ratio\"])\n }\n\n if return_fig is True:\n # edit axis labels in returned figure\n fig = embedding(adata, \"pca\", return_fig=return_fig, **kwargs)\n for ax in fig.axes:\n if xlabel := label_dict.get(ax.xaxis.get_label().get_text()):\n ax.set_xlabel(xlabel)\n if ylabel := label_dict.get(ax.yaxis.get_label().get_text()):\n ax.set_ylabel(ylabel)\n return fig\n\n # get the axs, edit the labels and apply show and save from user\n axs = embedding(adata, \"pca\", show=False, save=False, **kwargs)\n if isinstance(axs, list):\n for ax in axs:\n ax.set_xlabel(label_dict[ax.xaxis.get_label().get_text()])\n ax.set_ylabel(label_dict[ax.yaxis.get_label().get_text()])\n else:\n axs.set_xlabel(label_dict[axs.xaxis.get_label().get_text()])\n axs.set_ylabel(label_dict[axs.yaxis.get_label().get_text()])\n _utils.savefig_or_show(\"pca\", show=show, save=save)\n show = settings.autoshow if show is None else show\n if show:\n return None\n return axs\n\n\n@_wraps_plot_scatter\n@_doc_params(\n adata_color_etc=doc_adata_color_etc,\n scatter_spatial=doc_scatter_spatial,\n scatter_bulk=doc_scatter_embedding,\n show_save_ax=doc_show_save_ax,\n)\ndef spatial(\n adata: AnnData,\n *,\n basis: str = \"spatial\",\n img: np.ndarray | None = None,\n img_key: str | None | Empty = _empty,\n library_id: str | None | Empty = _empty,\n crop_coord: tuple[int, int, int, int] | None = None,\n alpha_img: float = 1.0,\n bw: bool | None = False,\n size: float = 1.0,\n scale_factor: float | None = None,\n spot_size: float | None = None,\n na_color: ColorLike | None = None,\n show: bool | None = None,\n return_fig: bool | None = None,\n save: bool | str | None = None,\n **kwargs,\n) -> Figure | Axes | list[Axes] | None:\n \"\"\"\\\n Scatter plot in spatial coordinates.\n\n This function allows overlaying data on top of images.\n Use the parameter `img_key` to see the image in the background\n And the parameter `library_id` to select the image.\n By default, `'hires'` and `'lowres'` are attempted.\n\n Use `crop_coord`, `alpha_img`, and `bw` to control how it is displayed.\n Use `size` to scale the size of the Visium spots plotted on top.\n\n As this function is designed to for imaging data, there are two key assumptions\n about how coordinates are handled:\n\n 1. The origin (e.g `(0, 0)`) is at the top left – as is common convention\n with image data.\n\n 2. Coordinates are in the pixel space of the source image, so an equal\n aspect ratio is assumed.\n\n If your anndata object has a `\"spatial\"` entry in `.uns`, the `img_key`\n and `library_id` parameters to find values for `img`, `scale_factor`,\n and `spot_size` arguments. Alternatively, these values be passed directly.\n\n Parameters\n ----------\n {adata_color_etc}\n {scatter_spatial}\n {scatter_bulk}\n {show_save_ax}\n\n Returns\n -------\n If `show==False` a :class:`~matplotlib.axes.Axes` or a list of it.\n\n Examples\n --------\n This function behaves very similarly to other embedding plots like\n :func:`~scanpy.pl.umap`\n\n >>> import scanpy as sc\n >>> adata = sc.datasets.visium_sge(\"Targeted_Visium_Human_Glioblastoma_Pan_Cancer\")\n >>> sc.pp.calculate_qc_metrics(adata, inplace=True)\n >>> sc.pl.spatial(adata, color=\"log1p_n_genes_by_counts\")\n\n See Also\n --------\n :func:`scanpy.datasets.visium_sge`\n Example visium data.\n :doc:`/tutorials/spatial/basic-analysis`\n Tutorial on spatial analysis.\n \"\"\"\n # get default image params if available\n library_id, spatial_data = _check_spatial_data(adata.uns, library_id)\n img, img_key = _check_img(spatial_data, img, img_key, bw=bw)\n spot_size = _check_spot_size(spatial_data, spot_size)\n scale_factor = _check_scale_factor(\n spatial_data, img_key=img_key, scale_factor=scale_factor\n )\n crop_coord = _check_crop_coord(crop_coord, scale_factor)\n na_color = _check_na_color(na_color, img=img)\n\n cmap_img = \"gray\" if bw else None\n circle_radius = size * scale_factor * spot_size * 0.5\n\n axs = embedding(\n adata,\n basis=basis,\n scale_factor=scale_factor,\n size=circle_radius,\n na_color=na_color,\n show=False,\n save=False,\n **kwargs,\n )\n if not isinstance(axs, list):\n axs = [axs]\n for ax in axs:\n cur_coords = np.concatenate([ax.get_xlim(), ax.get_ylim()])\n if img is not None:\n ax.imshow(img, cmap=cmap_img, alpha=alpha_img)\n else:\n ax.set_aspect(\"equal\")\n ax.invert_yaxis()\n if crop_coord is not None:\n ax.set_xlim(crop_coord[0], crop_coord[1])\n ax.set_ylim(crop_coord[3], crop_coord[2])\n else:\n ax.set_xlim(cur_coords[0], cur_coords[1])\n ax.set_ylim(cur_coords[3], cur_coords[2])\n _utils.savefig_or_show(\"show\", show=show, save=save)\n if return_fig:\n return axs[0].figure\n show = settings.autoshow if show is None else show\n if show:\n return None\n return axs\n\n\n# Helpers\ndef _components_to_dimensions(\n components: str | Collection[str] | None,\n dimensions: Collection[int] | Collection[Collection[int]] | None,\n *,\n projection: Literal[\"2d\", \"3d\"] = \"2d\",\n total_dims: int,\n) -> list[Collection[int]]:\n \"\"\"Normalize components/ dimensions args for embedding plots.\"\"\"\n # TODO: Deprecate components kwarg\n ndims = {\"2d\": 2, \"3d\": 3}[projection]\n if components is None and dimensions is None:\n dimensions = [tuple(i for i in range(ndims))]\n elif components is not None and dimensions is not None:\n raise ValueError(\"Cannot provide both dimensions and components\")\n\n # TODO: Consider deprecating this\n # If components is not None, parse them and set dimensions\n if components == \"all\":\n dimensions = list(combinations(range(total_dims), ndims))\n elif components is not None:\n if isinstance(components, str):\n components = [components]\n # Components use 1 based indexing\n dimensions = [[int(dim) - 1 for dim in c.split(\",\")] for c in components]\n\n if all(isinstance(el, Integral) for el in dimensions):\n dimensions = [dimensions]\n # if all(isinstance(el, Collection) for el in dimensions):\n for dims in dimensions:\n if len(dims) != ndims or not all(isinstance(d, Integral) for d in dims):\n raise ValueError()\n\n return dimensions\n\n\ndef _add_categorical_legend(\n ax: Axes,\n color_source_vector,\n *,\n palette: dict,\n legend_loc: _LegendLoc | None,\n legend_fontweight,\n legend_fontsize,\n legend_fontoutline,\n multi_panel,\n na_color,\n na_in_legend: bool,\n scatter_array=None,\n):\n \"\"\"Add a legend to the passed Axes.\"\"\"\n if na_in_legend and pd.isnull(color_source_vector).any():\n if \"NA\" in color_source_vector:\n raise NotImplementedError(\n \"No fallback for null labels has been defined if NA already in categories.\"\n )\n color_source_vector = color_source_vector.add_categories(\"NA\").fillna(\"NA\")\n palette = palette.copy()\n palette[\"NA\"] = na_color\n if color_source_vector.dtype == bool:\n cats = pd.Categorical(color_source_vector.astype(str)).categories\n else:\n cats = color_source_vector.categories\n\n if multi_panel is True:\n # Shrink current axis by 10% to fit legend and match\n # size of plots that are not categorical\n box = ax.get_position()\n ax.set_position([box.x0, box.y0, box.width * 0.91, box.height])\n\n if legend_loc == \"on data\":\n # identify centroids to put labels\n\n all_pos = (\n pd.DataFrame(scatter_array, columns=[\"x\", \"y\"])\n .groupby(color_source_vector, observed=True)\n .median()\n # Have to sort_index since if observed=True and categorical is unordered\n # the order of values in .index is undefined. Related issue:\n # https://github.com/pandas-dev/pandas/issues/25167\n .sort_index()\n )\n\n for label, x_pos, y_pos in all_pos.itertuples():\n ax.text(\n x_pos,\n y_pos,\n label,\n weight=legend_fontweight,\n verticalalignment=\"center\",\n horizontalalignment=\"center\",\n fontsize=legend_fontsize,\n path_effects=legend_fontoutline,\n )\n elif legend_loc not in {None, \"none\"}:\n for label in cats:\n ax.scatter([], [], c=palette[label], label=label)\n if legend_loc == \"right margin\":\n ax.legend(\n frameon=False,\n loc=\"center left\",\n bbox_to_anchor=(1, 0.5),\n ncol=(1 if len(cats) <= 14 else 2 if len(cats) <= 30 else 3),\n fontsize=legend_fontsize,\n )\n else:\n ax.legend(loc=legend_loc, fontsize=legend_fontsize)\n\n\ndef _get_basis(adata: AnnData, basis: str) -> np.ndarray:\n \"\"\"Get array for basis from anndata. Just tries to add 'X_'.\"\"\"\n if basis in adata.obsm:\n return adata.obsm[basis]\n elif f\"X_{basis}\" in adata.obsm:\n return adata.obsm[f\"X_{basis}\"]\n else:\n raise KeyError(f\"Could not find '{basis}' or 'X_{basis}' in .obsm\")\n\n\ndef _get_color_source_vector(\n adata: AnnData,\n value_to_plot: str,\n *,\n mask_obs: NDArray[np.bool_] | None = None,\n use_raw: bool = False,\n gene_symbols: str | None = None,\n layer: str | None = None,\n groups: Sequence[str] | None = None,\n) -> np.ndarray | pd.api.extensions.ExtensionArray:\n \"\"\"\n Get array from adata that colors will be based on.\n \"\"\"\n if value_to_plot is None:\n # Points will be plotted with `na_color`. Ideally this would work\n # with the \"bad color\" in a color map but that throws a warning. Instead\n # _color_vector handles this.\n # https://github.com/matplotlib/matplotlib/issues/18294\n return np.broadcast_to(np.nan, adata.n_obs)\n if (\n gene_symbols is not None\n and value_to_plot not in adata.obs.columns\n and value_to_plot not in adata.var_names\n ):\n # We should probably just make an index for this, and share it over runs\n # TODO: Throw helpful error if this doesn't work\n value_to_plot = adata.var.index[adata.var[gene_symbols] == value_to_plot][0]\n if use_raw and value_to_plot not in adata.obs.columns:\n values = adata.raw.obs_vector(value_to_plot)\n else:\n values = adata.obs_vector(value_to_plot, layer=layer)\n if mask_obs is not None:\n values[~mask_obs] = np.nan\n if groups and isinstance(values, pd.Categorical):\n values = values.remove_categories(values.categories.difference(groups))\n return values\n\n\ndef _get_palette(adata, values_key: str, palette=None):\n color_key = f\"{values_key}_colors\"\n if adata.obs[values_key].dtype == bool:\n values = pd.Categorical(adata.obs[values_key].astype(str))\n else:\n values = pd.Categorical(adata.obs[values_key])\n if palette:\n _utils._set_colors_for_categorical_obs(adata, values_key, palette)\n elif color_key not in adata.uns or len(adata.uns[color_key]) < len(\n values.categories\n ):\n # set a default palette in case that no colors or few colors are found\n _utils._set_default_colors_for_categorical_obs(adata, values_key)\n else:\n _utils._validate_palette(adata, values_key)\n return dict(zip(values.categories, adata.uns[color_key]))\n\n\ndef _color_vector(\n adata: AnnData,\n values_key: str | None,\n *,\n values: np.ndarray | pd.api.extensions.ExtensionArray,\n palette: str | Sequence[str] | Cycler | None,\n na_color: ColorLike = \"lightgray\",\n) -> tuple[np.ndarray | pd.api.extensions.ExtensionArray, Literal[\"cat\", \"na\", \"cont\"]]:\n \"\"\"\n Map array of values to array of hex (plus alpha) codes.\n\n For categorical data, the return value is list of colors taken\n from the category palette or from the given `palette` value.\n\n For continuous values, the input array is returned (may change in future).\n \"\"\"\n ###\n # when plotting, the color of the dots is determined for each plot\n # the data is either categorical or continuous and the data could be in\n # 'obs' or in 'var'\n to_hex = partial(colors.to_hex, keep_alpha=True)\n if values_key is None:\n return np.broadcast_to(to_hex(na_color), adata.n_obs), \"na\"\n if values.dtype == bool:\n values = pd.Categorical(values.astype(str))\n elif not isinstance(values, pd.Categorical):\n return values, \"cont\"\n\n color_map = {\n k: to_hex(v)\n for k, v in _get_palette(adata, values_key, palette=palette).items()\n }\n # If color_map does not have unique values, this can be slow as the\n # result is not categorical\n if Version(pd.__version__) < Version(\"2.1.0\"):\n color_vector = pd.Categorical(values.map(color_map))\n else:\n color_vector = pd.Categorical(values.map(color_map, na_action=\"ignore\"))\n # Set color to 'missing color' for all missing values\n if color_vector.isna().any():\n color_vector = color_vector.add_categories([to_hex(na_color)])\n color_vector = color_vector.fillna(to_hex(na_color))\n return color_vector, \"cat\"\n\n\ndef _basis2name(basis):\n \"\"\"\n converts the 'basis' into the proper name.\n \"\"\"\n\n component_name = (\n \"DC\"\n if basis == \"diffmap\"\n else \"tSNE\"\n if basis == \"tsne\"\n else \"UMAP\"\n if basis == \"umap\"\n else \"PC\"\n if basis == \"pca\"\n else basis.replace(\"draw_graph_\", \"\").upper()\n if \"draw_graph\" in basis\n else basis\n )\n return component_name\n\n\ndef _check_spot_size(spatial_data: Mapping | None, spot_size: float | None) -> float:\n \"\"\"\n Resolve spot_size value.\n\n This is a required argument for spatial plots.\n \"\"\"\n if spatial_data is None and spot_size is None:\n raise ValueError(\n \"When .uns['spatial'][library_id] does not exist, spot_size must be \"\n \"provided directly.\"\n )\n elif spot_size is None:\n return spatial_data[\"scalefactors\"][\"spot_diameter_fullres\"]\n else:\n return spot_size\n\n\ndef _check_scale_factor(\n spatial_data: Mapping | None,\n img_key: str | None,\n scale_factor: float | None,\n) -> float:\n \"\"\"Resolve scale_factor, defaults to 1.\"\"\"\n if scale_factor is not None:\n return scale_factor\n elif spatial_data is not None and img_key is not None:\n return spatial_data[\"scalefactors\"][f\"tissue_{img_key}_scalef\"]\n else:\n return 1.0\n\n\ndef _check_spatial_data(\n uns: Mapping, library_id: str | None | Empty\n) -> tuple[str | None, Mapping | None]:\n \"\"\"\n Given a mapping, try and extract a library id/ mapping with spatial data.\n\n Assumes this is `.uns` from how we parse visium data.\n \"\"\"\n spatial_mapping = uns.get(\"spatial\", {})\n if library_id is _empty:\n if len(spatial_mapping) > 1:\n raise ValueError(\n \"Found multiple possible libraries in `.uns['spatial']. Please specify.\"\n f\" Options are:\\n\\t{list(spatial_mapping.keys())}\"\n )\n elif len(spatial_mapping) == 1:\n library_id = list(spatial_mapping.keys())[0]\n else:\n library_id = None\n spatial_data = spatial_mapping[library_id] if library_id is not None else None\n return library_id, spatial_data\n\n\ndef _check_img(\n spatial_data: Mapping | None,\n img: np.ndarray | None,\n img_key: None | str | Empty,\n *,\n bw: bool = False,\n) -> tuple[np.ndarray | None, str | None]:\n \"\"\"\n Resolve image for spatial plots.\n \"\"\"\n if img is None and spatial_data is not None and img_key is _empty:\n img_key = next(\n (k for k in [\"hires\", \"lowres\"] if k in spatial_data[\"images\"]),\n ) # Throws StopIteration Error if keys not present\n if img is None and spatial_data is not None and img_key is not None:\n img = spatial_data[\"images\"][img_key]\n if bw:\n img = np.dot(img[..., :3], [0.2989, 0.5870, 0.1140])\n return img, img_key\n\n\ndef _check_crop_coord(\n crop_coord: tuple | None,\n scale_factor: float,\n) -> tuple[float, float, float, float]:\n \"\"\"Handle cropping with image or basis.\"\"\"\n if crop_coord is None:\n return None\n if len(crop_coord) != 4:\n raise ValueError(\"Invalid crop_coord of length {len(crop_coord)}(!=4)\")\n crop_coord = tuple(c * scale_factor for c in crop_coord)\n return crop_coord\n\n\ndef _check_na_color(\n na_color: ColorLike | None, *, img: np.ndarray | None = None\n) -> ColorLike:\n if na_color is None:\n na_color = (0.0, 0.0, 0.0, 0.0) if img is not None else \"lightgray\"\n return na_color\n\n\ndef _broadcast_args(*args):\n \"\"\"Broadcasts arguments to a common length.\"\"\"\n\n lens = [len(arg) for arg in args]\n longest = max(lens)\n if not (set(lens) == {1, longest} or set(lens) == {longest}):\n raise ValueError(f\"Could not broadcast together arguments with shapes: {lens}.\")\n return list(\n [[arg[0] for _ in range(longest)] if len(arg) == 1 else arg for arg in args]\n )\n\n\nfrom __future__ import annotations\n\nfrom collections.abc import Mapping, Sequence\nfrom copy import copy\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom matplotlib import colormaps, rcParams\nfrom matplotlib import pyplot as plt\n\nfrom scanpy.get import obs_df\n\nfrom ... import logging as logg\nfrom ..._compat import old_positionals\nfrom ..._settings import settings\nfrom ..._utils import _doc_params, _empty, sanitize_anndata, subsample\nfrom ...get import rank_genes_groups_df\nfrom .._anndata import ranking\nfrom .._docs import (\n doc_cm_palette,\n doc_panels,\n doc_rank_genes_groups_plot_args,\n doc_rank_genes_groups_values_to_plot,\n doc_scatter_embedding,\n doc_show_save,\n doc_show_save_ax,\n doc_vbound_percentile,\n)\nfrom .._utils import (\n _deprecated_scale,\n savefig_or_show,\n timeseries,\n timeseries_as_heatmap,\n timeseries_subplot,\n)\nfrom .scatterplots import _panel_grid, embedding, pca\n\nif TYPE_CHECKING:\n from collections.abc import Iterable\n from typing import Literal\n\n from anndata import AnnData\n from cycler import Cycler\n from matplotlib.axes import Axes\n from matplotlib.colors import Colormap, Normalize\n from matplotlib.figure import Figure\n\n from ..._utils import Empty\n from .._utils import DensityNorm\n\n# ------------------------------------------------------------------------------\n# PCA\n# ------------------------------------------------------------------------------\n\n\n@_doc_params(scatter_bulk=doc_scatter_embedding, show_save_ax=doc_show_save_ax)\ndef pca_overview(adata: AnnData, **params):\n \"\"\"\\\n Plot PCA results.\n\n The parameters are the ones of the scatter plot. Call pca_ranking separately\n if you want to change the default settings.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n color\n Keys for observation/cell annotation either as list `[\"ann1\", \"ann2\"]` or\n string `\"ann1,ann2,...\"`.\n use_raw\n Use `raw` attribute of `adata` if present.\n {scatter_bulk}\n show\n Show the plot, do not return axis.\n save\n If `True` or a `str`, save the figure.\n A string is appended to the default filename.\n Infer the filetype if ending on {{`'.pdf'`, `'.png'`, `'.svg'`}}.\n Examples\n --------\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc3k_processed()\n sc.pl.pca_overview(adata, color=\"louvain\")\n\n .. currentmodule:: scanpy\n\n See also\n --------\n pp.pca\n \"\"\"\n show = params.pop(\"show\", None)\n pca(adata, **params, show=False)\n pca_loadings(adata, show=False)\n pca_variance_ratio(adata, show=show)\n\n\n# backwards compat\npca_scatter = pca\n\n\n@old_positionals(\"include_lowest\", \"n_points\", \"show\", \"save\")\ndef pca_loadings(\n adata: AnnData,\n components: str | Sequence[int] | None = None,\n *,\n include_lowest: bool = True,\n n_points: int | None = None,\n show: bool | None = None,\n save: str | bool | None = None,\n):\n \"\"\"\\\n Rank genes according to contributions to PCs.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n components\n For example, ``'1,2,3'`` means ``[1, 2, 3]``, first, second, third\n principal component.\n include_lowest\n Whether to show the variables with both highest and lowest loadings.\n show\n Show the plot, do not return axis.\n n_points\n Number of variables to plot for each component.\n save\n If `True` or a `str`, save the figure.\n A string is appended to the default filename.\n Infer the filetype if ending on {`'.pdf'`, `'.png'`, `'.svg'`}.\n\n Examples\n --------\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc3k_processed()\n\n Show first 3 components loadings\n\n .. plot::\n :context: close-figs\n\n sc.pl.pca_loadings(adata, components = '1,2,3')\n\n\n \"\"\"\n if components is None:\n components = [1, 2, 3]\n elif isinstance(components, str):\n components = [int(x) for x in components.split(\",\")]\n components = np.array(components) - 1\n\n if np.any(components < 0):\n raise ValueError(\"Component indices must be greater than zero.\")\n\n if n_points is None:\n n_points = min(30, adata.n_vars)\n elif adata.n_vars < n_points:\n raise ValueError(\n f\"Tried to plot {n_points} variables, but passed anndata only has {adata.n_vars}.\"\n )\n\n ranking(\n adata,\n \"varm\",\n \"PCs\",\n n_points=n_points,\n indices=components,\n include_lowest=include_lowest,\n )\n savefig_or_show(\"pca_loadings\", show=show, save=save)\n\n\n@old_positionals(\"log\", \"show\", \"save\")\ndef pca_variance_ratio(\n adata: AnnData,\n n_pcs: int = 30,\n *,\n log: bool = False,\n show: bool | None = None,\n save: bool | str | None = None,\n):\n \"\"\"\\\n Plot the variance ratio.\n\n Parameters\n ----------\n n_pcs\n Number of PCs to show.\n log\n Plot on logarithmic scale..\n show\n Show the plot, do not return axis.\n save\n If `True` or a `str`, save the figure.\n A string is appended to the default filename.\n Infer the filetype if ending on {`'.pdf'`, `'.png'`, `'.svg'`}.\n \"\"\"\n ranking(\n adata,\n \"uns\",\n \"variance_ratio\",\n n_points=n_pcs,\n dictionary=\"pca\",\n labels=\"PC\",\n log=log,\n )\n savefig_or_show(\"pca_variance_ratio\", show=show, save=save)\n\n\n# ------------------------------------------------------------------------------\n# Subgroup identification and ordering – clustering, pseudotime, branching\n# and tree inference tools\n# ------------------------------------------------------------------------------\n\n\n@old_positionals(\"color_map\", \"show\", \"save\", \"as_heatmap\", \"marker\")\ndef dpt_timeseries(\n adata: AnnData,\n *,\n color_map: str | Colormap | None = None,\n show: bool | None = None,\n save: bool | None = None,\n as_heatmap: bool = True,\n marker: str | Sequence[str] = \".\",\n):\n \"\"\"\\\n Heatmap of pseudotime series.\n\n Parameters\n ----------\n as_heatmap\n Plot the timeseries as heatmap.\n \"\"\"\n if adata.n_vars > 100:\n logg.warning(\n \"Plotting more than 100 genes might take some while, \"\n \"consider selecting only highly variable genes, for example.\"\n )\n # only if number of genes is not too high\n if as_heatmap:\n # plot time series as heatmap, as in Haghverdi et al. (2016), Fig. 1d\n timeseries_as_heatmap(\n adata.X[adata.obs[\"dpt_order_indices\"].values],\n var_names=adata.var_names,\n highlights_x=adata.uns[\"dpt_changepoints\"],\n color_map=color_map,\n )\n else:\n # plot time series as gene expression vs time\n timeseries(\n adata.X[adata.obs[\"dpt_order_indices\"].values],\n var_names=adata.var_names,\n highlights_x=adata.uns[\"dpt_changepoints\"],\n xlim=[0, 1.3 * adata.X.shape[0]],\n marker=marker,\n )\n plt.xlabel(\"dpt order\")\n savefig_or_show(\"dpt_timeseries\", save=save, show=show)\n\n\n@old_positionals(\"color_map\", \"palette\", \"show\", \"save\", \"marker\")\n@_doc_params(cm_palette=doc_cm_palette, show_save=doc_show_save)\ndef dpt_groups_pseudotime(\n adata: AnnData,\n *,\n color_map: str | Colormap | None = None,\n palette: Sequence[str] | Cycler | None = None,\n show: bool | None = None,\n save: bool | str | None = None,\n marker: str | Sequence[str] = \".\",\n):\n \"\"\"\\\n Plot groups and pseudotime.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n {cm_palette}\n {show_save}\n marker\n Marker style. See :mod:`~matplotlib.markers` for details.\n \"\"\"\n _, (ax_grp, ax_ord) = plt.subplots(2, 1)\n timeseries_subplot(\n adata.obs[\"dpt_groups\"].cat.codes,\n time=adata.obs[\"dpt_order\"].values,\n color=np.asarray(adata.obs[\"dpt_groups\"]),\n highlights_x=adata.uns[\"dpt_changepoints\"],\n ylabel=\"dpt groups\",\n yticks=(\n np.arange(len(adata.obs[\"dpt_groups\"].cat.categories), dtype=int)\n if len(adata.obs[\"dpt_groups\"].cat.categories) < 5\n else None\n ),\n palette=palette,\n ax=ax_grp,\n marker=marker,\n )\n timeseries_subplot(\n adata.obs[\"dpt_pseudotime\"].values,\n time=adata.obs[\"dpt_order\"].values,\n color=adata.obs[\"dpt_pseudotime\"].values,\n xlabel=\"dpt order\",\n highlights_x=adata.uns[\"dpt_changepoints\"],\n ylabel=\"pseudotime\",\n yticks=[0, 1],\n color_map=color_map,\n ax=ax_ord,\n marker=marker,\n )\n savefig_or_show(\"dpt_groups_pseudotime\", save=save, show=show)\n\n\n@old_positionals(\n \"n_genes\",\n \"gene_symbols\",\n \"key\",\n \"fontsize\",\n \"ncols\",\n \"sharey\",\n \"show\",\n \"save\",\n \"ax\",\n)\n@_doc_params(show_save_ax=doc_show_save_ax)\ndef rank_genes_groups(\n adata: AnnData,\n groups: str | Sequence[str] | None = None,\n *,\n n_genes: int = 20,\n gene_symbols: str | None = None,\n key: str | None = \"rank_genes_groups\",\n fontsize: int = 8,\n ncols: int = 4,\n sharey: bool = True,\n show: bool | None = None,\n save: bool | None = None,\n ax: Axes | None = None,\n **kwds,\n):\n \"\"\"\\\n Plot ranking of genes.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n groups\n The groups for which to show the gene ranking.\n gene_symbols\n Key for field in `.var` that stores gene symbols if you do not want to\n use `.var_names`.\n n_genes\n Number of genes to show.\n fontsize\n Fontsize for gene names.\n ncols\n Number of panels shown per row.\n sharey\n Controls if the y-axis of each panels should be shared. But passing\n `sharey=False`, each panel has its own y-axis range.\n {show_save_ax}\n\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.pl.rank_genes_groups(adata)\n\n\n Plot top 10 genes (default 20 genes)\n\n .. plot::\n :context: close-figs\n\n sc.pl.rank_genes_groups(adata, n_genes=10)\n\n .. currentmodule:: scanpy\n\n See also\n --------\n tl.rank_genes_groups\n\n \"\"\"\n n_panels_per_row = kwds.get(\"n_panels_per_row\", ncols)\n if n_genes < 1:\n raise NotImplementedError(\n \"Specifying a negative number for n_genes has not been implemented for \"\n f\"this plot. Received n_genes={n_genes}.\"\n )\n\n reference = str(adata.uns[key][\"params\"][\"reference\"])\n group_names = adata.uns[key][\"names\"].dtype.names if groups is None else groups\n # one panel for each group\n # set up the figure\n n_panels_x = min(n_panels_per_row, len(group_names))\n n_panels_y = np.ceil(len(group_names) / n_panels_x).astype(int)\n\n from matplotlib import gridspec\n\n fig = plt.figure(\n figsize=(\n n_panels_x * rcParams[\"figure.figsize\"][0],\n n_panels_y * rcParams[\"figure.figsize\"][1],\n )\n )\n gs = gridspec.GridSpec(nrows=n_panels_y, ncols=n_panels_x, wspace=0.22, hspace=0.3)\n\n ax0 = None\n ymin = np.inf\n ymax = -np.inf\n for count, group_name in enumerate(group_names):\n gene_names = adata.uns[key][\"names\"][group_name][:n_genes]\n scores = adata.uns[key][\"scores\"][group_name][:n_genes]\n\n # Setting up axis, calculating y bounds\n if sharey:\n ymin = min(ymin, np.min(scores))\n ymax = max(ymax, np.max(scores))\n\n if ax0 is None:\n ax = fig.add_subplot(gs[count])\n ax0 = ax\n else:\n ax = fig.add_subplot(gs[count], sharey=ax0)\n else:\n ymin = np.min(scores)\n ymax = np.max(scores)\n ymax += 0.3 * (ymax - ymin)\n\n ax = fig.add_subplot(gs[count])\n ax.set_ylim(ymin, ymax)\n\n ax.set_xlim(-0.9, n_genes - 0.1)\n\n # Mapping to gene_symbols\n if gene_symbols is not None:\n if adata.raw is not None and adata.uns[key][\"params\"][\"use_raw\"]:\n gene_names = adata.raw.var[gene_symbols][gene_names]\n else:\n gene_names = adata.var[gene_symbols][gene_names]\n\n # Making labels\n for ig, gene_name in enumerate(gene_names):\n ax.text(\n ig,\n scores[ig],\n gene_name,\n rotation=\"vertical\",\n verticalalignment=\"bottom\",\n horizontalalignment=\"center\",\n fontsize=fontsize,\n )\n\n ax.set_title(f\"{group_name} vs. {reference}\")\n if count >= n_panels_x * (n_panels_y - 1):\n ax.set_xlabel(\"ranking\")\n\n # print the 'score' label only on the first panel per row.\n if count % n_panels_x == 0:\n ax.set_ylabel(\"score\")\n\n if sharey is True:\n ymax += 0.3 * (ymax - ymin)\n ax.set_ylim(ymin, ymax)\n\n writekey = f\"rank_genes_groups_{adata.uns[key]['params']['groupby']}\"\n savefig_or_show(writekey, show=show, save=save)\n\n\ndef _fig_show_save_or_axes(plot_obj, return_fig, show, save):\n \"\"\"\n Decides what to return\n \"\"\"\n if return_fig:\n return plot_obj\n plot_obj.make_figure()\n savefig_or_show(plot_obj.DEFAULT_SAVE_PREFIX, show=show, save=save)\n show = settings.autoshow if show is None else show\n if show:\n return None\n return plot_obj.get_axes()\n\n\ndef _rank_genes_groups_plot(\n adata: AnnData,\n plot_type: str = \"heatmap\",\n *,\n groups: str | Sequence[str] | None = None,\n n_genes: int | None = None,\n groupby: str | None = None,\n values_to_plot: str | None = None,\n var_names: Sequence[str] | Mapping[str, Sequence[str]] | None = None,\n min_logfoldchange: float | None = None,\n key: str | None = None,\n show: bool | None = None,\n save: bool | None = None,\n return_fig: bool | None = False,\n gene_symbols: str | None = None,\n **kwds,\n):\n \"\"\"\\\n Common function to call the different rank_genes_groups_* plots\n \"\"\"\n if var_names is not None and n_genes is not None:\n raise ValueError(\n \"The arguments n_genes and var_names are mutually exclusive. Please \"\n \"select only one.\"\n )\n\n if var_names is None and n_genes is None:\n # set n_genes = 10 as default when none of the options is given\n n_genes = 10\n\n if key is None:\n key = \"rank_genes_groups\"\n\n if groupby is None:\n groupby = str(adata.uns[key][\"params\"][\"groupby\"])\n group_names = adata.uns[key][\"names\"].dtype.names if groups is None else groups\n\n if var_names is not None:\n if isinstance(var_names, Mapping):\n # get a single list of all gene names in the dictionary\n var_names_list = sum([list(x) for x in var_names.values()], [])\n elif isinstance(var_names, str):\n var_names_list = [var_names]\n else:\n var_names_list = var_names\n else:\n # dict in which each group is the key and the n_genes are the values\n var_names = {}\n var_names_list = []\n for group in group_names:\n df = rank_genes_groups_df(\n adata,\n group,\n key=key,\n gene_symbols=gene_symbols,\n log2fc_min=min_logfoldchange,\n )\n\n if gene_symbols is not None:\n df[\"names\"] = df[gene_symbols]\n\n genes_list = df.names[df.names.notnull()].tolist()\n\n if len(genes_list) == 0:\n logg.warning(f\"No genes found for group {group}\")\n continue\n genes_list = genes_list[n_genes:] if n_genes < 0 else genes_list[:n_genes]\n var_names[group] = genes_list\n var_names_list.extend(genes_list)\n\n # by default add dendrogram to plots\n kwds.setdefault(\"dendrogram\", True)\n\n if plot_type in [\"dotplot\", \"matrixplot\"]:\n # these two types of plots can also\n # show score, logfoldchange and pvalues, in general any value from rank\n # genes groups\n title = None\n values_df = None\n if values_to_plot is not None:\n values_df = _get_values_to_plot(\n adata,\n values_to_plot,\n var_names_list,\n key=key,\n gene_symbols=gene_symbols,\n )\n title = values_to_plot\n if values_to_plot == \"logfoldchanges\":\n title = \"log fold change\"\n else:\n title = values_to_plot.replace(\"_\", \" \").replace(\"pvals\", \"p-value\")\n\n if plot_type == \"dotplot\":\n from .._dotplot import dotplot\n\n _pl = dotplot(\n adata,\n var_names,\n groupby,\n dot_color_df=values_df,\n return_fig=True,\n gene_symbols=gene_symbols,\n **kwds,\n )\n if title is not None and \"colorbar_title\" not in kwds:\n _pl.legend(colorbar_title=title)\n elif plot_type == \"matrixplot\":\n from .._matrixplot import matrixplot\n\n _pl = matrixplot(\n adata,\n var_names,\n groupby,\n values_df=values_df,\n return_fig=True,\n gene_symbols=gene_symbols,\n **kwds,\n )\n\n if title is not None and \"colorbar_title\" not in kwds:\n _pl.legend(title=title)\n\n return _fig_show_save_or_axes(_pl, return_fig, show, save)\n\n elif plot_type == \"stacked_violin\":\n from .._stacked_violin import stacked_violin\n\n _pl = stacked_violin(\n adata,\n var_names,\n groupby,\n return_fig=True,\n gene_symbols=gene_symbols,\n **kwds,\n )\n return _fig_show_save_or_axes(_pl, return_fig, show, save)\n elif plot_type == \"heatmap\":\n from .._anndata import heatmap\n\n return heatmap(\n adata,\n var_names,\n groupby,\n show=show,\n save=save,\n gene_symbols=gene_symbols,\n **kwds,\n )\n\n elif plot_type == \"tracksplot\":\n from .._anndata import tracksplot\n\n return tracksplot(\n adata,\n var_names,\n groupby,\n show=show,\n save=save,\n gene_symbols=gene_symbols,\n **kwds,\n )\n\n\n@old_positionals(\n \"n_genes\",\n \"groupby\",\n \"gene_symbols\",\n \"var_names\",\n \"min_logfoldchange\",\n \"key\",\n \"show\",\n \"save\",\n)\n@_doc_params(params=doc_rank_genes_groups_plot_args, show_save_ax=doc_show_save_ax)\ndef rank_genes_groups_heatmap(\n adata: AnnData,\n groups: str | Sequence[str] | None = None,\n *,\n n_genes: int | None = None,\n groupby: str | None = None,\n gene_symbols: str | None = None,\n var_names: Sequence[str] | Mapping[str, Sequence[str]] | None = None,\n min_logfoldchange: float | None = None,\n key: str | None = None,\n show: bool | None = None,\n save: bool | None = None,\n **kwds,\n):\n \"\"\"\\\n Plot ranking of genes using heatmap plot (see :func:`~scanpy.pl.heatmap`)\n\n Parameters\n ----------\n {params}\n {show_save_ax}\n **kwds\n Are passed to :func:`~scanpy.pl.heatmap`.\n {show_save_ax}\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.tl.rank_genes_groups(adata, 'bulk_labels')\n sc.pl.rank_genes_groups_heatmap(adata)\n\n Show gene names per group on the heatmap\n\n .. plot::\n :context: close-figs\n\n sc.pl.rank_genes_groups_heatmap(adata, show_gene_labels=True)\n\n Plot top 5 genes per group (default 10 genes)\n\n .. plot::\n :context: close-figs\n\n sc.pl.rank_genes_groups_heatmap(adata, n_genes=5, show_gene_labels=True)\n\n .. currentmodule:: scanpy\n\n See also\n --------\n tl.rank_genes_groups\n tl.dendrogram\n \"\"\"\n return _rank_genes_groups_plot(\n adata,\n plot_type=\"heatmap\",\n groups=groups,\n n_genes=n_genes,\n gene_symbols=gene_symbols,\n groupby=groupby,\n var_names=var_names,\n key=key,\n min_logfoldchange=min_logfoldchange,\n show=show,\n save=save,\n **kwds,\n )\n\n\n@old_positionals(\n \"n_genes\",\n \"groupby\",\n \"var_names\",\n \"gene_symbols\",\n \"min_logfoldchange\",\n \"key\",\n \"show\",\n \"save\",\n)\n@_doc_params(params=doc_rank_genes_groups_plot_args, show_save_ax=doc_show_save_ax)\ndef rank_genes_groups_tracksplot(\n adata: AnnData,\n groups: str | Sequence[str] | None = None,\n *,\n n_genes: int | None = None,\n groupby: str | None = None,\n var_names: Sequence[str] | Mapping[str, Sequence[str]] | None = None,\n gene_symbols: str | None = None,\n min_logfoldchange: float | None = None,\n key: str | None = None,\n show: bool | None = None,\n save: bool | None = None,\n **kwds,\n):\n \"\"\"\\\n Plot ranking of genes using heatmap plot (see :func:`~scanpy.pl.heatmap`)\n\n Parameters\n ----------\n {params}\n {show_save_ax}\n **kwds\n Are passed to :func:`~scanpy.pl.tracksplot`.\n {show_save_ax}\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.tl.rank_genes_groups(adata, 'bulk_labels')\n sc.pl.rank_genes_groups_tracksplot(adata)\n \"\"\"\n\n return _rank_genes_groups_plot(\n adata,\n plot_type=\"tracksplot\",\n groups=groups,\n n_genes=n_genes,\n var_names=var_names,\n gene_symbols=gene_symbols,\n groupby=groupby,\n key=key,\n min_logfoldchange=min_logfoldchange,\n show=show,\n save=save,\n **kwds,\n )\n\n\n@old_positionals(\n \"n_genes\",\n \"groupby\",\n \"values_to_plot\",\n \"var_names\",\n \"gene_symbols\",\n \"min_logfoldchange\",\n \"key\",\n \"show\",\n \"save\",\n \"return_fig\",\n)\n@_doc_params(\n params=doc_rank_genes_groups_plot_args,\n vals_to_plot=doc_rank_genes_groups_values_to_plot,\n show_save_ax=doc_show_save_ax,\n)\ndef rank_genes_groups_dotplot(\n adata: AnnData,\n groups: str | Sequence[str] | None = None,\n *,\n n_genes: int | None = None,\n groupby: str | None = None,\n values_to_plot: Literal[\n \"scores\",\n \"logfoldchanges\",\n \"pvals\",\n \"pvals_adj\",\n \"log10_pvals\",\n \"log10_pvals_adj\",\n ]\n | None = None,\n var_names: Sequence[str] | Mapping[str, Sequence[str]] | None = None,\n gene_symbols: str | None = None,\n min_logfoldchange: float | None = None,\n key: str | None = None,\n show: bool | None = None,\n save: bool | None = None,\n return_fig: bool | None = False,\n **kwds,\n):\n \"\"\"\\\n Plot ranking of genes using dotplot plot (see :func:`~scanpy.pl.dotplot`)\n\n Parameters\n ----------\n {params}\n {vals_to_plot}\n {show_save_ax}\n return_fig\n Returns :class:`DotPlot` object. Useful for fine-tuning\n the plot. Takes precedence over `show=False`.\n **kwds\n Are passed to :func:`~scanpy.pl.dotplot`.\n\n Returns\n -------\n If `return_fig` is `True`, returns a :class:`DotPlot` object,\n else if `show` is false, return axes dict\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.tl.rank_genes_groups(adata, 'bulk_labels', n_genes=adata.raw.shape[1])\n\n Plot top 2 genes per group.\n\n .. plot::\n :context: close-figs\n\n sc.pl.rank_genes_groups_dotplot(adata,n_genes=2)\n\n Plot with scaled expressions for easier identification of differences.\n\n .. plot::\n :context: close-figs\n\n sc.pl.rank_genes_groups_dotplot(adata, n_genes=2, standard_scale='var')\n\n Plot `logfoldchanges` instead of gene expression. In this case a diverging colormap\n like `bwr` or `seismic` works better. To center the colormap in zero, the minimum\n and maximum values to plot are set to -4 and 4 respectively.\n Also, only genes with a log fold change of 3 or more are shown.\n\n .. plot::\n :context: close-figs\n\n sc.pl.rank_genes_groups_dotplot(\n adata,\n n_genes=4,\n values_to_plot=\"logfoldchanges\", cmap='bwr',\n vmin=-4,\n vmax=4,\n min_logfoldchange=3,\n colorbar_title='log fold change'\n )\n\n Also, the last genes can be plotted. This can be useful to identify genes\n that are lowly expressed in a group. For this `n_genes=-4` is used\n\n .. plot::\n :context: close-figs\n\n sc.pl.rank_genes_groups_dotplot(\n adata,\n n_genes=-4,\n values_to_plot=\"logfoldchanges\",\n cmap='bwr',\n vmin=-4,\n vmax=4,\n min_logfoldchange=3,\n colorbar_title='log fold change',\n )\n\n A list specific genes can be given to check their log fold change. If a\n dictionary, the dictionary keys will be added as labels in the plot.\n\n .. plot::\n :context: close-figs\n\n var_names = {{'T-cell': ['CD3D', 'CD3E', 'IL32'],\n 'B-cell': ['CD79A', 'CD79B', 'MS4A1'],\n 'myeloid': ['CST3', 'LYZ'] }}\n sc.pl.rank_genes_groups_dotplot(\n adata,\n var_names=var_names,\n values_to_plot=\"logfoldchanges\",\n cmap='bwr',\n vmin=-4,\n vmax=4,\n min_logfoldchange=3,\n colorbar_title='log fold change',\n )\n\n .. currentmodule:: scanpy\n\n See also\n --------\n tl.rank_genes_groups\n \"\"\"\n return _rank_genes_groups_plot(\n adata,\n plot_type=\"dotplot\",\n groups=groups,\n n_genes=n_genes,\n groupby=groupby,\n values_to_plot=values_to_plot,\n var_names=var_names,\n gene_symbols=gene_symbols,\n key=key,\n min_logfoldchange=min_logfoldchange,\n show=show,\n save=save,\n return_fig=return_fig,\n **kwds,\n )\n\n\n@old_positionals(\"n_genes\", \"groupby\", \"gene_symbols\")\n@_doc_params(params=doc_rank_genes_groups_plot_args, show_save_ax=doc_show_save_ax)\ndef rank_genes_groups_stacked_violin(\n adata: AnnData,\n groups: str | Sequence[str] | None = None,\n *,\n n_genes: int | None = None,\n groupby: str | None = None,\n gene_symbols: str | None = None,\n var_names: Sequence[str] | Mapping[str, Sequence[str]] | None = None,\n min_logfoldchange: float | None = None,\n key: str | None = None,\n show: bool | None = None,\n save: bool | None = None,\n return_fig: bool | None = False,\n **kwds,\n):\n \"\"\"\\\n Plot ranking of genes using stacked_violin plot\n (see :func:`~scanpy.pl.stacked_violin`)\n\n Parameters\n ----------\n {params}\n {show_save_ax}\n return_fig\n Returns :class:`StackedViolin` object. Useful for fine-tuning\n the plot. Takes precedence over `show=False`.\n **kwds\n Are passed to :func:`~scanpy.pl.stacked_violin`.\n\n Returns\n -------\n If `return_fig` is `True`, returns a :class:`StackedViolin` object,\n else if `show` is false, return axes dict\n\n Examples\n --------\n >>> import scanpy as sc\n >>> adata = sc.datasets.pbmc68k_reduced()\n >>> sc.tl.rank_genes_groups(adata, 'bulk_labels')\n\n >>> sc.pl.rank_genes_groups_stacked_violin(adata, n_genes=4,\n ... min_logfoldchange=4, figsize=(8,6))\n\n \"\"\"\n\n return _rank_genes_groups_plot(\n adata,\n plot_type=\"stacked_violin\",\n groups=groups,\n n_genes=n_genes,\n gene_symbols=gene_symbols,\n groupby=groupby,\n var_names=var_names,\n key=key,\n min_logfoldchange=min_logfoldchange,\n show=show,\n save=save,\n return_fig=return_fig,\n **kwds,\n )\n\n\n@old_positionals(\n \"n_genes\",\n \"groupby\",\n \"values_to_plot\",\n \"var_names\",\n \"gene_symbols\",\n \"min_logfoldchange\",\n \"key\",\n \"show\",\n \"save\",\n \"return_fig\",\n)\n@_doc_params(\n params=doc_rank_genes_groups_plot_args,\n vals_to_plot=doc_rank_genes_groups_values_to_plot,\n show_save_ax=doc_show_save_ax,\n)\ndef rank_genes_groups_matrixplot(\n adata: AnnData,\n groups: str | Sequence[str] | None = None,\n *,\n n_genes: int | None = None,\n groupby: str | None = None,\n values_to_plot: Literal[\n \"scores\",\n \"logfoldchanges\",\n \"pvals\",\n \"pvals_adj\",\n \"log10_pvals\",\n \"log10_pvals_adj\",\n ]\n | None = None,\n var_names: Sequence[str] | Mapping[str, Sequence[str]] | None = None,\n gene_symbols: str | None = None,\n min_logfoldchange: float | None = None,\n key: str | None = None,\n show: bool | None = None,\n save: bool | None = None,\n return_fig: bool | None = False,\n **kwds,\n):\n \"\"\"\\\n Plot ranking of genes using matrixplot plot (see :func:`~scanpy.pl.matrixplot`)\n\n Parameters\n ----------\n {params}\n {vals_to_plot}\n {show_save_ax}\n return_fig\n Returns :class:`MatrixPlot` object. Useful for fine-tuning\n the plot. Takes precedence over `show=False`.\n **kwds\n Are passed to :func:`~scanpy.pl.matrixplot`.\n\n Returns\n -------\n If `return_fig` is `True`, returns a :class:`MatrixPlot` object,\n else if `show` is false, return axes dict\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.tl.rank_genes_groups(adata, 'bulk_labels', n_genes=adata.raw.shape[1])\n\n Plot `logfoldchanges` instead of gene expression. In this case a diverging colormap\n like `bwr` or `seismic` works better. To center the colormap in zero, the minimum\n and maximum values to plot are set to -4 and 4 respectively.\n Also, only genes with a log fold change of 3 or more are shown.\n\n\n .. plot::\n :context: close-figs\n\n sc.pl.rank_genes_groups_matrixplot(\n adata,\n n_genes=4,\n values_to_plot=\"logfoldchanges\",\n cmap='bwr',\n vmin=-4,\n vmax=4,\n min_logfoldchange=3,\n colorbar_title='log fold change',\n )\n\n Also, the last genes can be plotted. This can be useful to identify genes\n that are lowly expressed in a group. For this `n_genes=-4` is used\n\n .. plot::\n :context: close-figs\n\n sc.pl.rank_genes_groups_matrixplot(\n adata,\n n_genes=-4,\n values_to_plot=\"logfoldchanges\",\n cmap='bwr',\n vmin=-4,\n vmax=4,\n min_logfoldchange=3,\n colorbar_title='log fold change',\n )\n\n A list specific genes can be given to check their log fold change. If a\n dictionary, the dictionary keys will be added as labels in the plot.\n\n .. plot::\n :context: close-figs\n\n var_names = {{\"T-cell\": ['CD3D', 'CD3E', 'IL32'],\n 'B-cell': ['CD79A', 'CD79B', 'MS4A1'],\n 'myeloid': ['CST3', 'LYZ'] }}\n sc.pl.rank_genes_groups_matrixplot(\n adata,\n var_names=var_names,\n values_to_plot=\"logfoldchanges\",\n cmap='bwr',\n vmin=-4,\n vmax=4,\n min_logfoldchange=3,\n colorbar_title='log fold change',\n )\n \"\"\"\n\n return _rank_genes_groups_plot(\n adata,\n plot_type=\"matrixplot\",\n groups=groups,\n n_genes=n_genes,\n groupby=groupby,\n values_to_plot=values_to_plot,\n var_names=var_names,\n gene_symbols=gene_symbols,\n key=key,\n min_logfoldchange=min_logfoldchange,\n show=show,\n save=save,\n return_fig=return_fig,\n **kwds,\n )\n\n\n@old_positionals(\n \"n_genes\",\n \"gene_names\",\n \"gene_symbols\",\n \"use_raw\",\n \"key\",\n \"split\",\n \"density_norm\",\n \"strip\",\n \"jitter\",\n \"size\",\n \"ax\",\n \"show\",\n \"save\",\n)\n@_doc_params(show_save_ax=doc_show_save_ax)\ndef rank_genes_groups_violin(\n adata: AnnData,\n groups: Sequence[str] | None = None,\n *,\n n_genes: int = 20,\n gene_names: Iterable[str] | None = None,\n gene_symbols: str | None = None,\n use_raw: bool | None = None,\n key: str | None = None,\n split: bool = True,\n density_norm: DensityNorm = \"width\",\n strip: bool = True,\n jitter: int | float | bool = True,\n size: int = 1,\n ax: Axes | None = None,\n show: bool | None = None,\n save: bool | None = None,\n # deprecated\n scale: DensityNorm | Empty = _empty,\n):\n \"\"\"\\\n Plot ranking of genes for all tested comparisons.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n groups\n List of group names.\n n_genes\n Number of genes to show. Is ignored if `gene_names` is passed.\n gene_names\n List of genes to plot. Is only useful if interested in a custom gene list,\n which is not the result of :func:`scanpy.tl.rank_genes_groups`.\n gene_symbols\n Key for field in `.var` that stores gene symbols if you do not want to\n use `.var_names` displayed in the plot.\n use_raw\n Use `raw` attribute of `adata` if present. Defaults to the value that\n was used in :func:`~scanpy.tl.rank_genes_groups`.\n split\n Whether to split the violins or not.\n density_norm\n See :func:`~seaborn.violinplot`.\n strip\n Show a strip plot on top of the violin plot.\n jitter\n If set to 0, no points are drawn. See :func:`~seaborn.stripplot`.\n size\n Size of the jitter points.\n {show_save_ax}\n \"\"\"\n if key is None:\n key = \"rank_genes_groups\"\n groups_key = str(adata.uns[key][\"params\"][\"groupby\"])\n if use_raw is None:\n use_raw = bool(adata.uns[key][\"params\"][\"use_raw\"])\n reference = str(adata.uns[key][\"params\"][\"reference\"])\n groups_names = adata.uns[key][\"names\"].dtype.names if groups is None else groups\n if isinstance(groups_names, str):\n groups_names = [groups_names]\n density_norm = _deprecated_scale(density_norm, scale, default=\"width\")\n del scale\n axs = []\n for group_name in groups_names:\n if gene_names is None:\n _gene_names = adata.uns[key][\"names\"][group_name][:n_genes]\n else:\n _gene_names = gene_names\n if isinstance(_gene_names, np.ndarray):\n _gene_names = _gene_names.tolist()\n df = obs_df(adata, _gene_names, use_raw=use_raw, gene_symbols=gene_symbols)\n new_gene_names = df.columns\n df[\"hue\"] = adata.obs[groups_key].astype(str).values\n if reference == \"rest\":\n df.loc[df[\"hue\"] != group_name, \"hue\"] = \"rest\"\n else:\n df.loc[~df[\"hue\"].isin([group_name, reference]), \"hue\"] = np.nan\n df[\"hue\"] = df[\"hue\"].astype(\"category\")\n df_tidy = pd.melt(df, id_vars=\"hue\", value_vars=new_gene_names)\n x = \"variable\"\n y = \"value\"\n hue_order = [group_name, reference]\n import seaborn as sns\n\n _ax = sns.violinplot(\n x=x,\n y=y,\n data=df_tidy,\n inner=None,\n hue_order=hue_order,\n hue=\"hue\",\n split=split,\n density_norm=density_norm,\n orient=\"vertical\",\n ax=ax,\n )\n if strip:\n _ax = sns.stripplot(\n x=x,\n y=y,\n data=df_tidy,\n hue=\"hue\",\n dodge=True,\n hue_order=hue_order,\n jitter=jitter,\n palette=\"dark:black\",\n size=size,\n ax=_ax,\n )\n _ax.set_xlabel(\"genes\")\n _ax.set_title(f\"{group_name} vs. {reference}\")\n _ax.legend_.remove()\n _ax.set_ylabel(\"expression\")\n _ax.set_xticklabels(new_gene_names, rotation=\"vertical\")\n writekey = (\n f\"rank_genes_groups_\"\n f\"{adata.uns[key]['params']['groupby']}_\"\n f\"{group_name}\"\n )\n savefig_or_show(writekey, show=show, save=save)\n axs.append(_ax)\n show = settings.autoshow if show is None else show\n if show:\n return None\n return axs\n\n\n@old_positionals(\"tmax_realization\", \"as_heatmap\", \"shuffle\", \"show\", \"save\", \"marker\")\ndef sim(\n adata: AnnData,\n *,\n tmax_realization: int | None = None,\n as_heatmap: bool = False,\n shuffle: bool = False,\n show: bool | None = None,\n save: bool | str | None = None,\n marker: str | Sequence[str] = \".\",\n) -> None:\n \"\"\"\\\n Plot results of simulation.\n\n Parameters\n ----------\n tmax_realization\n Number of observations in one realization of the time series. The data matrix\n adata.X consists in concatenated realizations.\n as_heatmap\n Plot the timeseries as heatmap.\n shuffle\n Shuffle the data.\n show\n Show the plot, do not return axis.\n save\n If `True` or a `str`, save the figure.\n A string is appended to the default filename.\n Infer the filetype if ending on {{`'.pdf'`, `'.png'`, `'.svg'`}}.\n \"\"\"\n if tmax_realization is not None:\n tmax = tmax_realization\n elif \"tmax_write\" in adata.uns:\n tmax = adata.uns[\"tmax_write\"]\n else:\n tmax = adata.n_obs\n n_realizations = adata.n_obs / tmax\n if not shuffle:\n if not as_heatmap:\n timeseries(\n adata.X,\n var_names=adata.var_names,\n xlim=[0, 1.25 * adata.n_obs],\n highlights_x=np.arange(tmax, n_realizations * tmax, tmax),\n xlabel=\"realizations\",\n marker=marker,\n )\n else:\n # plot time series as heatmap, as in Haghverdi et al. (2016), Fig. 1d\n timeseries_as_heatmap(\n adata.X,\n var_names=adata.var_names,\n highlights_x=np.arange(tmax, n_realizations * tmax, tmax),\n )\n plt.xticks(\n np.arange(0, n_realizations * tmax, tmax),\n np.arange(n_realizations).astype(int) + 1,\n )\n savefig_or_show(\"sim\", save=save, show=show)\n else:\n # shuffled data\n X = adata.X\n X, rows = subsample(X, seed=1)\n timeseries(\n X,\n var_names=adata.var_names,\n xlim=[0, 1.25 * adata.n_obs],\n highlights_x=np.arange(tmax, n_realizations * tmax, tmax),\n xlabel=\"index (arbitrary order)\",\n marker=marker,\n )\n savefig_or_show(\"sim_shuffled\", save=save, show=show)\n\n\n@old_positionals(\n \"key\",\n \"groupby\",\n \"group\",\n \"color_map\",\n \"bg_dotsize\",\n \"fg_dotsize\",\n \"vmax\",\n \"vmin\",\n \"vcenter\",\n \"norm\",\n \"ncols\",\n \"hspace\",\n \"wspace\",\n \"title\",\n \"show\",\n \"save\",\n \"ax\",\n \"return_fig\",\n)\n@_doc_params(\n vminmax=doc_vbound_percentile, panels=doc_panels, show_save_ax=doc_show_save_ax\n)\ndef embedding_density(\n adata: AnnData,\n basis: str = \"umap\",\n *,\n key: str | None = None,\n groupby: str | None = None,\n group: str | Sequence[str] | None | None = \"all\",\n color_map: Colormap | str = \"YlOrRd\",\n bg_dotsize: int | None = 80,\n fg_dotsize: int | None = 180,\n vmax: int | None = 1,\n vmin: int | None = 0,\n vcenter: int | None = None,\n norm: Normalize | None = None,\n ncols: int | None = 4,\n hspace: float | None = 0.25,\n wspace: None = None,\n title: str | None = None,\n show: bool | None = None,\n save: bool | str | None = None,\n ax: Axes | None = None,\n return_fig: bool | None = None,\n **kwargs,\n) -> Figure | Axes | None:\n \"\"\"\\\n Plot the density of cells in an embedding (per condition).\n\n Plots the gaussian kernel density estimates (over condition) from the\n `sc.tl.embedding_density()` output.\n\n This function was written by Sophie Tritschler and implemented into\n Scanpy by Malte Luecken.\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n basis\n The embedding over which the density was calculated. This embedded\n representation should be found in `adata.obsm['X_[basis]']``.\n key\n Name of the `.obs` covariate that contains the density estimates. Alternatively, pass `groupby`.\n groupby\n Name of the condition used in `tl.embedding_density`. Alternatively, pass `key`.\n group\n The category in the categorical observation annotation to be plotted.\n For example, 'G1' in the cell cycle 'phase' covariate. If all categories\n are to be plotted use group='all' (default), If multiple categories\n want to be plotted use a list (e.g.: ['G1', 'S']. If the overall density\n wants to be ploted set group to 'None'.\n color_map\n Matplolib color map to use for density plotting.\n bg_dotsize\n Dot size for background data points not in the `group`.\n fg_dotsize\n Dot size for foreground data points in the `group`.\n {vminmax}\n {panels}\n {show_save_ax}\n\n Examples\n --------\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n adata = sc.datasets.pbmc68k_reduced()\n sc.tl.umap(adata)\n sc.tl.embedding_density(adata, basis='umap', groupby='phase')\n\n Plot all categories be default\n\n .. plot::\n :context: close-figs\n\n sc.pl.embedding_density(adata, basis='umap', key='umap_density_phase')\n\n Plot selected categories\n\n .. plot::\n :context: close-figs\n\n sc.pl.embedding_density(\n adata,\n basis='umap',\n key='umap_density_phase',\n group=['G1', 'S'],\n )\n\n .. currentmodule:: scanpy\n\n See also\n --------\n tl.embedding_density\n \"\"\"\n sanitize_anndata(adata)\n\n # Test user inputs\n basis = basis.lower()\n\n if basis == \"fa\":\n basis = \"draw_graph_fa\"\n\n if key is not None and groupby is not None:\n raise ValueError(\"either pass key or groupby but not both\")\n\n if key is None:\n key = \"umap_density\"\n if groupby is not None:\n key += f\"_{groupby}\"\n\n if f\"X_{basis}\" not in adata.obsm_keys():\n raise ValueError(\n f\"Cannot find the embedded representation `adata.obsm[X_{basis!r}]`. \"\n \"Compute the embedding first.\"\n )\n\n if key not in adata.obs or f\"{key}_params\" not in adata.uns:\n raise ValueError(\n \"Please run `sc.tl.embedding_density()` first \"\n \"and specify the correct key.\"\n )\n\n if \"components\" in kwargs:\n logg.warning(\n \"Components were specified, but will be ignored. Only the \"\n \"components used to calculate the density can be plotted.\"\n )\n del kwargs[\"components\"]\n\n components = adata.uns[f\"{key}_params\"][\"components\"]\n groupby = adata.uns[f\"{key}_params\"][\"covariate\"]\n\n # turn group into a list if needed\n if group == \"all\":\n group = None if groupby is None else list(adata.obs[groupby].cat.categories)\n elif isinstance(group, str):\n group = [group]\n\n if group is None and groupby is not None:\n raise ValueError(\n \"Densities were calculated over an `.obs` covariate. \"\n \"Please specify a group from this covariate to plot.\"\n )\n\n if group is not None and groupby is None:\n logg.warning(\n \"value of 'group' is ignored because densities \"\n \"were not calculated for an `.obs` covariate.\"\n )\n group = None\n\n if np.min(adata.obs[key]) < 0 or np.max(adata.obs[key]) > 1:\n raise ValueError(\"Densities should be scaled between 0 and 1.\")\n\n if wspace is None:\n # try to set a wspace that is not too large or too small given the\n # current figure size\n wspace = 0.75 / rcParams[\"figure.figsize\"][0] + 0.02\n\n # Make the color map\n if isinstance(color_map, str):\n color_map = copy(colormaps.get_cmap(color_map))\n\n color_map.set_over(\"black\")\n color_map.set_under(\"lightgray\")\n # a name to store the density values is needed. To avoid\n # overwriting a user name a new random name is created\n while True:\n col_id = np.random.randint(1000, 10000)\n density_col_name = f\"_tmp_embedding_density_column_{col_id}_\"\n if density_col_name not in adata.obs.columns:\n break\n\n # if group is set, then plot it using multiple panels\n # (even if only one group is set)\n if group is not None and not isinstance(group, str) and isinstance(group, Sequence):\n if ax is not None:\n raise ValueError(\"Can only specify `ax` if no `group` sequence is given.\")\n fig, gs = _panel_grid(hspace, wspace, ncols, len(group))\n\n axs = []\n for count, group_name in enumerate(group):\n if group_name not in adata.obs[groupby].cat.categories:\n raise ValueError(\n \"Please specify a group from the `.obs` category \"\n \"over which the density was calculated. \"\n f\"Invalid group name: {group_name}\"\n )\n\n ax = plt.subplot(gs[count])\n # Define plotting data\n dot_sizes = np.ones(adata.n_obs) * bg_dotsize\n group_mask = adata.obs[groupby] == group_name\n dens_values = -np.ones(adata.n_obs)\n dens_values[group_mask] = adata.obs[key][group_mask]\n adata.obs[density_col_name] = dens_values\n dot_sizes[group_mask] = np.ones(sum(group_mask)) * fg_dotsize\n\n _title = group_name if title is None else title\n\n ax = embedding(\n adata,\n basis,\n dimensions=np.array(components) - 1, # Saved with 1 based indexing\n color=density_col_name,\n color_map=color_map,\n size=dot_sizes,\n vmax=vmax,\n vmin=vmin,\n vcenter=vcenter,\n norm=norm,\n save=False,\n title=_title,\n ax=ax,\n show=False,\n **kwargs,\n )\n axs.append(ax)\n\n ax = axs\n else:\n dens_values = adata.obs[key]\n dot_sizes = np.ones(adata.n_obs) * fg_dotsize\n\n adata.obs[density_col_name] = dens_values\n\n # Ensure title is blank as default\n if title is None:\n title = group if group is not None else \"\"\n\n # Plot the graph\n fig_or_ax = embedding(\n adata,\n basis,\n dimensions=np.array(components) - 1, # Saved with 1 based indexing\n color=density_col_name,\n color_map=color_map,\n size=dot_sizes,\n vmax=vmax,\n vmin=vmin,\n vcenter=vcenter,\n norm=norm,\n save=False,\n show=False,\n title=title,\n ax=ax,\n return_fig=return_fig,\n **kwargs,\n )\n if return_fig:\n fig = fig_or_ax\n else:\n ax = fig_or_ax\n\n # remove temporary column name\n adata.obs = adata.obs.drop(columns=[density_col_name])\n\n if return_fig:\n return fig\n savefig_or_show(f\"{key}_\", show=show, save=save)\n show = settings.autoshow if show is None else show\n if show:\n return None\n return ax\n\n\ndef _get_values_to_plot(\n adata,\n values_to_plot: Literal[\n \"scores\",\n \"logfoldchanges\",\n \"pvals\",\n \"pvals_adj\",\n \"log10_pvals\",\n \"log10_pvals_adj\",\n ],\n gene_names: Sequence[str],\n *,\n groups: Sequence[str] | None = None,\n key: str | None = \"rank_genes_groups\",\n gene_symbols: str | None = None,\n):\n \"\"\"\n If rank_genes_groups has been called, this function\n prepares a dataframe containing scores, pvalues, logfoldchange etc to be plotted\n as dotplot or matrixplot.\n\n The dataframe index are the given groups and the columns are the gene_names\n\n used by rank_genes_groups_dotplot\n\n Parameters\n ----------\n adata\n values_to_plot\n name of the value to plot\n gene_names\n gene names\n groups\n groupby categories\n key\n adata.uns key where the rank_genes_groups is stored.\n By default 'rank_genes_groups'\n gene_symbols\n Key for field in .var that stores gene symbols.\n Returns\n -------\n pandas DataFrame index=groups, columns=gene_names\n\n \"\"\"\n valid_options = [\n \"scores\",\n \"logfoldchanges\",\n \"pvals\",\n \"pvals_adj\",\n \"log10_pvals\",\n \"log10_pvals_adj\",\n ]\n if values_to_plot not in valid_options:\n raise ValueError(\n f\"given value_to_plot: '{values_to_plot}' is not valid. Valid options are {valid_options}\"\n )\n\n values_df = None\n check_done = False\n if groups is None:\n groups = adata.uns[key][\"names\"].dtype.names\n if values_to_plot is not None:\n df_list = []\n for group in groups:\n df = rank_genes_groups_df(adata, group, key=key, gene_symbols=gene_symbols)\n if gene_symbols is not None:\n df[\"names\"] = df[gene_symbols]\n # check that all genes are present in the df as sc.tl.rank_genes_groups\n # can be called with only top genes\n if not check_done and df.shape[0] < adata.shape[1]:\n message = (\n \"Please run `sc.tl.rank_genes_groups` with \"\n \"'n_genes=adata.shape[1]' to save all gene \"\n f\"scores. Currently, only {df.shape[0]} \"\n \"are found\"\n )\n logg.error(message)\n raise ValueError(message)\n df[\"group\"] = group\n df_list.append(df)\n\n values_df = pd.concat(df_list)\n if values_to_plot.startswith(\"log10\"):\n column = values_to_plot.replace(\"log10_\", \"\")\n else:\n column = values_to_plot\n values_df = pd.pivot(\n values_df, index=\"names\", columns=\"group\", values=column\n ).fillna(1)\n\n if values_to_plot in [\"log10_pvals\", \"log10_pvals_adj\"]:\n values_df = -1 * np.log10(values_df)\n\n values_df = values_df.loc[gene_names].T\n\n return values_df\n\n\n\"\"\"Shared docstrings for preprocessing function parameters.\"\"\"\n\nfrom __future__ import annotations\n\ndoc_adata_basic = \"\"\"\\\nadata\n Annotated data matrix.\\\n\"\"\"\n\ndoc_expr_reps = \"\"\"\\\nlayer\n If provided, use `adata.layers[layer]` for expression values instead\n of `adata.X`.\nuse_raw\n If True, use `adata.raw.X` for expression values instead of `adata.X`.\\\n\"\"\"\n\ndoc_mask_var_hvg = \"\"\"\\\nmask_var\n To run only on a certain set of genes given by a boolean array\n or a string referring to an array in :attr:`~anndata.AnnData.var`.\n By default, uses `.var['highly_variable']` if available, else everything.\nuse_highly_variable\n Whether to use highly variable genes only, stored in\n `.var['highly_variable']`.\n By default uses them if they have been determined beforehand.\n\n .. deprecated:: 1.10.0\n Use `mask_var` instead\n\"\"\"\n\ndoc_obs_qc_args = \"\"\"\\\nqc_vars\n Keys for boolean columns of `.var` which identify variables you could\n want to control for (e.g. \"ERCC\" or \"mito\").\npercent_top\n List of ranks (where genes are ranked by expression) at which the cumulative\n proportion of expression will be reported as a percentage. This can be used to\n assess library complexity. Ranks are considered 1-indexed, and if empty or None\n don't calculate.\n\n E.g. `percent_top=[50]` finds cumulative proportion to the 50th most expressed gene.\n\"\"\"\n\ndoc_qc_metric_naming = \"\"\"\\\nexpr_type\n Name of kind of values in X.\nvar_type\n The kind of thing the variables are.\\\n\"\"\"\n\ndoc_obs_qc_returns = \"\"\"\\\nObservation level metrics include:\n\n`total_{var_type}_by_{expr_type}`\n E.g. \"total_genes_by_counts\". Number of genes with positive counts in a cell.\n`total_{expr_type}`\n E.g. \"total_counts\". Total number of counts for a cell.\n`pct_{expr_type}_in_top_{n}_{var_type}` – for `n` in `percent_top`\n E.g. \"pct_counts_in_top_50_genes\". Cumulative percentage of counts\n for 50 most expressed genes in a cell.\n`total_{expr_type}_{qc_var}` – for `qc_var` in `qc_vars`\n E.g. \"total_counts_mito\". Total number of counts for variables in\n `qc_vars`.\n`pct_{expr_type}_{qc_var}` – for `qc_var` in `qc_vars`\n E.g. \"pct_counts_mito\". Proportion of total counts for a cell which\n are mitochondrial.\\\n\"\"\"\n\ndoc_var_qc_returns = \"\"\"\\\nVariable level metrics include:\n\n`total_{expr_type}`\n E.g. \"total_counts\". Sum of counts for a gene.\n`n_genes_by_{expr_type}`\n E.g. \"n_genes_by_counts\". The number of genes with at least 1 count in a cell. Calculated for all cells.\n`mean_{expr_type}`\n E.g. \"mean_counts\". Mean expression over all cells.\n`n_cells_by_{expr_type}`\n E.g. \"n_cells_by_counts\". Number of cells this expression is\n measured in.\n`pct_dropout_by_{expr_type}`\n E.g. \"pct_dropout_by_counts\". Percentage of cells this feature does\n not appear in.\\\n\"\"\"\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom dataclasses import dataclass\nfrom inspect import signature\nfrom typing import TYPE_CHECKING, cast\n\nimport numba\nimport numpy as np\nimport pandas as pd\nimport scipy.sparse as sp_sparse\nfrom anndata import AnnData\n\nfrom .. import logging as logg\nfrom .._compat import DaskArray, old_positionals\nfrom .._settings import Verbosity, settings\nfrom .._utils import check_nonnegative_integers, sanitize_anndata\nfrom ..get import _get_obs_rep\nfrom ._distributed import materialize_as_ndarray\nfrom ._simple import filter_genes\nfrom ._utils import _get_mean_var\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from numpy.typing import NDArray\n\n\ndef _highly_variable_genes_seurat_v3(\n adata: AnnData,\n *,\n flavor: str = \"seurat_v3\",\n layer: str | None = None,\n n_top_genes: int = 2000,\n batch_key: str | None = None,\n check_values: bool = True,\n span: float = 0.3,\n subset: bool = False,\n inplace: bool = True,\n) -> pd.DataFrame | None:\n \"\"\"\\\n See `highly_variable_genes`.\n\n For further implementation details see https://www.overleaf.com/read/ckptrbgzzzpg\n\n Returns\n -------\n Depending on `inplace` returns calculated metrics (:class:`~pd.DataFrame`) or\n updates `.var` with the following fields:\n\n highly_variable : :class:`bool`\n boolean indicator of highly-variable genes.\n **means**\n means per gene.\n **variances**\n variance per gene.\n **variances_norm**\n normalized variance per gene, averaged in the case of multiple batches.\n highly_variable_rank : :class:`float`\n Rank of the gene according to normalized variance, median rank in the case of multiple batches.\n highly_variable_nbatches : :class:`int`\n If batch_key is given, this denotes in how many batches genes are detected as HVG.\n \"\"\"\n\n try:\n from skmisc.loess import loess\n except ImportError:\n raise ImportError(\n \"Please install skmisc package via `pip install --user scikit-misc\"\n )\n df = pd.DataFrame(index=adata.var_names)\n data = _get_obs_rep(adata, layer=layer)\n\n if check_values and not check_nonnegative_integers(data):\n warnings.warn(\n f\"`flavor='{flavor}'` expects raw count data, but non-integers were found.\",\n UserWarning,\n )\n\n df[\"means\"], df[\"variances\"] = _get_mean_var(data)\n\n if batch_key is None:\n batch_info = pd.Categorical(np.zeros(adata.shape[0], dtype=int))\n else:\n batch_info = adata.obs[batch_key].to_numpy()\n\n norm_gene_vars = []\n for b in np.unique(batch_info):\n data_batch = data[batch_info == b]\n\n mean, var = _get_mean_var(data_batch)\n not_const = var > 0\n estimat_var = np.zeros(data.shape[1], dtype=np.float64)\n\n y = np.log10(var[not_const])\n x = np.log10(mean[not_const])\n model = loess(x, y, span=span, degree=2)\n model.fit()\n estimat_var[not_const] = model.outputs.fitted_values\n reg_std = np.sqrt(10**estimat_var)\n\n # clip large values as in Seurat\n N = data_batch.shape[0]\n vmax = np.sqrt(N)\n clip_val = reg_std * vmax + mean\n if sp_sparse.issparse(data_batch):\n if sp_sparse.isspmatrix_csr(data_batch):\n batch_counts = data_batch\n else:\n batch_counts = sp_sparse.csr_matrix(data_batch)\n\n squared_batch_counts_sum, batch_counts_sum = _sum_and_sum_squares_clipped(\n batch_counts.indices,\n batch_counts.data,\n n_cols=batch_counts.shape[1],\n clip_val=clip_val,\n nnz=batch_counts.nnz,\n )\n else:\n batch_counts = data_batch.astype(np.float64).copy()\n clip_val_broad = np.broadcast_to(clip_val, batch_counts.shape)\n np.putmask(\n batch_counts,\n batch_counts > clip_val_broad,\n clip_val_broad,\n )\n\n squared_batch_counts_sum = np.square(batch_counts).sum(axis=0)\n batch_counts_sum = batch_counts.sum(axis=0)\n\n norm_gene_var = (1 / ((N - 1) * np.square(reg_std))) * (\n (N * np.square(mean))\n + squared_batch_counts_sum\n - 2 * batch_counts_sum * mean\n )\n norm_gene_vars.append(norm_gene_var.reshape(1, -1))\n\n norm_gene_vars = np.concatenate(norm_gene_vars, axis=0)\n # argsort twice gives ranks, small rank means most variable\n ranked_norm_gene_vars = np.argsort(np.argsort(-norm_gene_vars, axis=1), axis=1)\n\n # this is done in SelectIntegrationFeatures() in Seurat v3\n ranked_norm_gene_vars = ranked_norm_gene_vars.astype(np.float32)\n num_batches_high_var = np.sum(\n (ranked_norm_gene_vars < n_top_genes).astype(int), axis=0\n )\n ranked_norm_gene_vars[ranked_norm_gene_vars >= n_top_genes] = np.nan\n ma_ranked = np.ma.masked_invalid(ranked_norm_gene_vars)\n median_ranked = np.ma.median(ma_ranked, axis=0).filled(np.nan)\n\n df[\"gene_name\"] = df.index\n df[\"highly_variable_nbatches\"] = num_batches_high_var\n df[\"highly_variable_rank\"] = median_ranked\n df[\"variances_norm\"] = np.mean(norm_gene_vars, axis=0)\n if flavor == \"seurat_v3\":\n sort_cols = [\"highly_variable_rank\", \"highly_variable_nbatches\"]\n sort_ascending = [True, False]\n elif flavor == \"seurat_v3_paper\":\n sort_cols = [\"highly_variable_nbatches\", \"highly_variable_rank\"]\n sort_ascending = [False, True]\n else:\n raise ValueError(f\"Did not recognize flavor {flavor}\")\n sorted_index = (\n df[sort_cols]\n .sort_values(sort_cols, ascending=sort_ascending, na_position=\"last\")\n .index\n )\n df[\"highly_variable\"] = False\n df.loc[sorted_index[: int(n_top_genes)], \"highly_variable\"] = True\n\n if inplace:\n adata.uns[\"hvg\"] = {\"flavor\": flavor}\n logg.hint(\n \"added\\n\"\n \" 'highly_variable', boolean vector (adata.var)\\n\"\n \" 'highly_variable_rank', float vector (adata.var)\\n\"\n \" 'means', float vector (adata.var)\\n\"\n \" 'variances', float vector (adata.var)\\n\"\n \" 'variances_norm', float vector (adata.var)\"\n )\n adata.var[\"highly_variable\"] = df[\"highly_variable\"].to_numpy()\n adata.var[\"highly_variable_rank\"] = df[\"highly_variable_rank\"].to_numpy()\n adata.var[\"means\"] = df[\"means\"].to_numpy()\n adata.var[\"variances\"] = df[\"variances\"].to_numpy()\n adata.var[\"variances_norm\"] = (\n df[\"variances_norm\"].to_numpy().astype(\"float64\", copy=False)\n )\n if batch_key is not None:\n adata.var[\"highly_variable_nbatches\"] = df[\n \"highly_variable_nbatches\"\n ].to_numpy()\n if subset:\n adata._inplace_subset_var(df[\"highly_variable\"].to_numpy())\n else:\n if batch_key is None:\n df = df.drop([\"highly_variable_nbatches\"], axis=1)\n if subset:\n df = df.iloc[df[\"highly_variable\"].to_numpy(), :]\n\n return df\n\n\n@numba.njit(cache=True)\ndef _sum_and_sum_squares_clipped(\n indices: NDArray[np.integer],\n data: NDArray[np.floating],\n *,\n n_cols: int,\n clip_val: NDArray[np.float64],\n nnz: int,\n) -> tuple[NDArray[np.float64], NDArray[np.float64]]:\n squared_batch_counts_sum = np.zeros(n_cols, dtype=np.float64)\n batch_counts_sum = np.zeros(n_cols, dtype=np.float64)\n for i in range(nnz):\n idx = indices[i]\n element = min(np.float64(data[i]), clip_val[idx])\n squared_batch_counts_sum[idx] += element**2\n batch_counts_sum[idx] += element\n\n return squared_batch_counts_sum, batch_counts_sum\n\n\n@dataclass\nclass _Cutoffs:\n min_disp: float\n max_disp: float\n min_mean: float\n max_mean: float\n\n @classmethod\n def validate(\n cls,\n *,\n n_top_genes: int | None,\n min_disp: float,\n max_disp: float,\n min_mean: float,\n max_mean: float,\n ) -> _Cutoffs | int:\n if n_top_genes is None:\n return cls(min_disp, max_disp, min_mean, max_mean)\n\n cutoffs = {\"min_disp\", \"max_disp\", \"min_mean\", \"max_mean\"}\n defaults = {\n p.name: p.default\n for p in signature(highly_variable_genes).parameters.values()\n if p.name in cutoffs\n }\n if {k: v for k, v in locals().items() if k in cutoffs} != defaults:\n msg = \"If you pass `n_top_genes`, all cutoffs are ignored.\"\n warnings.warn(msg, UserWarning)\n return n_top_genes\n\n def in_bounds(\n self,\n mean: NDArray[np.floating] | DaskArray,\n dispersion_norm: NDArray[np.floating] | DaskArray,\n ) -> NDArray[np.bool_] | DaskArray:\n return (\n (mean > self.min_mean)\n & (mean < self.max_mean)\n & (dispersion_norm > self.min_disp)\n & (dispersion_norm < self.max_disp)\n )\n\n\ndef _highly_variable_genes_single_batch(\n adata: AnnData,\n *,\n layer: str | None = None,\n cutoff: _Cutoffs | int,\n n_bins: int = 20,\n flavor: Literal[\"seurat\", \"cell_ranger\"] = \"seurat\",\n) -> pd.DataFrame:\n \"\"\"\\\n See `highly_variable_genes`.\n\n Returns\n -------\n A DataFrame that contains the columns\n `highly_variable`, `means`, `dispersions`, and `dispersions_norm`.\n \"\"\"\n X = _get_obs_rep(adata, layer=layer)\n\n if hasattr(X, \"_view_args\"): # AnnData array view\n # For compatibility with anndata<0.9\n X = X.copy() # Doesn't actually copy memory, just removes View class wrapper\n\n if flavor == \"seurat\":\n X = X.copy()\n if (base := adata.uns.get(\"log1p\", {}).get(\"base\")) is not None:\n X *= np.log(base)\n # use out if possible. only possible since we copy the data matrix\n if isinstance(X, np.ndarray):\n np.expm1(X, out=X)\n else:\n X = np.expm1(X)\n\n mean, var = materialize_as_ndarray(_get_mean_var(X))\n # now actually compute the dispersion\n mean[mean == 0] = 1e-12 # set entries equal to zero to small value\n dispersion = var / mean\n if flavor == \"seurat\": # logarithmized mean as in Seurat\n dispersion[dispersion == 0] = np.nan\n dispersion = np.log(dispersion)\n mean = np.log1p(mean)\n\n # all of the following quantities are \"per-gene\" here\n df = pd.DataFrame(dict(zip([\"means\", \"dispersions\"], (mean, dispersion))))\n df[\"mean_bin\"] = _get_mean_bins(df[\"means\"], flavor, n_bins)\n disp_stats = _get_disp_stats(df, flavor)\n\n # actually do the normalization\n df[\"dispersions_norm\"] = (df[\"dispersions\"] - disp_stats[\"avg\"]) / disp_stats[\"dev\"]\n df[\"highly_variable\"] = _subset_genes(\n adata,\n mean=mean,\n dispersion_norm=df[\"dispersions_norm\"].to_numpy(),\n cutoff=cutoff,\n )\n\n df.index = adata.var_names\n return df\n\n\ndef _get_mean_bins(\n means: pd.Series, flavor: Literal[\"seurat\", \"cell_ranger\"], n_bins: int\n) -> pd.Series:\n if flavor == \"seurat\":\n bins = n_bins\n elif flavor == \"cell_ranger\":\n bins = np.r_[-np.inf, np.percentile(means, np.arange(10, 105, 5)), np.inf]\n else:\n raise ValueError('`flavor` needs to be \"seurat\" or \"cell_ranger\"')\n\n return pd.cut(means, bins=bins)\n\n\ndef _get_disp_stats(\n df: pd.DataFrame, flavor: Literal[\"seurat\", \"cell_ranger\"]\n) -> pd.DataFrame:\n disp_grouped = df.groupby(\"mean_bin\", observed=True)[\"dispersions\"]\n if flavor == \"seurat\":\n disp_bin_stats = disp_grouped.agg(avg=\"mean\", dev=\"std\")\n _postprocess_dispersions_seurat(disp_bin_stats, df[\"mean_bin\"])\n elif flavor == \"cell_ranger\":\n disp_bin_stats = disp_grouped.agg(avg=\"median\", dev=_mad)\n else:\n raise ValueError('`flavor` needs to be \"seurat\" or \"cell_ranger\"')\n return disp_bin_stats.loc[df[\"mean_bin\"]].set_index(df.index)\n\n\ndef _postprocess_dispersions_seurat(\n disp_bin_stats: pd.DataFrame, mean_bin: pd.Series\n) -> None:\n # retrieve those genes that have nan std, these are the ones where\n # only a single gene fell in the bin and implicitly set them to have\n # a normalized disperion of 1\n one_gene_per_bin = disp_bin_stats[\"dev\"].isnull()\n gen_indices = np.flatnonzero(one_gene_per_bin.loc[mean_bin])\n if len(gen_indices) == 0:\n return\n logg.debug(\n f\"Gene indices {gen_indices} fell into a single bin: their \"\n \"normalized dispersion was set to 1.\\n \"\n \"Decreasing `n_bins` will likely avoid this effect.\"\n )\n disp_bin_stats.loc[one_gene_per_bin, \"dev\"] = disp_bin_stats.loc[\n one_gene_per_bin, \"avg\"\n ]\n disp_bin_stats.loc[one_gene_per_bin, \"avg\"] = 0\n\n\ndef _mad(a):\n from statsmodels.robust import mad\n\n with warnings.catch_warnings():\n # MAD calculation raises the warning: \"Mean of empty slice\"\n warnings.simplefilter(\"ignore\", category=RuntimeWarning)\n return mad(a)\n\n\ndef _subset_genes(\n adata: AnnData,\n *,\n mean: NDArray[np.float64] | DaskArray,\n dispersion_norm: NDArray[np.float64] | DaskArray,\n cutoff: _Cutoffs | int,\n) -> NDArray[np.bool_] | DaskArray:\n \"\"\"Get boolean mask of genes with normalized dispersion in bounds.\"\"\"\n if isinstance(cutoff, _Cutoffs):\n dispersion_norm = np.nan_to_num(dispersion_norm) # similar to Seurat\n return cutoff.in_bounds(mean, dispersion_norm)\n n_top_genes = cutoff\n del cutoff\n\n if n_top_genes > adata.n_vars:\n logg.info(\"`n_top_genes` > `adata.n_var`, returning all genes.\")\n n_top_genes = adata.n_vars\n disp_cut_off = _nth_highest(dispersion_norm, n_top_genes)\n logg.debug(\n f\"the {n_top_genes} top genes correspond to a \"\n f\"normalized dispersion cutoff of {disp_cut_off}\"\n )\n return np.nan_to_num(dispersion_norm, nan=-np.inf) >= disp_cut_off\n\n\ndef _nth_highest(x: NDArray[np.float64] | DaskArray, n: int) -> float | DaskArray:\n x = x[~np.isnan(x)]\n if n > x.size:\n msg = \"`n_top_genes` > number of normalized dispersions, returning all genes with normalized dispersions.\"\n warnings.warn(msg, UserWarning)\n n = x.size\n if isinstance(x, DaskArray):\n return x.topk(n)[-1]\n # interestingly, np.argpartition is slightly slower\n x[::-1].sort()\n return x[n - 1]\n\n\ndef _highly_variable_genes_batched(\n adata: AnnData,\n batch_key: str,\n *,\n layer: str | None,\n n_bins: int,\n flavor: Literal[\"seurat\", \"cell_ranger\"],\n cutoff: _Cutoffs | int,\n) -> pd.DataFrame:\n sanitize_anndata(adata)\n batches = adata.obs[batch_key].cat.categories\n dfs = []\n gene_list = adata.var_names\n for batch in batches:\n adata_subset = adata[adata.obs[batch_key] == batch]\n\n # Filter to genes that are in the dataset\n with settings.verbosity.override(Verbosity.error):\n # TODO use groupby or so instead of materialize_as_ndarray\n filt, _ = materialize_as_ndarray(\n filter_genes(\n _get_obs_rep(adata_subset, layer=layer),\n min_cells=1,\n inplace=False,\n )\n )\n\n adata_subset = adata_subset[:, filt]\n\n hvg = _highly_variable_genes_single_batch(\n adata_subset, layer=layer, cutoff=cutoff, n_bins=n_bins, flavor=flavor\n )\n hvg.reset_index(drop=False, inplace=True, names=[\"gene\"])\n\n if (n_removed := np.sum(~filt)) > 0:\n # Add 0 values for genes that were filtered out\n missing_hvg = pd.DataFrame(\n np.zeros((n_removed, len(hvg.columns))),\n columns=hvg.columns,\n )\n missing_hvg[\"highly_variable\"] = missing_hvg[\"highly_variable\"].astype(bool)\n missing_hvg[\"gene\"] = gene_list[~filt]\n hvg = pd.concat([hvg, missing_hvg], ignore_index=True)\n\n dfs.append(hvg)\n\n df = pd.concat(dfs, axis=0)\n\n df[\"highly_variable\"] = df[\"highly_variable\"].astype(int)\n df = df.groupby(\"gene\", observed=True).agg(\n dict(\n means=\"mean\",\n dispersions=\"mean\",\n dispersions_norm=\"mean\",\n highly_variable=\"sum\",\n )\n )\n df[\"highly_variable_nbatches\"] = df[\"highly_variable\"]\n df[\"highly_variable_intersection\"] = df[\"highly_variable_nbatches\"] == len(batches)\n\n if isinstance(cutoff, int):\n # sort genes by how often they selected as hvg within each batch and\n # break ties with normalized dispersion across batches\n\n df_orig_ind = adata.var.index.copy()\n df.sort_values(\n [\"highly_variable_nbatches\", \"dispersions_norm\"],\n ascending=False,\n na_position=\"last\",\n inplace=True,\n )\n df[\"highly_variable\"] = np.arange(df.shape[0]) < cutoff\n df = df.loc[df_orig_ind]\n else:\n df[\"dispersions_norm\"] = df[\"dispersions_norm\"].fillna(0) # similar to Seurat\n df[\"highly_variable\"] = cutoff.in_bounds(df[\"means\"], df[\"dispersions_norm\"])\n\n return df\n\n\n@old_positionals(\n \"layer\",\n \"n_top_genes\",\n \"min_disp\",\n \"max_disp\",\n \"min_mean\",\n \"max_mean\",\n \"span\",\n \"n_bins\",\n \"flavor\",\n \"subset\",\n \"inplace\",\n \"batch_key\",\n \"check_values\",\n)\ndef highly_variable_genes(\n adata: AnnData,\n *,\n layer: str | None = None,\n n_top_genes: int | None = None,\n min_disp: float = 0.5,\n max_disp: float = np.inf,\n min_mean: float = 0.0125,\n max_mean: float = 3,\n span: float = 0.3,\n n_bins: int = 20,\n flavor: Literal[\"seurat\", \"cell_ranger\", \"seurat_v3\", \"seurat_v3_paper\"] = \"seurat\",\n subset: bool = False,\n inplace: bool = True,\n batch_key: str | None = None,\n check_values: bool = True,\n) -> pd.DataFrame | None:\n \"\"\"\\\n Annotate highly variable genes :cite:p:`Satija2015,Zheng2017,Stuart2019`.\n\n Expects logarithmized data, except when `flavor='seurat_v3'`/`'seurat_v3_paper'`, in which count\n data is expected.\n\n Depending on `flavor`, this reproduces the R-implementations of Seurat\n :cite:p:`Satija2015`, Cell Ranger :cite:p:`Zheng2017`, and Seurat v3 :cite:p:`Stuart2019`.\n\n `'seurat_v3'`/`'seurat_v3_paper'` requires `scikit-misc` package. If you plan to use this flavor, consider\n installing `scanpy` with this optional dependency: `scanpy[skmisc]`.\n\n For the dispersion-based methods (`flavor='seurat'` :cite:t:`Satija2015` and\n `flavor='cell_ranger'` :cite:t:`Zheng2017`), the normalized dispersion is obtained\n by scaling with the mean and standard deviation of the dispersions for genes\n falling into a given bin for mean expression of genes. This means that for each\n bin of mean expression, highly variable genes are selected.\n\n For `flavor='seurat_v3'`/`'seurat_v3_paper'` :cite:p:`Stuart2019`, a normalized variance for each gene\n is computed. First, the data are standardized (i.e., z-score normalization\n per feature) with a regularized standard deviation. Next, the normalized variance\n is computed as the variance of each gene after the transformation. Genes are ranked\n by the normalized variance.\n Only if `batch_key` is not `None`, the two flavors differ: For `flavor='seurat_v3'`, genes are first sorted by the median (across batches) rank, with ties broken by the number of batches a gene is a HVG.\n For `flavor='seurat_v3_paper'`, genes are first sorted by the number of batches a gene is a HVG, with ties broken by the median (across batches) rank.\n\n The following may help when comparing to Seurat's naming:\n If `batch_key=None` and `flavor='seurat'`, this mimics Seurat's `FindVariableFeatures(…, method='mean.var.plot')`.\n If `batch_key=None` and `flavor='seurat_v3'`/`flavor='seurat_v3_paper'`, this mimics Seurat's `FindVariableFeatures(..., method='vst')`.\n If `batch_key` is not `None` and `flavor='seurat_v3_paper'`, this mimics Seurat's `SelectIntegrationFeatures`.\n\n See also `scanpy.experimental.pp._highly_variable_genes` for additional flavors\n (e.g. Pearson residuals).\n\n Parameters\n ----------\n adata\n The annotated data matrix of shape `n_obs` × `n_vars`. Rows correspond\n to cells and columns to genes.\n layer\n If provided, use `adata.layers[layer]` for expression values instead of `adata.X`.\n n_top_genes\n Number of highly-variable genes to keep. Mandatory if `flavor='seurat_v3'`.\n min_mean\n If `n_top_genes` unequals `None`, this and all other cutoffs for the means and the\n normalized dispersions are ignored. Ignored if `flavor='seurat_v3'`.\n max_mean\n If `n_top_genes` unequals `None`, this and all other cutoffs for the means and the\n normalized dispersions are ignored. Ignored if `flavor='seurat_v3'`.\n min_disp\n If `n_top_genes` unequals `None`, this and all other cutoffs for the means and the\n normalized dispersions are ignored. Ignored if `flavor='seurat_v3'`.\n max_disp\n If `n_top_genes` unequals `None`, this and all other cutoffs for the means and the\n normalized dispersions are ignored. Ignored if `flavor='seurat_v3'`.\n span\n The fraction of the data (cells) used when estimating the variance in the loess\n model fit if `flavor='seurat_v3'`.\n n_bins\n Number of bins for binning the mean gene expression. Normalization is\n done with respect to each bin. If just a single gene falls into a bin,\n the normalized dispersion is artificially set to 1. You'll be informed\n about this if you set `settings.verbosity = 4`.\n flavor\n Choose the flavor for identifying highly variable genes. For the dispersion\n based methods in their default workflows, Seurat passes the cutoffs whereas\n Cell Ranger passes `n_top_genes`.\n subset\n Inplace subset to highly-variable genes if `True` otherwise merely indicate\n highly variable genes.\n inplace\n Whether to place calculated metrics in `.var` or return them.\n batch_key\n If specified, highly-variable genes are selected within each batch separately and merged.\n This simple process avoids the selection of batch-specific genes and acts as a\n lightweight batch correction method. For all flavors, except `seurat_v3`, genes are first sorted\n by how many batches they are a HVG. For dispersion-based flavors ties are broken\n by normalized dispersion. For `flavor = 'seurat_v3_paper'`, ties are broken by the median\n (across batches) rank based on within-batch normalized variance.\n check_values\n Check if counts in selected layer are integers. A Warning is returned if set to True.\n Only used if `flavor='seurat_v3'`/`'seurat_v3_paper'`.\n\n Returns\n -------\n Returns a :class:`pandas.DataFrame` with calculated metrics if `inplace=True`, else returns an `AnnData` object where it sets the following field:\n\n `adata.var['highly_variable']` : :class:`pandas.Series` (dtype `bool`)\n boolean indicator of highly-variable genes\n `adata.var['means']` : :class:`pandas.Series` (dtype `float`)\n means per gene\n `adata.var['dispersions']` : :class:`pandas.Series` (dtype `float`)\n For dispersion-based flavors, dispersions per gene\n `adata.var['dispersions_norm']` : :class:`pandas.Series` (dtype `float`)\n For dispersion-based flavors, normalized dispersions per gene\n `adata.var['variances']` : :class:`pandas.Series` (dtype `float`)\n For `flavor='seurat_v3'`/`'seurat_v3_paper'`, variance per gene\n `adata.var['variances_norm']`/`'seurat_v3_paper'` : :class:`pandas.Series` (dtype `float`)\n For `flavor='seurat_v3'`/`'seurat_v3_paper'`, normalized variance per gene, averaged in\n the case of multiple batches\n `adata.var['highly_variable_rank']` : :class:`pandas.Series` (dtype `float`)\n For `flavor='seurat_v3'`/`'seurat_v3_paper'`, rank of the gene according to normalized\n variance, in case of multiple batches description above\n `adata.var['highly_variable_nbatches']` : :class:`pandas.Series` (dtype `int`)\n If `batch_key` is given, this denotes in how many batches genes are detected as HVG\n `adata.var['highly_variable_intersection']` : :class:`pandas.Series` (dtype `bool`)\n If `batch_key` is given, this denotes the genes that are highly variable in all batches\n\n Notes\n -----\n This function replaces :func:`~scanpy.pp.filter_genes_dispersion`.\n \"\"\"\n\n start = logg.info(\"extracting highly variable genes\")\n\n if not isinstance(adata, AnnData):\n raise ValueError(\n \"`pp.highly_variable_genes` expects an `AnnData` argument, \"\n \"pass `inplace=False` if you want to return a `pd.DataFrame`.\"\n )\n\n if flavor in {\"seurat_v3\", \"seurat_v3_paper\"}:\n if n_top_genes is None:\n sig = signature(_highly_variable_genes_seurat_v3)\n n_top_genes = cast(int, sig.parameters[\"n_top_genes\"].default)\n return _highly_variable_genes_seurat_v3(\n adata,\n flavor=flavor,\n layer=layer,\n n_top_genes=n_top_genes,\n batch_key=batch_key,\n check_values=check_values,\n span=span,\n subset=subset,\n inplace=inplace,\n )\n\n cutoff = _Cutoffs.validate(\n n_top_genes=n_top_genes,\n min_disp=min_disp,\n max_disp=max_disp,\n min_mean=min_mean,\n max_mean=max_mean,\n )\n del min_disp, max_disp, min_mean, max_mean, n_top_genes\n\n if batch_key is None:\n df = _highly_variable_genes_single_batch(\n adata, layer=layer, cutoff=cutoff, n_bins=n_bins, flavor=flavor\n )\n else:\n df = _highly_variable_genes_batched(\n adata, batch_key, layer=layer, cutoff=cutoff, n_bins=n_bins, flavor=flavor\n )\n\n logg.info(\" finished\", time=start)\n\n if not inplace:\n if subset:\n df = df.loc[df[\"highly_variable\"]]\n\n return df\n\n adata.uns[\"hvg\"] = {\"flavor\": flavor}\n logg.hint(\n \"added\\n\"\n \" 'highly_variable', boolean vector (adata.var)\\n\"\n \" 'means', float vector (adata.var)\\n\"\n \" 'dispersions', float vector (adata.var)\\n\"\n \" 'dispersions_norm', float vector (adata.var)\"\n )\n adata.var[\"highly_variable\"] = df[\"highly_variable\"]\n adata.var[\"means\"] = df[\"means\"]\n adata.var[\"dispersions\"] = df[\"dispersions\"]\n adata.var[\"dispersions_norm\"] = df[\"dispersions_norm\"].astype(\n np.float32, copy=False\n )\n\n if batch_key is not None:\n adata.var[\"highly_variable_nbatches\"] = df[\"highly_variable_nbatches\"]\n adata.var[\"highly_variable_intersection\"] = df[\"highly_variable_intersection\"]\n if subset:\n adata._inplace_subset_var(df[\"highly_variable\"])\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING, overload\n\nimport numpy as np\n\nfrom scanpy._compat import DaskArray\n\nif TYPE_CHECKING:\n from numpy.typing import ArrayLike\n\n from scanpy._compat import ZappyArray\n\n\n@overload\ndef materialize_as_ndarray(a: ArrayLike) -> np.ndarray: ...\n\n\n@overload\ndef materialize_as_ndarray(a: tuple[ArrayLike]) -> tuple[np.ndarray]: ...\n\n\n@overload\ndef materialize_as_ndarray(\n a: tuple[ArrayLike, ArrayLike],\n) -> tuple[np.ndarray, np.ndarray]: ...\n\n\n@overload\ndef materialize_as_ndarray(\n a: tuple[ArrayLike, ArrayLike, ArrayLike],\n) -> tuple[np.ndarray, np.ndarray, np.ndarray]: ...\n\n\ndef materialize_as_ndarray(\n a: DaskArray | ArrayLike | tuple[ArrayLike | ZappyArray | DaskArray, ...],\n) -> tuple[np.ndarray] | np.ndarray:\n \"\"\"Compute distributed arrays and convert them to numpy ndarrays.\"\"\"\n if isinstance(a, DaskArray):\n return a.compute()\n if not isinstance(a, tuple):\n return np.asarray(a)\n\n if not any(isinstance(arr, DaskArray) for arr in a):\n return tuple(np.asarray(arr) for arr in a)\n\n import dask.array as da\n\n return da.compute(*a, sync=True)\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom numpy import linalg as la\nfrom scipy.sparse import issparse\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._utils import sanitize_anndata\n\nif TYPE_CHECKING:\n from collections.abc import Collection\n\n from anndata import AnnData\n\n\ndef _design_matrix(\n model: pd.DataFrame, batch_key: str, batch_levels: Collection[str]\n) -> pd.DataFrame:\n \"\"\"\\\n Computes a simple design matrix.\n\n Parameters\n --------\n model\n Contains the batch annotation\n batch_key\n Name of the batch column\n batch_levels\n Levels of the batch annotation\n\n Returns\n --------\n The design matrix for the regression problem\n \"\"\"\n import patsy\n\n design = patsy.dmatrix(\n f\"~ 0 + C(Q('{batch_key}'), levels=batch_levels)\",\n model,\n return_type=\"dataframe\",\n )\n model = model.drop([batch_key], axis=1)\n numerical_covariates = model.select_dtypes(\"number\").columns.values\n\n logg.info(f\"Found {design.shape[1]} batches\\n\")\n other_cols = [c for c in model.columns.values if c not in numerical_covariates]\n\n if other_cols:\n col_repr = \" + \".join(f\"Q('{x}')\" for x in other_cols)\n factor_matrix = patsy.dmatrix(\n f\"~ 0 + {col_repr}\", model[other_cols], return_type=\"dataframe\"\n )\n\n design = pd.concat((design, factor_matrix), axis=1)\n logg.info(f\"Found {len(other_cols)} categorical variables:\")\n logg.info(\"\\t\" + \", \".join(other_cols) + \"\\n\")\n\n if numerical_covariates is not None:\n logg.info(f\"Found {len(numerical_covariates)} numerical variables:\")\n logg.info(\"\\t\" + \", \".join(numerical_covariates) + \"\\n\")\n\n for nC in numerical_covariates:\n design[nC] = model[nC]\n\n return design\n\n\ndef _standardize_data(\n model: pd.DataFrame, data: pd.DataFrame, batch_key: str\n) -> tuple[pd.DataFrame, pd.DataFrame, np.ndarray, np.ndarray]:\n \"\"\"\\\n Standardizes the data per gene.\n\n The aim here is to make mean and variance be comparable across batches.\n\n Parameters\n --------\n model\n Contains the batch annotation\n data\n Contains the Data\n batch_key\n Name of the batch column in the model matrix\n\n Returns\n --------\n s_data\n Standardized Data\n design\n Batch assignment as one-hot encodings\n var_pooled\n Pooled variance per gene\n stand_mean\n Gene-wise mean\n \"\"\"\n\n # compute the design matrix\n batch_items = model.groupby(batch_key, observed=True).groups.items()\n batch_levels, batch_info = zip(*batch_items)\n n_batch = len(batch_info)\n n_batches = np.array([len(v) for v in batch_info])\n n_array = float(sum(n_batches))\n\n design = _design_matrix(model, batch_key, batch_levels)\n\n # compute pooled variance estimator\n B_hat = np.dot(np.dot(la.inv(np.dot(design.T, design)), design.T), data.T)\n grand_mean = np.dot((n_batches / n_array).T, B_hat[:n_batch, :])\n var_pooled = (data - np.dot(design, B_hat).T) ** 2\n var_pooled = np.dot(var_pooled, np.ones((int(n_array), 1)) / int(n_array))\n\n # Compute the means\n if np.sum(var_pooled == 0) > 0:\n print(f\"Found {np.sum(var_pooled == 0)} genes with zero variance.\")\n stand_mean = np.dot(\n grand_mean.T.reshape((len(grand_mean), 1)), np.ones((1, int(n_array)))\n )\n tmp = np.array(design.copy())\n tmp[:, :n_batch] = 0\n stand_mean += np.dot(tmp, B_hat).T\n\n # need to be a bit careful with the zero variance genes\n # just set the zero variance genes to zero in the standardized data\n s_data = np.where(\n var_pooled == 0,\n 0,\n ((data - stand_mean) / np.dot(np.sqrt(var_pooled), np.ones((1, int(n_array))))),\n )\n s_data = pd.DataFrame(s_data, index=data.index, columns=data.columns)\n\n return s_data, design, var_pooled, stand_mean\n\n\n@old_positionals(\"covariates\", \"inplace\")\ndef combat(\n adata: AnnData,\n key: str = \"batch\",\n *,\n covariates: Collection[str] | None = None,\n inplace: bool = True,\n) -> np.ndarray | None:\n \"\"\"\\\n ComBat function for batch effect correction :cite:p:`Johnson2006,Leek2012,Pedersen2012`.\n\n Corrects for batch effects by fitting linear models, gains statistical power\n via an EB framework where information is borrowed across genes.\n This uses the implementation `combat.py`_ :cite:p:`Pedersen2012`.\n\n .. _combat.py: https://github.com/brentp/combat.py\n\n Parameters\n ----------\n adata\n Annotated data matrix\n key\n Key to a categorical annotation from :attr:`~anndata.AnnData.obs`\n that will be used for batch effect removal.\n covariates\n Additional covariates besides the batch variable such as adjustment\n variables or biological condition. This parameter refers to the design\n matrix `X` in Equation 2.1 in :cite:t:`Johnson2006` and to the `mod` argument in\n the original combat function in the sva R package.\n Note that not including covariates may introduce bias or lead to the\n removal of biological signal in unbalanced designs.\n inplace\n Whether to replace adata.X or to return the corrected data\n\n Returns\n -------\n Returns :class:`numpy.ndarray` if `inplace=True`, else returns `None` and sets the following field in the `adata` object:\n\n `adata.X` : :class:`numpy.ndarray` (dtype `float`)\n Corrected data matrix.\n \"\"\"\n\n # check the input\n if key not in adata.obs_keys():\n raise ValueError(f\"Could not find the key {key!r} in adata.obs\")\n\n if covariates is not None:\n cov_exist = np.isin(covariates, adata.obs_keys())\n if np.any(~cov_exist):\n missing_cov = np.array(covariates)[~cov_exist].tolist()\n raise ValueError(\n f\"Could not find the covariate(s) {missing_cov!r} in adata.obs\"\n )\n\n if key in covariates:\n raise ValueError(\"Batch key and covariates cannot overlap\")\n\n if len(covariates) != len(set(covariates)):\n raise ValueError(\"Covariates must be unique\")\n\n # only works on dense matrices so far\n X = adata.X.toarray().T if issparse(adata.X) else adata.X.T\n data = pd.DataFrame(data=X, index=adata.var_names, columns=adata.obs_names)\n\n sanitize_anndata(adata)\n\n # construct a pandas series of the batch annotation\n model = adata.obs[[key, *(covariates if covariates else [])]]\n batch_info = model.groupby(key, observed=True).indices.values()\n n_batch = len(batch_info)\n n_batches = np.array([len(v) for v in batch_info])\n n_array = float(sum(n_batches))\n\n # standardize across genes using a pooled variance estimator\n logg.info(\"Standardizing Data across genes.\\n\")\n s_data, design, var_pooled, stand_mean = _standardize_data(model, data, key)\n\n # fitting the parameters on the standardized data\n logg.info(\"Fitting L/S model and finding priors\\n\")\n batch_design = design[design.columns[:n_batch]]\n # first estimate of the additive batch effect\n gamma_hat = (\n la.inv(batch_design.T @ batch_design) @ batch_design.T @ s_data.T\n ).values\n delta_hat = []\n\n # first estimate for the multiplicative batch effect\n for i, batch_idxs in enumerate(batch_info):\n delta_hat.append(s_data.iloc[:, batch_idxs].var(axis=1))\n\n # empirically fix the prior hyperparameters\n gamma_bar = gamma_hat.mean(axis=1)\n t2 = gamma_hat.var(axis=1)\n # a_prior and b_prior are the priors on lambda and theta from Johnson and Li (2006)\n a_prior = list(map(_aprior, delta_hat))\n b_prior = list(map(_bprior, delta_hat))\n\n logg.info(\"Finding parametric adjustments\\n\")\n # gamma star and delta star will be our empirical bayes (EB) estimators\n # for the additive and multiplicative batch effect per batch and cell\n gamma_star, delta_star = [], []\n for i, batch_idxs in enumerate(batch_info):\n # temp stores our estimates for the batch effect parameters.\n # temp[0] is the additive batch effect\n # temp[1] is the multiplicative batch effect\n gamma, delta = _it_sol(\n s_data.iloc[:, batch_idxs].values,\n gamma_hat[i],\n delta_hat[i].values,\n g_bar=gamma_bar[i],\n t2=t2[i],\n a=a_prior[i],\n b=b_prior[i],\n )\n\n gamma_star.append(gamma)\n delta_star.append(delta)\n\n logg.info(\"Adjusting data\\n\")\n bayesdata = s_data\n gamma_star = np.array(gamma_star)\n delta_star = np.array(delta_star)\n\n # we now apply the parametric adjustment to the standardized data from above\n # loop over all batches in the data\n for j, batch_idxs in enumerate(batch_info):\n # we basically substract the additive batch effect, rescale by the ratio\n # of multiplicative batch effect to pooled variance and add the overall gene\n # wise mean\n dsq = np.sqrt(delta_star[j, :])\n dsq = dsq.reshape((len(dsq), 1))\n denom = np.dot(dsq, np.ones((1, n_batches[j])))\n numer = np.array(\n bayesdata.iloc[:, batch_idxs]\n - np.dot(batch_design.iloc[batch_idxs], gamma_star).T\n )\n bayesdata.iloc[:, batch_idxs] = numer / denom\n\n vpsq = np.sqrt(var_pooled).reshape((len(var_pooled), 1))\n bayesdata = bayesdata * np.dot(vpsq, np.ones((1, int(n_array)))) + stand_mean\n\n # put back into the adata object or return\n if inplace:\n adata.X = bayesdata.values.transpose()\n else:\n return bayesdata.values.transpose()\n\n\ndef _it_sol(\n s_data: np.ndarray,\n g_hat: np.ndarray,\n d_hat: np.ndarray,\n *,\n g_bar: float,\n t2: float,\n a: float,\n b: float,\n conv: float = 0.0001,\n) -> tuple[np.ndarray, np.ndarray]:\n \"\"\"\\\n Iteratively compute the conditional posterior means for gamma and delta.\n\n gamma is an estimator for the additive batch effect, deltat is an estimator\n for the multiplicative batch effect. We use an EB framework to estimate these\n two. Analytical expressions exist for both parameters, which however depend on each other.\n We therefore iteratively evalutate these two expressions until convergence is reached.\n\n Parameters\n --------\n s_data\n Contains the standardized Data\n g_hat\n Initial guess for gamma\n d_hat\n Initial guess for delta\n g_bar, t2, a, b\n Hyperparameters\n conv: float, optional (default: `0.0001`)\n convergence criterium\n\n Returns:\n --------\n gamma\n estimated value for gamma\n delta\n estimated value for delta\n \"\"\"\n\n n = (1 - np.isnan(s_data)).sum(axis=1)\n g_old = g_hat.copy()\n d_old = d_hat.copy()\n\n change = 1\n count = 0\n\n # They need to be initialized for numba to properly infer types\n g_new = g_old\n d_new = d_old\n # we place a normally distributed prior on gamma and and inverse gamma prior on delta\n # in the loop, gamma and delta are updated together. they depend on each other. we iterate until convergence.\n while change > conv:\n g_new = (t2 * n * g_hat + d_old * g_bar) / (t2 * n + d_old)\n sum2 = s_data - g_new.reshape((g_new.shape[0], 1)) @ np.ones(\n (1, s_data.shape[1])\n )\n sum2 = sum2**2\n sum2 = sum2.sum(axis=1)\n d_new = (0.5 * sum2 + b) / (n / 2.0 + a - 1.0)\n\n change = max(\n (abs(g_new - g_old) / g_old).max(), (abs(d_new - d_old) / d_old).max()\n )\n g_old = g_new # .copy()\n d_old = d_new # .copy()\n count = count + 1\n\n return g_new, d_new\n\n\ndef _aprior(delta_hat):\n m = delta_hat.mean()\n s2 = delta_hat.var()\n return (2 * s2 + m**2) / s2\n\n\ndef _bprior(delta_hat):\n m = delta_hat.mean()\n s2 = delta_hat.var()\n return (m * s2 + m**3) / s2\n\n\n\"\"\"Preprocessing recipes from the literature\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nfrom .. import logging as logg\nfrom .. import preprocessing as pp\nfrom .._compat import old_positionals\nfrom ._deprecated.highly_variable_genes import (\n filter_genes_cv_deprecated,\n filter_genes_dispersion,\n)\nfrom ._normalization import normalize_total\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n from .._utils import AnyRandom\n\n\n@old_positionals(\n \"log\",\n \"mean_threshold\",\n \"cv_threshold\",\n \"n_pcs\",\n \"svd_solver\",\n \"random_state\",\n \"copy\",\n)\ndef recipe_weinreb17(\n adata: AnnData,\n *,\n log: bool = True,\n mean_threshold: float = 0.01,\n cv_threshold: int = 2,\n n_pcs: int = 50,\n svd_solver=\"randomized\",\n random_state: AnyRandom = 0,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Normalization and filtering as of :cite:p:`Weinreb2017`.\n\n Expects non-logarithmized data.\n If using logarithmized data, pass `log=False`.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n log\n Logarithmize data?\n copy\n Return a copy if true.\n \"\"\"\n from scipy.sparse import issparse\n\n from ._deprecated import normalize_per_cell_weinreb16_deprecated, zscore_deprecated\n\n if issparse(adata.X):\n raise ValueError(\"`recipe_weinreb16 does not support sparse matrices.\")\n if copy:\n adata = adata.copy()\n if log:\n pp.log1p(adata)\n adata.X = normalize_per_cell_weinreb16_deprecated(\n adata.X, max_fraction=0.05, mult_with_mean=True\n )\n gene_subset = filter_genes_cv_deprecated(adata.X, mean_threshold, cv_threshold)\n adata._inplace_subset_var(gene_subset) # this modifies the object itself\n X_pca = pp.pca(\n zscore_deprecated(adata.X),\n n_comps=n_pcs,\n svd_solver=svd_solver,\n random_state=random_state,\n )\n # update adata\n adata.obsm[\"X_pca\"] = X_pca\n return adata if copy else None\n\n\n@old_positionals(\"log\", \"plot\", \"copy\")\ndef recipe_seurat(\n adata: AnnData, *, log: bool = True, plot: bool = False, copy: bool = False\n) -> AnnData | None:\n \"\"\"\\\n Normalization and filtering as of Seurat :cite:p:`Satija2015`.\n\n This uses a particular preprocessing.\n\n Expects non-logarithmized data.\n If using logarithmized data, pass `log=False`.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n log\n Logarithmize data?\n plot\n Show a plot of the gene dispersion vs. mean relation.\n copy\n Return a copy if true.\n \"\"\"\n if copy:\n adata = adata.copy()\n pp.filter_cells(adata, min_genes=200)\n pp.filter_genes(adata, min_cells=3)\n normalize_total(adata, target_sum=1e4)\n filter_result = filter_genes_dispersion(\n adata.X, min_mean=0.0125, max_mean=3, min_disp=0.5, log=not log\n )\n if plot:\n from ..plotting import (\n _preprocessing as ppp,\n )\n\n ppp.filter_genes_dispersion(filter_result, log=not log)\n adata._inplace_subset_var(filter_result.gene_subset) # filter genes\n if log:\n pp.log1p(adata)\n pp.scale(adata, max_value=10)\n return adata if copy else None\n\n\n@old_positionals(\"n_top_genes\", \"log\", \"plot\", \"copy\")\ndef recipe_zheng17(\n adata: AnnData,\n *,\n n_top_genes: int = 1000,\n log: bool = True,\n plot: bool = False,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Normalization and filtering as of :cite:t:`Zheng2017`.\n\n Reproduces the preprocessing of :cite:t:`Zheng2017` – the Cell Ranger R Kit of 10x\n Genomics.\n\n Expects non-logarithmized data.\n If using logarithmized data, pass `log=False`.\n\n The recipe runs the following steps\n\n .. code:: python\n\n sc.pp.filter_genes(adata, min_counts=1) # only consider genes with more than 1 count\n sc.pp.normalize_per_cell( # normalize with total UMI count per cell\n adata, key_n_counts='n_counts_all'\n )\n filter_result = sc.pp.filter_genes_dispersion( # select highly-variable genes\n adata.X, flavor='cell_ranger', n_top_genes=n_top_genes, log=False\n )\n adata = adata[:, filter_result.gene_subset] # subset the genes\n sc.pp.normalize_per_cell(adata) # renormalize after filtering\n if log: sc.pp.log1p(adata) # log transform: adata.X = log(adata.X + 1)\n sc.pp.scale(adata) # scale to unit variance and shift to zero mean\n\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n n_top_genes\n Number of genes to keep.\n log\n Take logarithm.\n plot\n Show a plot of the gene dispersion vs. mean relation.\n copy\n Return a copy of `adata` instead of updating it.\n\n Returns\n -------\n Returns or updates `adata` depending on `copy`.\n \"\"\"\n start = logg.info(\"running recipe zheng17\")\n if copy:\n adata = adata.copy()\n # only consider genes with more than 1 count\n pp.filter_genes(adata, min_counts=1)\n # normalize with total UMI count per cell\n normalize_total(adata, key_added=\"n_counts_all\")\n filter_result = filter_genes_dispersion(\n adata.X, flavor=\"cell_ranger\", n_top_genes=n_top_genes, log=False\n )\n if plot: # should not import at the top of the file\n from ..plotting import _preprocessing as ppp\n\n ppp.filter_genes_dispersion(filter_result, log=True)\n # actually filter the genes, the following is the inplace version of\n # adata = adata[:, filter_result.gene_subset]\n adata._inplace_subset_var(filter_result.gene_subset) # filter genes\n normalize_total(adata) # renormalize after filtering\n if log:\n pp.log1p(adata) # log transform: X = log(X + 1)\n pp.scale(adata)\n logg.info(\" finished\", time=start)\n return adata if copy else None\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\nfrom warnings import warn\n\nimport numba\nimport numpy as np\nimport pandas as pd\nfrom scipy.sparse import csr_matrix, issparse, isspmatrix_coo, isspmatrix_csr\nfrom sklearn.utils.sparsefuncs import mean_variance_axis\n\nfrom .._utils import _doc_params\nfrom ._docs import (\n doc_adata_basic,\n doc_expr_reps,\n doc_obs_qc_args,\n doc_obs_qc_returns,\n doc_qc_metric_naming,\n doc_var_qc_returns,\n)\n\nif TYPE_CHECKING:\n from collections.abc import Collection\n\n from anndata import AnnData\n from scipy.sparse import spmatrix\n\n\ndef _choose_mtx_rep(adata, *, use_raw: bool = False, layer: str | None = None):\n is_layer = layer is not None\n if use_raw and is_layer:\n raise ValueError(\n \"Cannot use expression from both layer and raw. You provided:\"\n f\"'use_raw={use_raw}' and 'layer={layer}'\"\n )\n if is_layer:\n return adata.layers[layer]\n elif use_raw:\n return adata.raw.X\n else:\n return adata.X\n\n\n@_doc_params(\n doc_adata_basic=doc_adata_basic,\n doc_expr_reps=doc_expr_reps,\n doc_obs_qc_args=doc_obs_qc_args,\n doc_qc_metric_naming=doc_qc_metric_naming,\n doc_obs_qc_returns=doc_obs_qc_returns,\n)\ndef describe_obs(\n adata: AnnData,\n *,\n expr_type: str = \"counts\",\n var_type: str = \"genes\",\n qc_vars: Collection[str] = (),\n percent_top: Collection[int] | None = (50, 100, 200, 500),\n layer: str | None = None,\n use_raw: bool = False,\n log1p: bool | None = True,\n inplace: bool = False,\n X=None,\n parallel=None,\n) -> pd.DataFrame | None:\n \"\"\"\\\n Describe observations of anndata.\n\n Calculates a number of qc metrics for observations in AnnData object. See\n section `Returns` for a description of those metrics.\n\n Note that this method can take a while to compile on the first call. That\n result is then cached to disk to be used later.\n\n Params\n ------\n {doc_adata_basic}\n {doc_qc_metric_naming}\n {doc_obs_qc_args}\n {doc_expr_reps}\n log1p\n Add `log1p` transformed metrics.\n inplace\n Whether to place calculated metrics in `adata.obs`.\n X\n Matrix to calculate values on. Meant for internal usage.\n\n Returns\n -------\n QC metrics for observations in adata. If inplace, values are placed into\n the AnnData's `.obs` dataframe.\n\n {doc_obs_qc_returns}\n \"\"\"\n if parallel is not None:\n warn(\n \"Argument `parallel` is deprecated, and currently has no effect.\",\n FutureWarning,\n )\n # Handle whether X is passed\n if X is None:\n X = _choose_mtx_rep(adata, use_raw=use_raw, layer=layer)\n if isspmatrix_coo(X):\n X = csr_matrix(X) # COO not subscriptable\n if issparse(X):\n X.eliminate_zeros()\n obs_metrics = pd.DataFrame(index=adata.obs_names)\n if issparse(X):\n obs_metrics[f\"n_{var_type}_by_{expr_type}\"] = X.getnnz(axis=1)\n else:\n obs_metrics[f\"n_{var_type}_by_{expr_type}\"] = np.count_nonzero(X, axis=1)\n if log1p:\n obs_metrics[f\"log1p_n_{var_type}_by_{expr_type}\"] = np.log1p(\n obs_metrics[f\"n_{var_type}_by_{expr_type}\"]\n )\n obs_metrics[f\"total_{expr_type}\"] = np.ravel(X.sum(axis=1))\n if log1p:\n obs_metrics[f\"log1p_total_{expr_type}\"] = np.log1p(\n obs_metrics[f\"total_{expr_type}\"]\n )\n if percent_top:\n percent_top = sorted(percent_top)\n proportions = top_segment_proportions(X, percent_top)\n for i, n in enumerate(percent_top):\n obs_metrics[f\"pct_{expr_type}_in_top_{n}_{var_type}\"] = (\n proportions[:, i] * 100\n )\n for qc_var in qc_vars:\n obs_metrics[f\"total_{expr_type}_{qc_var}\"] = np.ravel(\n X[:, adata.var[qc_var].values].sum(axis=1)\n )\n if log1p:\n obs_metrics[f\"log1p_total_{expr_type}_{qc_var}\"] = np.log1p(\n obs_metrics[f\"total_{expr_type}_{qc_var}\"]\n )\n obs_metrics[f\"pct_{expr_type}_{qc_var}\"] = (\n obs_metrics[f\"total_{expr_type}_{qc_var}\"]\n / obs_metrics[f\"total_{expr_type}\"]\n * 100\n )\n if inplace:\n adata.obs[obs_metrics.columns] = obs_metrics\n else:\n return obs_metrics\n\n\n@_doc_params(\n doc_adata_basic=doc_adata_basic,\n doc_expr_reps=doc_expr_reps,\n doc_qc_metric_naming=doc_qc_metric_naming,\n doc_var_qc_returns=doc_var_qc_returns,\n)\ndef describe_var(\n adata: AnnData,\n *,\n expr_type: str = \"counts\",\n var_type: str = \"genes\",\n layer: str | None = None,\n use_raw: bool = False,\n inplace: bool = False,\n log1p: bool = True,\n X: spmatrix | np.ndarray | None = None,\n) -> pd.DataFrame | None:\n \"\"\"\\\n Describe variables of anndata.\n\n Calculates a number of qc metrics for variables in AnnData object. See\n section `Returns` for a description of those metrics.\n\n Params\n ------\n {doc_adata_basic}\n {doc_qc_metric_naming}\n {doc_expr_reps}\n inplace\n Whether to place calculated metrics in `adata.var`.\n X\n Matrix to calculate values on. Meant for internal usage.\n\n Returns\n -------\n QC metrics for variables in adata. If inplace, values are placed into the\n AnnData's `.var` dataframe.\n\n {doc_var_qc_returns}\n \"\"\"\n # Handle whether X is passed\n if X is None:\n X = _choose_mtx_rep(adata, use_raw=use_raw, layer=layer)\n if isspmatrix_coo(X):\n X = csr_matrix(X) # COO not subscriptable\n if issparse(X):\n X.eliminate_zeros()\n var_metrics = pd.DataFrame(index=adata.var_names)\n if issparse(X):\n # Current memory bottleneck for csr matrices:\n var_metrics[\"n_cells_by_{expr_type}\"] = X.getnnz(axis=0)\n var_metrics[\"mean_{expr_type}\"] = mean_variance_axis(X, axis=0)[0]\n else:\n var_metrics[\"n_cells_by_{expr_type}\"] = np.count_nonzero(X, axis=0)\n var_metrics[\"mean_{expr_type}\"] = X.mean(axis=0)\n if log1p:\n var_metrics[\"log1p_mean_{expr_type}\"] = np.log1p(\n var_metrics[\"mean_{expr_type}\"]\n )\n var_metrics[\"pct_dropout_by_{expr_type}\"] = (\n 1 - var_metrics[\"n_cells_by_{expr_type}\"] / X.shape[0]\n ) * 100\n var_metrics[\"total_{expr_type}\"] = np.ravel(X.sum(axis=0))\n if log1p:\n var_metrics[\"log1p_total_{expr_type}\"] = np.log1p(\n var_metrics[\"total_{expr_type}\"]\n )\n # Relabel\n new_colnames = []\n for col in var_metrics.columns:\n new_colnames.append(col.format(**locals()))\n var_metrics.columns = new_colnames\n if inplace:\n adata.var[var_metrics.columns] = var_metrics\n else:\n return var_metrics\n\n\n@_doc_params(\n doc_adata_basic=doc_adata_basic,\n doc_expr_reps=doc_expr_reps,\n doc_obs_qc_args=doc_obs_qc_args,\n doc_qc_metric_naming=doc_qc_metric_naming,\n doc_obs_qc_returns=doc_obs_qc_returns,\n doc_var_qc_returns=doc_var_qc_returns,\n)\ndef calculate_qc_metrics(\n adata: AnnData,\n *,\n expr_type: str = \"counts\",\n var_type: str = \"genes\",\n qc_vars: Collection[str] | str = (),\n percent_top: Collection[int] | None = (50, 100, 200, 500),\n layer: str | None = None,\n use_raw: bool = False,\n inplace: bool = False,\n log1p: bool = True,\n parallel: bool | None = None,\n) -> tuple[pd.DataFrame, pd.DataFrame] | None:\n \"\"\"\\\n Calculate quality control metrics.\n\n Calculates a number of qc metrics for an AnnData object, see section\n `Returns` for specifics. Largely based on `calculateQCMetrics` from scater\n :cite:p:`McCarthy2017`. Currently is most efficient on a sparse CSR or dense matrix.\n\n Note that this method can take a while to compile on the first call. That\n result is then cached to disk to be used later.\n\n Parameters\n ----------\n {doc_adata_basic}\n {doc_qc_metric_naming}\n {doc_obs_qc_args}\n {doc_expr_reps}\n inplace\n Whether to place calculated metrics in `adata`'s `.obs` and `.var`.\n log1p\n Set to `False` to skip computing `log1p` transformed annotations.\n\n Returns\n -------\n Depending on `inplace` returns calculated metrics\n (as :class:`~pandas.DataFrame`) or updates `adata`'s `obs` and `var`.\n\n {doc_obs_qc_returns}\n\n {doc_var_qc_returns}\n\n Example\n -------\n Calculate qc metrics for visualization.\n\n .. plot::\n :context: close-figs\n\n import scanpy as sc\n import seaborn as sns\n\n pbmc = sc.datasets.pbmc3k()\n pbmc.var[\"mito\"] = pbmc.var_names.str.startswith(\"MT-\")\n sc.pp.calculate_qc_metrics(pbmc, qc_vars=[\"mito\"], inplace=True)\n sns.jointplot(\n data=pbmc.obs,\n x=\"log1p_total_counts\",\n y=\"log1p_n_genes_by_counts\",\n kind=\"hex\",\n )\n\n .. plot::\n :context: close-figs\n\n sns.histplot(pbmc.obs[\"pct_counts_mito\"])\n \"\"\"\n if parallel is not None:\n warn(\n \"Argument `parallel` is deprecated, and currently has no effect.\",\n FutureWarning,\n )\n # Pass X so I only have to do it once\n X = _choose_mtx_rep(adata, use_raw=use_raw, layer=layer)\n if isspmatrix_coo(X):\n X = csr_matrix(X) # COO not subscriptable\n if issparse(X):\n X.eliminate_zeros()\n\n # Convert qc_vars to list if str\n if isinstance(qc_vars, str):\n qc_vars = [qc_vars]\n\n obs_metrics = describe_obs(\n adata,\n expr_type=expr_type,\n var_type=var_type,\n qc_vars=qc_vars,\n percent_top=percent_top,\n inplace=inplace,\n X=X,\n log1p=log1p,\n )\n var_metrics = describe_var(\n adata,\n expr_type=expr_type,\n var_type=var_type,\n inplace=inplace,\n X=X,\n log1p=log1p,\n )\n\n if not inplace:\n return obs_metrics, var_metrics\n\n\ndef top_proportions(mtx: np.ndarray | spmatrix, n: int):\n \"\"\"\\\n Calculates cumulative proportions of top expressed genes\n\n Parameters\n ----------\n mtx\n Matrix, where each row is a sample, each column a feature.\n n\n Rank to calculate proportions up to. Value is treated as 1-indexed,\n `n=50` will calculate cumulative proportions up to the 50th most\n expressed gene.\n \"\"\"\n if issparse(mtx):\n if not isspmatrix_csr(mtx):\n mtx = csr_matrix(mtx)\n # Allowing numba to do more\n return top_proportions_sparse_csr(mtx.data, mtx.indptr, np.array(n))\n else:\n return top_proportions_dense(mtx, n)\n\n\ndef top_proportions_dense(mtx, n):\n sums = mtx.sum(axis=1)\n partitioned = np.apply_along_axis(np.argpartition, 1, -mtx, n - 1)\n partitioned = partitioned[:, :n]\n values = np.zeros_like(partitioned, dtype=np.float64)\n for i in range(partitioned.shape[0]):\n vec = mtx[i, partitioned[i, :]] # Not a view\n vec[::-1].sort() # Sorting on a reversed view (e.g. a descending sort)\n vec = np.cumsum(vec) / sums[i]\n values[i, :] = vec\n return values\n\n\ndef top_proportions_sparse_csr(data, indptr, n):\n values = np.zeros((indptr.size - 1, n), dtype=np.float64)\n for i in numba.prange(indptr.size - 1):\n start, end = indptr[i], indptr[i + 1]\n vec = np.zeros(n, dtype=np.float64)\n if end - start <= n:\n vec[: end - start] = data[start:end]\n total = vec.sum()\n else:\n vec[:] = -(np.partition(-data[start:end], n - 1)[:n])\n total = (data[start:end]).sum() # Is this not just vec.sum()?\n vec[::-1].sort()\n values[i, :] = vec.cumsum() / total\n return values\n\n\ndef top_segment_proportions(\n mtx: np.ndarray | spmatrix, ns: Collection[int]\n) -> np.ndarray:\n \"\"\"\n Calculates total percentage of counts in top ns genes.\n\n Parameters\n ----------\n mtx\n Matrix, where each row is a sample, each column a feature.\n ns\n Positions to calculate cumulative proportion at. Values are considered\n 1-indexed, e.g. `ns=[50]` will calculate cumulative proportion up to\n the 50th most expressed gene.\n \"\"\"\n # Pretty much just does dispatch\n if not (max(ns) <= mtx.shape[1] and min(ns) > 0):\n raise IndexError(\"Positions outside range of features.\")\n if issparse(mtx):\n if not isspmatrix_csr(mtx):\n mtx = csr_matrix(mtx)\n return top_segment_proportions_sparse_csr(mtx.data, mtx.indptr, np.array(ns))\n else:\n return top_segment_proportions_dense(mtx, ns)\n\n\ndef top_segment_proportions_dense(\n mtx: np.ndarray | spmatrix, ns: Collection[int]\n) -> np.ndarray:\n # Currently ns is considered to be 1 indexed\n ns = np.sort(ns)\n sums = mtx.sum(axis=1)\n partitioned = np.apply_along_axis(np.partition, 1, mtx, mtx.shape[1] - ns)[:, ::-1][\n :, : ns[-1]\n ]\n values = np.zeros((mtx.shape[0], len(ns)))\n acc = np.zeros(mtx.shape[0])\n prev = 0\n for j, n in enumerate(ns):\n acc += partitioned[:, prev:n].sum(axis=1)\n values[:, j] = acc\n prev = n\n return values / sums[:, None]\n\n\n@numba.njit(cache=True, parallel=True)\ndef top_segment_proportions_sparse_csr(data, indptr, ns):\n # work around https://github.com/numba/numba/issues/5056\n indptr = indptr.astype(np.int64)\n ns = ns.astype(np.int64)\n ns = np.sort(ns)\n maxidx = ns[-1]\n sums = np.zeros((indptr.size - 1), dtype=data.dtype)\n values = np.zeros((indptr.size - 1, len(ns)), dtype=np.float64)\n # Just to keep it simple, as a dense matrix\n partitioned = np.zeros((indptr.size - 1, maxidx), dtype=data.dtype)\n for i in numba.prange(indptr.size - 1):\n start, end = indptr[i], indptr[i + 1]\n sums[i] = np.sum(data[start:end])\n if end - start <= maxidx:\n partitioned[i, : end - start] = data[start:end]\n elif (end - start) > maxidx:\n partitioned[i, :] = -(np.partition(-data[start:end], maxidx))[:maxidx]\n partitioned[i, :] = np.partition(partitioned[i, :], maxidx - ns)\n partitioned = partitioned[:, ::-1][:, : ns[-1]]\n acc = np.zeros((indptr.size - 1), dtype=data.dtype)\n prev = 0\n # can’t use enumerate due to https://github.com/numba/numba/issues/2625\n for j in range(ns.size):\n acc += partitioned[:, prev : ns[j]].sum(axis=1)\n values[:, j] = acc\n prev = ns[j]\n return values / sums.reshape((indptr.size - 1, 1))\n\n\nfrom __future__ import annotations\n\nfrom functools import singledispatch\nfrom typing import TYPE_CHECKING\n\nimport numba\nimport numpy as np\nfrom scipy import sparse\nfrom sklearn.random_projection import sample_without_replacement\n\nfrom .._utils import axis_sum, elem_mul\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from numpy.typing import DTypeLike, NDArray\n\n from .._compat import DaskArray\n from .._utils import AnyRandom, _SupportedArray\n\n\n@singledispatch\ndef axis_mean(X: DaskArray, *, axis: Literal[0, 1], dtype: DTypeLike) -> DaskArray:\n total = axis_sum(X, axis=axis, dtype=dtype)\n return total / X.shape[axis]\n\n\n@axis_mean.register(np.ndarray)\ndef _(X: np.ndarray, *, axis: Literal[0, 1], dtype: DTypeLike) -> np.ndarray:\n return X.mean(axis=axis, dtype=dtype)\n\n\ndef _get_mean_var(\n X: _SupportedArray, *, axis: Literal[0, 1] = 0\n) -> tuple[NDArray[np.float64], NDArray[np.float64]]:\n if isinstance(X, sparse.spmatrix):\n mean, var = sparse_mean_variance_axis(X, axis=axis)\n else:\n mean = axis_mean(X, axis=axis, dtype=np.float64)\n mean_sq = axis_mean(elem_mul(X, X), axis=axis, dtype=np.float64)\n var = mean_sq - mean**2\n # enforce R convention (unbiased estimator) for variance\n var *= X.shape[axis] / (X.shape[axis] - 1)\n return mean, var\n\n\ndef sparse_mean_variance_axis(mtx: sparse.spmatrix, axis: int):\n \"\"\"\n This code and internal functions are based on sklearns\n `sparsefuncs.mean_variance_axis`.\n\n Modifications:\n * allow deciding on the output type, which can increase accuracy when calculating the mean and variance of 32bit floats.\n * This doesn't currently implement support for null values, but could.\n * Uses numba not cython\n \"\"\"\n assert axis in (0, 1)\n if isinstance(mtx, sparse.csr_matrix):\n ax_minor = 1\n shape = mtx.shape\n elif isinstance(mtx, sparse.csc_matrix):\n ax_minor = 0\n shape = mtx.shape[::-1]\n else:\n raise ValueError(\"This function only works on sparse csr and csc matrices\")\n if axis == ax_minor:\n return sparse_mean_var_major_axis(\n mtx.data,\n mtx.indptr,\n major_len=shape[0],\n minor_len=shape[1],\n n_threads=numba.get_num_threads(),\n )\n else:\n return sparse_mean_var_minor_axis(\n mtx.data,\n mtx.indices,\n mtx.indptr,\n major_len=shape[0],\n minor_len=shape[1],\n n_threads=numba.get_num_threads(),\n )\n\n\n@numba.njit(cache=True, parallel=True)\ndef sparse_mean_var_minor_axis(\n data, indices, indptr, *, major_len, minor_len, n_threads\n):\n \"\"\"\n Computes mean and variance for a sparse matrix for the minor axis.\n\n Given arrays for a csr matrix, returns the means and variances for each\n column back.\n \"\"\"\n rows = len(indptr) - 1\n sums_minor = np.zeros((n_threads, minor_len))\n squared_sums_minor = np.zeros((n_threads, minor_len))\n means = np.zeros(minor_len)\n variances = np.zeros(minor_len)\n for i in numba.prange(n_threads):\n for r in range(i, rows, n_threads):\n for j in range(indptr[r], indptr[r + 1]):\n minor_index = indices[j]\n if minor_index >= minor_len:\n continue\n value = data[j]\n sums_minor[i, minor_index] += value\n squared_sums_minor[i, minor_index] += value * value\n for c in numba.prange(minor_len):\n sum_minor = sums_minor[:, c].sum()\n means[c] = sum_minor / major_len\n variances[c] = (\n squared_sums_minor[:, c].sum() / major_len - (sum_minor / major_len) ** 2\n )\n return means, variances\n\n\n@numba.njit(cache=True, parallel=True)\ndef sparse_mean_var_major_axis(data, indptr, *, major_len, minor_len, n_threads):\n \"\"\"\n Computes mean and variance for a sparse array for the major axis.\n\n Given arrays for a csr matrix, returns the means and variances for each\n row back.\n \"\"\"\n rows = len(indptr) - 1\n means = np.zeros(major_len)\n variances = np.zeros_like(means)\n\n for i in numba.prange(n_threads):\n for r in range(i, rows, n_threads):\n sum_major = 0.0\n squared_sum_minor = 0.0\n for j in range(indptr[r], indptr[r + 1]):\n value = np.float64(data[j])\n sum_major += value\n squared_sum_minor += value * value\n means[r] = sum_major\n variances[r] = squared_sum_minor\n for c in numba.prange(major_len):\n mean = means[c] / minor_len\n means[c] = mean\n variances[c] = variances[c] / minor_len - mean * mean\n return means, variances\n\n\ndef sample_comb(\n dims: tuple[int, ...],\n nsamp: int,\n *,\n random_state: AnyRandom = None,\n method: Literal[\n \"auto\", \"tracking_selection\", \"reservoir_sampling\", \"pool\"\n ] = \"auto\",\n) -> NDArray[np.int64]:\n \"\"\"Randomly sample indices from a grid, without repeating the same tuple.\"\"\"\n idx = sample_without_replacement(\n np.prod(dims), nsamp, random_state=random_state, method=method\n )\n return np.vstack(np.unravel_index(idx, dims)).T\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom typing import TYPE_CHECKING\nfrom warnings import warn\n\nimport anndata as ad\nimport numpy as np\nfrom anndata import AnnData\nfrom packaging.version import Version\nfrom scipy.sparse import issparse\nfrom scipy.sparse.linalg import LinearOperator, svds\nfrom sklearn.utils import check_array, check_random_state\nfrom sklearn.utils.extmath import svd_flip\n\nfrom .. import logging as logg\nfrom .._compat import DaskArray, pkg_version\nfrom .._settings import settings\nfrom .._utils import _doc_params, _empty, is_backed_type\nfrom ..get import _check_mask, _get_obs_rep\nfrom ._docs import doc_mask_var_hvg\nfrom ._utils import _get_mean_var\n\nif TYPE_CHECKING:\n from numpy.typing import DTypeLike, NDArray\n from scipy.sparse import spmatrix\n from sklearn.decomposition import PCA\n\n from .._utils import AnyRandom, Empty\n\n\n@_doc_params(\n mask_var_hvg=doc_mask_var_hvg,\n)\ndef pca(\n data: AnnData | np.ndarray | spmatrix,\n n_comps: int | None = None,\n *,\n layer: str | None = None,\n zero_center: bool | None = True,\n svd_solver: str | None = None,\n random_state: AnyRandom = 0,\n return_info: bool = False,\n mask_var: NDArray[np.bool_] | str | None | Empty = _empty,\n use_highly_variable: bool | None = None,\n dtype: DTypeLike = \"float32\",\n chunked: bool = False,\n chunk_size: int | None = None,\n key_added: str | None = None,\n copy: bool = False,\n) -> AnnData | np.ndarray | spmatrix | None:\n \"\"\"\\\n Principal component analysis :cite:p:`Pedregosa2011`.\n\n Computes PCA coordinates, loadings and variance decomposition.\n Uses the implementation of *scikit-learn* :cite:p:`Pedregosa2011`.\n\n .. versionchanged:: 1.5.0\n\n In previous versions, computing a PCA on a sparse matrix would make\n a dense copy of the array for mean centering.\n As of scanpy 1.5.0, mean centering is implicit.\n While results are extremely similar, they are not exactly the same.\n If you would like to reproduce the old results, pass a dense array.\n\n Parameters\n ----------\n data\n The (annotated) data matrix of shape `n_obs` × `n_vars`.\n Rows correspond to cells and columns to genes.\n n_comps\n Number of principal components to compute. Defaults to 50, or 1 - minimum\n dimension size of selected representation.\n layer\n If provided, which element of layers to use for PCA.\n zero_center\n If `True`, compute standard PCA from covariance matrix.\n If `False`, omit zero-centering variables\n (uses *scikit-learn* :class:`~sklearn.decomposition.TruncatedSVD` or\n *dask-ml* :class:`~dask_ml.decomposition.TruncatedSVD`),\n which allows to handle sparse input efficiently.\n Passing `None` decides automatically based on sparseness of the data.\n svd_solver\n SVD solver to use:\n\n `None`\n See `chunked` and `zero_center` descriptions to determine which class will be used.\n Depending on the class and the type of X different values for default will be set.\n If *scikit-learn* :class:`~sklearn.decomposition.PCA` is used, will give `'arpack'`,\n if *scikit-learn* :class:`~sklearn.decomposition.TruncatedSVD` is used, will give `'randomized'`,\n if *dask-ml* :class:`~dask_ml.decomposition.PCA` or :class:`~dask_ml.decomposition.IncrementalPCA` is used, will give `'auto'`,\n if *dask-ml* :class:`~dask_ml.decomposition.TruncatedSVD` is used, will give `'tsqr'`\n `'arpack'`\n for the ARPACK wrapper in SciPy (:func:`~scipy.sparse.linalg.svds`)\n Not available with *dask* arrays.\n `'randomized'`\n for the randomized algorithm due to Halko (2009). For *dask* arrays,\n this will use :func:`~dask.array.linalg.svd_compressed`.\n `'auto'`\n chooses automatically depending on the size of the problem.\n `'lobpcg'`\n An alternative SciPy solver. Not available with dask arrays.\n `'tsqr'`\n Only available with *dask* arrays. \"tsqr\"\n algorithm from Benson et. al. (2013).\n\n .. versionchanged:: 1.9.3\n Default value changed from `'arpack'` to None.\n .. versionchanged:: 1.4.5\n Default value changed from `'auto'` to `'arpack'`.\n\n Efficient computation of the principal components of a sparse matrix\n currently only works with the `'arpack`' or `'lobpcg'` solvers.\n\n If X is a *dask* array, *dask-ml* classes :class:`~dask_ml.decomposition.PCA`,\n :class:`~dask_ml.decomposition.IncrementalPCA`, or\n :class:`~dask_ml.decomposition.TruncatedSVD` will be used.\n Otherwise their *scikit-learn* counterparts :class:`~sklearn.decomposition.PCA`,\n :class:`~sklearn.decomposition.IncrementalPCA`, or\n :class:`~sklearn.decomposition.TruncatedSVD` will be used.\n random_state\n Change to use different initial states for the optimization.\n return_info\n Only relevant when not passing an :class:`~anndata.AnnData`:\n see “Returns”.\n {mask_var_hvg}\n layer\n Layer of `adata` to use as expression values.\n dtype\n Numpy data type string to which to convert the result.\n chunked\n If `True`, perform an incremental PCA on segments of `chunk_size`.\n The incremental PCA automatically zero centers and ignores settings of\n `random_seed` and `svd_solver`. Uses sklearn :class:`~sklearn.decomposition.IncrementalPCA` or\n *dask-ml* :class:`~dask_ml.decomposition.IncrementalPCA`. If `False`, perform a full PCA and\n use sklearn :class:`~sklearn.decomposition.PCA` or\n *dask-ml* :class:`~dask_ml.decomposition.PCA`\n chunk_size\n Number of observations to include in each chunk.\n Required if `chunked=True` was passed.\n key_added\n If not specified, the embedding is stored as\n :attr:`~anndata.AnnData.obsm`\\\\ `['X_pca']`, the loadings as\n :attr:`~anndata.AnnData.varm`\\\\ `['PCs']`, and the the parameters in\n :attr:`~anndata.AnnData.uns`\\\\ `['pca']`.\n If specified, the embedding is stored as\n :attr:`~anndata.AnnData.obsm`\\\\ ``[key_added]``, the loadings as\n :attr:`~anndata.AnnData.varm`\\\\ ``[key_added]``, and the the parameters in\n :attr:`~anndata.AnnData.uns`\\\\ ``[key_added]``.\n copy\n If an :class:`~anndata.AnnData` is passed, determines whether a copy\n is returned. Is ignored otherwise.\n\n Returns\n -------\n If `data` is array-like and `return_info=False` was passed,\n this function returns the PCA representation of `data` as an\n array of the same type as the input array.\n\n Otherwise, it returns `None` if `copy=False`, else an updated `AnnData` object.\n Sets the following fields:\n\n `.obsm['X_pca' | key_added]` : :class:`~scipy.sparse.spmatrix` | :class:`~numpy.ndarray` (shape `(adata.n_obs, n_comps)`)\n PCA representation of data.\n `.varm['PCs' | key_added]` : :class:`~numpy.ndarray` (shape `(adata.n_vars, n_comps)`)\n The principal components containing the loadings.\n `.uns['pca' | key_added]['variance_ratio']` : :class:`~numpy.ndarray` (shape `(n_comps,)`)\n Ratio of explained variance.\n `.uns['pca' | key_added]['variance']` : :class:`~numpy.ndarray` (shape `(n_comps,)`)\n Explained variance, equivalent to the eigenvalues of the\n covariance matrix.\n \"\"\"\n logg_start = logg.info(\"computing PCA\")\n if layer is not None and chunked:\n # Current chunking implementation relies on pca being called on X\n raise NotImplementedError(\"Cannot use `layer` and `chunked` at the same time.\")\n\n # chunked calculation is not randomized, anyways\n if svd_solver in {\"auto\", \"randomized\"} and not chunked:\n logg.info(\n \"Note that scikit-learn's randomized PCA might not be exactly \"\n \"reproducible across different computational platforms. For exact \"\n \"reproducibility, choose `svd_solver='arpack'.`\"\n )\n data_is_AnnData = isinstance(data, AnnData)\n if data_is_AnnData:\n if layer is None and not chunked and is_backed_type(data.X):\n raise NotImplementedError(\n f\"PCA is not implemented for matrices of type {type(data.X)} with chunked as False\"\n )\n adata = data.copy() if copy else data\n else:\n if pkg_version(\"anndata\") < Version(\"0.8.0rc1\"):\n adata = AnnData(data, dtype=data.dtype)\n else:\n adata = AnnData(data)\n\n # Unify new mask argument and deprecated use_highly_varible argument\n mask_var_param, mask_var = _handle_mask_var(adata, mask_var, use_highly_variable)\n del use_highly_variable\n adata_comp = adata[:, mask_var] if mask_var is not None else adata\n\n if n_comps is None:\n min_dim = min(adata_comp.n_vars, adata_comp.n_obs)\n n_comps = min_dim - 1 if min_dim <= settings.N_PCS else settings.N_PCS\n\n logg.info(f\" with n_comps={n_comps}\")\n\n X = _get_obs_rep(adata_comp, layer=layer)\n if is_backed_type(X) and layer is not None:\n raise NotImplementedError(\n f\"PCA is not implemented for matrices of type {type(X)} from layers\"\n )\n # See: https://github.com/scverse/scanpy/pull/2816#issuecomment-1932650529\n if (\n Version(ad.__version__) < Version(\"0.9\")\n and mask_var is not None\n and isinstance(X, np.ndarray)\n ):\n warnings.warn(\n \"When using a mask parameter with anndata<0.9 on a dense array, the PCA\"\n \"can have slightly different results due the array being column major \"\n \"instead of row major.\",\n UserWarning,\n )\n\n is_dask = isinstance(X, DaskArray)\n\n # check_random_state returns a numpy RandomState when passed an int but\n # dask needs an int for random state\n if not is_dask:\n random_state = check_random_state(random_state)\n elif not isinstance(random_state, int):\n msg = f\"random_state needs to be an int, not a {type(random_state).__name__} when passing a dask array\"\n raise TypeError(msg)\n\n if chunked:\n if (\n not zero_center\n or random_state\n or (svd_solver is not None and svd_solver != \"arpack\")\n ):\n logg.debug(\"Ignoring zero_center, random_state, svd_solver\")\n\n incremental_pca_kwargs = dict()\n if is_dask:\n from dask.array import zeros\n from dask_ml.decomposition import IncrementalPCA\n\n incremental_pca_kwargs[\"svd_solver\"] = _handle_dask_ml_args(\n svd_solver, \"IncrementalPCA\"\n )\n else:\n from numpy import zeros\n from sklearn.decomposition import IncrementalPCA\n\n X_pca = zeros((X.shape[0], n_comps), X.dtype)\n\n pca_ = IncrementalPCA(n_components=n_comps, **incremental_pca_kwargs)\n\n for chunk, _, _ in adata_comp.chunked_X(chunk_size):\n chunk = chunk.toarray() if issparse(chunk) else chunk\n pca_.partial_fit(chunk)\n\n for chunk, start, end in adata_comp.chunked_X(chunk_size):\n chunk = chunk.toarray() if issparse(chunk) else chunk\n X_pca[start:end] = pca_.transform(chunk)\n elif (not issparse(X) or svd_solver == \"randomized\") and zero_center:\n if is_dask:\n from dask_ml.decomposition import PCA\n\n svd_solver = _handle_dask_ml_args(svd_solver, \"PCA\")\n else:\n from sklearn.decomposition import PCA\n\n svd_solver = _handle_sklearn_args(svd_solver, \"PCA\")\n\n if issparse(X) and svd_solver == \"randomized\":\n # This is for backwards compat. Better behaviour would be to either error or use arpack.\n warnings.warn(\n \"svd_solver 'randomized' does not work with sparse input. Densifying the array. \"\n \"This may take a very large amount of memory.\"\n )\n X = X.toarray()\n pca_ = PCA(\n n_components=n_comps, svd_solver=svd_solver, random_state=random_state\n )\n X_pca = pca_.fit_transform(X)\n elif issparse(X) and zero_center:\n svd_solver = _handle_sklearn_args(svd_solver, \"PCA (with sparse input)\")\n\n X_pca, pca_ = _pca_with_sparse(\n X, n_comps, solver=svd_solver, random_state=random_state\n )\n elif not zero_center:\n if is_dask:\n from dask_ml.decomposition import TruncatedSVD\n\n svd_solver = _handle_dask_ml_args(svd_solver, \"TruncatedSVD\")\n else:\n from sklearn.decomposition import TruncatedSVD\n\n svd_solver = _handle_sklearn_args(svd_solver, \"TruncatedSVD\")\n\n logg.debug(\n \" without zero-centering: \\n\"\n \" the explained variance does not correspond to the exact statistical defintion\\n\"\n \" the first component, e.g., might be heavily influenced by different means\\n\"\n \" the following components often resemble the exact PCA very closely\"\n )\n pca_ = TruncatedSVD(\n n_components=n_comps, random_state=random_state, algorithm=svd_solver\n )\n X_pca = pca_.fit_transform(X)\n else:\n msg = \"This shouldn’t happen. Please open a bug report.\"\n raise AssertionError(msg)\n\n if X_pca.dtype.descr != np.dtype(dtype).descr:\n X_pca = X_pca.astype(dtype)\n\n if data_is_AnnData:\n key_obsm, key_varm, key_uns = (\n (\"X_pca\", \"PCs\", \"pca\") if key_added is None else [key_added] * 3\n )\n adata.obsm[key_obsm] = X_pca\n\n if mask_var is not None:\n adata.varm[key_varm] = np.zeros(shape=(adata.n_vars, n_comps))\n adata.varm[key_varm][mask_var] = pca_.components_.T\n else:\n adata.varm[key_varm] = pca_.components_.T\n\n params = dict(\n zero_center=zero_center,\n use_highly_variable=mask_var_param == \"highly_variable\",\n mask_var=mask_var_param,\n )\n if layer is not None:\n params[\"layer\"] = layer\n adata.uns[key_uns] = dict(\n params=params,\n variance=pca_.explained_variance_,\n variance_ratio=pca_.explained_variance_ratio_,\n )\n\n logg.info(\" finished\", time=logg_start)\n logg.debug(\n \"and added\\n\"\n f\" {key_obsm!r}, the PCA coordinates (adata.obs)\\n\"\n f\" {key_varm!r}, the loadings (adata.varm)\\n\"\n f\" 'pca_variance', the variance / eigenvalues (adata.uns[{key_uns!r}])\\n\"\n f\" 'pca_variance_ratio', the variance ratio (adata.uns[{key_uns!r}])\"\n )\n return adata if copy else None\n else:\n logg.info(\" finished\", time=logg_start)\n if return_info:\n return (\n X_pca,\n pca_.components_,\n pca_.explained_variance_ratio_,\n pca_.explained_variance_,\n )\n else:\n return X_pca\n\n\ndef _handle_mask_var(\n adata: AnnData,\n mask_var: NDArray[np.bool_] | str | Empty | None,\n use_highly_variable: bool | None,\n) -> tuple[np.ndarray | str | None, np.ndarray | None]:\n \"\"\"\\\n Unify new mask argument and deprecated use_highly_varible argument.\n\n Returns both the normalized mask parameter and the validated mask array.\n \"\"\"\n # First, verify and possibly warn\n if use_highly_variable is not None:\n hint = (\n 'Use_highly_variable=True can be called through mask_var=\"highly_variable\". '\n \"Use_highly_variable=False can be called through mask_var=None\"\n )\n msg = f\"Argument `use_highly_variable` is deprecated, consider using the mask argument. {hint}\"\n warn(msg, FutureWarning)\n if mask_var is not _empty:\n msg = f\"These arguments are incompatible. {hint}\"\n raise ValueError(msg)\n\n # Handle default case and explicit use_highly_variable=True\n if use_highly_variable or (\n use_highly_variable is None\n and mask_var is _empty\n and \"highly_variable\" in adata.var.columns\n ):\n mask_var = \"highly_variable\"\n\n # Without highly variable genes, we don’t use a mask by default\n if mask_var is _empty or mask_var is None:\n return None, None\n return mask_var, _check_mask(adata, mask_var, \"var\")\n\n\ndef _pca_with_sparse(\n X: spmatrix,\n n_pcs: int,\n *,\n solver: str = \"arpack\",\n mu: NDArray[np.floating] | None = None,\n random_state: AnyRandom = None,\n) -> tuple[NDArray[np.floating], PCA]:\n random_state = check_random_state(random_state)\n np.random.set_state(random_state.get_state())\n random_init = np.random.rand(np.min(X.shape))\n X = check_array(X, accept_sparse=[\"csr\", \"csc\"])\n\n if mu is None:\n mu = np.asarray(X.mean(0)).flatten()[None, :]\n mdot = mu.dot\n mmat = mdot\n mhdot = mu.T.dot\n mhmat = mu.T.dot\n Xdot = X.dot\n Xmat = Xdot\n XHdot = X.T.conj().dot\n XHmat = XHdot\n ones = np.ones(X.shape[0])[None, :].dot\n\n def matvec(x):\n return Xdot(x) - mdot(x)\n\n def matmat(x):\n return Xmat(x) - mmat(x)\n\n def rmatvec(x):\n return XHdot(x) - mhdot(ones(x))\n\n def rmatmat(x):\n return XHmat(x) - mhmat(ones(x))\n\n XL = LinearOperator(\n matvec=matvec,\n dtype=X.dtype,\n matmat=matmat,\n shape=X.shape,\n rmatvec=rmatvec,\n rmatmat=rmatmat,\n )\n\n u, s, v = svds(XL, solver=solver, k=n_pcs, v0=random_init)\n # u_based_decision was changed in https://github.com/scikit-learn/scikit-learn/pull/27491\n u, v = svd_flip(\n u, v, u_based_decision=pkg_version(\"scikit-learn\") < Version(\"1.5.0rc1\")\n )\n idx = np.argsort(-s)\n v = v[idx, :]\n\n X_pca = (u * s)[:, idx]\n ev = s[idx] ** 2 / (X.shape[0] - 1)\n\n total_var = _get_mean_var(X)[1].sum()\n ev_ratio = ev / total_var\n\n from sklearn.decomposition import PCA\n\n pca = PCA(n_components=n_pcs, svd_solver=solver, random_state=random_state)\n pca.explained_variance_ = ev\n pca.explained_variance_ratio_ = ev_ratio\n pca.components_ = v\n return X_pca, pca\n\n\ndef _handle_dask_ml_args(svd_solver: str, method: str) -> str:\n method2args = {\n \"PCA\": {\"auto\", \"full\", \"tsqr\", \"randomized\"},\n \"IncrementalPCA\": {\"auto\", \"full\", \"tsqr\", \"randomized\"},\n \"TruncatedSVD\": {\"tsqr\", \"randomized\"},\n }\n method2default = {\n \"PCA\": \"auto\",\n \"IncrementalPCA\": \"auto\",\n \"TruncatedSVD\": \"tsqr\",\n }\n\n return _handle_x_args(\"dask_ml\", svd_solver, method, method2args, method2default)\n\n\ndef _handle_sklearn_args(svd_solver: str | None, method: str) -> str:\n method2args = {\n \"PCA\": {\"auto\", \"full\", \"arpack\", \"randomized\"},\n \"TruncatedSVD\": {\"arpack\", \"randomized\"},\n \"PCA (with sparse input)\": {\"lobpcg\", \"arpack\"},\n }\n method2default = {\n \"PCA\": \"arpack\",\n \"TruncatedSVD\": \"randomized\",\n \"PCA (with sparse input)\": \"arpack\",\n }\n\n return _handle_x_args(\"sklearn\", svd_solver, method, method2args, method2default)\n\n\ndef _handle_x_args(lib, svd_solver: str | None, method, method2args, method2default):\n if svd_solver not in method2args[method]:\n if svd_solver is not None:\n warnings.warn(\n f\"Ignoring {svd_solver} and using {method2default[method]}, {lib}.decomposition.{method} only supports {method2args[method]}\"\n )\n svd_solver = method2default[method]\n return svd_solver\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom functools import singledispatch\nfrom operator import truediv\nfrom typing import TYPE_CHECKING\n\nimport numba\nimport numpy as np\nfrom anndata import AnnData\nfrom scipy.sparse import issparse, isspmatrix_csc, spmatrix\n\nfrom .. import logging as logg\nfrom .._compat import DaskArray, old_positionals\nfrom .._utils import (\n _check_array_function_arguments,\n axis_mul_or_truediv,\n raise_not_implemented_error_if_backed_type,\n renamed_arg,\n view_to_actual,\n)\nfrom ..get import _check_mask, _get_obs_rep, _set_obs_rep\nfrom ._utils import _get_mean_var\n\n# install dask if available\ntry:\n import dask.array as da\nexcept ImportError:\n da = None\n\nif TYPE_CHECKING:\n from numpy.typing import NDArray\n\n\n@numba.njit(cache=True, parallel=True)\ndef _scale_sparse_numba(indptr, indices, data, *, std, mask_obs, clip):\n for i in numba.prange(len(indptr) - 1):\n if mask_obs[i]:\n for j in range(indptr[i], indptr[i + 1]):\n if clip:\n data[j] = min(clip, data[j] / std[indices[j]])\n else:\n data[j] /= std[indices[j]]\n\n\n@numba.njit(parallel=True, cache=True)\ndef clip_array(X: np.ndarray, *, max_value: float = 10, zero_center: bool = True):\n a_min, a_max = -max_value, max_value\n if X.ndim > 1:\n for r, c in numba.pndindex(X.shape):\n if X[r, c] > a_max:\n X[r, c] = a_max\n elif X[r, c] < a_min and zero_center:\n X[r, c] = a_min\n else:\n for i in numba.prange(X.size):\n if X[i] > a_max:\n X[i] = a_max\n elif X[i] < a_min and zero_center:\n X[i] = a_min\n return X\n\n\n@renamed_arg(\"X\", \"data\", pos_0=True)\n@old_positionals(\"zero_center\", \"max_value\", \"copy\", \"layer\", \"obsm\")\n@singledispatch\ndef scale(\n data: AnnData | spmatrix | np.ndarray | DaskArray,\n *,\n zero_center: bool = True,\n max_value: float | None = None,\n copy: bool = False,\n layer: str | None = None,\n obsm: str | None = None,\n mask_obs: NDArray[np.bool_] | str | None = None,\n) -> AnnData | spmatrix | np.ndarray | DaskArray | None:\n \"\"\"\\\n Scale data to unit variance and zero mean.\n\n .. note::\n Variables (genes) that do not display any variation (are constant across\n all observations) are retained and (for zero_center==True) set to 0\n during this operation. In the future, they might be set to NaNs.\n\n Parameters\n ----------\n data\n The (annotated) data matrix of shape `n_obs` × `n_vars`.\n Rows correspond to cells and columns to genes.\n zero_center\n If `False`, omit zero-centering variables, which allows to handle sparse\n input efficiently.\n max_value\n Clip (truncate) to this value after scaling. If `None`, do not clip.\n copy\n Whether this function should be performed inplace. If an AnnData object\n is passed, this also determines if a copy is returned.\n layer\n If provided, which element of layers to scale.\n obsm\n If provided, which element of obsm to scale.\n mask_obs\n Restrict both the derivation of scaling parameters and the scaling itself\n to a certain set of observations. The mask is specified as a boolean array\n or a string referring to an array in :attr:`~anndata.AnnData.obs`.\n This will transform data from csc to csr format if `issparse(data)`.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an updated `AnnData` object. Sets the following fields:\n\n `adata.X` | `adata.layers[layer]` : :class:`numpy.ndarray` | :class:`scipy.sparse._csr.csr_matrix` (dtype `float`)\n Scaled count data matrix.\n `adata.var['mean']` : :class:`pandas.Series` (dtype `float`)\n Means per gene before scaling.\n `adata.var['std']` : :class:`pandas.Series` (dtype `float`)\n Standard deviations per gene before scaling.\n `adata.var['var']` : :class:`pandas.Series` (dtype `float`)\n Variances per gene before scaling.\n \"\"\"\n _check_array_function_arguments(layer=layer, obsm=obsm)\n if layer is not None:\n raise ValueError(\n f\"`layer` argument inappropriate for value of type {type(data)}\"\n )\n if obsm is not None:\n raise ValueError(\n f\"`obsm` argument inappropriate for value of type {type(data)}\"\n )\n return scale_array(\n data, zero_center=zero_center, max_value=max_value, copy=copy, mask_obs=mask_obs\n )\n\n\n@scale.register(np.ndarray)\n@scale.register(DaskArray)\ndef scale_array(\n X: np.ndarray | DaskArray,\n *,\n zero_center: bool = True,\n max_value: float | None = None,\n copy: bool = False,\n return_mean_std: bool = False,\n mask_obs: NDArray[np.bool_] | None = None,\n) -> (\n np.ndarray\n | DaskArray\n | tuple[\n np.ndarray | DaskArray, NDArray[np.float64] | DaskArray, NDArray[np.float64]\n ]\n | DaskArray\n):\n if copy:\n X = X.copy()\n if mask_obs is not None:\n mask_obs = _check_mask(X, mask_obs, \"obs\")\n scale_rv = scale_array(\n X[mask_obs, :],\n zero_center=zero_center,\n max_value=max_value,\n copy=False,\n return_mean_std=return_mean_std,\n mask_obs=None,\n )\n\n if return_mean_std:\n X[mask_obs, :], mean, std = scale_rv\n return X, mean, std\n else:\n X[mask_obs, :] = scale_rv\n return X\n\n if not zero_center and max_value is not None:\n logg.info( # Be careful of what? This should be more specific\n \"... be careful when using `max_value` \" \"without `zero_center`.\"\n )\n\n if np.issubdtype(X.dtype, np.integer):\n logg.info(\n \"... as scaling leads to float results, integer \"\n \"input is cast to float, returning copy.\"\n )\n X = X.astype(float)\n\n mean, var = _get_mean_var(X)\n std = np.sqrt(var)\n std[std == 0] = 1\n if zero_center:\n if isinstance(X, DaskArray) and issparse(X._meta):\n warnings.warn(\n \"zero-center being used with `DaskArray` sparse chunks. This can be bad if you have large chunks or intend to eventually read the whole data into memory.\",\n UserWarning,\n )\n X -= mean\n\n X = axis_mul_or_truediv(\n X,\n std,\n op=truediv,\n out=X if isinstance(X, np.ndarray) or issparse(X) else None,\n axis=1,\n )\n\n # do the clipping\n if max_value is not None:\n logg.debug(f\"... clipping at max_value {max_value}\")\n if isinstance(X, DaskArray) and issparse(X._meta):\n\n def clip_set(x):\n x = x.copy()\n x[x > max_value] = max_value\n if zero_center:\n x[x < -max_value] = -max_value\n return x\n\n X = da.map_blocks(clip_set, X)\n else:\n if isinstance(X, DaskArray):\n X = X.map_blocks(\n clip_array, max_value=max_value, zero_center=zero_center\n )\n elif issparse(X):\n X.data = clip_array(X.data, max_value=max_value, zero_center=False)\n else:\n X = clip_array(X, max_value=max_value, zero_center=zero_center)\n if return_mean_std:\n return X, mean, std\n else:\n return X\n\n\n@scale.register(spmatrix)\ndef scale_sparse(\n X: spmatrix,\n *,\n zero_center: bool = True,\n max_value: float | None = None,\n copy: bool = False,\n return_mean_std: bool = False,\n mask_obs: NDArray[np.bool_] | None = None,\n) -> np.ndarray | tuple[np.ndarray, NDArray[np.float64], NDArray[np.float64]]:\n # need to add the following here to make inplace logic work\n if zero_center:\n logg.info(\n \"... as `zero_center=True`, sparse input is \"\n \"densified and may lead to large memory consumption\"\n )\n X = X.toarray()\n copy = False # Since the data has been copied\n return scale_array(\n X,\n zero_center=zero_center,\n copy=copy,\n max_value=max_value,\n return_mean_std=return_mean_std,\n mask_obs=mask_obs,\n )\n elif mask_obs is None:\n return scale_array(\n X,\n zero_center=zero_center,\n copy=copy,\n max_value=max_value,\n return_mean_std=return_mean_std,\n mask_obs=mask_obs,\n )\n else:\n if isspmatrix_csc(X):\n X = X.tocsr()\n elif copy:\n X = X.copy()\n\n if mask_obs is not None:\n mask_obs = _check_mask(X, mask_obs, \"obs\")\n\n mean, var = _get_mean_var(X[mask_obs, :])\n\n std = np.sqrt(var)\n std[std == 0] = 1\n\n if max_value is None:\n max_value = 0\n\n _scale_sparse_numba(\n X.indptr,\n X.indices,\n X.data,\n std=std.astype(X.dtype),\n mask_obs=mask_obs,\n clip=max_value,\n )\n\n if return_mean_std:\n return X, mean, std\n else:\n return X\n\n\n@scale.register(AnnData)\ndef scale_anndata(\n adata: AnnData,\n *,\n zero_center: bool = True,\n max_value: float | None = None,\n copy: bool = False,\n layer: str | None = None,\n obsm: str | None = None,\n mask_obs: NDArray[np.bool_] | str | None = None,\n) -> AnnData | None:\n adata = adata.copy() if copy else adata\n str_mean_std = (\"mean\", \"std\")\n if mask_obs is not None:\n if isinstance(mask_obs, str):\n str_mean_std = (f\"mean of {mask_obs}\", f\"std of {mask_obs}\")\n else:\n str_mean_std = (\"mean with mask\", \"std with mask\")\n mask_obs = _check_mask(adata, mask_obs, \"obs\")\n view_to_actual(adata)\n X = _get_obs_rep(adata, layer=layer, obsm=obsm)\n raise_not_implemented_error_if_backed_type(X, \"scale\")\n X, adata.var[str_mean_std[0]], adata.var[str_mean_std[1]] = scale(\n X,\n zero_center=zero_center,\n max_value=max_value,\n copy=False, # because a copy has already been made, if it were to be made\n return_mean_std=True,\n mask_obs=mask_obs,\n )\n _set_obs_rep(adata, X, layer=layer, obsm=obsm)\n return adata if copy else None\n\n\nfrom __future__ import annotations\n\nfrom operator import truediv\nfrom typing import TYPE_CHECKING\nfrom warnings import warn\n\nimport numpy as np\nfrom scipy.sparse import issparse\n\nfrom .. import logging as logg\nfrom .._compat import DaskArray, old_positionals\nfrom .._utils import axis_mul_or_truediv, axis_sum, view_to_actual\nfrom ..get import _get_obs_rep, _set_obs_rep\n\ntry:\n import dask\n import dask.array as da\nexcept ImportError:\n da = None\n dask = None\n\nif TYPE_CHECKING:\n from collections.abc import Iterable\n from typing import Literal\n\n from anndata import AnnData\n\n\ndef _normalize_data(X, counts, after=None, *, copy: bool = False):\n X = X.copy() if copy else X\n if issubclass(X.dtype.type, (int, np.integer)):\n X = X.astype(np.float32) # TODO: Check if float64 should be used\n if after is None:\n if isinstance(counts, DaskArray):\n\n def nonzero_median(x):\n return np.ma.median(np.ma.masked_array(x, x == 0)).item()\n\n after = da.from_delayed(\n dask.delayed(nonzero_median)(counts),\n shape=(),\n meta=counts._meta,\n dtype=counts.dtype,\n )\n else:\n counts_greater_than_zero = counts[counts > 0]\n after = np.median(counts_greater_than_zero, axis=0)\n counts = counts / after\n return axis_mul_or_truediv(\n X,\n counts,\n op=truediv,\n out=X if isinstance(X, np.ndarray) or issparse(X) else None,\n allow_divide_by_zero=False,\n axis=0,\n )\n\n\n@old_positionals(\n \"target_sum\",\n \"exclude_highly_expressed\",\n \"max_fraction\",\n \"key_added\",\n \"layer\",\n \"layers\",\n \"layer_norm\",\n \"inplace\",\n \"copy\",\n)\ndef normalize_total(\n adata: AnnData,\n *,\n target_sum: float | None = None,\n exclude_highly_expressed: bool = False,\n max_fraction: float = 0.05,\n key_added: str | None = None,\n layer: str | None = None,\n layers: Literal[\"all\"] | Iterable[str] | None = None,\n layer_norm: str | None = None,\n inplace: bool = True,\n copy: bool = False,\n) -> AnnData | dict[str, np.ndarray] | None:\n \"\"\"\\\n Normalize counts per cell.\n\n Normalize each cell by total counts over all genes,\n so that every cell has the same total count after normalization.\n If choosing `target_sum=1e6`, this is CPM normalization.\n\n If `exclude_highly_expressed=True`, very highly expressed genes are excluded\n from the computation of the normalization factor (size factor) for each\n cell. This is meaningful as these can strongly influence the resulting\n normalized values for all other genes :cite:p:`Weinreb2017`.\n\n Similar functions are used, for example, by Seurat :cite:p:`Satija2015`, Cell Ranger\n :cite:p:`Zheng2017` or SPRING :cite:p:`Weinreb2017`.\n\n .. note::\n When used with a :class:`~dask.array.Array` in `adata.X`, this function will have to\n call functions that trigger `.compute()` on the :class:`~dask.array.Array` if `exclude_highly_expressed`\n is `True`, `layer_norm` is not `None`, or if `key_added` is not `None`.\n\n Params\n ------\n adata\n The annotated data matrix of shape `n_obs` × `n_vars`.\n Rows correspond to cells and columns to genes.\n target_sum\n If `None`, after normalization, each observation (cell) has a total\n count equal to the median of total counts for observations (cells)\n before normalization.\n exclude_highly_expressed\n Exclude (very) highly expressed genes for the computation of the\n normalization factor (size factor) for each cell. A gene is considered\n highly expressed, if it has more than `max_fraction` of the total counts\n in at least one cell. The not-excluded genes will sum up to\n `target_sum`. Providing this argument when `adata.X` is a :class:`~dask.array.Array`\n will incur blocking `.compute()` calls on the array.\n max_fraction\n If `exclude_highly_expressed=True`, consider cells as highly expressed\n that have more counts than `max_fraction` of the original total counts\n in at least one cell.\n key_added\n Name of the field in `adata.obs` where the normalization factor is\n stored.\n layer\n Layer to normalize instead of `X`. If `None`, `X` is normalized.\n inplace\n Whether to update `adata` or return dictionary with normalized copies of\n `adata.X` and `adata.layers`.\n copy\n Whether to modify copied input object. Not compatible with inplace=False.\n\n Returns\n -------\n Returns dictionary with normalized copies of `adata.X` and `adata.layers`\n or updates `adata` with normalized version of the original\n `adata.X` and `adata.layers`, depending on `inplace`.\n\n Example\n --------\n >>> import sys\n >>> from anndata import AnnData\n >>> import scanpy as sc\n >>> sc.settings.verbosity = 'info'\n >>> sc.settings.logfile = sys.stdout # for doctests\n >>> np.set_printoptions(precision=2)\n >>> adata = AnnData(np.array([\n ... [3, 3, 3, 6, 6],\n ... [1, 1, 1, 2, 2],\n ... [1, 22, 1, 2, 2],\n ... ], dtype='float32'))\n >>> adata.X\n array([[ 3., 3., 3., 6., 6.],\n [ 1., 1., 1., 2., 2.],\n [ 1., 22., 1., 2., 2.]], dtype=float32)\n >>> X_norm = sc.pp.normalize_total(adata, target_sum=1, inplace=False)['X']\n normalizing counts per cell\n finished (0:00:00)\n >>> X_norm\n array([[0.14, 0.14, 0.14, 0.29, 0.29],\n [0.14, 0.14, 0.14, 0.29, 0.29],\n [0.04, 0.79, 0.04, 0.07, 0.07]], dtype=float32)\n >>> X_norm = sc.pp.normalize_total(\n ... adata, target_sum=1, exclude_highly_expressed=True,\n ... max_fraction=0.2, inplace=False\n ... )['X']\n normalizing counts per cell. The following highly-expressed genes are not considered during normalization factor computation:\n ['1', '3', '4']\n finished (0:00:00)\n >>> X_norm\n array([[ 0.5, 0.5, 0.5, 1. , 1. ],\n [ 0.5, 0.5, 0.5, 1. , 1. ],\n [ 0.5, 11. , 0.5, 1. , 1. ]], dtype=float32)\n \"\"\"\n if copy:\n if not inplace:\n raise ValueError(\"`copy=True` cannot be used with `inplace=False`.\")\n adata = adata.copy()\n\n if max_fraction < 0 or max_fraction > 1:\n raise ValueError(\"Choose max_fraction between 0 and 1.\")\n\n # Deprecated features\n if layers is not None:\n warn(\n FutureWarning(\n \"The `layers` argument is deprecated. Instead, specify individual \"\n \"layers to normalize with `layer`.\"\n )\n )\n if layer_norm is not None:\n warn(\n FutureWarning(\n \"The `layer_norm` argument is deprecated. Specify the target size \"\n \"factor directly with `target_sum`.\"\n )\n )\n\n if layers == \"all\":\n layers = adata.layers.keys()\n elif isinstance(layers, str):\n raise ValueError(\n f\"`layers` needs to be a list of strings or 'all', not {layers!r}\"\n )\n\n view_to_actual(adata)\n\n x = _get_obs_rep(adata, layer=layer)\n\n gene_subset = None\n msg = \"normalizing counts per cell\"\n\n counts_per_cell = axis_sum(x, axis=1)\n if exclude_highly_expressed:\n counts_per_cell = np.ravel(counts_per_cell)\n\n # at least one cell as more than max_fraction of counts per cell\n\n gene_subset = axis_sum((x > counts_per_cell[:, None] * max_fraction), axis=0)\n gene_subset = np.asarray(np.ravel(gene_subset) == 0)\n\n msg += (\n \". The following highly-expressed genes are not considered during \"\n f\"normalization factor computation:\\n{adata.var_names[~gene_subset].tolist()}\"\n )\n counts_per_cell = axis_sum(x[:, gene_subset], axis=1)\n\n start = logg.info(msg)\n counts_per_cell = np.ravel(counts_per_cell)\n\n cell_subset = counts_per_cell > 0\n if not isinstance(cell_subset, DaskArray) and not np.all(cell_subset):\n warn(UserWarning(\"Some cells have zero counts\"))\n\n if inplace:\n if key_added is not None:\n adata.obs[key_added] = counts_per_cell\n _set_obs_rep(\n adata, _normalize_data(x, counts_per_cell, target_sum), layer=layer\n )\n else:\n # not recarray because need to support sparse\n dat = dict(\n X=_normalize_data(x, counts_per_cell, target_sum, copy=True),\n norm_factor=counts_per_cell,\n )\n\n # Deprecated features\n if layer_norm == \"after\":\n after = target_sum\n elif layer_norm == \"X\":\n after = np.median(counts_per_cell[cell_subset])\n elif layer_norm is None:\n after = None\n else:\n raise ValueError('layer_norm should be \"after\", \"X\" or None')\n\n for layer_to_norm in layers if layers is not None else ():\n res = normalize_total(\n adata, layer=layer_to_norm, target_sum=after, inplace=inplace\n )\n if not inplace:\n dat[layer_to_norm] = res[\"X\"]\n\n logg.info(\n \" finished ({time_passed})\",\n time=start,\n )\n if key_added is not None:\n logg.debug(\n f\"and added {key_added!r}, counts per cell before normalization (adata.obs)\"\n )\n\n if copy:\n return adata\n elif not inplace:\n return dat\n\n\nfrom __future__ import annotations\n\nfrom ..neighbors import neighbors\nfrom ._combat import combat\nfrom ._deprecated.highly_variable_genes import filter_genes_dispersion\nfrom ._highly_variable_genes import highly_variable_genes\nfrom ._normalization import normalize_total\nfrom ._pca import pca\nfrom ._qc import calculate_qc_metrics\nfrom ._recipes import recipe_seurat, recipe_weinreb17, recipe_zheng17\nfrom ._scale import scale\nfrom ._scrublet import scrublet, scrublet_simulate_doublets\nfrom ._simple import (\n downsample_counts,\n filter_cells,\n filter_genes,\n log1p,\n normalize_per_cell,\n regress_out,\n sqrt,\n subsample,\n)\n\n__all__ = [\n \"neighbors\",\n \"combat\",\n \"filter_genes_dispersion\",\n \"highly_variable_genes\",\n \"normalize_total\",\n \"pca\",\n \"calculate_qc_metrics\",\n \"recipe_seurat\",\n \"recipe_weinreb17\",\n \"recipe_zheng17\",\n \"scrublet\",\n \"scrublet_simulate_doublets\",\n \"downsample_counts\",\n \"filter_cells\",\n \"filter_genes\",\n \"log1p\",\n \"normalize_per_cell\",\n \"regress_out\",\n \"scale\",\n \"sqrt\",\n \"subsample\",\n]\n\n\n\"\"\"Simple Preprocessing Functions\n\nCompositions of these functions are found in sc.preprocess.recipes.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport warnings\nfrom functools import singledispatch\nfrom typing import TYPE_CHECKING\n\nimport numba\nimport numpy as np\nimport scipy as sp\nfrom anndata import AnnData\nfrom pandas.api.types import CategoricalDtype\nfrom scipy.sparse import csr_matrix, issparse, isspmatrix_csr, spmatrix\nfrom sklearn.utils import check_array, sparsefuncs\n\nfrom .. import logging as logg\nfrom .._compat import old_positionals\nfrom .._settings import settings as sett\nfrom .._utils import (\n _check_array_function_arguments,\n axis_sum,\n is_backed_type,\n raise_not_implemented_error_if_backed_type,\n renamed_arg,\n sanitize_anndata,\n view_to_actual,\n)\nfrom ..get import _get_obs_rep, _set_obs_rep\nfrom ._distributed import materialize_as_ndarray\n\n# install dask if available\ntry:\n import dask.array as da\nexcept ImportError:\n da = None\n\n# backwards compat\nfrom ._deprecated.highly_variable_genes import filter_genes_dispersion # noqa: F401\n\nif TYPE_CHECKING:\n from collections.abc import Collection, Iterable, Sequence\n from numbers import Number\n from typing import Literal\n\n from numpy.typing import NDArray\n\n from .._compat import DaskArray\n from .._utils import AnyRandom\n\n\n@old_positionals(\n \"min_counts\", \"min_genes\", \"max_counts\", \"max_genes\", \"inplace\", \"copy\"\n)\ndef filter_cells(\n data: AnnData | spmatrix | np.ndarray | DaskArray,\n *,\n min_counts: int | None = None,\n min_genes: int | None = None,\n max_counts: int | None = None,\n max_genes: int | None = None,\n inplace: bool = True,\n copy: bool = False,\n) -> AnnData | tuple[np.ndarray, np.ndarray] | None:\n \"\"\"\\\n Filter cell outliers based on counts and numbers of genes expressed.\n\n For instance, only keep cells with at least `min_counts` counts or\n `min_genes` genes expressed. This is to filter measurement outliers,\n i.e. “unreliable” observations.\n\n Only provide one of the optional parameters `min_counts`, `min_genes`,\n `max_counts`, `max_genes` per call.\n\n Parameters\n ----------\n data\n The (annotated) data matrix of shape `n_obs` × `n_vars`.\n Rows correspond to cells and columns to genes.\n min_counts\n Minimum number of counts required for a cell to pass filtering.\n min_genes\n Minimum number of genes expressed required for a cell to pass filtering.\n max_counts\n Maximum number of counts required for a cell to pass filtering.\n max_genes\n Maximum number of genes expressed required for a cell to pass filtering.\n inplace\n Perform computation inplace or return result.\n\n Returns\n -------\n Depending on `inplace`, returns the following arrays or directly subsets\n and annotates the data matrix:\n\n cells_subset\n Boolean index mask that does filtering. `True` means that the\n cell is kept. `False` means the cell is removed.\n number_per_cell\n Depending on what was thresholded (`counts` or `genes`),\n the array stores `n_counts` or `n_cells` per gene.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> adata = sc.datasets.krumsiek11()\n UserWarning: Observation names are not unique. To make them unique, call `.obs_names_make_unique`.\n utils.warn_names_duplicates(\"obs\")\n >>> adata.obs_names_make_unique()\n >>> adata.n_obs\n 640\n >>> adata.var_names.tolist() # doctest: +NORMALIZE_WHITESPACE\n ['Gata2', 'Gata1', 'Fog1', 'EKLF', 'Fli1', 'SCL',\n 'Cebpa', 'Pu.1', 'cJun', 'EgrNab', 'Gfi1']\n >>> # add some true zeros\n >>> adata.X[adata.X < 0.3] = 0\n >>> # simply compute the number of genes per cell\n >>> sc.pp.filter_cells(adata, min_genes=0)\n >>> adata.n_obs\n 640\n >>> int(adata.obs['n_genes'].min())\n 1\n >>> # filter manually\n >>> adata_copy = adata[adata.obs['n_genes'] >= 3]\n >>> adata_copy.n_obs\n 554\n >>> int(adata_copy.obs['n_genes'].min())\n 3\n >>> # actually do some filtering\n >>> sc.pp.filter_cells(adata, min_genes=3)\n >>> adata.n_obs\n 554\n >>> int(adata.obs['n_genes'].min())\n 3\n \"\"\"\n if copy:\n logg.warning(\"`copy` is deprecated, use `inplace` instead.\")\n n_given_options = sum(\n option is not None for option in [min_genes, min_counts, max_genes, max_counts]\n )\n if n_given_options != 1:\n raise ValueError(\n \"Only provide one of the optional parameters `min_counts`, \"\n \"`min_genes`, `max_counts`, `max_genes` per call.\"\n )\n if isinstance(data, AnnData):\n raise_not_implemented_error_if_backed_type(data.X, \"filter_cells\")\n adata = data.copy() if copy else data\n cell_subset, number = materialize_as_ndarray(\n filter_cells(\n adata.X,\n min_counts=min_counts,\n min_genes=min_genes,\n max_counts=max_counts,\n max_genes=max_genes,\n ),\n )\n if not inplace:\n return cell_subset, number\n if min_genes is None and max_genes is None:\n adata.obs[\"n_counts\"] = number\n else:\n adata.obs[\"n_genes\"] = number\n adata._inplace_subset_obs(cell_subset)\n return adata if copy else None\n X = data # proceed with processing the data matrix\n min_number = min_counts if min_genes is None else min_genes\n max_number = max_counts if max_genes is None else max_genes\n number_per_cell = axis_sum(\n X if min_genes is None and max_genes is None else X > 0, axis=1\n )\n if issparse(X):\n number_per_cell = number_per_cell.A1\n if min_number is not None:\n cell_subset = number_per_cell >= min_number\n if max_number is not None:\n cell_subset = number_per_cell <= max_number\n\n s = axis_sum(~cell_subset)\n if s > 0:\n msg = f\"filtered out {s} cells that have \"\n if min_genes is not None or min_counts is not None:\n msg += \"less than \"\n msg += (\n f\"{min_genes} genes expressed\"\n if min_counts is None\n else f\"{min_counts} counts\"\n )\n if max_genes is not None or max_counts is not None:\n msg += \"more than \"\n msg += (\n f\"{max_genes} genes expressed\"\n if max_counts is None\n else f\"{max_counts} counts\"\n )\n logg.info(msg)\n return cell_subset, number_per_cell\n\n\n@old_positionals(\n \"min_counts\", \"min_cells\", \"max_counts\", \"max_cells\", \"inplace\", \"copy\"\n)\ndef filter_genes(\n data: AnnData | spmatrix | np.ndarray | DaskArray,\n *,\n min_counts: int | None = None,\n min_cells: int | None = None,\n max_counts: int | None = None,\n max_cells: int | None = None,\n inplace: bool = True,\n copy: bool = False,\n) -> AnnData | tuple[np.ndarray, np.ndarray] | None:\n \"\"\"\\\n Filter genes based on number of cells or counts.\n\n Keep genes that have at least `min_counts` counts or are expressed in at\n least `min_cells` cells or have at most `max_counts` counts or are expressed\n in at most `max_cells` cells.\n\n Only provide one of the optional parameters `min_counts`, `min_cells`,\n `max_counts`, `max_cells` per call.\n\n Parameters\n ----------\n data\n An annotated data matrix of shape `n_obs` × `n_vars`. Rows correspond\n to cells and columns to genes.\n min_counts\n Minimum number of counts required for a gene to pass filtering.\n min_cells\n Minimum number of cells expressed required for a gene to pass filtering.\n max_counts\n Maximum number of counts required for a gene to pass filtering.\n max_cells\n Maximum number of cells expressed required for a gene to pass filtering.\n inplace\n Perform computation inplace or return result.\n\n Returns\n -------\n Depending on `inplace`, returns the following arrays or directly subsets\n and annotates the data matrix\n\n gene_subset\n Boolean index mask that does filtering. `True` means that the\n gene is kept. `False` means the gene is removed.\n number_per_gene\n Depending on what was thresholded (`counts` or `cells`), the array stores\n `n_counts` or `n_cells` per gene.\n \"\"\"\n if copy:\n logg.warning(\"`copy` is deprecated, use `inplace` instead.\")\n n_given_options = sum(\n option is not None for option in [min_cells, min_counts, max_cells, max_counts]\n )\n if n_given_options != 1:\n raise ValueError(\n \"Only provide one of the optional parameters `min_counts`, \"\n \"`min_cells`, `max_counts`, `max_cells` per call.\"\n )\n\n if isinstance(data, AnnData):\n raise_not_implemented_error_if_backed_type(data.X, \"filter_genes\")\n adata = data.copy() if copy else data\n gene_subset, number = materialize_as_ndarray(\n filter_genes(\n adata.X,\n min_cells=min_cells,\n min_counts=min_counts,\n max_cells=max_cells,\n max_counts=max_counts,\n )\n )\n if not inplace:\n return gene_subset, number\n if min_cells is None and max_cells is None:\n adata.var[\"n_counts\"] = number\n else:\n adata.var[\"n_cells\"] = number\n adata._inplace_subset_var(gene_subset)\n return adata if copy else None\n\n X = data # proceed with processing the data matrix\n min_number = min_counts if min_cells is None else min_cells\n max_number = max_counts if max_cells is None else max_cells\n number_per_gene = axis_sum(\n X if min_cells is None and max_cells is None else X > 0, axis=0\n )\n if issparse(X):\n number_per_gene = number_per_gene.A1\n if min_number is not None:\n gene_subset = number_per_gene >= min_number\n if max_number is not None:\n gene_subset = number_per_gene <= max_number\n\n s = axis_sum(~gene_subset)\n if s > 0:\n msg = f\"filtered out {s} genes that are detected \"\n if min_cells is not None or min_counts is not None:\n msg += \"in less than \"\n msg += (\n f\"{min_cells} cells\" if min_counts is None else f\"{min_counts} counts\"\n )\n if max_cells is not None or max_counts is not None:\n msg += \"in more than \"\n msg += (\n f\"{max_cells} cells\" if max_counts is None else f\"{max_counts} counts\"\n )\n logg.info(msg)\n return gene_subset, number_per_gene\n\n\n@renamed_arg(\"X\", \"data\", pos_0=True)\n@singledispatch\ndef log1p(\n data: AnnData | np.ndarray | spmatrix,\n *,\n base: Number | None = None,\n copy: bool = False,\n chunked: bool | None = None,\n chunk_size: int | None = None,\n layer: str | None = None,\n obsm: str | None = None,\n) -> AnnData | np.ndarray | spmatrix | None:\n \"\"\"\\\n Logarithmize the data matrix.\n\n Computes :math:`X = \\\\log(X + 1)`,\n where :math:`log` denotes the natural logarithm unless a different base is given.\n\n Parameters\n ----------\n data\n The (annotated) data matrix of shape `n_obs` × `n_vars`.\n Rows correspond to cells and columns to genes.\n base\n Base of the logarithm. Natural logarithm is used by default.\n copy\n If an :class:`~anndata.AnnData` is passed, determines whether a copy\n is returned.\n chunked\n Process the data matrix in chunks, which will save memory.\n Applies only to :class:`~anndata.AnnData`.\n chunk_size\n `n_obs` of the chunks to process the data in.\n layer\n Entry of layers to transform.\n obsm\n Entry of obsm to transform.\n\n Returns\n -------\n Returns or updates `data`, depending on `copy`.\n \"\"\"\n _check_array_function_arguments(\n chunked=chunked, chunk_size=chunk_size, layer=layer, obsm=obsm\n )\n return log1p_array(data, copy=copy, base=base)\n\n\n@log1p.register(spmatrix)\ndef log1p_sparse(X: spmatrix, *, base: Number | None = None, copy: bool = False):\n X = check_array(\n X, accept_sparse=(\"csr\", \"csc\"), dtype=(np.float64, np.float32), copy=copy\n )\n X.data = log1p(X.data, copy=False, base=base)\n return X\n\n\n@log1p.register(np.ndarray)\ndef log1p_array(X: np.ndarray, *, base: Number | None = None, copy: bool = False):\n # Can force arrays to be np.ndarrays, but would be useful to not\n # X = check_array(X, dtype=(np.float64, np.float32), ensure_2d=False, copy=copy)\n if copy:\n X = X.astype(float) if not np.issubdtype(X.dtype, np.floating) else X.copy()\n elif not (np.issubdtype(X.dtype, np.floating) or np.issubdtype(X.dtype, complex)):\n X = X.astype(float)\n np.log1p(X, out=X)\n if base is not None:\n np.divide(X, np.log(base), out=X)\n return X\n\n\n@log1p.register(AnnData)\ndef log1p_anndata(\n adata: AnnData,\n *,\n base: Number | None = None,\n copy: bool = False,\n chunked: bool = False,\n chunk_size: int | None = None,\n layer: str | None = None,\n obsm: str | None = None,\n) -> AnnData | None:\n if \"log1p\" in adata.uns:\n logg.warning(\"adata.X seems to be already log-transformed.\")\n\n adata = adata.copy() if copy else adata\n view_to_actual(adata)\n\n if chunked:\n if (layer is not None) or (obsm is not None):\n raise NotImplementedError(\n \"Currently cannot perform chunked operations on arrays not stored in X.\"\n )\n if adata.isbacked and adata.file._filemode != \"r+\":\n raise NotImplementedError(\n \"log1p is not implemented for backed AnnData with backed mode not r+\"\n )\n for chunk, start, end in adata.chunked_X(chunk_size):\n adata.X[start:end] = log1p(chunk, base=base, copy=False)\n else:\n X = _get_obs_rep(adata, layer=layer, obsm=obsm)\n if is_backed_type(X):\n msg = f\"log1p is not implemented for matrices of type {type(X)}\"\n if layer is not None:\n raise NotImplementedError(f\"{msg} from layers\")\n raise NotImplementedError(f\"{msg} without `chunked=True`\")\n X = log1p(X, copy=False, base=base)\n _set_obs_rep(adata, X, layer=layer, obsm=obsm)\n\n adata.uns[\"log1p\"] = {\"base\": base}\n if copy:\n return adata\n\n\n@old_positionals(\"copy\", \"chunked\", \"chunk_size\")\ndef sqrt(\n data: AnnData | spmatrix | np.ndarray,\n *,\n copy: bool = False,\n chunked: bool = False,\n chunk_size: int | None = None,\n) -> AnnData | spmatrix | np.ndarray | None:\n \"\"\"\\\n Square root the data matrix.\n\n Computes :math:`X = \\\\sqrt(X)`.\n\n Parameters\n ----------\n data\n The (annotated) data matrix of shape `n_obs` × `n_vars`.\n Rows correspond to cells and columns to genes.\n copy\n If an :class:`~anndata.AnnData` object is passed,\n determines whether a copy is returned.\n chunked\n Process the data matrix in chunks, which will save memory.\n Applies only to :class:`~anndata.AnnData`.\n chunk_size\n `n_obs` of the chunks to process the data in.\n\n Returns\n -------\n Returns or updates `data`, depending on `copy`.\n \"\"\"\n if isinstance(data, AnnData):\n adata = data.copy() if copy else data\n if chunked:\n for chunk, start, end in adata.chunked_X(chunk_size):\n adata.X[start:end] = sqrt(chunk)\n else:\n adata.X = sqrt(data.X)\n return adata if copy else None\n X = data # proceed with data matrix\n if not issparse(X):\n return np.sqrt(X)\n else:\n return X.sqrt()\n\n\n@old_positionals(\n \"counts_per_cell_after\",\n \"counts_per_cell\",\n \"key_n_counts\",\n \"copy\",\n \"layers\",\n \"use_rep\",\n \"min_counts\",\n)\ndef normalize_per_cell(\n data: AnnData | np.ndarray | spmatrix,\n *,\n counts_per_cell_after: float | None = None,\n counts_per_cell: np.ndarray | None = None,\n key_n_counts: str = \"n_counts\",\n copy: bool = False,\n layers: Literal[\"all\"] | Iterable[str] = (),\n use_rep: Literal[\"after\", \"X\"] | None = None,\n min_counts: int = 1,\n) -> AnnData | np.ndarray | spmatrix | None:\n \"\"\"\\\n Normalize total counts per cell.\n\n .. warning::\n .. deprecated:: 1.3.7\n Use :func:`~scanpy.pp.normalize_total` instead.\n The new function is equivalent to the present\n function, except that\n\n * the new function doesn't filter cells based on `min_counts`,\n use :func:`~scanpy.pp.filter_cells` if filtering is needed.\n * some arguments were renamed\n * `copy` is replaced by `inplace`\n\n Normalize each cell by total counts over all genes, so that every cell has\n the same total count after normalization.\n\n Similar functions are used, for example, by Seurat :cite:p:`Satija2015`, Cell Ranger\n :cite:p:`Zheng2017` or SPRING :cite:p:`Weinreb2017`.\n\n Parameters\n ----------\n data\n The (annotated) data matrix of shape `n_obs` × `n_vars`. Rows correspond\n to cells and columns to genes.\n counts_per_cell_after\n If `None`, after normalization, each cell has a total count equal\n to the median of the *counts_per_cell* before normalization.\n counts_per_cell\n Precomputed counts per cell.\n key_n_counts\n Name of the field in `adata.obs` where the total counts per cell are\n stored.\n copy\n If an :class:`~anndata.AnnData` is passed, determines whether a copy\n is returned.\n min_counts\n Cells with counts less than `min_counts` are filtered out during\n normalization.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an updated `AnnData` object. Sets the following fields:\n\n `adata.X` : :class:`numpy.ndarray` | :class:`scipy.sparse._csr.csr_matrix` (dtype `float`)\n Normalized count data matrix.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> adata = AnnData(np.array([[1, 0], [3, 0], [5, 6]], dtype=np.float32))\n >>> print(adata.X.sum(axis=1))\n [ 1. 3. 11.]\n >>> sc.pp.normalize_per_cell(adata)\n >>> print(adata.obs)\n n_counts\n 0 1.0\n 1 3.0\n 2 11.0\n >>> print(adata.X.sum(axis=1))\n [3. 3. 3.]\n >>> sc.pp.normalize_per_cell(\n ... adata, counts_per_cell_after=1,\n ... key_n_counts='n_counts2',\n ... )\n >>> print(adata.obs)\n n_counts n_counts2\n 0 1.0 3.0\n 1 3.0 3.0\n 2 11.0 3.0\n >>> print(adata.X.sum(axis=1))\n [1. 1. 1.]\n \"\"\"\n if isinstance(data, AnnData):\n start = logg.info(\"normalizing by total count per cell\")\n adata = data.copy() if copy else data\n if counts_per_cell is None:\n cell_subset, counts_per_cell = materialize_as_ndarray(\n filter_cells(adata.X, min_counts=min_counts)\n )\n adata.obs[key_n_counts] = counts_per_cell\n adata._inplace_subset_obs(cell_subset)\n counts_per_cell = counts_per_cell[cell_subset]\n normalize_per_cell(\n adata.X,\n counts_per_cell_after=counts_per_cell_after,\n counts_per_cell=counts_per_cell,\n )\n\n layers = adata.layers.keys() if layers == \"all\" else layers\n if use_rep == \"after\":\n after = counts_per_cell_after\n elif use_rep == \"X\":\n after = np.median(counts_per_cell[cell_subset])\n elif use_rep is None:\n after = None\n else:\n raise ValueError('use_rep should be \"after\", \"X\" or None')\n for layer in layers:\n _subset, counts = filter_cells(adata.layers[layer], min_counts=min_counts)\n temp = normalize_per_cell(adata.layers[layer], after, counts, copy=True)\n adata.layers[layer] = temp\n\n logg.info(\n \" finished ({time_passed}): normalized adata.X and added\\n\"\n f\" {key_n_counts!r}, counts per cell before normalization (adata.obs)\",\n time=start,\n )\n return adata if copy else None\n # proceed with data matrix\n X = data.copy() if copy else data\n if counts_per_cell is None:\n if not copy:\n raise ValueError(\"Can only be run with copy=True\")\n cell_subset, counts_per_cell = filter_cells(X, min_counts=min_counts)\n X = X[cell_subset]\n counts_per_cell = counts_per_cell[cell_subset]\n if counts_per_cell_after is None:\n counts_per_cell_after = np.median(counts_per_cell)\n with warnings.catch_warnings():\n warnings.simplefilter(\"ignore\")\n counts_per_cell += counts_per_cell == 0\n counts_per_cell /= counts_per_cell_after\n if not issparse(X):\n X /= counts_per_cell[:, np.newaxis]\n else:\n sparsefuncs.inplace_row_scale(X, 1 / counts_per_cell)\n return X if copy else None\n\n\n@old_positionals(\"layer\", \"n_jobs\", \"copy\")\ndef regress_out(\n adata: AnnData,\n keys: str | Sequence[str],\n *,\n layer: str | None = None,\n n_jobs: int | None = None,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Regress out (mostly) unwanted sources of variation.\n\n Uses simple linear regression. This is inspired by Seurat's `regressOut`\n function in R :cite:p:`Satija2015`. Note that this function tends to overcorrect\n in certain circumstances as described in :issue:`526`.\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n keys\n Keys for observation annotation on which to regress on.\n layer\n If provided, which element of layers to regress on.\n n_jobs\n Number of jobs for parallel computation.\n `None` means using :attr:`scanpy._settings.ScanpyConfig.n_jobs`.\n copy\n Determines whether a copy of `adata` is returned.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an updated `AnnData` object. Sets the following fields:\n\n `adata.X` | `adata.layers[layer]` : :class:`numpy.ndarray` | :class:`scipy.sparse._csr.csr_matrix` (dtype `float`)\n Corrected count data matrix.\n \"\"\"\n start = logg.info(f\"regressing out {keys}\")\n adata = adata.copy() if copy else adata\n\n sanitize_anndata(adata)\n\n view_to_actual(adata)\n\n if isinstance(keys, str):\n keys = [keys]\n\n X = _get_obs_rep(adata, layer=layer)\n raise_not_implemented_error_if_backed_type(X, \"regress_out\")\n\n if issparse(X):\n logg.info(\" sparse input is densified and may \" \"lead to high memory use\")\n X = X.toarray()\n\n n_jobs = sett.n_jobs if n_jobs is None else n_jobs\n\n # regress on a single categorical variable\n variable_is_categorical = False\n if keys[0] in adata.obs_keys() and isinstance(\n adata.obs[keys[0]].dtype, CategoricalDtype\n ):\n if len(keys) > 1:\n raise ValueError(\n \"If providing categorical variable, \"\n \"only a single one is allowed. For this one \"\n \"we regress on the mean for each category.\"\n )\n logg.debug(\"... regressing on per-gene means within categories\")\n regressors = np.zeros(X.shape, dtype=\"float32\")\n for category in adata.obs[keys[0]].cat.categories:\n mask = (category == adata.obs[keys[0]]).values\n for ix, x in enumerate(X.T):\n regressors[mask, ix] = x[mask].mean()\n variable_is_categorical = True\n # regress on one or several ordinal variables\n else:\n # create data frame with selected keys (if given)\n regressors = adata.obs[keys] if keys else adata.obs.copy()\n\n # add column of ones at index 0 (first column)\n regressors.insert(0, \"ones\", 1.0)\n\n len_chunk = np.ceil(min(1000, X.shape[1]) / n_jobs).astype(int)\n n_chunks = np.ceil(X.shape[1] / len_chunk).astype(int)\n\n tasks = []\n # split the adata.X matrix by columns in chunks of size n_chunk\n # (the last chunk could be of smaller size than the others)\n chunk_list = np.array_split(X, n_chunks, axis=1)\n if variable_is_categorical:\n regressors_chunk = np.array_split(regressors, n_chunks, axis=1)\n for idx, data_chunk in enumerate(chunk_list):\n # each task is a tuple of a data_chunk eg. (adata.X[:,0:100]) and\n # the regressors. This data will be passed to each of the jobs.\n regres = regressors_chunk[idx] if variable_is_categorical else regressors\n tasks.append(tuple((data_chunk, regres, variable_is_categorical)))\n\n from joblib import Parallel, delayed\n\n # TODO: figure out how to test that this doesn't oversubscribe resources\n res = Parallel(n_jobs=n_jobs)(delayed(_regress_out_chunk)(task) for task in tasks)\n\n # res is a list of vectors (each corresponding to a regressed gene column).\n # The transpose is needed to get the matrix in the shape needed\n _set_obs_rep(adata, np.vstack(res).T, layer=layer)\n logg.info(\" finished\", time=start)\n return adata if copy else None\n\n\ndef _regress_out_chunk(data):\n # data is a tuple containing the selected columns from adata.X\n # and the regressors dataFrame\n data_chunk = data[0]\n regressors = data[1]\n variable_is_categorical = data[2]\n\n responses_chunk_list = []\n import statsmodels.api as sm\n from statsmodels.tools.sm_exceptions import PerfectSeparationError\n\n for col_index in range(data_chunk.shape[1]):\n # if all values are identical, the statsmodel.api.GLM throws an error;\n # but then no regression is necessary anyways...\n if not (data_chunk[:, col_index] != data_chunk[0, col_index]).any():\n responses_chunk_list.append(data_chunk[:, col_index])\n continue\n\n if variable_is_categorical:\n regres = np.c_[np.ones(regressors.shape[0]), regressors[:, col_index]]\n else:\n regres = regressors\n try:\n result = sm.GLM(\n data_chunk[:, col_index], regres, family=sm.families.Gaussian()\n ).fit()\n new_column = result.resid_response\n except PerfectSeparationError: # this emulates R's behavior\n logg.warning(\"Encountered PerfectSeparationError, setting to 0 as in R.\")\n new_column = np.zeros(data_chunk.shape[0])\n\n responses_chunk_list.append(new_column)\n\n return np.vstack(responses_chunk_list)\n\n\n@old_positionals(\"n_obs\", \"random_state\", \"copy\")\ndef subsample(\n data: AnnData | np.ndarray | spmatrix,\n fraction: float | None = None,\n *,\n n_obs: int | None = None,\n random_state: AnyRandom = 0,\n copy: bool = False,\n) -> AnnData | tuple[np.ndarray | spmatrix, NDArray[np.int64]] | None:\n \"\"\"\\\n Subsample to a fraction of the number of observations.\n\n Parameters\n ----------\n data\n The (annotated) data matrix of shape `n_obs` × `n_vars`.\n Rows correspond to cells and columns to genes.\n fraction\n Subsample to this `fraction` of the number of observations.\n n_obs\n Subsample to this number of observations.\n random_state\n Random seed to change subsampling.\n copy\n If an :class:`~anndata.AnnData` is passed,\n determines whether a copy is returned.\n\n Returns\n -------\n Returns `X[obs_indices], obs_indices` if data is array-like, otherwise\n subsamples the passed :class:`~anndata.AnnData` (`copy == False`) or\n returns a subsampled copy of it (`copy == True`).\n \"\"\"\n np.random.seed(random_state)\n old_n_obs = data.n_obs if isinstance(data, AnnData) else data.shape[0]\n if n_obs is not None:\n new_n_obs = n_obs\n elif fraction is not None:\n if fraction > 1 or fraction < 0:\n raise ValueError(f\"`fraction` needs to be within [0, 1], not {fraction}\")\n new_n_obs = int(fraction * old_n_obs)\n logg.debug(f\"... subsampled to {new_n_obs} data points\")\n else:\n raise ValueError(\"Either pass `n_obs` or `fraction`.\")\n obs_indices = np.random.choice(old_n_obs, size=new_n_obs, replace=False)\n if isinstance(data, AnnData):\n if data.isbacked:\n if copy:\n return data[obs_indices].to_memory()\n else:\n raise NotImplementedError(\n \"Inplace subsampling is not implemented for backed objects.\"\n )\n else:\n if copy:\n return data[obs_indices].copy()\n else:\n data._inplace_subset_obs(obs_indices)\n else:\n X = data\n return X[obs_indices], obs_indices\n\n\n@renamed_arg(\"target_counts\", \"counts_per_cell\")\ndef downsample_counts(\n adata: AnnData,\n counts_per_cell: int | Collection[int] | None = None,\n total_counts: int | None = None,\n *,\n random_state: AnyRandom = 0,\n replace: bool = False,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Downsample counts from count matrix.\n\n If `counts_per_cell` is specified, each cell will downsampled.\n If `total_counts` is specified, expression matrix will be downsampled to\n contain at most `total_counts`.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n counts_per_cell\n Target total counts per cell. If a cell has more than 'counts_per_cell',\n it will be downsampled to this number. Resulting counts can be specified\n on a per cell basis by passing an array.Should be an integer or integer\n ndarray with same length as number of obs.\n total_counts\n Target total counts. If the count matrix has more than `total_counts`\n it will be downsampled to have this number.\n random_state\n Random seed for subsampling.\n replace\n Whether to sample the counts with replacement.\n copy\n Determines whether a copy of `adata` is returned.\n\n Returns\n -------\n Returns `None` if `copy=False`, else returns an `AnnData` object. Sets the following fields:\n\n `adata.X` : :class:`numpy.ndarray` | :class:`scipy.sparse.spmatrix` (dtype `float`)\n Downsampled counts matrix.\n \"\"\"\n raise_not_implemented_error_if_backed_type(adata.X, \"downsample_counts\")\n # This logic is all dispatch\n total_counts_call = total_counts is not None\n counts_per_cell_call = counts_per_cell is not None\n if total_counts_call is counts_per_cell_call:\n raise ValueError(\n \"Must specify exactly one of `total_counts` or `counts_per_cell`.\"\n )\n if copy:\n adata = adata.copy()\n if total_counts_call:\n adata.X = _downsample_total_counts(adata.X, total_counts, random_state, replace)\n elif counts_per_cell_call:\n adata.X = _downsample_per_cell(adata.X, counts_per_cell, random_state, replace)\n if copy:\n return adata\n\n\ndef _downsample_per_cell(X, counts_per_cell, random_state, replace):\n n_obs = X.shape[0]\n if isinstance(counts_per_cell, int):\n counts_per_cell = np.full(n_obs, counts_per_cell)\n else:\n counts_per_cell = np.asarray(counts_per_cell)\n # np.random.choice needs int arguments in numba code:\n counts_per_cell = counts_per_cell.astype(np.int_, copy=False)\n if not isinstance(counts_per_cell, np.ndarray) or len(counts_per_cell) != n_obs:\n raise ValueError(\n \"If provided, 'counts_per_cell' must be either an integer, or \"\n \"coercible to an `np.ndarray` of length as number of observations\"\n \" by `np.asarray(counts_per_cell)`.\"\n )\n if issparse(X):\n original_type = type(X)\n if not isspmatrix_csr(X):\n X = csr_matrix(X)\n totals = np.ravel(axis_sum(X, axis=1)) # Faster for csr matrix\n under_target = np.nonzero(totals > counts_per_cell)[0]\n rows = np.split(X.data, X.indptr[1:-1])\n for rowidx in under_target:\n row = rows[rowidx]\n _downsample_array(\n row,\n counts_per_cell[rowidx],\n random_state=random_state,\n replace=replace,\n inplace=True,\n )\n X.eliminate_zeros()\n if original_type is not csr_matrix: # Put it back\n X = original_type(X)\n else:\n totals = np.ravel(axis_sum(X, axis=1))\n under_target = np.nonzero(totals > counts_per_cell)[0]\n for rowidx in under_target:\n row = X[rowidx, :]\n _downsample_array(\n row,\n counts_per_cell[rowidx],\n random_state=random_state,\n replace=replace,\n inplace=True,\n )\n return X\n\n\ndef _downsample_total_counts(X, total_counts, random_state, replace):\n total_counts = int(total_counts)\n total = X.sum()\n if total < total_counts:\n return X\n if issparse(X):\n original_type = type(X)\n if not isspmatrix_csr(X):\n X = csr_matrix(X)\n _downsample_array(\n X.data,\n total_counts,\n random_state=random_state,\n replace=replace,\n inplace=True,\n )\n X.eliminate_zeros()\n if original_type is not csr_matrix:\n X = original_type(X)\n else:\n v = X.reshape(np.multiply(*X.shape))\n _downsample_array(\n v, total_counts, random_state=random_state, replace=replace, inplace=True\n )\n return X\n\n\n@numba.njit(cache=True)\ndef _downsample_array(\n col: np.ndarray,\n target: int,\n *,\n random_state: AnyRandom = 0,\n replace: bool = True,\n inplace: bool = False,\n):\n \"\"\"\\\n Evenly reduce counts in cell to target amount.\n\n This is an internal function and has some restrictions:\n\n * total counts in cell must be less than target\n \"\"\"\n np.random.seed(random_state)\n cumcounts = col.cumsum()\n if inplace:\n col[:] = 0\n else:\n col = np.zeros_like(col)\n total = np.int_(cumcounts[-1])\n sample = np.random.choice(total, target, replace=replace)\n sample.sort()\n geneptr = 0\n for count in sample:\n while count >= cumcounts[geneptr]:\n geneptr += 1\n col[geneptr] += 1\n return col\n\n\n# --------------------------------------------------------------------------------\n# Helper Functions\n# --------------------------------------------------------------------------------\n\n\ndef _pca_fallback(data, n_comps=2):\n # mean center the data\n data -= data.mean(axis=0)\n # calculate the covariance matrix\n C = np.cov(data, rowvar=False)\n # calculate eigenvectors & eigenvalues of the covariance matrix\n # use 'eigh' rather than 'eig' since C is symmetric,\n # the performance gain is substantial\n # evals, evecs = np.linalg.eigh(C)\n evals, evecs = sp.sparse.linalg.eigsh(C, k=n_comps)\n # sort eigenvalues in decreasing order\n idcs = np.argsort(evals)[::-1]\n evecs = evecs[:, idcs]\n evals = evals[idcs]\n # select the first n eigenvectors (n is desired dimension\n # of rescaled data array, or n_comps)\n evecs = evecs[:, :n_comps]\n # project data points on eigenvectors\n return np.dot(evecs.T, data.T).T\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nfrom scipy import sparse\n\nfrom scanpy.preprocessing._utils import _get_mean_var\n\nfrom .sparse_utils import sparse_multiply, sparse_zscore\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from ..._utils import AnyRandom\n from .core import Scrublet\n\n\ndef mean_center(self: Scrublet) -> None:\n gene_means = self._counts_obs_norm.mean(0)\n self._counts_obs_norm = sparse.csc_matrix(self._counts_obs_norm - gene_means)\n if self._counts_sim_norm is not None:\n self._counts_sim_norm = sparse.csc_matrix(self._counts_sim_norm - gene_means)\n\n\ndef normalize_variance(self: Scrublet) -> None:\n _, gene_vars = _get_mean_var(self._counts_obs_norm, axis=0)\n gene_stdevs = np.sqrt(gene_vars)\n self._counts_obs_norm = sparse_multiply(self._counts_obs_norm.T, 1 / gene_stdevs).T\n if self._counts_sim_norm is not None:\n self._counts_sim_norm = sparse_multiply(\n self._counts_sim_norm.T, 1 / gene_stdevs\n ).T\n\n\ndef zscore(self: Scrublet) -> None:\n gene_means, gene_vars = _get_mean_var(self._counts_obs_norm, axis=0)\n gene_stdevs = np.sqrt(gene_vars)\n self._counts_obs_norm = sparse_zscore(\n self._counts_obs_norm, gene_mean=gene_means, gene_stdev=gene_stdevs\n )\n if self._counts_sim_norm is not None:\n self._counts_sim_norm = sparse_zscore(\n self._counts_sim_norm, gene_mean=gene_means, gene_stdev=gene_stdevs\n )\n\n\ndef truncated_svd(\n self: Scrublet,\n n_prin_comps: int = 30,\n *,\n random_state: AnyRandom = 0,\n algorithm: Literal[\"arpack\", \"randomized\"] = \"arpack\",\n) -> None:\n if self._counts_sim_norm is None:\n raise RuntimeError(\"_counts_sim_norm is not set\")\n from sklearn.decomposition import TruncatedSVD\n\n svd = TruncatedSVD(\n n_components=n_prin_comps, random_state=random_state, algorithm=algorithm\n ).fit(self._counts_obs_norm)\n self.set_manifold(\n svd.transform(self._counts_obs_norm), svd.transform(self._counts_sim_norm)\n )\n\n\ndef pca(\n self: Scrublet,\n n_prin_comps: int = 50,\n *,\n random_state: AnyRandom = 0,\n svd_solver: Literal[\"auto\", \"full\", \"arpack\", \"randomized\"] = \"arpack\",\n) -> None:\n if self._counts_sim_norm is None:\n raise RuntimeError(\"_counts_sim_norm is not set\")\n from sklearn.decomposition import PCA\n\n X_obs = self._counts_obs_norm.toarray()\n X_sim = self._counts_sim_norm.toarray()\n\n pca = PCA(\n n_components=n_prin_comps, random_state=random_state, svd_solver=svd_solver\n ).fit(X_obs)\n self.set_manifold(pca.transform(X_obs), pca.transform(X_sim))\n\n\nfrom __future__ import annotations\n\nimport sys\nfrom dataclasses import InitVar, dataclass, field\nfrom typing import TYPE_CHECKING, cast\n\nimport numpy as np\nimport pandas as pd\nfrom anndata import AnnData, concat\nfrom scipy import sparse\n\nfrom ... import logging as logg\nfrom ..._utils import get_random_state\nfrom ...neighbors import (\n Neighbors,\n _get_indices_distances_from_sparse_matrix,\n)\nfrom .._utils import sample_comb\nfrom .sparse_utils import subsample_counts\n\nif TYPE_CHECKING:\n from numpy.random import RandomState\n from numpy.typing import NDArray\n\n from ..._utils import AnyRandom\n from ...neighbors import _Metric, _MetricFn\n\n__all__ = [\"Scrublet\"]\n\n\nif sys.version_info > (3, 10):\n kw_only = lambda yes: {\"kw_only\": yes} # noqa: E731\nelse:\n kw_only = lambda _: {} # noqa: E731\n\n\n@dataclass(**kw_only(True)) # noqa: FBT003\nclass Scrublet:\n \"\"\"\\\n Initialize Scrublet object with counts matrix and doublet prediction parameters\n\n Parameters\n ----------\n counts_obs\n Matrix with shape (n_cells, n_genes) containing raw (unnormalized)\n UMI-based transcript counts.\n Converted into a :class:`scipy.sparse.csc_matrix`.\n\n total_counts_obs\n Array with shape (n_cells,) of total UMI counts per cell.\n If `None`, this is calculated as the row sums of `counts_obs`.\n\n sim_doublet_ratio\n Number of doublets to simulate relative to the number of observed\n transcriptomes.\n\n n_neighbors\n Number of neighbors used to construct the KNN graph of observed\n transcriptomes and simulated doublets.\n If `None`, this is set to round(0.5 * sqrt(n_cells))\n\n expected_doublet_rate\n The estimated doublet rate for the experiment.\n\n stdev_doublet_rate\n Uncertainty in the expected doublet rate.\n\n random_state\n Random state for doublet simulation, approximate\n nearest neighbor search, and PCA/TruncatedSVD.\n \"\"\"\n\n # init fields\n\n counts_obs: InitVar[sparse.csr_matrix | sparse.csc_matrix | NDArray[np.integer]] = (\n field(**kw_only(False)) # noqa: FBT003\n )\n total_counts_obs: InitVar[NDArray[np.integer] | None] = None\n sim_doublet_ratio: float = 2.0\n n_neighbors: InitVar[int | None] = None\n expected_doublet_rate: float = 0.1\n stdev_doublet_rate: float = 0.02\n random_state: InitVar[AnyRandom] = 0\n\n # private fields\n\n _n_neighbors: int = field(init=False, repr=False)\n _random_state: RandomState = field(init=False, repr=False)\n\n _counts_obs: sparse.csc_matrix = field(init=False, repr=False)\n _total_counts_obs: NDArray[np.integer] = field(init=False, repr=False)\n _counts_obs_norm: sparse.csr_matrix | sparse.csc_matrix = field(\n init=False, repr=False\n )\n\n _counts_sim: sparse.csr_matrix | sparse.csc_matrix = field(init=False, repr=False)\n _total_counts_sim: NDArray[np.integer] = field(init=False, repr=False)\n _counts_sim_norm: sparse.csr_matrix | sparse.csc_matrix | None = field(\n default=None, init=False, repr=False\n )\n\n # Fields set by methods\n\n predicted_doublets_: NDArray[np.bool_] | None = field(init=False)\n \"\"\"(shape: n_cells)\n Boolean mask of predicted doublets in the observed transcriptomes.\n \"\"\"\n\n doublet_scores_obs_: NDArray[np.float64] = field(init=False)\n \"\"\"(shape: n_cells)\n Doublet scores for observed transcriptomes.\n \"\"\"\n\n doublet_scores_sim_: NDArray[np.float64] = field(init=False)\n \"\"\"(shape: n_doublets)\n Doublet scores for simulated doublets.\n \"\"\"\n\n doublet_errors_obs_: NDArray[np.float64] = field(init=False)\n \"\"\"(shape: n_cells)\n Standard error in the doublet scores for observed transcriptomes.\n \"\"\"\n\n doublet_errors_sim_: NDArray[np.float64] = field(init=False)\n \"\"\"(shape: n_doublets)\n Standard error in the doublet scores for simulated doublets.\n \"\"\"\n\n threshold_: float = field(init=False)\n \"\"\"Doublet score threshold for calling a transcriptome a doublet.\"\"\"\n\n z_scores_: NDArray[np.float64] = field(init=False)\n \"\"\"(shape: n_cells)\n Z-score conveying confidence in doublet calls.\n Z = `(doublet_score_obs_ - threhsold_) / doublet_errors_obs_`\n \"\"\"\n\n detected_doublet_rate_: float = field(init=False)\n \"\"\"Fraction of observed transcriptomes that have been called doublets.\"\"\"\n\n detectable_doublet_fraction_: float = field(init=False)\n \"\"\"Estimated fraction of doublets that are detectable, i.e.,\n fraction of simulated doublets with doublet scores above `threshold_`\n \"\"\"\n\n overall_doublet_rate_: float = field(init=False)\n \"\"\"Estimated overall doublet rate,\n `detected_doublet_rate_ / detectable_doublet_fraction_`.\n Should agree (roughly) with `expected_doublet_rate`.\n \"\"\"\n\n manifold_obs_: NDArray[np.float64] = field(init=False)\n \"\"\"(shape: n_cells × n_features)\n The single-cell \"manifold\" coordinates (e.g., PCA coordinates)\n for observed transcriptomes. Nearest neighbors are found using\n the union of `manifold_obs_` and `manifold_sim_` (see below).\n \"\"\"\n\n manifold_sim_: NDArray[np.float64] = field(init=False)\n \"\"\"shape (n_doublets × n_features)\n The single-cell \"manifold\" coordinates (e.g., PCA coordinates)\n for simulated doublets. Nearest neighbors are found using\n the union of `manifold_obs_` (see above) and `manifold_sim_`.\n \"\"\"\n\n doublet_parents_: NDArray[np.intp] = field(init=False)\n \"\"\"(shape: n_doublets × 2)\n Indices of the observed transcriptomes used to generate the\n simulated doublets.\n \"\"\"\n\n doublet_neighbor_parents_: list[NDArray[np.intp]] = field(init=False)\n \"\"\"(length: n_cells)\n A list of arrays of the indices of the doublet neighbors of\n each observed transcriptome (the ith entry is an array of\n the doublet neighbors of transcriptome i).\n \"\"\"\n\n def __post_init__(\n self,\n counts_obs: sparse.csr_matrix | sparse.csc_matrix | NDArray[np.integer],\n total_counts_obs: NDArray[np.integer] | None,\n n_neighbors: int | None,\n random_state: AnyRandom,\n ) -> None:\n self._counts_obs = sparse.csc_matrix(counts_obs)\n self._total_counts_obs = (\n np.asarray(self._counts_obs.sum(1)).squeeze()\n if total_counts_obs is None\n else total_counts_obs\n )\n self._n_neighbors = (\n int(round(0.5 * np.sqrt(self._counts_obs.shape[0])))\n if n_neighbors is None\n else n_neighbors\n )\n self._random_state = get_random_state(random_state)\n\n def simulate_doublets(\n self,\n *,\n sim_doublet_ratio: float | None = None,\n synthetic_doublet_umi_subsampling: float = 1.0,\n ) -> None:\n \"\"\"Simulate doublets by adding the counts of random observed transcriptome pairs.\n\n Arguments\n ---------\n sim_doublet_ratio\n Number of doublets to simulate relative to the number of observed\n transcriptomes. If `None`, self.sim_doublet_ratio is used.\n\n synthetic_doublet_umi_subsampling\n Rate for sampling UMIs when creating synthetic doublets.\n If 1.0, each doublet is created by simply adding the UMIs from two randomly\n sampled observed transcriptomes.\n For values less than 1, the UMI counts are added and then randomly sampled\n at the specified rate.\n\n Sets\n ----\n doublet_parents_\n \"\"\"\n\n if sim_doublet_ratio is None:\n sim_doublet_ratio = self.sim_doublet_ratio\n else:\n self.sim_doublet_ratio = sim_doublet_ratio\n\n n_obs = self._counts_obs.shape[0]\n n_sim = int(n_obs * sim_doublet_ratio)\n\n pair_ix = sample_comb((n_obs, n_obs), n_sim, random_state=self._random_state)\n\n E1 = cast(sparse.csc_matrix, self._counts_obs[pair_ix[:, 0], :])\n E2 = cast(sparse.csc_matrix, self._counts_obs[pair_ix[:, 1], :])\n tots1 = self._total_counts_obs[pair_ix[:, 0]]\n tots2 = self._total_counts_obs[pair_ix[:, 1]]\n if synthetic_doublet_umi_subsampling < 1:\n self._counts_sim, self._total_counts_sim = subsample_counts(\n E1 + E2,\n rate=synthetic_doublet_umi_subsampling,\n original_totals=tots1 + tots2,\n random_seed=self._random_state,\n )\n else:\n self._counts_sim = E1 + E2\n self._total_counts_sim = tots1 + tots2\n self.doublet_parents_ = pair_ix\n\n def set_manifold(\n self, manifold_obs: NDArray[np.float64], manifold_sim: NDArray[np.float64]\n ) -> None:\n \"\"\"\\\n Set the manifold coordinates used in k-nearest-neighbor graph construction\n\n Arguments\n ---------\n manifold_obs\n (shape: n_cells × n_features)\n The single-cell \"manifold\" coordinates (e.g., PCA coordinates)\n for observed transcriptomes. Nearest neighbors are found using\n the union of `manifold_obs` and `manifold_sim` (see below).\n\n manifold_sim\n (shape: n_doublets × n_features)\n The single-cell \"manifold\" coordinates (e.g., PCA coordinates)\n for simulated doublets. Nearest neighbors are found using\n the union of `manifold_obs` (see above) and `manifold_sim`.\n\n Sets\n ----\n manifold_obs_, manifold_sim_,\n \"\"\"\n\n self.manifold_obs_ = manifold_obs\n self.manifold_sim_ = manifold_sim\n\n def calculate_doublet_scores(\n self,\n *,\n use_approx_neighbors: bool | None = None,\n distance_metric: _Metric | _MetricFn = \"euclidean\",\n get_doublet_neighbor_parents: bool = False,\n ) -> NDArray[np.float64]:\n \"\"\"\\\n Calculate doublet scores for observed transcriptomes and simulated doublets\n\n Requires that manifold_obs_ and manifold_sim_ have already been set.\n\n Arguments\n ---------\n use_approx_neighbors\n Use approximate nearest neighbor method (annoy) for the KNN\n classifier.\n\n distance_metric\n Distance metric used when finding nearest neighbors. For list of\n valid values, see the documentation for annoy (if `use_approx_neighbors`\n is True) or sklearn.neighbors.NearestNeighbors (if `use_approx_neighbors`\n is False).\n\n get_doublet_neighbor_parents\n If True, return the parent transcriptomes that generated the\n doublet neighbors of each observed transcriptome. This information can\n be used to infer the cell states that generated a given\n doublet state.\n\n Sets\n ----\n doublet_scores_obs_, doublet_scores_sim_,\n doublet_errors_obs_, doublet_errors_sim_,\n doublet_neighbor_parents_\n \"\"\"\n\n self._nearest_neighbor_classifier(\n k=self._n_neighbors,\n exp_doub_rate=self.expected_doublet_rate,\n stdev_doub_rate=self.stdev_doublet_rate,\n use_approx_neighbors=use_approx_neighbors,\n distance_metric=distance_metric,\n get_neighbor_parents=get_doublet_neighbor_parents,\n )\n return self.doublet_scores_obs_\n\n def _nearest_neighbor_classifier(\n self,\n k: int = 40,\n *,\n use_approx_neighbors: bool | None = None,\n distance_metric: _Metric | _MetricFn = \"euclidean\",\n exp_doub_rate: float = 0.1,\n stdev_doub_rate: float = 0.03,\n get_neighbor_parents: bool = False,\n ) -> None:\n adatas = [\n AnnData(\n (arr := getattr(self, f\"manifold_{n}_\")),\n obs=dict(\n obs_names=pd.RangeIndex(arr.shape[0]).astype(\"string\") + n,\n doub_labels=n,\n ),\n )\n for n in [\"obs\", \"sim\"]\n ]\n manifold = concat(adatas)\n\n n_obs: int = (manifold.obs[\"doub_labels\"] == \"obs\").sum()\n n_sim: int = (manifold.obs[\"doub_labels\"] == \"sim\").sum()\n\n # Adjust k (number of nearest neighbors) based on the ratio of simulated to observed cells\n k_adj = int(round(k * (1 + n_sim / float(n_obs))))\n\n # Find k_adj nearest neighbors\n knn = Neighbors(manifold)\n transformer = None\n if use_approx_neighbors is not None:\n transformer = \"pynndescent\" if use_approx_neighbors else \"sklearn\"\n knn.compute_neighbors(\n k_adj,\n metric=distance_metric,\n knn=True,\n transformer=transformer,\n method=None,\n random_state=self._random_state,\n )\n neighbors, _ = _get_indices_distances_from_sparse_matrix(knn.distances, k_adj)\n if use_approx_neighbors:\n neighbors = neighbors[:, 1:]\n # Calculate doublet score based on ratio of simulated cell neighbors vs. observed cell neighbors\n doub_neigh_mask: NDArray[np.bool_] = (\n manifold.obs[\"doub_labels\"].to_numpy()[neighbors] == \"sim\"\n )\n n_sim_neigh: NDArray[np.int64] = doub_neigh_mask.sum(axis=1)\n\n rho = exp_doub_rate\n r = n_sim / float(n_obs)\n nd = n_sim_neigh.astype(np.float64)\n N = float(k_adj)\n\n # Bayesian\n q = (nd + 1) / (N + 2)\n Ld = q * rho / r / (1 - rho - q * (1 - rho - rho / r))\n\n se_q = np.sqrt(q * (1 - q) / (N + 3))\n se_rho = stdev_doub_rate\n\n se_Ld = (\n q\n * rho\n / r\n / (1 - rho - q * (1 - rho - rho / r)) ** 2\n * np.sqrt((se_q / q * (1 - rho)) ** 2 + (se_rho / rho * (1 - q)) ** 2)\n )\n\n self.doublet_scores_obs_ = Ld[manifold.obs[\"doub_labels\"] == \"obs\"]\n self.doublet_scores_sim_ = Ld[manifold.obs[\"doub_labels\"] == \"sim\"]\n self.doublet_errors_obs_ = se_Ld[manifold.obs[\"doub_labels\"] == \"obs\"]\n self.doublet_errors_sim_ = se_Ld[manifold.obs[\"doub_labels\"] == \"sim\"]\n\n # get parents of doublet neighbors, if requested\n neighbor_parents = None\n if get_neighbor_parents:\n parent_cells = self.doublet_parents_\n neighbors = neighbors - n_obs\n neighbor_parents = []\n for iCell in range(n_obs):\n this_doub_neigh = neighbors[iCell, :][neighbors[iCell, :] > -1]\n if len(this_doub_neigh) > 0:\n this_doub_neigh_parents = np.unique(\n parent_cells[this_doub_neigh, :].flatten()\n )\n neighbor_parents.append(this_doub_neigh_parents)\n else:\n neighbor_parents.append(np.array([], dtype=np.intp))\n self.doublet_neighbor_parents_ = neighbor_parents\n\n def call_doublets(\n self, *, threshold: float | None = None, verbose: bool = True\n ) -> NDArray[np.bool_] | None:\n \"\"\"\\\n Call trancriptomes as doublets or singlets\n\n Arguments\n ---------\n threshold\n Doublet score threshold for calling a transcriptome\n a doublet. If `None`, this is set automatically by looking\n for the minimum between the two modes of the `doublet_scores_sim_`\n histogram. It is best practice to check the threshold visually\n using the `doublet_scores_sim_` histogram and/or based on\n co-localization of predicted doublets in a 2-D embedding.\n\n verbose\n If True, log summary statistics.\n\n Sets\n ----\n predicted_doublets_, z_scores_, threshold_,\n detected_doublet_rate_, detectable_doublet_fraction,\n overall_doublet_rate_\n \"\"\"\n\n if threshold is None:\n # automatic threshold detection\n # http://scikit-image.org/docs/dev/api/skimage.filters.html\n from skimage.filters import threshold_minimum\n\n try:\n threshold = cast(float, threshold_minimum(self.doublet_scores_sim_))\n if verbose:\n logg.info(\n f\"Automatically set threshold at doublet score = {threshold:.2f}\"\n )\n except Exception:\n self.predicted_doublets_ = None\n if verbose:\n logg.warning(\n \"Failed to automatically identify doublet score threshold. \"\n \"Run `call_doublets` with user-specified threshold.\"\n )\n return self.predicted_doublets_\n\n Ld_obs = self.doublet_scores_obs_\n Ld_sim = self.doublet_scores_sim_\n se_obs = self.doublet_errors_obs_\n Z = (Ld_obs - threshold) / se_obs\n self.predicted_doublets_ = Ld_obs > threshold\n self.z_scores_ = Z\n self.threshold_ = threshold\n self.detected_doublet_rate_ = (Ld_obs > threshold).sum() / float(len(Ld_obs))\n self.detectable_doublet_fraction_ = (Ld_sim > threshold).sum() / float(\n len(Ld_sim)\n )\n self.overall_doublet_rate_ = (\n self.detected_doublet_rate_ / self.detectable_doublet_fraction_\n )\n\n if verbose:\n logg.info(\n f\"Detected doublet rate = {100 * self.detected_doublet_rate_:.1f}%\\n\"\n f\"Estimated detectable doublet fraction = {100 * self.detectable_doublet_fraction_:.1f}%\\n\"\n \"Overall doublet rate:\\n\"\n f\"\\tExpected = {100 * self.expected_doublet_rate:.1f}%\\n\"\n f\"\\tEstimated = {100 * self.overall_doublet_rate_:.1f}%\"\n )\n\n return self.predicted_doublets_\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom anndata import AnnData\nfrom scipy import sparse\n\nfrom ... import logging as logg\nfrom ... import preprocessing as pp\nfrom ..._compat import old_positionals\nfrom ...get import _get_obs_rep\nfrom . import pipeline\nfrom .core import Scrublet\n\nif TYPE_CHECKING:\n from ..._utils import AnyRandom\n from ...neighbors import _Metric, _MetricFn\n\n\n@old_positionals(\n \"batch_key\",\n \"sim_doublet_ratio\",\n \"expected_doublet_rate\",\n \"stdev_doublet_rate\",\n \"synthetic_doublet_umi_subsampling\",\n \"knn_dist_metric\",\n \"normalize_variance\",\n \"log_transform\",\n \"mean_center\",\n \"n_prin_comps\",\n \"use_approx_neighbors\",\n \"get_doublet_neighbor_parents\",\n \"n_neighbors\",\n \"threshold\",\n \"verbose\",\n \"copy\",\n \"random_state\",\n)\ndef scrublet(\n adata: AnnData,\n adata_sim: AnnData | None = None,\n *,\n batch_key: str | None = None,\n sim_doublet_ratio: float = 2.0,\n expected_doublet_rate: float = 0.05,\n stdev_doublet_rate: float = 0.02,\n synthetic_doublet_umi_subsampling: float = 1.0,\n knn_dist_metric: _Metric | _MetricFn = \"euclidean\",\n normalize_variance: bool = True,\n log_transform: bool = False,\n mean_center: bool = True,\n n_prin_comps: int = 30,\n use_approx_neighbors: bool | None = None,\n get_doublet_neighbor_parents: bool = False,\n n_neighbors: int | None = None,\n threshold: float | None = None,\n verbose: bool = True,\n copy: bool = False,\n random_state: AnyRandom = 0,\n) -> AnnData | None:\n \"\"\"\\\n Predict doublets using Scrublet :cite:p:`Wolock2019`.\n\n Predict cell doublets using a nearest-neighbor classifier of observed\n transcriptomes and simulated doublets. Works best if the input is a raw\n (unnormalized) counts matrix from a single sample or a collection of\n similar samples from the same experiment.\n This function is a wrapper around functions that pre-process using Scanpy\n and directly call functions of Scrublet(). You may also undertake your own\n preprocessing, simulate doublets with\n :func:`~scanpy.pp.scrublet_simulate_doublets`, and run the core scrublet\n function :func:`~scanpy.pp.scrublet` with ``adata_sim`` set.\n\n Parameters\n ----------\n adata\n The annotated data matrix of shape ``n_obs`` × ``n_vars``. Rows\n correspond to cells and columns to genes. Expected to be un-normalised\n where adata_sim is not supplied, in which case doublets will be\n simulated and pre-processing applied to both objects. If adata_sim is\n supplied, this should be the observed transcriptomes processed\n consistently (filtering, transform, normalisaton, hvg) with adata_sim.\n adata_sim\n (Advanced use case) Optional annData object generated by\n :func:`~scanpy.pp.scrublet_simulate_doublets`, with same number of vars\n as adata. This should have been built from adata_obs after\n filtering genes and cells and selcting highly-variable genes.\n batch_key\n Optional :attr:`~anndata.AnnData.obs` column name discriminating between batches.\n sim_doublet_ratio\n Number of doublets to simulate relative to the number of observed\n transcriptomes.\n expected_doublet_rate\n Where adata_sim not suplied, the estimated doublet rate for the\n experiment.\n stdev_doublet_rate\n Where adata_sim not suplied, uncertainty in the expected doublet rate.\n synthetic_doublet_umi_subsampling\n Where adata_sim not suplied, rate for sampling UMIs when creating\n synthetic doublets. If 1.0, each doublet is created by simply adding\n the UMI counts from two randomly sampled observed transcriptomes. For\n values less than 1, the UMI counts are added and then randomly sampled\n at the specified rate.\n knn_dist_metric\n Distance metric used when finding nearest neighbors. For list of\n valid values, see the documentation for annoy (if `use_approx_neighbors`\n is True) or sklearn.neighbors.NearestNeighbors (if `use_approx_neighbors`\n is False).\n normalize_variance\n If True, normalize the data such that each gene has a variance of 1.\n :class:`sklearn.decomposition.TruncatedSVD` will be used for dimensionality\n reduction, unless `mean_center` is True.\n log_transform\n Whether to use :func:`~scanpy.pp.log1p` to log-transform the data\n prior to PCA.\n mean_center\n If True, center the data such that each gene has a mean of 0.\n :class:`sklearn.decomposition.PCA` will be used for dimensionality\n reduction.\n n_prin_comps\n Number of principal components used to embed the transcriptomes prior\n to k-nearest-neighbor graph construction.\n use_approx_neighbors\n Use approximate nearest neighbor method (annoy) for the KNN\n classifier.\n get_doublet_neighbor_parents\n If True, return (in .uns) the parent transcriptomes that generated the\n doublet neighbors of each observed transcriptome. This information can\n be used to infer the cell states that generated a given doublet state.\n n_neighbors\n Number of neighbors used to construct the KNN graph of observed\n transcriptomes and simulated doublets. If ``None``, this is\n automatically set to ``np.round(0.5 * np.sqrt(n_obs))``.\n threshold\n Doublet score threshold for calling a transcriptome a doublet. If\n `None`, this is set automatically by looking for the minimum between\n the two modes of the `doublet_scores_sim_` histogram. It is best\n practice to check the threshold visually using the\n `doublet_scores_sim_` histogram and/or based on co-localization of\n predicted doublets in a 2-D embedding.\n verbose\n If :data:`True`, log progress updates.\n copy\n If :data:`True`, return a copy of the input ``adata`` with Scrublet results\n added. Otherwise, Scrublet results are added in place.\n random_state\n Initial state for doublet simulation and nearest neighbors.\n\n Returns\n -------\n if ``copy=True`` it returns or else adds fields to ``adata``. Those fields:\n\n ``.obs['doublet_score']``\n Doublet scores for each observed transcriptome\n\n ``.obs['predicted_doublet']``\n Boolean indicating predicted doublet status\n\n ``.uns['scrublet']['doublet_scores_sim']``\n Doublet scores for each simulated doublet transcriptome\n\n ``.uns['scrublet']['doublet_parents']``\n Pairs of ``.obs_names`` used to generate each simulated doublet\n transcriptome\n\n ``.uns['scrublet']['parameters']``\n Dictionary of Scrublet parameters\n\n See also\n --------\n :func:`~scanpy.pp.scrublet_simulate_doublets`: Run Scrublet's doublet\n simulation separately for advanced usage.\n :func:`~scanpy.pl.scrublet_score_distribution`: Plot histogram of doublet\n scores for observed transcriptomes and simulated doublets.\n \"\"\"\n\n if copy:\n adata = adata.copy()\n\n start = logg.info(\"Running Scrublet\")\n\n adata_obs = adata.copy()\n\n def _run_scrublet(ad_obs: AnnData, ad_sim: AnnData | None = None):\n # With no adata_sim we assume the regular use case, starting with raw\n # counts and simulating doublets\n\n if ad_sim is None:\n pp.filter_genes(ad_obs, min_cells=3)\n pp.filter_cells(ad_obs, min_genes=3)\n\n # Doublet simulation will be based on the un-normalised counts, but on the\n # selection of genes following normalisation and variability filtering. So\n # we need to save the raw and subset at the same time.\n\n ad_obs.layers[\"raw\"] = ad_obs.X.copy()\n pp.normalize_total(ad_obs)\n\n # HVG process needs log'd data.\n ad_obs.layers[\"log1p\"] = ad_obs.X.copy()\n pp.log1p(ad_obs, layer=\"log1p\")\n pp.highly_variable_genes(ad_obs, layer=\"log1p\")\n del ad_obs.layers[\"log1p\"]\n ad_obs = ad_obs[:, ad_obs.var[\"highly_variable\"]].copy()\n\n # Simulate the doublets based on the raw expressions from the normalised\n # and filtered object.\n\n ad_sim = scrublet_simulate_doublets(\n ad_obs,\n layer=\"raw\",\n sim_doublet_ratio=sim_doublet_ratio,\n synthetic_doublet_umi_subsampling=synthetic_doublet_umi_subsampling,\n random_seed=random_state,\n )\n del ad_obs.layers[\"raw\"]\n if log_transform:\n pp.log1p(ad_obs)\n pp.log1p(ad_sim)\n\n # Now normalise simulated and observed in the same way\n\n pp.normalize_total(ad_obs, target_sum=1e6)\n pp.normalize_total(ad_sim, target_sum=1e6)\n\n ad_obs = _scrublet_call_doublets(\n adata_obs=ad_obs,\n adata_sim=ad_sim,\n n_neighbors=n_neighbors,\n expected_doublet_rate=expected_doublet_rate,\n stdev_doublet_rate=stdev_doublet_rate,\n mean_center=mean_center,\n normalize_variance=normalize_variance,\n n_prin_comps=n_prin_comps,\n use_approx_neighbors=use_approx_neighbors,\n knn_dist_metric=knn_dist_metric,\n get_doublet_neighbor_parents=get_doublet_neighbor_parents,\n threshold=threshold,\n random_state=random_state,\n verbose=verbose,\n )\n\n return {\"obs\": ad_obs.obs, \"uns\": ad_obs.uns[\"scrublet\"]}\n\n if batch_key is not None:\n if batch_key not in adata.obs.columns:\n msg = (\n \"`batch_key` must be a column of .obs in the input AnnData object,\"\n f\"but {batch_key!r} is not in {adata.obs.keys()!r}.\"\n )\n raise ValueError(msg)\n\n # Run Scrublet independently on batches and return just the\n # scrublet-relevant parts of the objects to add to the input object\n\n batches = np.unique(adata.obs[batch_key])\n scrubbed = [\n _run_scrublet(\n adata_obs[adata_obs.obs[batch_key] == batch].copy(),\n adata_sim,\n )\n for batch in batches\n ]\n scrubbed_obs = pd.concat([scrub[\"obs\"] for scrub in scrubbed])\n\n # Now reset the obs to get the scrublet scores\n\n adata.obs = scrubbed_obs.loc[adata.obs_names.values]\n\n # Save the .uns from each batch separately\n\n adata.uns[\"scrublet\"] = {}\n adata.uns[\"scrublet\"][\"batches\"] = dict(\n zip(batches, [scrub[\"uns\"] for scrub in scrubbed])\n )\n\n # Record that we've done batched analysis, so e.g. the plotting\n # function knows what to do.\n\n adata.uns[\"scrublet\"][\"batched_by\"] = batch_key\n\n else:\n scrubbed = _run_scrublet(adata_obs, adata_sim)\n\n # Copy outcomes to input object from our processed version\n\n adata.obs[\"doublet_score\"] = scrubbed[\"obs\"][\"doublet_score\"]\n adata.obs[\"predicted_doublet\"] = scrubbed[\"obs\"][\"predicted_doublet\"]\n adata.uns[\"scrublet\"] = scrubbed[\"uns\"]\n\n logg.info(\" Scrublet finished\", time=start)\n\n return adata if copy else None\n\n\ndef _scrublet_call_doublets(\n adata_obs: AnnData,\n adata_sim: AnnData,\n *,\n n_neighbors: int | None = None,\n expected_doublet_rate: float = 0.05,\n stdev_doublet_rate: float = 0.02,\n mean_center: bool = True,\n normalize_variance: bool = True,\n n_prin_comps: int = 30,\n use_approx_neighbors: bool | None = None,\n knn_dist_metric: _Metric | _MetricFn = \"euclidean\",\n get_doublet_neighbor_parents: bool = False,\n threshold: float | None = None,\n random_state: AnyRandom = 0,\n verbose: bool = True,\n) -> AnnData:\n \"\"\"\\\n Core function for predicting doublets using Scrublet :cite:p:`Wolock2019`.\n\n Predict cell doublets using a nearest-neighbor classifier of observed\n transcriptomes and simulated doublets.\n\n Parameters\n ----------\n adata_obs\n The annotated data matrix of shape ``n_obs`` × ``n_vars``. Rows\n correspond to cells and columns to genes. Should be normalised with\n :func:`~scanpy.pp.normalize_total` and filtered to include only highly\n variable genes.\n adata_sim\n Anndata object generated by\n :func:`~scanpy.pp.scrublet_simulate_doublets`, with same number of vars\n as adata_obs. This should have been built from adata_obs after\n filtering genes and cells and selcting highly-variable genes.\n n_neighbors\n Number of neighbors used to construct the KNN graph of observed\n transcriptomes and simulated doublets. If ``None``, this is\n automatically set to ``np.round(0.5 * np.sqrt(n_obs))``.\n expected_doublet_rate\n The estimated doublet rate for the experiment.\n stdev_doublet_rate\n Uncertainty in the expected doublet rate.\n mean_center\n If True, center the data such that each gene has a mean of 0.\n `sklearn.decomposition.PCA` will be used for dimensionality\n reduction.\n normalize_variance\n If True, normalize the data such that each gene has a variance of 1.\n `sklearn.decomposition.TruncatedSVD` will be used for dimensionality\n reduction, unless `mean_center` is True.\n n_prin_comps\n Number of principal components used to embed the transcriptomes prior\n to k-nearest-neighbor graph construction.\n use_approx_neighbors\n Use approximate nearest neighbor method (annoy) for the KNN\n classifier.\n knn_dist_metric\n Distance metric used when finding nearest neighbors. For list of\n valid values, see the documentation for annoy (if `use_approx_neighbors`\n is True) or sklearn.neighbors.NearestNeighbors (if `use_approx_neighbors`\n is False).\n get_doublet_neighbor_parents\n If True, return the parent transcriptomes that generated the\n doublet neighbors of each observed transcriptome. This information can\n be used to infer the cell states that generated a given\n doublet state.\n threshold\n Doublet score threshold for calling a transcriptome a doublet. If\n `None`, this is set automatically by looking for the minimum between\n the two modes of the `doublet_scores_sim_` histogram. It is best\n practice to check the threshold visually using the\n `doublet_scores_sim_` histogram and/or based on co-localization of\n predicted doublets in a 2-D embedding.\n random_state\n Initial state for doublet simulation and nearest neighbors.\n verbose\n If :data:`True`, log progress updates.\n\n Returns\n -------\n if ``copy=True`` it returns or else adds fields to ``adata``:\n\n ``.obs['doublet_score']``\n Doublet scores for each observed transcriptome\n\n ``.obs['predicted_doublets']``\n Boolean indicating predicted doublet status\n\n ``.uns['scrublet']['doublet_scores_sim']``\n Doublet scores for each simulated doublet transcriptome\n\n ``.uns['scrublet']['doublet_parents']``\n Pairs of ``.obs_names`` used to generate each simulated doublet transcriptome\n\n ``.uns['scrublet']['parameters']``\n Dictionary of Scrublet parameters\n \"\"\"\n\n # Estimate n_neighbors if not provided, and create scrublet object.\n\n if n_neighbors is None:\n n_neighbors = int(round(0.5 * np.sqrt(adata_obs.shape[0])))\n\n # Note: Scrublet() will sparse adata_obs.X if it's not already, but this\n # matrix won't get used if we pre-set the normalised slots.\n\n scrub = Scrublet(\n adata_obs.X,\n n_neighbors=n_neighbors,\n expected_doublet_rate=expected_doublet_rate,\n stdev_doublet_rate=stdev_doublet_rate,\n random_state=random_state,\n )\n\n # Ensure normalised matrix sparseness as Scrublet does\n # https://github.com/swolock/scrublet/blob/67f8ecbad14e8e1aa9c89b43dac6638cebe38640/src/scrublet/scrublet.py#L100\n\n scrub._counts_obs_norm = sparse.csc_matrix(adata_obs.X)\n scrub._counts_sim_norm = sparse.csc_matrix(adata_sim.X)\n\n scrub.doublet_parents_ = adata_sim.obsm[\"doublet_parents\"]\n\n # Call scrublet-specific preprocessing where specified\n\n if mean_center and normalize_variance:\n pipeline.zscore(scrub)\n elif mean_center:\n pipeline.mean_center(scrub)\n elif normalize_variance:\n pipeline.normalize_variance(scrub)\n\n # Do PCA. Scrublet fits to the observed matrix and decomposes both observed\n # and simulated based on that fit, so we'll just let it do its thing rather\n # than trying to use Scanpy's PCA wrapper of the same functions.\n\n if mean_center:\n logg.info(\"Embedding transcriptomes using PCA...\")\n pipeline.pca(scrub, n_prin_comps=n_prin_comps, random_state=scrub._random_state)\n else:\n logg.info(\"Embedding transcriptomes using Truncated SVD...\")\n pipeline.truncated_svd(\n scrub, n_prin_comps=n_prin_comps, random_state=scrub._random_state\n )\n\n # Score the doublets\n\n scrub.calculate_doublet_scores(\n use_approx_neighbors=use_approx_neighbors,\n distance_metric=knn_dist_metric,\n get_doublet_neighbor_parents=get_doublet_neighbor_parents,\n )\n\n # Actually call doublets\n\n scrub.call_doublets(threshold=threshold, verbose=verbose)\n\n # Store results in AnnData for return\n\n adata_obs.obs[\"doublet_score\"] = scrub.doublet_scores_obs_\n\n # Store doublet Scrublet metadata\n\n adata_obs.uns[\"scrublet\"] = {\n \"doublet_scores_sim\": scrub.doublet_scores_sim_,\n \"doublet_parents\": adata_sim.obsm[\"doublet_parents\"],\n \"parameters\": {\n \"expected_doublet_rate\": expected_doublet_rate,\n \"sim_doublet_ratio\": (\n adata_sim.uns.get(\"scrublet\", {})\n .get(\"parameters\", {})\n .get(\"sim_doublet_ratio\", None)\n ),\n \"n_neighbors\": n_neighbors,\n \"random_state\": random_state,\n },\n }\n\n # If threshold hasn't been located successfully then we couldn't make any\n # predictions. The user will get a warning from Scrublet, but we need to\n # set the boolean so that any downstream filtering on\n # predicted_doublet=False doesn't incorrectly filter cells. The user can\n # still use this object to generate the plot and derive a threshold\n # manually.\n\n if hasattr(scrub, \"threshold_\"):\n adata_obs.uns[\"scrublet\"][\"threshold\"] = scrub.threshold_\n adata_obs.obs[\"predicted_doublet\"] = scrub.predicted_doublets_\n else:\n adata_obs.obs[\"predicted_doublet\"] = False\n\n if get_doublet_neighbor_parents:\n adata_obs.uns[\"scrublet\"][\"doublet_neighbor_parents\"] = (\n scrub.doublet_neighbor_parents_\n )\n\n return adata_obs\n\n\n@old_positionals(\n \"layer\", \"sim_doublet_ratio\", \"synthetic_doublet_umi_subsampling\", \"random_seed\"\n)\ndef scrublet_simulate_doublets(\n adata: AnnData,\n *,\n layer: str | None = None,\n sim_doublet_ratio: float = 2.0,\n synthetic_doublet_umi_subsampling: float = 1.0,\n random_seed: AnyRandom = 0,\n) -> AnnData:\n \"\"\"\\\n Simulate doublets by adding the counts of random observed transcriptome pairs.\n\n Parameters\n ----------\n adata\n The annotated data matrix of shape ``n_obs`` × ``n_vars``. Rows\n correspond to cells and columns to genes. Genes should have been\n filtered for expression and variability, and the object should contain\n raw expression of the same dimensions.\n layer\n Layer of adata where raw values are stored, or 'X' if values are in .X.\n sim_doublet_ratio\n Number of doublets to simulate relative to the number of observed\n transcriptomes. If `None`, self.sim_doublet_ratio is used.\n synthetic_doublet_umi_subsampling\n Rate for sampling UMIs when creating synthetic doublets. If 1.0,\n each doublet is created by simply adding the UMIs from two randomly\n sampled observed transcriptomes. For values less than 1, the\n UMI counts are added and then randomly sampled at the specified\n rate.\n\n Returns\n -------\n adata : anndata.AnnData with simulated doublets in .X\n Adds fields to ``adata``:\n\n ``.obsm['scrublet']['doublet_parents']``\n Pairs of ``.obs_names`` used to generate each simulated doublet transcriptome\n\n ``.uns['scrublet']['parameters']``\n Dictionary of Scrublet parameters\n\n See also\n --------\n :func:`~scanpy.pp.scrublet`: Main way of running Scrublet, runs\n preprocessing, doublet simulation (this function) and calling.\n :func:`~scanpy.pl.scrublet_score_distribution`: Plot histogram of doublet\n scores for observed transcriptomes and simulated doublets.\n \"\"\"\n\n X = _get_obs_rep(adata, layer=layer)\n scrub = Scrublet(X, random_state=random_seed)\n\n scrub.simulate_doublets(\n sim_doublet_ratio=sim_doublet_ratio,\n synthetic_doublet_umi_subsampling=synthetic_doublet_umi_subsampling,\n )\n\n adata_sim = AnnData(scrub._counts_sim)\n adata_sim.obs[\"n_counts\"] = scrub._total_counts_sim\n adata_sim.obsm[\"doublet_parents\"] = scrub.doublet_parents_\n adata_sim.uns[\"scrublet\"] = {\"parameters\": {\"sim_doublet_ratio\": sim_doublet_ratio}}\n return adata_sim\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nfrom scipy import sparse\n\nfrom scanpy.preprocessing._utils import _get_mean_var\n\nfrom ..._utils import get_random_state\n\nif TYPE_CHECKING:\n from numpy.typing import NDArray\n\n from ..._utils import AnyRandom\n\n\ndef sparse_multiply(\n E: sparse.csr_matrix | sparse.csc_matrix | NDArray[np.float64],\n a: float | int | NDArray[np.float64],\n) -> sparse.csr_matrix | sparse.csc_matrix:\n \"\"\"multiply each row of E by a scalar\"\"\"\n\n nrow = E.shape[0]\n w = sparse.dia_matrix((a, 0), shape=(nrow, nrow), dtype=a.dtype)\n r = w @ E\n if isinstance(r, np.ndarray):\n return sparse.csc_matrix(r)\n return r\n\n\ndef sparse_zscore(\n E: sparse.csr_matrix | sparse.csc_matrix,\n *,\n gene_mean: NDArray[np.float64] | None = None,\n gene_stdev: NDArray[np.float64] | None = None,\n) -> sparse.csr_matrix | sparse.csc_matrix:\n \"\"\"z-score normalize each column of E\"\"\"\n if gene_mean is None or gene_stdev is None:\n gene_means, gene_stdevs = _get_mean_var(E, axis=0)\n gene_stdevs = np.sqrt(gene_stdevs)\n return sparse_multiply(np.asarray((E - gene_mean).T), 1 / gene_stdev).T\n\n\ndef subsample_counts(\n E: sparse.csr_matrix | sparse.csc_matrix,\n *,\n rate: float,\n original_totals,\n random_seed: AnyRandom = 0,\n) -> tuple[sparse.csr_matrix | sparse.csc_matrix, NDArray[np.int64]]:\n if rate < 1:\n random_seed = get_random_state(random_seed)\n E.data = random_seed.binomial(np.round(E.data).astype(int), rate)\n current_totals = np.asarray(E.sum(1)).squeeze()\n unsampled_orig_totals = original_totals - current_totals\n unsampled_downsamp_totals = np.random.binomial(\n np.round(unsampled_orig_totals).astype(int), rate\n )\n final_downsamp_totals = current_totals + unsampled_downsamp_totals\n else:\n final_downsamp_totals = original_totals\n return E, final_downsamp_totals\n\n\nfrom __future__ import annotations\n\nimport numpy as np\nfrom scipy.sparse import csr_matrix, issparse\n\nfrom ..._compat import old_positionals\n\n\n@old_positionals(\"max_fraction\", \"mult_with_mean\")\ndef normalize_per_cell_weinreb16_deprecated(\n x: np.ndarray,\n *,\n max_fraction: float = 1,\n mult_with_mean: bool = False,\n) -> np.ndarray:\n \"\"\"\\\n Normalize each cell :cite:p:`Weinreb2017`.\n\n This is a deprecated version. See `normalize_per_cell` instead.\n\n Normalize each cell by UMI count, so that every cell has the same total\n count.\n\n Parameters\n ----------\n X\n Expression matrix. Rows correspond to cells and columns to genes.\n max_fraction\n Only use genes that make up more than max_fraction of the total\n reads in every cell.\n mult_with_mean\n Multiply the result with the mean of total counts.\n\n Returns\n -------\n Normalized version of the original expression matrix.\n \"\"\"\n if max_fraction < 0 or max_fraction > 1:\n raise ValueError(\"Choose max_fraction between 0 and 1.\")\n\n counts_per_cell = x.sum(1).A1 if issparse(x) else x.sum(1)\n gene_subset = np.all(x <= counts_per_cell[:, None] * max_fraction, axis=0)\n if issparse(x):\n gene_subset = gene_subset.A1\n tc_include = (\n x[:, gene_subset].sum(1).A1 if issparse(x) else x[:, gene_subset].sum(1)\n )\n\n x_norm = (\n x.multiply(csr_matrix(1 / tc_include[:, None]))\n if issparse(x)\n else x / tc_include[:, None]\n )\n if mult_with_mean:\n x_norm *= np.mean(counts_per_cell)\n\n return x_norm\n\n\ndef zscore_deprecated(X: np.ndarray) -> np.ndarray:\n \"\"\"\\\n Z-score standardize each variable/gene in X :cite:p:`Weinreb2017`.\n\n Use `scale` instead.\n\n Parameters\n ----------\n X\n Data matrix. Rows correspond to cells and columns to genes.\n\n Returns\n -------\n Z-score standardized version of the data matrix.\n \"\"\"\n means = np.tile(np.mean(X, axis=0)[None, :], (X.shape[0], 1))\n stds = np.tile(np.std(X, axis=0)[None, :], (X.shape[0], 1))\n return (X - means) / (stds + 0.0001)\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom anndata import AnnData\nfrom scipy.sparse import issparse\n\nfrom ... import logging as logg\nfrom ..._compat import old_positionals\nfrom .._distributed import materialize_as_ndarray\nfrom .._utils import _get_mean_var\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from scipy.sparse import spmatrix\n\n\n@old_positionals(\n \"flavor\",\n \"min_disp\",\n \"max_disp\",\n \"min_mean\",\n \"max_mean\",\n \"n_bins\",\n \"n_top_genes\",\n \"log\",\n \"subset\",\n \"copy\",\n)\ndef filter_genes_dispersion(\n data: AnnData | spmatrix | np.ndarray,\n *,\n flavor: Literal[\"seurat\", \"cell_ranger\"] = \"seurat\",\n min_disp: float | None = None,\n max_disp: float | None = None,\n min_mean: float | None = None,\n max_mean: float | None = None,\n n_bins: int = 20,\n n_top_genes: int | None = None,\n log: bool = True,\n subset: bool = True,\n copy: bool = False,\n) -> AnnData | np.recarray | None:\n \"\"\"\\\n Extract highly variable genes :cite:p:`Satija2015,Zheng2017`.\n\n .. warning::\n .. deprecated:: 1.3.6\n Use :func:`~scanpy.pp.highly_variable_genes`\n instead. The new function is equivalent to the present\n function, except that\n\n * the new function always expects logarithmized data\n * `subset=False` in the new function, it suffices to\n merely annotate the genes, tools like `pp.pca` will\n detect the annotation\n * you can now call: `sc.pl.highly_variable_genes(adata)`\n * `copy` is replaced by `inplace`\n\n If trying out parameters, pass the data matrix instead of AnnData.\n\n Depending on `flavor`, this reproduces the R-implementations of Seurat\n :cite:p:`Satija2015` and Cell Ranger :cite:p:`Zheng2017`.\n\n The normalized dispersion is obtained by scaling with the mean and standard\n deviation of the dispersions for genes falling into a given bin for mean\n expression of genes. This means that for each bin of mean expression, highly\n variable genes are selected.\n\n Use `flavor='cell_ranger'` with care and in the same way as in\n :func:`~scanpy.pp.recipe_zheng17`.\n\n Parameters\n ----------\n data\n The (annotated) data matrix of shape `n_obs` × `n_vars`. Rows correspond\n to cells and columns to genes.\n flavor\n Choose the flavor for computing normalized dispersion. If choosing\n 'seurat', this expects non-logarithmized data – the logarithm of mean\n and dispersion is taken internally when `log` is at its default value\n `True`. For 'cell_ranger', this is usually called for logarithmized data\n – in this case you should set `log` to `False`. In their default\n workflows, Seurat passes the cutoffs whereas Cell Ranger passes\n `n_top_genes`.\n min_mean\n max_mean\n min_disp\n max_disp\n If `n_top_genes` unequals `None`, these cutoffs for the means and the\n normalized dispersions are ignored.\n n_bins\n Number of bins for binning the mean gene expression. Normalization is\n done with respect to each bin. If just a single gene falls into a bin,\n the normalized dispersion is artificially set to 1. You'll be informed\n about this if you set `settings.verbosity = 4`.\n n_top_genes\n Number of highly-variable genes to keep.\n log\n Use the logarithm of the mean to variance ratio.\n subset\n Keep highly-variable genes only (if True) else write a bool array for h\n ighly-variable genes while keeping all genes\n copy\n If an :class:`~anndata.AnnData` is passed, determines whether a copy\n is returned.\n\n Returns\n -------\n If an AnnData `adata` is passed, returns or updates `adata` depending on\n `copy`. It filters the `adata` and adds the annotations\n\n **means** : adata.var\n Means per gene. Logarithmized when `log` is `True`.\n **dispersions** : adata.var\n Dispersions per gene. Logarithmized when `log` is `True`.\n **dispersions_norm** : adata.var\n Normalized dispersions per gene. Logarithmized when `log` is `True`.\n\n If a data matrix `X` is passed, the annotation is returned as `np.recarray`\n with the same information stored in fields: `gene_subset`, `means`, `dispersions`, `dispersion_norm`.\n \"\"\"\n if n_top_genes is not None and not all(\n x is None for x in [min_disp, max_disp, min_mean, max_mean]\n ):\n msg = \"If you pass `n_top_genes`, all cutoffs are ignored.\"\n warnings.warn(msg, UserWarning)\n if min_disp is None:\n min_disp = 0.5\n if min_mean is None:\n min_mean = 0.0125\n if max_mean is None:\n max_mean = 3\n if isinstance(data, AnnData):\n adata = data.copy() if copy else data\n result = filter_genes_dispersion(\n adata.X,\n log=log,\n min_disp=min_disp,\n max_disp=max_disp,\n min_mean=min_mean,\n max_mean=max_mean,\n n_top_genes=n_top_genes,\n flavor=flavor,\n )\n adata.var[\"means\"] = result[\"means\"]\n adata.var[\"dispersions\"] = result[\"dispersions\"]\n adata.var[\"dispersions_norm\"] = result[\"dispersions_norm\"]\n if subset:\n adata._inplace_subset_var(result[\"gene_subset\"])\n else:\n adata.var[\"highly_variable\"] = result[\"gene_subset\"]\n return adata if copy else None\n start = logg.info(\"extracting highly variable genes\")\n X = data # no copy necessary, X remains unchanged in the following\n mean, var = materialize_as_ndarray(_get_mean_var(X))\n # now actually compute the dispersion\n mean[mean == 0] = 1e-12 # set entries equal to zero to small value\n dispersion = var / mean\n if log: # logarithmized mean as in Seurat\n dispersion[dispersion == 0] = np.nan\n dispersion = np.log(dispersion)\n mean = np.log1p(mean)\n # all of the following quantities are \"per-gene\" here\n df = pd.DataFrame()\n df[\"mean\"] = mean\n df[\"dispersion\"] = dispersion\n if flavor == \"seurat\":\n df[\"mean_bin\"] = pd.cut(df[\"mean\"], bins=n_bins)\n disp_grouped = df.groupby(\"mean_bin\", observed=True)[\"dispersion\"]\n disp_mean_bin = disp_grouped.mean()\n disp_std_bin = disp_grouped.std(ddof=1)\n # retrieve those genes that have nan std, these are the ones where\n # only a single gene fell in the bin and implicitly set them to have\n # a normalized disperion of 1\n one_gene_per_bin = disp_std_bin.isnull()\n gen_indices = np.where(one_gene_per_bin[df[\"mean_bin\"].values])[0].tolist()\n if len(gen_indices) > 0:\n logg.debug(\n f\"Gene indices {gen_indices} fell into a single bin: their \"\n \"normalized dispersion was set to 1.\\n \"\n \"Decreasing `n_bins` will likely avoid this effect.\"\n )\n # Circumvent pandas 0.23 bug. Both sides of the assignment have dtype==float32,\n # but there’s still a dtype error without “.value”.\n disp_std_bin[one_gene_per_bin] = disp_mean_bin[one_gene_per_bin.values].values\n disp_mean_bin[one_gene_per_bin] = 0\n # actually do the normalization\n df[\"dispersion_norm\"] = (\n # use values here as index differs\n df[\"dispersion\"].values - disp_mean_bin[df[\"mean_bin\"].values].values\n ) / disp_std_bin[df[\"mean_bin\"].values].values\n elif flavor == \"cell_ranger\":\n from statsmodels import robust\n\n df[\"mean_bin\"] = pd.cut(\n df[\"mean\"],\n np.r_[-np.inf, np.percentile(df[\"mean\"], np.arange(10, 105, 5)), np.inf],\n )\n disp_grouped = df.groupby(\"mean_bin\", observed=True)[\"dispersion\"]\n disp_median_bin = disp_grouped.median()\n # the next line raises the warning: \"Mean of empty slice\"\n with warnings.catch_warnings():\n warnings.simplefilter(\"ignore\")\n disp_mad_bin = disp_grouped.apply(robust.mad)\n df[\"dispersion_norm\"] = (\n np.abs(\n df[\"dispersion\"].values - disp_median_bin[df[\"mean_bin\"].values].values\n )\n / disp_mad_bin[df[\"mean_bin\"].values].values\n )\n else:\n raise ValueError('`flavor` needs to be \"seurat\" or \"cell_ranger\"')\n dispersion_norm = df[\"dispersion_norm\"].values.astype(\"float32\")\n if n_top_genes is not None:\n dispersion_norm = dispersion_norm[~np.isnan(dispersion_norm)]\n dispersion_norm[\n ::-1\n ].sort() # interestingly, np.argpartition is slightly slower\n disp_cut_off = dispersion_norm[n_top_genes - 1]\n gene_subset = df[\"dispersion_norm\"].values >= disp_cut_off\n logg.debug(\n f\"the {n_top_genes} top genes correspond to a \"\n f\"normalized dispersion cutoff of {disp_cut_off}\"\n )\n else:\n max_disp = np.inf if max_disp is None else max_disp\n dispersion_norm[np.isnan(dispersion_norm)] = 0 # similar to Seurat\n gene_subset = np.logical_and.reduce(\n (\n mean > min_mean,\n mean < max_mean,\n dispersion_norm > min_disp,\n dispersion_norm < max_disp,\n )\n )\n logg.info(\" finished\", time=start)\n return np.rec.fromarrays(\n (\n gene_subset,\n df[\"mean\"].values,\n df[\"dispersion\"].values,\n df[\"dispersion_norm\"].values.astype(\"float32\", copy=False),\n ),\n dtype=[\n (\"gene_subset\", bool),\n (\"means\", \"float32\"),\n (\"dispersions\", \"float32\"),\n (\"dispersions_norm\", \"float32\"),\n ],\n )\n\n\ndef filter_genes_cv_deprecated(X, Ecutoff, cvFilter):\n \"\"\"Filter genes by coefficient of variance and mean.\"\"\"\n return _filter_genes(X, Ecutoff, cvFilter, np.std)\n\n\ndef filter_genes_fano_deprecated(X, Ecutoff, Vcutoff):\n \"\"\"Filter genes by fano factor and mean.\"\"\"\n return _filter_genes(X, Ecutoff, Vcutoff, np.var)\n\n\ndef _filter_genes(X, e_cutoff, v_cutoff, meth):\n \"\"\"See `filter_genes_dispersion` :cite:p:`Weinreb2017`.\"\"\"\n if issparse(X):\n raise ValueError(\"Not defined for sparse input. See `filter_genes_dispersion`.\")\n mean_filter = np.mean(X, axis=0) > e_cutoff\n var_filter = meth(X, axis=0) / (np.mean(X, axis=0) + 0.0001) > v_cutoff\n gene_subset = np.nonzero(np.all([mean_filter, var_filter], axis=0))[0]\n return gene_subset\n\n\n# See Table 1 in Krumsiek et al. (2011), p. 3 or\n# Table 1, in Suppl. Mat. of Moignard et al. (2015), p. 28.\n#\n# For each \"variable = \", there must be a right hand side:\n# either an empty string or a python-style logical expression\n# involving variable names, \"or\", \"and\", \"(\", \")\".\n# The order of equations matters!\n#\n# modelType = hill\n# invTimeStep = 0.02\n#\n# boolean update rules:\nGata2 = Gata2 and not (Gata1 and Fog1) and not Pu.1\nGata1 = (Gata1 or Gata2 or Fli1) and not Pu.1\nFog1 = Gata1\nEKLF = Gata1 and not Fli1\nFli1 = Gata1 and not EKLF\nSCL = Gata1 and not Pu.1\nCebpa = Cebpa and not (Gata1 and Fog1 and SCL)\nPu.1 = (Cebpa or Pu.1) and not (Gata1 or Gata2)\ncJun = Pu.1 and not Gfi1\nEgrNab = (Pu.1 and cJun) and not Gfi1\nGfi1 = Cebpa and not EgrNab\n# coupling list:\nGata2 Gata2 1.0\nGata2 Gata1 -0.1\nGata2 Fog1 -1.0\nGata2 Pu.1 -1.15\nGata1 Gata2 1.0\nGata1 Gata1 0.1\nGata1 Fli1 1.0\nGata1 Pu.1 -1.21\nFog1 Gata1 0.1\nEKLF Gata1 0.2\nEKLF Fli1 -1.0\nFli1 Gata1 0.2\nFli1 EKLF -1.0\nSCL Gata1 1.0\nSCL Pu.1 -1.0\nCebpa Gata1 -1.0\nCebpa Fog1 -1.0\nCebpa SCL -1.0\nCebpa Cebpa 10.0\nPu.1 Gata2 -1.0\nPu.1 Gata1 -1.0\nPu.1 Cebpa 10.0\nPu.1 Pu.1 10.0\ncJun Pu.1 1.0\ncJun Gfi1 -1.0\nEgrNab Pu.1 1.0\nEgrNab cJun 1.0\nEgrNab Gfi1 -1.3\nGfi1 Cebpa 1.0\nGfi1 EgrNab -5.0\n\n\nmodel = SCANPY_PATH/sim_models/krumsiek11.txt\ntmax = 800\nbranching = True\nnrRealizations = 4\nnoiseObs = 0\nnoiseDyn = 0.001\nstep = 5\nseed = 0\n\n\n# For each \"variable = \", there must be a right hand side:\n# either an empty string or a python-style logical expression\n# involving variable names, \"or\", \"and\", \"(\", \")\".\n# The order of equations matters!\n#\n# modelType = hill\n# invTimeStep = 0.1\n#\n# boolean update rules:\n0 = 0 and not 1\n1 = 1 and not 0\n# coupling list:\n0 0 1.0\n0 1 -1.0\n1 1 1.0\n1 0 -1.0\n\n\nmodel = SCANPY_PATH/sim_models/toggleswitch.txt\ntmax = 100\nbranching = True\nnrRealizations = 2\nnoiseObs = 0.01\nnoiseDyn = 0.001\nstep = 1\nseed = 0\n\n\n\n\nfrom __future__ import annotations\n\nfrom collections.abc import Callable\nfrom typing import TypeVar\n\nF = TypeVar(\"F\", bound=Callable)\n\n\ndef doctest_needs(mod: str) -> Callable[[F], F]:\n \"\"\"Mark function with doctest dependency.\"\"\"\n\n def decorator(func: F) -> F:\n func._doctest_needs = mod\n return func\n\n return decorator\n\n\ndef doctest_skip(reason: str) -> Callable[[F], F]:\n \"\"\"Mark function so doctest is skipped.\"\"\"\n if not reason:\n raise ValueError(\"reason must not be empty\")\n\n def decorator(func: F) -> F:\n func._doctest_skip_reason = reason\n return func\n\n return decorator\n\n\ndef doctest_internet(func: F) -> F:\n \"\"\"Mark function so doctest gets the internet mark.\"\"\"\n\n func._doctest_internet = True\n return func\n\n\n\"\"\"Utility functions and classes\n\nThis file largely consists of the old _utils.py file. Over time, these functions\nshould be moved of this file.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport importlib.util\nimport inspect\nimport random\nimport re\nimport sys\nimport warnings\nfrom collections import namedtuple\nfrom contextlib import contextmanager, suppress\nfrom enum import Enum\nfrom functools import partial, singledispatch, wraps\nfrom operator import mul, truediv\nfrom textwrap import dedent\nfrom types import MethodType, ModuleType\nfrom typing import TYPE_CHECKING, Union, overload\nfrom weakref import WeakSet\n\nimport h5py\nimport numpy as np\nfrom anndata import __version__ as anndata_version\nfrom packaging.version import Version\nfrom scipy import sparse\nfrom sklearn.utils import check_random_state\n\nfrom .. import logging as logg\nfrom .._compat import DaskArray\nfrom .._settings import settings\nfrom .compute.is_constant import is_constant # noqa: F401\n\nif Version(anndata_version) >= Version(\"0.10.0\"):\n from anndata._core.sparse_dataset import (\n BaseCompressedSparseDataset as SparseDataset,\n )\nelse:\n from anndata._core.sparse_dataset import SparseDataset\n\nif TYPE_CHECKING:\n from collections.abc import Mapping\n from pathlib import Path\n from typing import Any, Callable, Literal, TypeVar\n\n from anndata import AnnData\n from numpy.typing import DTypeLike, NDArray\n\n from ..neighbors import NeighborsParams, RPForestDict\n\n\n# e.g. https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html\n# maybe in the future random.Generator\nAnyRandom = Union[int, np.random.RandomState, None]\n\n\nclass Empty(Enum):\n token = 0\n\n def __repr__(self) -> str:\n return \"_empty\"\n\n\n_empty = Empty.token\n\n\nclass RNGIgraph:\n \"\"\"\n Random number generator for ipgraph so global seed is not changed.\n See :func:`igraph.set_random_number_generator` for the requirements.\n \"\"\"\n\n def __init__(self, random_state: int = 0) -> None:\n self._rng = check_random_state(random_state)\n\n def __getattr__(self, attr: str):\n return getattr(self._rng, \"normal\" if attr == \"gauss\" else attr)\n\n\ndef ensure_igraph() -> None:\n if importlib.util.find_spec(\"igraph\"):\n return\n raise ImportError(\n \"Please install the igraph package: \"\n \"`conda install -c conda-forge python-igraph` or \"\n \"`pip3 install igraph`.\"\n )\n\n\n@contextmanager\ndef set_igraph_random_state(random_state: int):\n ensure_igraph()\n import igraph\n\n rng = RNGIgraph(random_state)\n try:\n igraph.set_random_number_generator(rng)\n yield None\n finally:\n igraph.set_random_number_generator(random)\n\n\nEPS = 1e-15\n\n\ndef check_versions():\n if Version(anndata_version) < Version(\"0.6.10\"):\n from .. import __version__\n\n raise ImportError(\n f\"Scanpy {__version__} needs anndata version >=0.6.10, \"\n f\"not {anndata_version}.\\nRun `pip install anndata -U --no-deps`.\"\n )\n\n\ndef getdoc(c_or_f: Callable | type) -> str | None:\n if getattr(c_or_f, \"__doc__\", None) is None:\n return None\n doc = inspect.getdoc(c_or_f)\n if isinstance(c_or_f, type) and hasattr(c_or_f, \"__init__\"):\n sig = inspect.signature(c_or_f.__init__)\n else:\n sig = inspect.signature(c_or_f)\n\n def type_doc(name: str):\n param: inspect.Parameter = sig.parameters[name]\n cls = getattr(param.annotation, \"__qualname__\", repr(param.annotation))\n if param.default is not param.empty:\n return f\"{cls}, optional (default: {param.default!r})\"\n else:\n return cls\n\n return \"\\n\".join(\n f\"{line} : {type_doc(line)}\" if line.strip() in sig.parameters else line\n for line in doc.split(\"\\n\")\n )\n\n\ndef renamed_arg(old_name, new_name, *, pos_0: bool = False):\n def decorator(func):\n @wraps(func)\n def wrapper(*args, **kwargs):\n if old_name in kwargs:\n f_name = func.__name__\n pos_str = (\n (\n f\" at first position. Call it as `{f_name}(val, ...)` \"\n f\"instead of `{f_name}({old_name}=val, ...)`\"\n )\n if pos_0\n else \"\"\n )\n msg = (\n f\"In function `{f_name}`, argument `{old_name}` \"\n f\"was renamed to `{new_name}`{pos_str}.\"\n )\n warnings.warn(msg, FutureWarning, stacklevel=3)\n if pos_0:\n args = (kwargs.pop(old_name), *args)\n else:\n kwargs[new_name] = kwargs.pop(old_name)\n return func(*args, **kwargs)\n\n return wrapper\n\n return decorator\n\n\ndef _import_name(name: str) -> Any:\n from importlib import import_module\n\n parts = name.split(\".\")\n obj = import_module(parts[0])\n for i, name in enumerate(parts[1:]):\n try:\n obj = import_module(f\"{obj.__name__}.{name}\")\n except ModuleNotFoundError:\n break\n else:\n i = len(parts)\n for name in parts[i + 1 :]:\n try:\n obj = getattr(obj, name)\n except AttributeError:\n raise RuntimeError(f\"{parts[:i]}, {parts[i + 1:]}, {obj} {name}\")\n return obj\n\n\ndef _one_of_ours(obj, root: str):\n return (\n hasattr(obj, \"__name__\")\n and not obj.__name__.split(\".\")[-1].startswith(\"_\")\n and getattr(\n obj, \"__module__\", getattr(obj, \"__qualname__\", obj.__name__)\n ).startswith(root)\n )\n\n\ndef descend_classes_and_funcs(mod: ModuleType, root: str, encountered=None):\n if encountered is None:\n encountered = WeakSet()\n for obj in vars(mod).values():\n if not _one_of_ours(obj, root) or obj in encountered:\n continue\n encountered.add(obj)\n if callable(obj) and not isinstance(obj, MethodType):\n yield obj\n if isinstance(obj, type):\n for m in vars(obj).values():\n if callable(m) and _one_of_ours(m, root):\n yield m\n elif isinstance(obj, ModuleType):\n if obj.__name__.startswith(\"scanpy.tests\"):\n # Python’s import mechanism seems to add this to `scanpy`’s attributes\n continue\n yield from descend_classes_and_funcs(obj, root, encountered)\n\n\ndef annotate_doc_types(mod: ModuleType, root: str):\n for c_or_f in descend_classes_and_funcs(mod, root):\n c_or_f.getdoc = partial(getdoc, c_or_f)\n\n\ndef _doc_params(**kwds):\n \"\"\"\\\n Docstrings should start with ``\\\\`` in the first line for proper formatting.\n \"\"\"\n\n def dec(obj):\n obj.__orig_doc__ = obj.__doc__\n obj.__doc__ = dedent(obj.__doc__).format_map(kwds)\n return obj\n\n return dec\n\n\ndef _check_array_function_arguments(**kwargs):\n \"\"\"Checks for invalid arguments when an array is passed.\n\n Helper for functions that work on either AnnData objects or array-likes.\n \"\"\"\n # TODO: Figure out a better solution for documenting dispatched functions\n invalid_args = [k for k, v in kwargs.items() if v is not None]\n if len(invalid_args) > 0:\n raise TypeError(\n f\"Arguments {invalid_args} are only valid if an AnnData object is passed.\"\n )\n\n\ndef _check_use_raw(\n adata: AnnData, use_raw: None | bool, *, layer: str | None = None\n) -> bool:\n \"\"\"\n Normalize checking `use_raw`.\n\n My intentention here is to also provide a single place to throw a deprecation warning from in future.\n \"\"\"\n if use_raw is not None:\n return use_raw\n if layer is not None:\n return False\n return adata.raw is not None\n\n\n# --------------------------------------------------------------------------------\n# Graph stuff\n# --------------------------------------------------------------------------------\n\n\ndef get_igraph_from_adjacency(adjacency, directed=None):\n \"\"\"Get igraph graph from adjacency matrix.\"\"\"\n import igraph as ig\n\n sources, targets = adjacency.nonzero()\n weights = adjacency[sources, targets]\n if isinstance(weights, np.matrix):\n weights = weights.A1\n g = ig.Graph(directed=directed)\n g.add_vertices(adjacency.shape[0]) # this adds adjacency.shape[0] vertices\n g.add_edges(list(zip(sources, targets)))\n with suppress(KeyError):\n g.es[\"weight\"] = weights\n if g.vcount() != adjacency.shape[0]:\n logg.warning(\n f\"The constructed graph has only {g.vcount()} nodes. \"\n \"Your adjacency matrix contained redundant nodes.\"\n )\n return g\n\n\n# --------------------------------------------------------------------------------\n# Group stuff\n# --------------------------------------------------------------------------------\n\n\ndef compute_association_matrix_of_groups(\n adata: AnnData,\n prediction: str,\n reference: str,\n *,\n normalization: Literal[\"prediction\", \"reference\"] = \"prediction\",\n threshold: float = 0.01,\n max_n_names: int | None = 2,\n):\n \"\"\"Compute overlaps between groups.\n\n See ``identify_groups`` for identifying the groups.\n\n Parameters\n ----------\n adata\n prediction\n Field name of adata.obs.\n reference\n Field name of adata.obs.\n normalization\n Whether to normalize with respect to the predicted groups or the\n reference groups.\n threshold\n Do not consider associations whose overlap is below this fraction.\n max_n_names\n Control how many reference names you want to be associated with per\n predicted name. Set to `None`, if you want all.\n\n Returns\n -------\n asso_names\n List of associated reference names\n (`max_n_names` for each predicted name).\n asso_matrix\n Matrix where rows correspond to the predicted labels and columns to the\n reference labels, entries are proportional to degree of association.\n \"\"\"\n if normalization not in {\"prediction\", \"reference\"}:\n raise ValueError(\n '`normalization` needs to be either \"prediction\" or \"reference\".'\n )\n sanitize_anndata(adata)\n cats = adata.obs[reference].cat.categories\n for cat in cats:\n if cat in settings.categories_to_ignore:\n logg.info(\n f\"Ignoring category {cat!r} \"\n \"as it’s in `settings.categories_to_ignore`.\"\n )\n asso_names = []\n asso_matrix = []\n for ipred_group, pred_group in enumerate(adata.obs[prediction].cat.categories):\n if \"?\" in pred_group:\n pred_group = str(ipred_group)\n # starting from numpy version 1.13, subtractions of boolean arrays are deprecated\n mask_pred = adata.obs[prediction].values == pred_group\n mask_pred_int = mask_pred.astype(np.int8)\n asso_matrix += [[]]\n for ref_group in adata.obs[reference].cat.categories:\n mask_ref = (adata.obs[reference].values == ref_group).astype(np.int8)\n mask_ref_or_pred = mask_ref.copy()\n mask_ref_or_pred[mask_pred] = 1\n # e.g. if the pred group is contained in mask_ref, mask_ref and\n # mask_ref_or_pred are the same\n if normalization == \"prediction\":\n # compute which fraction of the predicted group is contained in\n # the ref group\n ratio_contained = (\n np.sum(mask_pred_int) - np.sum(mask_ref_or_pred - mask_ref)\n ) / np.sum(mask_pred_int)\n else:\n # compute which fraction of the reference group is contained in\n # the predicted group\n ratio_contained = (\n np.sum(mask_ref) - np.sum(mask_ref_or_pred - mask_pred_int)\n ) / np.sum(mask_ref)\n asso_matrix[-1] += [ratio_contained]\n name_list_pred = [\n cats[i] if cats[i] not in settings.categories_to_ignore else \"\"\n for i in np.argsort(asso_matrix[-1])[::-1]\n if asso_matrix[-1][i] > threshold\n ]\n asso_names += [\"\\n\".join(name_list_pred[:max_n_names])]\n Result = namedtuple(\n \"compute_association_matrix_of_groups\", [\"asso_names\", \"asso_matrix\"]\n )\n return Result(asso_names=asso_names, asso_matrix=np.array(asso_matrix))\n\n\ndef get_associated_colors_of_groups(reference_colors, asso_matrix):\n return [\n {\n reference_colors[i_ref]: asso_matrix[i_pred, i_ref]\n for i_ref in range(asso_matrix.shape[1])\n }\n for i_pred in range(asso_matrix.shape[0])\n ]\n\n\ndef identify_groups(ref_labels, pred_labels, *, return_overlaps: bool = False):\n \"\"\"Which predicted label explains which reference label?\n\n A predicted label explains the reference label which maximizes the minimum\n of ``relative_overlaps_pred`` and ``relative_overlaps_ref``.\n\n Compare this with ``compute_association_matrix_of_groups``.\n\n Returns\n -------\n A dictionary of length ``len(np.unique(ref_labels))`` that stores for each\n reference label the predicted label that best explains it.\n\n If ``return_overlaps`` is ``True``, this will in addition return the overlap\n of the reference group with the predicted group; normalized with respect to\n the reference group size and the predicted group size, respectively.\n \"\"\"\n ref_unique, ref_counts = np.unique(ref_labels, return_counts=True)\n ref_dict = dict(zip(ref_unique, ref_counts))\n pred_unique, pred_counts = np.unique(pred_labels, return_counts=True)\n pred_dict = dict(zip(pred_unique, pred_counts))\n associated_predictions = {}\n associated_overlaps = {}\n for ref_label in ref_unique:\n sub_pred_unique, sub_pred_counts = np.unique(\n pred_labels[ref_label == ref_labels], return_counts=True\n )\n relative_overlaps_pred = [\n sub_pred_counts[i] / pred_dict[n] for i, n in enumerate(sub_pred_unique)\n ]\n relative_overlaps_ref = [\n sub_pred_counts[i] / ref_dict[ref_label]\n for i, n in enumerate(sub_pred_unique)\n ]\n relative_overlaps = np.c_[relative_overlaps_pred, relative_overlaps_ref]\n relative_overlaps_min = np.min(relative_overlaps, axis=1)\n pred_best_index = np.argsort(relative_overlaps_min)[::-1]\n associated_predictions[ref_label] = sub_pred_unique[pred_best_index]\n associated_overlaps[ref_label] = relative_overlaps[pred_best_index]\n if return_overlaps:\n return associated_predictions, associated_overlaps\n else:\n return associated_predictions\n\n\n# --------------------------------------------------------------------------------\n# Other stuff\n# --------------------------------------------------------------------------------\n\n\n# backwards compat... remove this in the future\ndef sanitize_anndata(adata: AnnData) -> None:\n \"\"\"Transform string annotations to categoricals.\"\"\"\n adata._sanitize()\n\n\ndef view_to_actual(adata: AnnData) -> None:\n if adata.is_view:\n warnings.warn(\n \"Received a view of an AnnData. Making a copy.\",\n stacklevel=2,\n )\n adata._init_as_actual(adata.copy())\n\n\ndef moving_average(a: np.ndarray, n: int):\n \"\"\"Moving average over one-dimensional array.\n\n Parameters\n ----------\n a\n One-dimensional array.\n n\n Number of entries to average over. n=2 means averaging over the currrent\n the previous entry.\n\n Returns\n -------\n An array view storing the moving average.\n \"\"\"\n ret = np.cumsum(a, dtype=float)\n ret[n:] = ret[n:] - ret[:-n]\n return ret[n - 1 :] / n\n\n\ndef get_random_state(seed: AnyRandom) -> np.random.RandomState:\n if isinstance(seed, np.random.RandomState):\n return seed\n return np.random.RandomState(seed)\n\n\n# --------------------------------------------------------------------------------\n# Deal with tool parameters\n# --------------------------------------------------------------------------------\n\n\ndef update_params(\n old_params: Mapping[str, Any],\n new_params: Mapping[str, Any],\n *,\n check: bool = False,\n) -> dict[str, Any]:\n \"\"\"\\\n Update old_params with new_params.\n\n If check==False, this merely adds and overwrites the content of old_params.\n\n If check==True, this only allows updating of parameters that are already\n present in old_params.\n\n Parameters\n ----------\n old_params\n new_params\n check\n\n Returns\n -------\n updated_params\n \"\"\"\n updated_params = dict(old_params)\n if new_params: # allow for new_params to be None\n for key, val in new_params.items():\n if key not in old_params and check:\n raise ValueError(\n \"'\"\n + key\n + \"' is not a valid parameter key, \"\n + \"consider one of \\n\"\n + str(list(old_params.keys()))\n )\n if val is not None:\n updated_params[key] = val\n return updated_params\n\n\n# --------------------------------------------------------------------------------\n# Others\n# --------------------------------------------------------------------------------\n\n\nif TYPE_CHECKING:\n _SparseMatrix = Union[sparse.csr_matrix, sparse.csc_matrix]\n _MemoryArray = Union[NDArray, _SparseMatrix]\n _SupportedArray = Union[_MemoryArray, DaskArray]\n\n\n@singledispatch\ndef elem_mul(x: _SupportedArray, y: _SupportedArray) -> _SupportedArray:\n raise NotImplementedError\n\n\n@elem_mul.register(np.ndarray)\n@elem_mul.register(sparse.spmatrix)\ndef _elem_mul_in_mem(x: _MemoryArray, y: _MemoryArray) -> _MemoryArray:\n if isinstance(x, sparse.spmatrix):\n # returns coo_matrix, so cast back to input type\n return type(x)(x.multiply(y))\n return x * y\n\n\n@elem_mul.register(DaskArray)\ndef _elem_mul_dask(x: DaskArray, y: DaskArray) -> DaskArray:\n import dask.array as da\n\n return da.map_blocks(elem_mul, x, y)\n\n\nif TYPE_CHECKING:\n Scaling_T = TypeVar(\"Scaling_T\", DaskArray, np.ndarray)\n\n\ndef broadcast_axis(divisor: Scaling_T, axis: Literal[0, 1]) -> Scaling_T:\n divisor = np.ravel(divisor)\n if axis:\n return divisor[None, :]\n return divisor[:, None]\n\n\ndef check_op(op):\n if op not in {truediv, mul}:\n raise ValueError(f\"{op} not one of truediv or mul\")\n\n\n@singledispatch\ndef axis_mul_or_truediv(\n X: np.ndarray,\n scaling_array: np.ndarray,\n axis: Literal[0, 1],\n op: Callable[[Any, Any], Any],\n *,\n allow_divide_by_zero: bool = True,\n out: np.ndarray | None = None,\n) -> np.ndarray:\n check_op(op)\n scaling_array = broadcast_axis(scaling_array, axis)\n if op is mul:\n return np.multiply(X, scaling_array, out=out)\n if not allow_divide_by_zero:\n scaling_array = scaling_array.copy() + (scaling_array == 0)\n return np.true_divide(X, scaling_array, out=out)\n\n\n@axis_mul_or_truediv.register(sparse.csr_matrix)\n@axis_mul_or_truediv.register(sparse.csc_matrix)\ndef _(\n X: sparse.csr_matrix | sparse.csc_matrix,\n scaling_array,\n axis: Literal[0, 1],\n op: Callable[[Any, Any], Any],\n *,\n allow_divide_by_zero: bool = True,\n out: sparse.csr_matrix | sparse.csc_matrix | None = None,\n) -> sparse.csr_matrix | sparse.csc_matrix:\n check_op(op)\n if out is not None and X.data is not out.data:\n raise ValueError(\n \"`out` argument provided but not equal to X. This behavior is not supported for sparse matrix scaling.\"\n )\n if not allow_divide_by_zero and op is truediv:\n scaling_array = scaling_array.copy() + (scaling_array == 0)\n\n row_scale = axis == 0\n column_scale = axis == 1\n if row_scale:\n\n def new_data_op(x):\n return op(x.data, np.repeat(scaling_array, np.diff(x.indptr)))\n\n elif column_scale:\n\n def new_data_op(x):\n return op(x.data, scaling_array.take(x.indices, mode=\"clip\"))\n\n if X.format == \"csr\":\n indices = X.indices\n indptr = X.indptr\n if out is not None:\n X.data = new_data_op(X)\n return X\n return sparse.csr_matrix(\n (new_data_op(X), indices.copy(), indptr.copy()), shape=X.shape\n )\n transposed = X.T\n return axis_mul_or_truediv(\n transposed,\n scaling_array,\n op=op,\n axis=1 - axis,\n out=transposed,\n allow_divide_by_zero=allow_divide_by_zero,\n ).T\n\n\ndef make_axis_chunks(\n X: DaskArray, axis: Literal[0, 1]\n) -> tuple[tuple[int], tuple[int]]:\n if axis == 0:\n return (X.chunks[axis], (1,))\n return ((1,), X.chunks[axis])\n\n\n@axis_mul_or_truediv.register(DaskArray)\ndef _(\n X: DaskArray,\n scaling_array: Scaling_T,\n axis: Literal[0, 1],\n op: Callable[[Any, Any], Any],\n *,\n allow_divide_by_zero: bool = True,\n out: None = None,\n) -> DaskArray:\n check_op(op)\n if out is not None:\n raise TypeError(\n \"`out` is not `None`. Do not do in-place modifications on dask arrays.\"\n )\n\n import dask.array as da\n\n scaling_array = broadcast_axis(scaling_array, axis)\n row_scale = axis == 0\n column_scale = axis == 1\n\n if isinstance(scaling_array, DaskArray):\n if (row_scale and X.chunksize[0] != scaling_array.chunksize[0]) or (\n column_scale\n and (\n (\n len(scaling_array.chunksize) == 1\n and X.chunksize[1] != scaling_array.chunksize[0]\n )\n or (\n len(scaling_array.chunksize) == 2\n and X.chunksize[1] != scaling_array.chunksize[1]\n )\n )\n ):\n warnings.warn(\"Rechunking scaling_array in user operation\", UserWarning)\n scaling_array = scaling_array.rechunk(make_axis_chunks(X, axis))\n else:\n scaling_array = da.from_array(\n scaling_array,\n chunks=make_axis_chunks(X, axis),\n )\n return da.map_blocks(\n axis_mul_or_truediv,\n X,\n scaling_array,\n axis,\n op,\n meta=X._meta,\n out=out,\n allow_divide_by_zero=allow_divide_by_zero,\n )\n\n\n@overload\ndef axis_sum(\n X: sparse.spmatrix,\n *,\n axis: tuple[Literal[0, 1], ...] | Literal[0, 1] | None = None,\n dtype: DTypeLike | None = None,\n) -> np.matrix: ...\n\n\n@singledispatch\ndef axis_sum(\n X: np.ndarray,\n *,\n axis: tuple[Literal[0, 1], ...] | Literal[0, 1] | None = None,\n dtype: DTypeLike | None = None,\n) -> np.ndarray:\n return np.sum(X, axis=axis, dtype=dtype)\n\n\n@axis_sum.register(DaskArray)\ndef _(\n X: DaskArray,\n *,\n axis: tuple[Literal[0, 1], ...] | Literal[0, 1] | None = None,\n dtype: DTypeLike | None = None,\n) -> DaskArray:\n import dask.array as da\n\n if dtype is None:\n dtype = getattr(np.zeros(1, dtype=X.dtype).sum(), \"dtype\", object)\n\n if isinstance(X._meta, np.ndarray) and not isinstance(X._meta, np.matrix):\n return X.sum(axis=axis, dtype=dtype)\n\n def sum_drop_keepdims(*args, **kwargs):\n kwargs.pop(\"computing_meta\", None)\n # masked operations on sparse produce which numpy matrices gives the same API issues handled here\n if isinstance(X._meta, (sparse.spmatrix, np.matrix)) or isinstance(\n args[0], (sparse.spmatrix, np.matrix)\n ):\n kwargs.pop(\"keepdims\", None)\n axis = kwargs[\"axis\"]\n if isinstance(axis, tuple):\n if len(axis) != 1:\n raise ValueError(\n f\"`axis_sum` can only sum over one axis when `axis` arg is provided but got {axis} instead\"\n )\n kwargs[\"axis\"] = axis[0]\n # returns a np.matrix normally, which is undesireable\n return np.array(np.sum(*args, dtype=dtype, **kwargs))\n\n def aggregate_sum(*args, **kwargs):\n return np.sum(args[0], dtype=dtype, **kwargs)\n\n return da.reduction(\n X,\n sum_drop_keepdims,\n aggregate_sum,\n axis=axis,\n dtype=dtype,\n meta=np.array([], dtype=dtype),\n )\n\n\n@singledispatch\ndef check_nonnegative_integers(X: _SupportedArray) -> bool | DaskArray:\n \"\"\"Checks values of X to ensure it is count data\"\"\"\n raise NotImplementedError\n\n\n@check_nonnegative_integers.register(np.ndarray)\n@check_nonnegative_integers.register(sparse.spmatrix)\ndef _check_nonnegative_integers_in_mem(X: _MemoryArray) -> bool:\n from numbers import Integral\n\n data = X if isinstance(X, np.ndarray) else X.data\n # Check no negatives\n if np.signbit(data).any():\n return False\n # Check all are integers\n elif issubclass(data.dtype.type, Integral):\n return True\n return not np.any((data % 1) != 0)\n\n\n@check_nonnegative_integers.register(DaskArray)\ndef _check_nonnegative_integers_dask(X: DaskArray) -> DaskArray:\n return X.map_blocks(check_nonnegative_integers, dtype=bool, drop_axis=(0, 1))\n\n\ndef select_groups(\n adata: AnnData,\n groups_order_subset: list[str] | Literal[\"all\"] = \"all\",\n key: str = \"groups\",\n) -> tuple[list[str], NDArray[np.bool_]]:\n \"\"\"Get subset of groups in adata.obs[key].\"\"\"\n groups_order = adata.obs[key].cat.categories\n if key + \"_masks\" in adata.uns:\n groups_masks_obs = adata.uns[key + \"_masks\"]\n else:\n groups_masks_obs = np.zeros(\n (len(adata.obs[key].cat.categories), adata.obs[key].values.size), dtype=bool\n )\n for iname, name in enumerate(adata.obs[key].cat.categories):\n # if the name is not found, fallback to index retrieval\n if adata.obs[key].cat.categories[iname] in adata.obs[key].values:\n mask_obs = adata.obs[key].cat.categories[iname] == adata.obs[key].values\n else:\n mask_obs = str(iname) == adata.obs[key].values\n groups_masks_obs[iname] = mask_obs\n groups_ids = list(range(len(groups_order)))\n if groups_order_subset != \"all\":\n groups_ids = []\n for name in groups_order_subset:\n groups_ids.append(\n np.where(adata.obs[key].cat.categories.values == name)[0][0]\n )\n if len(groups_ids) == 0:\n # fallback to index retrieval\n groups_ids = np.where(\n np.in1d(\n np.arange(len(adata.obs[key].cat.categories)).astype(str),\n np.array(groups_order_subset),\n )\n )[0]\n if len(groups_ids) == 0:\n logg.debug(\n f\"{np.array(groups_order_subset)} invalid! specify valid \"\n f\"groups_order (or indices) from {adata.obs[key].cat.categories}\",\n )\n from sys import exit\n\n exit(0)\n groups_masks_obs = groups_masks_obs[groups_ids]\n groups_order_subset = adata.obs[key].cat.categories[groups_ids].values\n else:\n groups_order_subset = groups_order.values\n return groups_order_subset, groups_masks_obs\n\n\ndef warn_with_traceback(message, category, filename, lineno, file=None, line=None): # noqa: PLR0917\n \"\"\"Get full tracebacks when warning is raised by setting\n\n warnings.showwarning = warn_with_traceback\n\n See also\n --------\n https://stackoverflow.com/questions/22373927/get-traceback-of-warnings\n \"\"\"\n import traceback\n\n traceback.print_stack()\n log = ( # noqa: F841 # TODO Does this need fixing?\n file if hasattr(file, \"write\") else sys.stderr\n )\n settings.write(warnings.formatwarning(message, category, filename, lineno, line))\n\n\ndef warn_once(msg: str, category: type[Warning], stacklevel: int = 1):\n warnings.warn(msg, category, stacklevel=stacklevel)\n # You'd think `'once'` works, but it doesn't at the repl and in notebooks\n warnings.filterwarnings(\"ignore\", category=category, message=re.escape(msg))\n\n\ndef subsample(\n X: np.ndarray,\n subsample: int = 1,\n seed: int = 0,\n) -> tuple[np.ndarray, np.ndarray]:\n \"\"\"\\\n Subsample a fraction of 1/subsample samples from the rows of X.\n\n Parameters\n ----------\n X\n Data array.\n subsample\n 1/subsample is the fraction of data sampled, n = X.shape[0]/subsample.\n seed\n Seed for sampling.\n\n Returns\n -------\n Xsampled\n Subsampled X.\n rows\n Indices of rows that are stored in Xsampled.\n \"\"\"\n if subsample == 1 and seed == 0:\n return X, np.arange(X.shape[0], dtype=int)\n if seed == 0:\n # this sequence is defined simply by skipping rows\n # is faster than sampling\n rows = np.arange(0, X.shape[0], subsample, dtype=int)\n n = rows.size\n Xsampled = np.array(X[rows])\n else:\n if seed < 0:\n raise ValueError(f\"Invalid seed value < 0: {seed}\")\n n = int(X.shape[0] / subsample)\n np.random.seed(seed)\n Xsampled, rows = subsample_n(X, n=n)\n logg.debug(f\"... subsampled to {n} of {X.shape[0]} data points\")\n return Xsampled, rows\n\n\ndef subsample_n(\n X: np.ndarray, n: int = 0, seed: int = 0\n) -> tuple[np.ndarray, np.ndarray]:\n \"\"\"Subsample n samples from rows of array.\n\n Parameters\n ----------\n X\n Data array.\n n\n Sample size.\n seed\n Seed for sampling.\n\n Returns\n -------\n Xsampled\n Subsampled X.\n rows\n Indices of rows that are stored in Xsampled.\n \"\"\"\n if n < 0:\n raise ValueError(\"n must be greater 0\")\n np.random.seed(seed)\n n = X.shape[0] if (n == 0 or n > X.shape[0]) else n\n rows = np.random.choice(X.shape[0], size=n, replace=False)\n Xsampled = X[rows]\n return Xsampled, rows\n\n\ndef check_presence_download(filename: Path, backup_url):\n \"\"\"Check if file is present otherwise download.\"\"\"\n if not filename.is_file():\n from ..readwrite import _download\n\n _download(backup_url, filename)\n\n\ndef lazy_import(full_name):\n \"\"\"Imports a module in a way that it’s only executed on member access\"\"\"\n try:\n return sys.modules[full_name]\n except KeyError:\n spec = importlib.util.find_spec(full_name)\n module = importlib.util.module_from_spec(spec)\n loader = importlib.util.LazyLoader(spec.loader)\n # Make module with proper locking and get it inserted into sys.modules.\n loader.exec_module(module)\n return module\n\n\n# --------------------------------------------------------------------------------\n# Neighbors\n# --------------------------------------------------------------------------------\n\n\ndef _fallback_to_uns(dct, conns, dists, conns_key, dists_key):\n if conns is None and conns_key in dct:\n conns = dct[conns_key]\n if dists is None and dists_key in dct:\n dists = dct[dists_key]\n\n return conns, dists\n\n\nclass NeighborsView:\n \"\"\"Convenience class for accessing neighbors graph representations.\n\n Allows to access neighbors distances, connectivities and settings\n dictionary in a uniform manner.\n\n Parameters\n ----------\n\n adata\n AnnData object.\n key\n This defines where to look for neighbors dictionary,\n connectivities, distances.\n\n neigh = NeighborsView(adata, key)\n neigh['distances']\n neigh['connectivities']\n neigh['params']\n 'connectivities' in neigh\n 'params' in neigh\n\n is the same as\n\n adata.obsp[adata.uns[key]['distances_key']]\n adata.obsp[adata.uns[key]['connectivities_key']]\n adata.uns[key]['params']\n adata.uns[key]['connectivities_key'] in adata.obsp\n 'params' in adata.uns[key]\n \"\"\"\n\n def __init__(self, adata: AnnData, key=None):\n self._connectivities = None\n self._distances = None\n\n if key is None or key == \"neighbors\":\n if \"neighbors\" not in adata.uns:\n raise KeyError('No \"neighbors\" in .uns')\n self._neighbors_dict = adata.uns[\"neighbors\"]\n self._conns_key = \"connectivities\"\n self._dists_key = \"distances\"\n else:\n if key not in adata.uns:\n raise KeyError(f'No \"{key}\" in .uns')\n self._neighbors_dict = adata.uns[key]\n self._conns_key = self._neighbors_dict[\"connectivities_key\"]\n self._dists_key = self._neighbors_dict[\"distances_key\"]\n\n if self._conns_key in adata.obsp:\n self._connectivities = adata.obsp[self._conns_key]\n if self._dists_key in adata.obsp:\n self._distances = adata.obsp[self._dists_key]\n\n # fallback to uns\n self._connectivities, self._distances = _fallback_to_uns(\n self._neighbors_dict,\n self._connectivities,\n self._distances,\n self._conns_key,\n self._dists_key,\n )\n\n @overload\n def __getitem__(\n self, key: Literal[\"distances\", \"connectivities\"]\n ) -> sparse.csr_matrix: ...\n @overload\n def __getitem__(self, key: Literal[\"params\"]) -> NeighborsParams: ...\n @overload\n def __getitem__(self, key: Literal[\"rp_forest\"]) -> RPForestDict: ...\n @overload\n def __getitem__(self, key: Literal[\"connectivities_key\"]) -> str: ...\n\n def __getitem__(self, key: str):\n if key == \"distances\":\n if \"distances\" not in self:\n raise KeyError(f'No \"{self._dists_key}\" in .obsp')\n return self._distances\n elif key == \"connectivities\":\n if \"connectivities\" not in self:\n raise KeyError(f'No \"{self._conns_key}\" in .obsp')\n return self._connectivities\n elif key == \"connectivities_key\":\n return self._conns_key\n else:\n return self._neighbors_dict[key]\n\n def __contains__(self, key: str) -> bool:\n if key == \"distances\":\n return self._distances is not None\n elif key == \"connectivities\":\n return self._connectivities is not None\n else:\n return key in self._neighbors_dict\n\n\ndef _choose_graph(adata, obsp, neighbors_key):\n \"\"\"Choose connectivities from neighbbors or another obsp column\"\"\"\n if obsp is not None and neighbors_key is not None:\n raise ValueError(\n \"You can't specify both obsp, neighbors_key. \" \"Please select only one.\"\n )\n\n if obsp is not None:\n return adata.obsp[obsp]\n else:\n neighbors = NeighborsView(adata, neighbors_key)\n if \"connectivities\" not in neighbors:\n raise ValueError(\n \"You need to run `pp.neighbors` first \"\n \"to compute a neighborhood graph.\"\n )\n return neighbors[\"connectivities\"]\n\n\ndef _resolve_axis(\n axis: Literal[\"obs\", 0, \"var\", 1],\n) -> tuple[Literal[0], Literal[\"obs\"]] | tuple[Literal[1], Literal[\"var\"]]:\n if axis in {0, \"obs\"}:\n return (0, \"obs\")\n if axis in {1, \"var\"}:\n return (1, \"var\")\n raise ValueError(f\"`axis` must be either 0, 1, 'obs', or 'var', was {axis!r}\")\n\n\ndef is_backed_type(X: object) -> bool:\n return isinstance(X, (SparseDataset, h5py.File, h5py.Dataset))\n\n\ndef raise_not_implemented_error_if_backed_type(X: object, method_name: str) -> None:\n if is_backed_type(X):\n raise NotImplementedError(\n f\"{method_name} is not implemented for matrices of type {type(X)}\"\n )\n\n\nfrom __future__ import annotations\n\nfrom collections.abc import Callable\nfrom functools import partial, singledispatch, wraps\nfrom numbers import Integral\nfrom typing import TYPE_CHECKING, TypeVar, overload\n\nimport numpy as np\nfrom numba import njit\nfrom scipy import sparse\n\nfrom ..._compat import DaskArray\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from numpy.typing import NDArray\n\nC = TypeVar(\"C\", bound=Callable)\n\n\ndef _check_axis_supported(wrapped: C) -> C:\n @wraps(wrapped)\n def func(a, axis=None):\n if axis is not None:\n if not isinstance(axis, Integral):\n raise TypeError(\"axis must be integer or None.\")\n if axis not in (0, 1):\n raise NotImplementedError(\"We only support axis 0 and 1 at the moment\")\n return wrapped(a, axis)\n\n return func\n\n\n@overload\ndef is_constant(a: NDArray, axis: None = None) -> bool: ...\n\n\n@overload\ndef is_constant(a: NDArray, axis: Literal[0, 1]) -> NDArray[np.bool_]: ...\n\n\n@_check_axis_supported\n@singledispatch\ndef is_constant(\n a: NDArray, axis: Literal[0, 1] | None = None\n) -> bool | NDArray[np.bool_]:\n \"\"\"\n Check whether values in array are constant.\n\n Params\n ------\n a\n Array to check\n axis\n Axis to reduce over.\n\n\n Returns\n -------\n Boolean array, True values were constant.\n\n Example\n -------\n\n >>> a = np.array([[0, 1], [0, 0]])\n >>> a\n array([[0, 1],\n [0, 0]])\n >>> is_constant(a)\n False\n >>> is_constant(a, axis=0)\n array([ True, False])\n >>> is_constant(a, axis=1)\n array([False, True])\n \"\"\"\n raise NotImplementedError()\n\n\n@is_constant.register(np.ndarray)\ndef _(a: NDArray, axis: Literal[0, 1] | None = None) -> bool | NDArray[np.bool_]:\n # Should eventually support nd, not now.\n if axis is None:\n return bool((a == a.flat[0]).all())\n if axis == 0:\n return _is_constant_rows(a.T)\n elif axis == 1:\n return _is_constant_rows(a)\n\n\ndef _is_constant_rows(a: NDArray) -> NDArray[np.bool_]:\n b = np.broadcast_to(a[:, 0][:, np.newaxis], a.shape)\n return (a == b).all(axis=1)\n\n\n@is_constant.register(sparse.csr_matrix)\ndef _(\n a: sparse.csr_matrix, axis: Literal[0, 1] | None = None\n) -> bool | NDArray[np.bool_]:\n if axis is None:\n if len(a.data) == np.multiply(*a.shape):\n return is_constant(a.data)\n else:\n return (a.data == 0).all()\n if axis == 1:\n return _is_constant_csr_rows(a.data, a.indices, a.indptr, a.shape)\n elif axis == 0:\n a = a.T.tocsr()\n return _is_constant_csr_rows(a.data, a.indices, a.indptr, a.shape)\n\n\n@njit\ndef _is_constant_csr_rows(\n data: NDArray[np.number],\n indices: NDArray[np.integer],\n indptr: NDArray[np.integer],\n shape: tuple[int, int],\n):\n n = len(indptr) - 1\n result = np.ones(n, dtype=np.bool_)\n for i in range(n):\n start = indptr[i]\n stop = indptr[i + 1]\n val = data[start] if stop - start == shape[1] else 0\n for j in range(start, stop):\n if data[j] != val:\n result[i] = False\n break\n return result\n\n\n@is_constant.register(sparse.csc_matrix)\ndef _(\n a: sparse.csc_matrix, axis: Literal[0, 1] | None = None\n) -> bool | NDArray[np.bool_]:\n if axis is None:\n if len(a.data) == np.multiply(*a.shape):\n return is_constant(a.data)\n else:\n return (a.data == 0).all()\n if axis == 0:\n return _is_constant_csr_rows(a.data, a.indices, a.indptr, a.shape[::-1])\n elif axis == 1:\n a = a.T.tocsc()\n return _is_constant_csr_rows(a.data, a.indices, a.indptr, a.shape[::-1])\n\n\n@is_constant.register(DaskArray)\ndef _(a: DaskArray, axis: Literal[0, 1] | None = None) -> bool | NDArray[np.bool_]:\n if axis is None:\n v = a[tuple(0 for _ in range(a.ndim))].compute()\n return (a == v).all()\n # TODO: use overlapping blocks and reduction instead of `drop_axis`\n return a.map_blocks(partial(is_constant, axis=axis), drop_axis=axis)\n\n\n\n\n\"\"\"\\\nExporting to formats for other software.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport json\nimport logging as logg\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nimport h5py\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport scipy.sparse\nfrom pandas.api.types import CategoricalDtype\n\nfrom .._compat import old_positionals\nfrom .._utils import NeighborsView\nfrom ..preprocessing._utils import _get_mean_var\n\nif TYPE_CHECKING:\n from collections.abc import Iterable, Mapping\n\n from anndata import AnnData\n\n__all__ = [\"spring_project\", \"cellbrowser\"]\n\n\n@old_positionals(\n \"subplot_name\",\n \"cell_groupings\",\n \"custom_color_tracks\",\n \"total_counts_key\",\n \"neighbors_key\",\n \"overwrite\",\n)\ndef spring_project(\n adata: AnnData,\n project_dir: Path | str,\n embedding_method: str,\n *,\n subplot_name: str | None = None,\n cell_groupings: str | Iterable[str] | None = None,\n custom_color_tracks: str | Iterable[str] | None = None,\n total_counts_key: str = \"n_counts\",\n neighbors_key: str | None = None,\n overwrite: bool = False,\n) -> None:\n \"\"\"\\\n Exports to a SPRING project directory :cite:p:`Weinreb2017`.\n\n Visualize annotation present in `adata`. By default, export all gene expression data\n from `adata.raw` and categorical and continuous annotations present in `adata.obs`.\n\n See `SPRING `__ or :cite:t:`Weinreb2017` for details.\n\n Parameters\n ----------\n adata\n Annotated data matrix: `adata.uns['neighbors']` needs to\n be present.\n project_dir\n Path to directory for exported SPRING files.\n embedding_method\n Name of a 2-D embedding in `adata.obsm`\n subplot_name\n Name of subplot folder to be created at `project_dir+\"/\"+subplot_name`\n cell_groupings\n Instead of importing all categorical annotations when `None`,\n pass a list of keys for `adata.obs`.\n custom_color_tracks\n Specify specific `adata.obs` keys for continuous coloring.\n total_counts_key\n Name of key for total transcript counts in `adata.obs`.\n overwrite\n When `True`, existing counts matrices in `project_dir` are overwritten.\n\n Examples\n --------\n See this `tutorial `__.\n \"\"\"\n\n # need to get nearest neighbors first\n if neighbors_key is None:\n neighbors_key = \"neighbors\"\n\n if neighbors_key not in adata.uns:\n raise ValueError(\"Run `sc.pp.neighbors` first.\")\n\n # check that requested 2-D embedding has been generated\n if embedding_method not in adata.obsm_keys():\n if \"X_\" + embedding_method in adata.obsm_keys():\n embedding_method = \"X_\" + embedding_method\n else:\n if embedding_method in adata.uns:\n embedding_method = (\n \"X_\"\n + embedding_method\n + \"_\"\n + adata.uns[embedding_method][\"params\"][\"layout\"]\n )\n else:\n raise ValueError(\n f\"Run the specified embedding method `{embedding_method}` first.\"\n )\n\n coords = adata.obsm[embedding_method]\n\n # Make project directory and subplot directory (subplot has same name as project)\n # For now, the subplot is just all cells in adata\n project_dir = Path(project_dir)\n subplot_dir = (\n project_dir.parent if subplot_name is None else project_dir / subplot_name\n )\n subplot_dir.mkdir(parents=True, exist_ok=True)\n print(f\"Writing subplot to {subplot_dir}\")\n\n # Write counts matrices as hdf5 files and npz if they do not already exist\n # or if user requires overwrite.\n # To do: check if Alex's h5sparse format will allow fast loading from just\n # one file.\n write_counts_matrices = True\n base_dir_filelist = [\n \"counts_norm_sparse_genes.hdf5\",\n \"counts_norm_sparse_cells.hdf5\",\n \"counts_norm.npz\",\n \"total_counts.txt\",\n \"genes.txt\",\n ]\n if all((project_dir / f).is_file() for f in base_dir_filelist):\n if not overwrite:\n logg.warning(\n f\"{project_dir} is an existing SPRING folder. A new subplot will be created, but \"\n \"you must set `overwrite=True` to overwrite counts matrices.\"\n )\n write_counts_matrices = False\n else:\n logg.warning(f\"Overwriting the files in {project_dir}.\")\n\n # Ideally, all genes will be written from adata.raw\n if adata.raw is not None:\n E = adata.raw.X.tocsc()\n gene_list = list(adata.raw.var_names)\n else:\n E = adata.X.tocsc()\n gene_list = list(adata.var_names)\n\n # Keep track of total counts per cell if present\n if total_counts_key in adata.obs:\n total_counts = np.array(adata.obs[total_counts_key])\n else:\n total_counts = E.sum(1).A1\n\n # Write the counts matrices to project directory\n if write_counts_matrices:\n write_hdf5_genes(E, gene_list, project_dir / \"counts_norm_sparse_genes.hdf5\")\n write_hdf5_cells(E, project_dir / \"counts_norm_sparse_cells.hdf5\")\n write_sparse_npz(E, project_dir / \"counts_norm.npz\")\n with (project_dir / \"genes.txt\").open(\"w\") as o:\n for g in gene_list:\n o.write(g + \"\\n\")\n np.savetxt(project_dir / \"total_counts.txt\", total_counts)\n\n # Get categorical and continuous metadata\n categorical_extras = {}\n continuous_extras = {}\n if cell_groupings is None:\n for obs_name in adata.obs:\n if isinstance(adata.obs[obs_name].dtype, CategoricalDtype):\n categorical_extras[obs_name] = [str(x) for x in adata.obs[obs_name]]\n else:\n if isinstance(cell_groupings, str):\n cell_groupings = [cell_groupings]\n for obs_name in cell_groupings:\n if obs_name not in adata.obs:\n logg.warning(f\"Cell grouping {obs_name!r} is not in adata.obs\")\n elif isinstance(adata.obs[obs_name].dtype, CategoricalDtype):\n categorical_extras[obs_name] = [str(x) for x in adata.obs[obs_name]]\n else:\n logg.warning(\n f\"Cell grouping {obs_name!r} is not a categorical variable\"\n )\n if custom_color_tracks is None:\n for obs_name in adata.obs:\n if not isinstance(adata.obs[obs_name].dtype, CategoricalDtype):\n continuous_extras[obs_name] = np.array(adata.obs[obs_name])\n else:\n if isinstance(custom_color_tracks, str):\n custom_color_tracks = [custom_color_tracks]\n for obs_name in custom_color_tracks:\n if obs_name not in adata.obs:\n logg.warning(f\"Custom color track {obs_name!r} is not in adata.obs\")\n elif not isinstance(adata.obs[obs_name].dtype, CategoricalDtype):\n continuous_extras[obs_name] = np.array(adata.obs[obs_name])\n else:\n logg.warning(\n f\"Custom color track {obs_name!r} is not a continuous variable\"\n )\n\n # Write continuous colors\n continuous_extras[\"Uniform\"] = np.zeros(E.shape[0])\n _write_color_tracks(continuous_extras, subplot_dir / \"color_data_gene_sets.csv\")\n\n # Create and write a dictionary of color profiles to be used by the visualizer\n color_stats = {}\n color_stats = _get_color_stats_genes(color_stats, E, gene_list)\n color_stats = _get_color_stats_custom(color_stats, continuous_extras)\n _write_color_stats(subplot_dir / \"color_stats.json\", color_stats)\n\n # Write categorical data\n categorical_coloring_data = {}\n categorical_coloring_data = _build_categ_colors(\n categorical_coloring_data, categorical_extras\n )\n _write_cell_groupings(\n subplot_dir / \"categorical_coloring_data.json\", categorical_coloring_data\n )\n\n # Write graph in two formats for backwards compatibility\n edges = _get_edges(adata, neighbors_key)\n _write_graph(subplot_dir / \"graph_data.json\", E.shape[0], edges)\n _write_edges(subplot_dir / \"edges.csv\", edges)\n\n # Write cell filter; for now, subplots must be generated from within SPRING,\n # so cell filter includes all cells.\n np.savetxt(subplot_dir / \"cell_filter.txt\", np.arange(E.shape[0]), fmt=\"%i\")\n np.save(subplot_dir / \"cell_filter.npy\", np.arange(E.shape[0]))\n\n # Write 2-D coordinates, after adjusting to roughly match SPRING's default d3js force layout parameters\n coords = coords - coords.min(0)[None, :]\n coords = (\n coords * (np.array([1000, 1000]) / coords.ptp(0))[None, :]\n + np.array([200, -200])[None, :]\n )\n np.savetxt(\n subplot_dir / \"coordinates.txt\",\n np.hstack((np.arange(E.shape[0])[:, None], coords)),\n fmt=\"%i,%.6f,%.6f\",\n )\n\n # Write some useful intermediates, if they exist\n if \"X_pca\" in adata.obsm_keys():\n np.savez_compressed(\n subplot_dir / \"intermediates.npz\",\n Epca=adata.obsm[\"X_pca\"],\n total_counts=total_counts,\n )\n\n # Write PAGA data, if present\n if \"paga\" in adata.uns:\n clusts = np.array(adata.obs[adata.uns[\"paga\"][\"groups\"]].cat.codes)\n uniq_clusts = adata.obs[adata.uns[\"paga\"][\"groups\"]].cat.categories\n paga_coords = [coords[clusts == i, :].mean(0) for i in range(len(uniq_clusts))]\n _export_PAGA_to_SPRING(adata, paga_coords, subplot_dir / \"PAGA_data.json\")\n\n\n# --------------------------------------------------------------------------------\n# Helper Functions\n# --------------------------------------------------------------------------------\n\n\ndef _get_edges(adata, neighbors_key=None):\n neighbors = NeighborsView(adata, neighbors_key)\n if \"distances\" in neighbors: # these are sparse matrices\n matrix = neighbors[\"distances\"]\n else:\n matrix = neighbors[\"connectivities\"]\n matrix = matrix.tocoo()\n edges = [(i, j) for i, j in zip(matrix.row, matrix.col)]\n\n return edges\n\n\ndef write_hdf5_genes(E, gene_list, filename):\n '''SPRING standard: filename = main_spring_dir + \"counts_norm_sparse_genes.hdf5\"'''\n\n E = E.tocsc()\n\n hf = h5py.File(filename, \"w\")\n counts_group = hf.create_group(\"counts\")\n cix_group = hf.create_group(\"cell_ix\")\n\n hf.attrs[\"ncells\"] = E.shape[0]\n hf.attrs[\"ngenes\"] = E.shape[1]\n\n for iG, g in enumerate(gene_list):\n counts = E[:, iG].toarray().squeeze()\n cell_ix = np.nonzero(counts)[0]\n counts = counts[cell_ix]\n counts_group.create_dataset(g, data=counts)\n cix_group.create_dataset(g, data=cell_ix)\n\n hf.close()\n\n\ndef write_hdf5_cells(E, filename):\n '''SPRING standard: filename = main_spring_dir + \"counts_norm_sparse_cells.hdf5\"'''\n\n E = E.tocsr()\n\n hf = h5py.File(filename, \"w\")\n counts_group = hf.create_group(\"counts\")\n gix_group = hf.create_group(\"gene_ix\")\n\n hf.attrs[\"ncells\"] = E.shape[0]\n hf.attrs[\"ngenes\"] = E.shape[1]\n\n for iC in range(E.shape[0]):\n counts = E[iC, :].toarray().squeeze()\n gene_ix = np.nonzero(counts)[0]\n counts = counts[gene_ix]\n counts_group.create_dataset(str(iC), data=counts)\n gix_group.create_dataset(str(iC), data=gene_ix)\n\n hf.close()\n\n\ndef write_sparse_npz(E, filename, *, compressed: bool = False):\n \"\"\"SPRING standard: filename = f\"{main_spring_dir}/counts_norm.npz\".\"\"\"\n E = E.tocsc()\n scipy.sparse.save_npz(filename, E, compressed=compressed)\n\n\ndef _write_graph(filename, n_nodes, edges):\n nodes = [{\"name\": int(i), \"number\": int(i)} for i in range(n_nodes)]\n edges = [{\"source\": int(i), \"target\": int(j), \"distance\": 0} for i, j in edges]\n out = {\"nodes\": nodes, \"links\": edges}\n Path(filename).write_text(json.dumps(out, indent=4, separators=(\",\", \": \")))\n\n\ndef _write_edges(filename, edges):\n with Path(filename).open(\"w\") as f:\n for e in edges:\n f.write(f\"{e[0]};{e[1]}\\n\")\n\n\ndef _write_color_tracks(ctracks, fname):\n out = []\n for name, score in ctracks.items():\n line = f\"{name},\" + \",\".join(f\"{x:.3f}\" for x in score)\n out += [line]\n out = sorted(out, key=lambda x: x.split(\",\")[0])\n Path(fname).write_text(\"\\n\".join(out))\n\n\ndef _frac_to_hex(frac):\n rgb = tuple(np.array(np.array(plt.cm.jet(frac)[:3]) * 255, dtype=int))\n return \"#{:02x}{:02x}{:02x}\".format(*rgb)\n\n\ndef _get_color_stats_genes(color_stats, E, gene_list):\n means, variances = _get_mean_var(E)\n stdevs = np.zeros(variances.shape, dtype=float)\n stdevs[variances > 0] = np.sqrt(variances[variances > 0])\n mins = E.min(0).todense().A1\n maxes = E.max(0).todense().A1\n\n pctl = 99.6\n pctl_n = (100 - pctl) / 100.0 * E.shape[0]\n pctls = np.zeros(E.shape[1], dtype=float)\n for iG in range(E.shape[1]):\n n_nonzero = E.indptr[iG + 1] - E.indptr[iG]\n if n_nonzero > pctl_n:\n pctls[iG] = np.percentile(\n E.data[E.indptr[iG] : E.indptr[iG + 1]], 100 - 100 * pctl_n / n_nonzero\n )\n else:\n pctls[iG] = 0\n color_stats[gene_list[iG]] = tuple(\n map(float, (means[iG], stdevs[iG], mins[iG], maxes[iG], pctls[iG]))\n )\n return color_stats\n\n\ndef _get_color_stats_custom(color_stats, custom_colors):\n for k, v in custom_colors.items():\n color_stats[k] = tuple(\n map(\n float,\n (np.mean(v), np.std(v), np.min(v), np.max(v), np.percentile(v, 99)),\n )\n )\n return color_stats\n\n\ndef _write_color_stats(filename, color_stats):\n Path(filename).write_text(json.dumps(color_stats, indent=4, sort_keys=True))\n\n\ndef _build_categ_colors(categorical_coloring_data, cell_groupings):\n for k, labels in cell_groupings.items():\n label_colors = {\n l: _frac_to_hex(float(i) / len(set(labels)))\n for i, l in enumerate(list(set(labels)))\n }\n categorical_coloring_data[k] = {\n \"label_colors\": label_colors,\n \"label_list\": labels,\n }\n return categorical_coloring_data\n\n\ndef _write_cell_groupings(filename, categorical_coloring_data):\n Path(filename).write_text(\n json.dumps(categorical_coloring_data, indent=4, sort_keys=True)\n )\n\n\ndef _export_PAGA_to_SPRING(adata, paga_coords, outpath):\n # retrieve node data\n group_key = adata.uns[\"paga\"][\"groups\"]\n names = adata.obs[group_key].cat.categories\n coords = [list(xy) for xy in paga_coords]\n\n sizes = list(adata.uns[group_key + \"_sizes\"])\n clus_labels = adata.obs[group_key].cat.codes.values\n cell_groups = [\n [int(j) for j in np.nonzero(clus_labels == i)[0]] for i in range(len(names))\n ]\n\n if group_key + \"_colors\" in adata.uns:\n colors = list(adata.uns[group_key + \"_colors\"])\n else:\n import scanpy.plotting.utils\n\n scanpy.plotting.utils.add_colors_for_categorical_sample_annotation(\n adata, group_key\n )\n colors = list(adata.uns[group_key + \"_colors\"])\n\n # retrieve edge level data\n sources, targets = adata.uns[\"paga\"][\"connectivities\"].nonzero()\n weights = np.sqrt(adata.uns[\"paga\"][\"connectivities\"].data) / 3\n\n # save a threshold weight for showing edges so that by default,\n # the number of edges shown is 8X the number of nodes\n if len(names) * 8 > len(weights):\n min_edge_weight_view = 0\n else:\n min_edge_weight_view = sorted(weights)[-len(names) * 8]\n\n # save another threshold for even saving edges at all, with 100 edges per node\n if len(weights) < 100 * len(names):\n min_edge_weight_save = 0\n else:\n min_edge_weight_save = sorted(weights)[-len(names) * 100]\n\n # make node list\n nodes = []\n for i, name, xy, color, size, cells in zip(\n range(len(names)), names, coords, colors, sizes, cell_groups\n ):\n nodes.append(\n {\n \"index\": i,\n \"size\": int(size),\n \"color\": color,\n \"coordinates\": xy,\n \"cells\": cells,\n \"name\": name,\n }\n )\n\n # make link list, avoid redundant encoding (graph is undirected)\n links = []\n for source, target, weight in zip(sources, targets, weights):\n if source < target and weight > min_edge_weight_save:\n links.append(\n {\"source\": int(source), \"target\": int(target), \"weight\": float(weight)}\n )\n\n # save data about edge weights\n edge_weight_meta = {\n \"min_edge_weight\": min_edge_weight_view,\n \"max_edge_weight\": np.max(weights),\n }\n\n PAGA_data = {\"nodes\": nodes, \"links\": links, \"edge_weight_meta\": edge_weight_meta}\n\n import json\n\n Path(outpath).write_text(json.dumps(PAGA_data, indent=4))\n\n return None\n\n\n@old_positionals(\n \"embedding_keys\",\n \"annot_keys\",\n \"cluster_field\",\n \"nb_marker\",\n \"skip_matrix\",\n \"html_dir\",\n \"port\",\n \"do_debug\",\n)\ndef cellbrowser(\n adata: AnnData,\n data_dir: Path | str,\n data_name: str,\n *,\n embedding_keys: Iterable[str] | Mapping[str, str] | str | None = None,\n annot_keys: Iterable[str] | Mapping[str, str] | None = (\n \"louvain\",\n \"percent_mito\",\n \"n_genes\",\n \"n_counts\",\n ),\n cluster_field: str = \"louvain\",\n nb_marker: int = 50,\n skip_matrix: bool = False,\n html_dir: Path | str | None = None,\n port: int | None = None,\n do_debug: bool = False,\n):\n \"\"\"\\\n Export adata to a UCSC Cell Browser project directory. If `html_dir` is\n set, subsequently build the html files from the project directory into\n `html_dir`. If `port` is set, start an HTTP server in the background and\n serve `html_dir` on `port`.\n\n By default, export all gene expression data from `adata.raw`, the\n annotations `louvain`, `percent_mito`, `n_genes` and `n_counts` and the top\n `nb_marker` cluster markers. All existing files in data_dir are\n overwritten, except `cellbrowser.conf`.\n\n See `UCSC Cellbrowser `__ for\n details.\n\n Parameters\n ----------\n adata\n Annotated data matrix\n data_dir\n Path to directory for exported Cell Browser files.\n Usually these are the files `exprMatrix.tsv.gz`, `meta.tsv`,\n coordinate files like `tsne.coords.tsv`,\n and cluster marker gene lists like `markers.tsv`.\n A file `cellbrowser.conf` is also created with pointers to these files.\n As a result, each adata object should have its own project_dir.\n data_name\n Name of dataset in Cell Browser, a string without special characters.\n This is written to `data_dir/cellbrowser.conf`.\n Ideally this is a short unique name for the dataset,\n like `\"pbmc3k\"` or `\"tabulamuris\"`.\n embedding_keys\n 2-D embeddings in `adata.obsm` to export.\n The prefix `X_` or `X_draw_graph_` is not necessary.\n Coordinates missing from `adata` are skipped.\n By default (or when specifying `'all'` or `None`), these keys are tried:\n [`\"tsne\"`, `\"umap\"`, `\"pagaFa\"`, `\"pagaFr\"`, `\"pagaUmap\"`, `\"phate\"`,\n `\"fa\"`, `\"fr\"`, `\"kk\"`, `\"drl\"`, `\"rt\"`, `\"trimap\"`].\n For these, default display labels are automatically used.\n For other values, you can specify a mapping from coordinate name to\n display label, e.g. `{\"tsne\": \"t-SNE by Scanpy\"}`.\n annot_keys\n Annotations in `adata.obsm` to export.\n Can be a mapping from annotation column name to display label.\n Specify `None` for all available columns in `.obs`.\n skip_matrix\n Do not export the matrix.\n If you had previously exported this adata into the same `data_dir`,\n then there is no need to export the whole matrix again.\n This option will make the export a lot faster,\n e.g. when only coordinates or meta data were changed.\n html_dir\n If this variable is set, the export will build html\n files from `data_dir` to `html_dir`, creating html/js/json files.\n Usually there is one global html output directory for all datasets.\n Often, `html_dir` is located under a webserver's (like Apache)\n htdocs directory or is copied to one.\n A directory `html_dir`/`project_name` will be created and\n an index.html will be created under `html_dir` for all subdirectories.\n Existing files will be overwritten.\n If do not to use html_dir,\n you can use the command line tool `cbBuild` to build the html directory.\n port\n If this variable and `html_dir` are set,\n Python's built-in web server will be spawned as a daemon in the\n background and serve the files under `html_dir`.\n To kill the process, call `cellbrowser.cellbrowser.stop()`.\n do_debug\n Activate debugging output\n\n Examples\n --------\n See this\n `tutorial `__.\n \"\"\"\n\n try:\n import cellbrowser.cellbrowser as cb\n except ImportError:\n logg.error(\n \"The package cellbrowser is not installed. \"\n \"Install with 'pip install cellbrowser' and retry.\"\n )\n raise\n\n data_dir = str(data_dir)\n\n cb.setDebug(do_debug)\n cb.scanpyToCellbrowser(\n adata,\n data_dir,\n data_name,\n coordFields=embedding_keys,\n metaFields=annot_keys,\n clusterField=cluster_field,\n nb_marker=nb_marker,\n skipMatrix=skip_matrix,\n doDebug=None,\n )\n\n if html_dir is not None:\n html_dir = str(html_dir)\n cb.build(data_dir, html_dir, doDebug=None)\n if port is not None:\n cb.serve(html_dir, port)\n\n\nfrom __future__ import annotations\n\nimport contextlib\nfrom typing import TYPE_CHECKING\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom anndata import AnnData # noqa: TCH002\nfrom matplotlib.axes import Axes # noqa: TCH002\nfrom sklearn.utils import deprecated\n\nfrom .._compat import old_positionals\nfrom .._utils import _doc_params\nfrom .._utils._doctests import doctest_needs\nfrom ..plotting import _scrublet, _utils, embedding\nfrom ..plotting._docs import (\n doc_adata_color_etc,\n doc_edges_arrows,\n doc_scatter_embedding,\n doc_show_save_ax,\n)\nfrom ..plotting._tools.scatterplots import _wraps_plot_scatter\nfrom .tl._wishbone import _anndata_to_wishbone\n\nif TYPE_CHECKING:\n from collections.abc import Collection\n from typing import Any\n\n\n__all__ = [\n \"phate\",\n \"trimap\",\n \"harmony_timeseries\",\n \"sam\",\n \"wishbone_marker_trajectory\",\n]\n\n\n@doctest_needs(\"phate\")\n@_wraps_plot_scatter\n@_doc_params(\n adata_color_etc=doc_adata_color_etc,\n edges_arrows=doc_edges_arrows,\n scatter_bulk=doc_scatter_embedding,\n show_save_ax=doc_show_save_ax,\n)\ndef phate(adata: AnnData, **kwargs) -> list[Axes] | None:\n \"\"\"\\\n Scatter plot in PHATE basis.\n\n Parameters\n ----------\n {adata_color_etc}\n {edges_arrows}\n {scatter_bulk}\n {show_save_ax}\n\n Returns\n -------\n If `show==False`, a list of :class:`~matplotlib.axes.Axes` objects.\n Every second element corresponds to the 'right margin'\n drawing area for color bars and legends.\n\n Examples\n --------\n >>> from anndata import AnnData\n >>> import scanpy.external as sce\n >>> import phate\n >>> data, branches = phate.tree.gen_dla(\n ... n_dim=100,\n ... n_branch=20,\n ... branch_length=100,\n ... )\n >>> data.shape\n (2000, 100)\n >>> adata = AnnData(data)\n >>> adata.obs['branches'] = branches\n >>> sce.tl.phate(adata, k=5, a=20, t=150)\n >>> adata.obsm['X_phate'].shape\n (2000, 2)\n >>> sce.pl.phate(\n ... adata,\n ... color='branches',\n ... color_map='tab20',\n ... )\n \"\"\"\n return embedding(adata, \"phate\", **kwargs)\n\n\n@_wraps_plot_scatter\n@_doc_params(\n adata_color_etc=doc_adata_color_etc,\n edges_arrows=doc_edges_arrows,\n scatter_bulk=doc_scatter_embedding,\n show_save_ax=doc_show_save_ax,\n)\ndef trimap(adata: AnnData, **kwargs) -> Axes | list[Axes] | None:\n \"\"\"\\\n Scatter plot in TriMap basis.\n\n Parameters\n ----------\n {adata_color_etc}\n {edges_arrows}\n {scatter_bulk}\n {show_save_ax}\n\n Returns\n -------\n If `show==False` a :class:`~matplotlib.axes.Axes` or a list of it.\n \"\"\"\n return embedding(adata, \"trimap\", **kwargs)\n\n\n@_wraps_plot_scatter\n@_doc_params(\n adata_color_etc=doc_adata_color_etc,\n edges_arrows=doc_edges_arrows,\n scatter_bulk=doc_scatter_embedding,\n show_save_ax=doc_show_save_ax,\n)\ndef harmony_timeseries(\n adata: AnnData, *, show: bool = True, return_fig: bool = False, **kwargs\n) -> Axes | list[Axes] | None:\n \"\"\"\\\n Scatter plot in Harmony force-directed layout basis.\n\n Parameters\n ----------\n {adata_color_etc}\n {edges_arrows}\n {scatter_bulk}\n {show_save_ax}\n\n Returns\n -------\n If `return_fig` is True, a :class:`~matplotlib.figure.Figure`.\n If `show==False` a :class:`~matplotlib.axes.Axes` or a list of it.\n \"\"\"\n\n tp_name = adata.uns[\"harmony_timepoint_var\"]\n tps = adata.obs[tp_name].unique()\n\n fig, axes = plt.subplots(1, len(tps))\n for i, tp in enumerate(tps):\n p = embedding(\n adata,\n \"harmony\",\n color=tp_name,\n groups=tp,\n title=tp,\n show=False,\n ax=axes[i],\n legend_loc=\"none\",\n )\n p.set_axis_off()\n if return_fig:\n return fig\n if show:\n return None\n return axes\n\n\n@old_positionals(\"c\", \"cmap\", \"linewidth\", \"edgecolor\", \"axes\", \"colorbar\", \"s\")\ndef sam(\n adata: AnnData,\n projection: str | np.ndarray = \"X_umap\",\n *,\n c: str | np.ndarray | None = None,\n cmap: str = \"Spectral_r\",\n linewidth: float = 0.0,\n edgecolor: str = \"k\",\n axes: Axes | None = None,\n colorbar: bool = True,\n s: float = 10.0,\n **kwargs: Any,\n) -> Axes:\n \"\"\"\\\n Scatter plot using the SAM projection or another input projection.\n\n Parameters\n ----------\n projection\n A case-sensitive string indicating the projection to display (a key\n in adata.obsm) or a 2D numpy array with cell coordinates. If None,\n projection defaults to UMAP.\n c\n Cell color values overlaid on the projection. Can be a string from adata.obs\n to overlay cluster assignments / annotations or a 1D numpy array.\n axes\n Plot output to the specified, existing axes. If None, create new\n figure window.\n kwargs\n all keyword arguments in matplotlib.pyplot.scatter are eligible.\n \"\"\"\n\n if isinstance(projection, str):\n try:\n dt = adata.obsm[projection]\n except KeyError:\n raise ValueError(\n \"Please create a projection first using run_umap or run_tsne\"\n )\n else:\n dt = projection\n\n if axes is None:\n plt.figure()\n axes = plt.gca()\n\n if c is None:\n axes.scatter(\n dt[:, 0], dt[:, 1], s=s, linewidth=linewidth, edgecolor=edgecolor, **kwargs\n )\n return axes\n\n if isinstance(c, str):\n with contextlib.suppress(KeyError):\n c = np.array(list(adata.obs[c]))\n\n if isinstance(c[0], (str, np.str_)) and isinstance(c, (np.ndarray, list)):\n import samalg.utilities as ut\n\n i = ut.convert_annotations(c)\n ui, ai = np.unique(i, return_index=True)\n cax = axes.scatter(\n dt[:, 0],\n dt[:, 1],\n c=i,\n cmap=cmap,\n s=s,\n linewidth=linewidth,\n edgecolor=edgecolor,\n **kwargs,\n )\n\n if colorbar:\n cbar = plt.colorbar(cax, ax=axes, ticks=ui)\n cbar.ax.set_yticklabels(c[ai])\n else:\n if not isinstance(c, (np.ndarray, list)):\n colorbar = False\n i = c\n\n cax = axes.scatter(\n dt[:, 0],\n dt[:, 1],\n c=i,\n cmap=cmap,\n s=s,\n linewidth=linewidth,\n edgecolor=edgecolor,\n **kwargs,\n )\n\n if colorbar:\n plt.colorbar(cax, ax=axes)\n return axes\n\n\n@old_positionals(\n \"no_bins\",\n \"smoothing_factor\",\n \"min_delta\",\n \"show_variance\",\n \"figsize\",\n \"return_fig\",\n \"show\",\n \"save\",\n \"ax\",\n)\n@_doc_params(show_save_ax=doc_show_save_ax)\ndef wishbone_marker_trajectory(\n adata: AnnData,\n markers: Collection[str],\n *,\n no_bins: int = 150,\n smoothing_factor: int = 1,\n min_delta: float = 0.1,\n show_variance: bool = False,\n figsize: tuple[float, float] | None = None,\n return_fig: bool = False,\n show: bool = True,\n save: str | bool | None = None,\n ax: Axes | None = None,\n):\n \"\"\"\\\n Plot marker trends along trajectory, and return trajectory branches for further\n analysis and visualization (heatmap, etc..)\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n markers\n Iterable of markers/genes to be plotted.\n show_variance\n Logical indicating if the trends should be accompanied with variance.\n no_bins\n Number of bins for calculating marker density.\n smoothing_factor\n Parameter controlling the degree of smoothing.\n min_delta\n Minimum difference in marker expression after normalization to show\n separate trends for the two branches.\n figsize\n width, height\n return_fig\n Return the matplotlib figure.\n {show_save_ax}\n\n Returns\n -------\n Updates `adata` with the following fields:\n\n `trunk_wishbone` : :class:`pandas.DataFrame` (`adata.uns`)\n Computed values before branching\n `branch1_wishbone` : :class:`pandas.DataFrame` (`adata.uns`)\n Computed values for the first branch\n `branch2_wishbone` : :class:`pandas.DataFrame` (`adata.uns`)\n Computed values for the second branch.\n \"\"\"\n\n wb = _anndata_to_wishbone(adata)\n\n if figsize is None:\n width = 2 * len(markers)\n height = 0.75 * len(markers)\n else:\n width, height = figsize\n\n if ax:\n fig = ax.figure\n else:\n fig = plt.figure(figsize=(width, height))\n ax = plt.gca()\n\n ret_values, fig, ax = wb.plot_marker_trajectory(\n markers=markers,\n show_variance=show_variance,\n no_bins=no_bins,\n smoothing_factor=smoothing_factor,\n min_delta=min_delta,\n fig=fig,\n ax=ax,\n )\n\n adata.uns[\"trunk_wishbone\"] = ret_values[\"Trunk\"]\n adata.uns[\"branch1_wishbone\"] = ret_values[\"Branch1\"]\n adata.uns[\"branch2_wishbone\"] = ret_values[\"Branch2\"]\n\n _utils.savefig_or_show(\"wishbone_trajectory\", show=show, save=save)\n\n if return_fig:\n return fig\n if show:\n return None\n return ax\n\n\nscrublet_score_distribution = deprecated(\"Import from sc.pl instead\")(\n _scrublet.scrublet_score_distribution\n)\n\n\nfrom __future__ import annotations\n\nimport sys\n\nfrom .. import _utils\nfrom . import exporting, pl, pp, tl\n\n_utils.annotate_doc_types(sys.modules[__name__], \"scanpy\")\ndel sys, _utils\n\n__all__ = [\"exporting\", \"pl\", \"pp\", \"tl\"]\n\n\n\"\"\"\\\nRun Diffusion maps using the adaptive anisotropic kernel\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport pandas as pd\n\nfrom ... import logging as logg\nfrom ..._compat import old_positionals\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n\n@old_positionals(\n \"n_components\",\n \"knn\",\n \"alpha\",\n \"use_adjacency_matrix\",\n \"distances_key\",\n \"n_eigs\",\n \"impute_data\",\n \"n_steps\",\n \"copy\",\n)\n@doctest_needs(\"palantir\")\ndef palantir(\n adata: AnnData,\n *,\n n_components: int = 10,\n knn: int = 30,\n alpha: float = 0,\n use_adjacency_matrix: bool = False,\n distances_key: str | None = None,\n n_eigs: int | None = None,\n impute_data: bool = True,\n n_steps: int = 3,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Run Diffusion maps using the adaptive anisotropic kernel :cite:p:`Setty2019`.\n\n Palantir is an algorithm to align cells along differentiation trajectories.\n Palantir models differentiation as a stochastic process where stem cells\n differentiate to terminally differentiated cells by a series of steps through\n a low dimensional phenotypic manifold. Palantir effectively captures the\n continuity in cell states and the stochasticity in cell fate determination.\n Palantir has been designed to work with multidimensional single cell data\n from diverse technologies such as Mass cytometry and single cell RNA-seq.\n\n .. note::\n More information and bug reports `here `__.\n\n Parameters\n ----------\n adata\n An AnnData object.\n n_components\n Number of diffusion components.\n knn\n Number of nearest neighbors for graph construction.\n alpha\n Normalization parameter for the diffusion operator.\n use_adjacency_matrix\n Use adaptive anisotropic adjacency matrix, instead of PCA projections\n (default) to compute diffusion components.\n distances_key\n With `use_adjacency_matrix=True`, use the indicated distances key for `.obsp`.\n If `None`, `'distances'`.\n n_eigs\n Number of eigen vectors to use. If `None` specified, the number of eigen\n vectors will be determined using eigen gap. Passed to\n `palantir.utils.determine_multiscale_space`.\n impute_data\n Impute data using MAGIC.\n n_steps\n Number of steps in the diffusion operator. Passed to\n `palantir.utils.run_magic_imputation`.\n copy\n Return a copy instead of writing to `adata`.\n\n Returns\n -------\n Depending on `copy`, returns or updates `adata` with the following fields:\n\n **Diffusion maps**,\n used for magic imputation, and to generate multi-scale data matrix,\n\n - X_palantir_diff_comp - :class:`~numpy.ndarray` (:attr:`~anndata.AnnData.obsm`, dtype `float`)\n Array of Diffusion components.\n - palantir_EigenValues - :class:`~numpy.ndarray` (:attr:`~anndata.AnnData.uns`, dtype `float`)\n Array of corresponding eigen values.\n - palantir_diff_op - :class:`~scipy.sparse.spmatrix` (:attr:`~anndata.AnnData.obsp`, dtype `float`)\n The diffusion operator matrix.\n\n **Multi scale space results**,\n used to build tsne on diffusion components, and to compute branch probabilities\n and waypoints,\n\n - X_palantir_multiscale - :class:`~numpy.ndarray` (:attr:`~anndata.AnnData.obsm`, dtype `float`)\n Multi scale data matrix.\n\n **MAGIC imputation**,\n used for plotting gene expression on tsne, and gene expression trends,\n\n - palantir_imp - :class:`~numpy.ndarray` (:attr:`~anndata.AnnData.layers`, dtype `float`)\n Imputed data matrix (MAGIC imputation).\n\n Example\n -------\n >>> import scanpy.external as sce\n >>> import scanpy as sc\n\n A sample data is available `here `_.\n\n **Load sample data**\n\n >>> adata = sc.read_csv(filename=\"Palantir/data/marrow_sample_scseq_counts.csv.gz\")\n\n *Cleanup and normalize*\n\n >>> sc.pp.filter_cells(adata, min_counts=1000)\n >>> sc.pp.filter_genes(adata, min_counts=10)\n >>> sc.pp.normalize_per_cell(adata)\n >>> sc.pp.log1p(adata)\n\n **Data preprocessing**\n\n Palantir builds diffusion maps using one of two optional inputs:\n\n *Principal component analysis*\n\n >>> sc.pp.pca(adata, n_comps=300)\n\n or,\n\n *Nearist neighbors graph*\n\n >>> sc.pp.neighbors(adata, knn=30)\n\n *Diffusion maps*\n\n Palantir determines the diffusion maps of the data as an estimate of the low\n dimensional phenotypic manifold of the data.\n\n >>> sce.tl.palantir(adata, n_components=5, knn=30)\n\n if pre-computed distances are to be used,\n\n >>> sce.tl.palantir(\n ... adata,\n ... n_components=5,\n ... knn=30,\n ... use_adjacency_matrix=True,\n ... distances_key=\"distances\",\n ... )\n\n **Visualizing Palantir results**\n\n *tSNE visualization*\n\n important for Palantir!\n\n Palantir constructs the tSNE map in the embedded space since these maps better\n represent the differentiation trajectories.\n\n >>> sc.tl.tsne(adata, n_pcs=2, use_rep='X_palantir_multiscale', perplexity=150)\n\n *tsne by cell size*\n\n >>> sc.pl.tsne(adata, color=\"n_counts\")\n\n *Imputed gene expression visualized on tSNE maps*\n\n >>> sc.pl.tsne(\n ... adata,\n ... gene_symbols=['CD34', 'MPO', 'GATA1', 'IRF8'],\n ... layer='palantir_imp',\n ... color=['CD34', 'MPO', 'GATA1', 'IRF8']\n ... )\n\n **Running Palantir**\n\n Palantir can be run by specifying an approximate early cell. While Palantir\n automatically determines the terminal states, they can also be specified using the\n `termine_states` parameter.\n\n >>> start_cell = 'Run5_164698952452459'\n >>> pr_res = sce.tl.palantir_results(\n ... adata,\n ... early_cell=start_cell,\n ... ms_data='X_palantir_multiscale',\n ... num_waypoints=500,\n ... )\n\n .. note::\n A `start_cell` must be defined for every data set. The start cell for\n this dataset was chosen based on high expression of CD34.\n\n At this point the returned Palantir object `pr_res` can be used for all downstream\n analysis and plotting. Please consult this notebook\n `Palantir_sample_notebook.ipynb\n `_.\n It provides a comprehensive guide to draw *gene expression trends*, amongst other\n things.\n \"\"\"\n\n _check_import()\n from palantir.utils import (\n determine_multiscale_space,\n run_diffusion_maps,\n run_magic_imputation,\n )\n\n adata = adata.copy() if copy else adata\n\n logg.info(\"Palantir Diffusion Maps in progress ...\")\n\n if use_adjacency_matrix:\n df = adata.obsp[distances_key] if distances_key else adata.obsp[\"distances\"]\n else:\n df = pd.DataFrame(adata.obsm[\"X_pca\"], index=adata.obs_names)\n\n # Diffusion maps\n dm_res = run_diffusion_maps(\n df,\n n_components=n_components,\n knn=knn,\n alpha=alpha,\n )\n # Determine the multi scale space of the data\n ms_data = determine_multiscale_space(dm_res=dm_res, n_eigs=n_eigs)\n\n # MAGIC imputation\n if impute_data:\n imp_df = run_magic_imputation(\n data=adata.to_df(), dm_res=dm_res, n_steps=n_steps\n )\n adata.layers[\"palantir_imp\"] = imp_df\n\n (\n adata.obsm[\"X_palantir_diff_comp\"],\n adata.uns[\"palantir_EigenValues\"],\n adata.obsp[\"palantir_diff_op\"],\n adata.obsm[\"X_palantir_multiscale\"],\n ) = (\n dm_res[\"EigenVectors\"].to_numpy(),\n dm_res[\"EigenValues\"].to_numpy(),\n dm_res[\"T\"],\n ms_data.to_numpy(),\n )\n\n return adata if copy else None\n\n\n@old_positionals(\n \"ms_data\",\n \"terminal_states\",\n \"knn\",\n \"num_waypoints\",\n \"n_jobs\",\n \"scale_components\",\n \"use_early_cell_as_start\",\n \"max_iterations\",\n)\ndef palantir_results(\n adata: AnnData,\n early_cell: str,\n *,\n ms_data: str = \"X_palantir_multiscale\",\n terminal_states: list | None = None,\n knn: int = 30,\n num_waypoints: int = 1200,\n n_jobs: int = -1,\n scale_components: bool = True,\n use_early_cell_as_start: bool = False,\n max_iterations: int = 25,\n) -> AnnData | None:\n \"\"\"\\\n **Running Palantir**\n\n A convenience function that wraps `palantir.core.run_palantir` to compute branch\n probabilities and waypoints.\n\n Parameters\n ----------\n adata\n An AnnData object.\n early_cell\n Start cell for pseudotime construction.\n ms_data\n Palantir multi scale data matrix,\n terminal_states\n List of user defined terminal states\n knn\n Number of nearest neighbors for graph construction.\n num_waypoints\n Number of waypoints to sample.\n n_jobs\n Number of jobs for parallel processing.\n scale_components\n Transform features by scaling each feature to a given range. Consult the\n documentation for `sklearn.preprocessing.minmax_scale`.\n use_early_cell_as_start\n Use `early_cell` as `start_cell`, instead of determining it from the boundary\n cells closest to the defined `early_cell`.\n max_iterations\n Maximum number of iterations for pseudotime convergence.\n\n Returns\n -------\n PResults object with pseudotime, entropy, branch probabilities and waypoints.\n \"\"\"\n logg.info(\"Palantir computing waypoints..\")\n\n _check_import()\n from palantir.core import run_palantir\n\n ms_data = pd.DataFrame(adata.obsm[ms_data], index=adata.obs_names)\n pr_res = run_palantir(\n ms_data,\n early_cell=early_cell,\n terminal_states=terminal_states,\n knn=knn,\n num_waypoints=num_waypoints,\n n_jobs=n_jobs,\n scale_components=scale_components,\n use_early_cell_as_start=use_early_cell_as_start,\n max_iterations=max_iterations,\n )\n\n return pr_res\n\n\ndef _check_import():\n try:\n import palantir # noqa: F401\n except ImportError:\n raise ImportError(\"\\nplease install palantir:\\n\\tpip install palantir\")\n\n\nfrom __future__ import annotations\n\nfrom collections.abc import Collection\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\n\nfrom ... import logging\nfrom ..._compat import old_positionals\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from collections.abc import Iterable\n\n from anndata import AnnData\n\n\n@old_positionals(\"branch\", \"k\", \"components\", \"num_waypoints\")\n@doctest_needs(\"wishbone\")\ndef wishbone(\n adata: AnnData,\n start_cell: str,\n *,\n branch: bool = True,\n k: int = 15,\n components: Iterable[int] = (1, 2, 3),\n num_waypoints: int | Collection = 250,\n):\n \"\"\"\\\n Wishbone identifies bifurcating developmental trajectories from single-cell data\n :cite:p:`Setty2016`.\n\n Wishbone is an algorithm for positioning single cells along bifurcating\n developmental trajectories with high resolution. Wishbone uses multi-dimensional\n single-cell data, such as mass cytometry or RNA-Seq data, as input and orders cells\n according to their developmental progression, and it pinpoints bifurcation points\n by labeling each cell as pre-bifurcation or as one of two post-bifurcation cell\n fates.\n\n .. note::\n More information and bug reports `here\n `__.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n start_cell\n Desired start cell from `obs_names`.\n branch\n Use True for Wishbone and False for Wanderlust.\n k\n Number of nearest neighbors for graph construction.\n components\n Components to use for running Wishbone.\n num_waypoints\n Number of waypoints to sample.\n\n Returns\n -------\n Updates `adata` with the following fields:\n\n `trajectory_wishbone` : (`adata.obs`, dtype `float64`)\n Computed trajectory positions.\n `branch_wishbone` : (`adata.obs`, dtype `int64`)\n Assigned branches.\n\n Example\n -------\n\n >>> import scanpy.external as sce\n >>> import scanpy as sc\n\n **Loading Data and Pre-processing**\n\n >>> adata = sc.datasets.pbmc3k()\n >>> sc.pp.normalize_per_cell(adata)\n >>> sc.pp.pca(adata)\n >>> sc.tl.tsne(adata=adata, n_pcs=5, perplexity=30)\n >>> sc.pp.neighbors(adata, n_pcs=15, n_neighbors=10)\n >>> sc.tl.diffmap(adata, n_comps=10)\n\n **Running Wishbone Core Function**\n\n Usually, the start cell for a dataset should be chosen based on high expression of\n the gene of interest:\n\n >>> sce.tl.wishbone(\n ... adata=adata, start_cell='ACAAGAGACTTATC-1',\n ... components=[2, 3], num_waypoints=150,\n ... )\n\n **Visualizing Wishbone results**\n\n >>> sc.pl.tsne(adata, color=['trajectory_wishbone', 'branch_wishbone'])\n >>> markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ', 'MALAT1']\n >>> sce.pl.wishbone_marker_trajectory(adata, markers, show=True)\n\n For further demonstration of Wishbone methods and visualization please follow the\n notebooks in the package `Wishbone_for_single_cell_RNAseq.ipynb\n `_.\\\n \"\"\"\n try:\n from wishbone.core import wishbone as c_wishbone\n except ImportError:\n raise ImportError(\n \"\\nplease install wishbone:\\n\\n\\thttps://github.com/dpeerlab/wishbone\"\n )\n\n # Start cell index\n s = np.where(adata.obs_names == start_cell)[0]\n if len(s) == 0:\n raise RuntimeError(\n f\"Start cell {start_cell} not found in data. \"\n \"Please rerun with correct start cell.\"\n )\n if isinstance(num_waypoints, Collection):\n diff = np.setdiff1d(num_waypoints, adata.obs.index)\n if diff.size > 0:\n logging.warning(\n \"Some of the specified waypoints are not in the data. \"\n \"These will be removed\"\n )\n num_waypoints = diff.tolist()\n elif num_waypoints > adata.shape[0]:\n raise RuntimeError(\n \"num_waypoints parameter is higher than the number of cells in the \"\n \"dataset. Please select a smaller number\"\n )\n s = s[0]\n\n # Run the algorithm\n components = list(components)\n res = c_wishbone(\n adata.obsm[\"X_diffmap\"][:, components],\n s=s,\n k=k,\n l=k,\n num_waypoints=num_waypoints,\n branch=branch,\n )\n\n # Assign results\n trajectory = res[\"Trajectory\"]\n trajectory = (trajectory - np.min(trajectory)) / (\n np.max(trajectory) - np.min(trajectory)\n )\n adata.obs[\"trajectory_wishbone\"] = np.asarray(trajectory)\n\n # branch_ = None\n if branch:\n branches = res[\"Branches\"].astype(int)\n adata.obs[\"branch_wishbone\"] = np.asarray(branches)\n\n\ndef _anndata_to_wishbone(adata: AnnData):\n from wishbone.wb import SCData, Wishbone\n\n scdata = SCData(adata.to_df())\n scdata.diffusion_eigenvectors = pd.DataFrame(\n adata.obsm[\"X_diffmap\"], index=adata.obs_names\n )\n wb = Wishbone(scdata)\n wb.trajectory = adata.obs[\"trajectory_wishbone\"]\n wb.branch = adata.obs[\"branch_wishbone\"]\n return wb\n\n\n\"\"\"\\\nPerform clustering using PhenoGraph\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport pandas as pd\nfrom anndata import AnnData\n\nfrom ... import logging as logg\nfrom ..._compat import old_positionals\nfrom ..._utils import renamed_arg\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from typing import Any, Literal\n\n import numpy as np\n from scipy.sparse import spmatrix\n\n from ...tools._leiden import MutableVertexPartition\n\n\n@renamed_arg(\"adata\", \"data\", pos_0=True)\n@old_positionals(\n \"k\",\n \"directed\",\n \"prune\",\n \"min_cluster_size\",\n \"jaccard\",\n \"primary_metric\",\n \"n_jobs\",\n \"q_tol\",\n \"louvain_time_limit\",\n \"nn_method\",\n \"partition_type\",\n \"resolution_parameter\",\n \"n_iterations\",\n \"use_weights\",\n \"seed\",\n \"copy\",\n)\n@doctest_needs(\"phenograph\")\ndef phenograph(\n data: AnnData | np.ndarray | spmatrix,\n clustering_algo: Literal[\"louvain\", \"leiden\"] | None = \"louvain\",\n *,\n k: int = 30,\n directed: bool = False,\n prune: bool = False,\n min_cluster_size: int = 10,\n jaccard: bool = True,\n primary_metric: Literal[\n \"euclidean\",\n \"manhattan\",\n \"correlation\",\n \"cosine\",\n ] = \"euclidean\",\n n_jobs: int = -1,\n q_tol: float = 1e-3,\n louvain_time_limit: int = 2000,\n nn_method: Literal[\"kdtree\", \"brute\"] = \"kdtree\",\n partition_type: type[MutableVertexPartition] | None = None,\n resolution_parameter: float = 1,\n n_iterations: int = -1,\n use_weights: bool = True,\n seed: int | None = None,\n copy: bool = False,\n **kargs: Any,\n) -> tuple[np.ndarray | None, spmatrix, float | None] | None:\n \"\"\"\\\n PhenoGraph clustering :cite:p:`Levine2015`.\n\n **PhenoGraph** is a clustering method designed for high-dimensional single-cell\n data. It works by creating a graph (\"network\") representing phenotypic similarities\n between cells and then identifying communities in this graph. It supports both\n Louvain_ and Leiden_ algorithms for community detection.\n\n .. _Louvain: https://louvain-igraph.readthedocs.io/en/latest/\n\n .. _Leiden: https://leidenalg.readthedocs.io/en/latest/reference.html\n\n .. note::\n More information and bug reports `here\n `__.\n\n Parameters\n ----------\n data\n AnnData, or Array of data to cluster, or sparse matrix of k-nearest neighbor\n graph. If ndarray, n-by-d array of n cells in d dimensions. if sparse matrix,\n n-by-n adjacency matrix.\n clustering_algo\n Choose between `'Louvain'` or `'Leiden'` algorithm for clustering.\n k\n Number of nearest neighbors to use in first step of graph construction.\n directed\n Whether to use a symmetric (default) or asymmetric (`'directed'`) graph.\n The graph construction process produces a directed graph, which is symmetrized\n by one of two methods (see `prune` below).\n prune\n `prune=False`, symmetrize by taking the average between the graph and its\n transpose. `prune=True`, symmetrize by taking the product between the graph\n and its transpose.\n min_cluster_size\n Cells that end up in a cluster smaller than min_cluster_size are considered\n outliers and are assigned to -1 in the cluster labels.\n jaccard\n If `True`, use Jaccard metric between k-neighborhoods to build graph. If\n `False`, use a Gaussian kernel.\n primary_metric\n Distance metric to define nearest neighbors. Note that performance will be\n slower for correlation and cosine.\n n_jobs\n Nearest Neighbors and Jaccard coefficients will be computed in parallel using\n n_jobs. If 1 is given, no parallelism is used. If set to -1, all CPUs are used.\n For n_jobs below -1, `n_cpus + 1 + n_jobs` are used.\n q_tol\n Tolerance, i.e. precision, for monitoring modularity optimization.\n louvain_time_limit\n Maximum number of seconds to run modularity optimization. If exceeded the best\n result so far is returned.\n nn_method\n Whether to use brute force or kdtree for nearest neighbor search.\n For very large high-dimensional data sets, brute force, with parallel\n computation, performs faster than kdtree.\n partition_type\n Defaults to :class:`~leidenalg.RBConfigurationVertexPartition`. For the\n available options, consult the documentation for\n :func:`~leidenalg.find_partition`.\n resolution_parameter\n A parameter value controlling the coarseness of the clustering in Leiden. Higher\n values lead to more clusters. Set to `None` if overriding `partition_type` to\n one that does not accept a `resolution_parameter`.\n n_iterations\n Number of iterations to run the Leiden algorithm. If the number of iterations is\n negative, the Leiden algorithm is run until an iteration in which there was no\n improvement.\n use_weights\n Use vertices in the Leiden computation.\n seed\n Leiden initialization of the optimization.\n copy\n Return a copy or write to `adata`.\n kargs\n Additional arguments passed to :func:`~leidenalg.find_partition` and the\n constructor of the `partition_type`.\n\n Returns\n -------\n Depending on `copy`, returns or updates `adata` with the following fields:\n\n **communities** - :class:`~numpy.ndarray` (:attr:`~anndata.AnnData.obs`, dtype `int`)\n integer array of community assignments for each row in data.\n\n **graph** - :class:`~scipy.sparse.spmatrix` (:attr:`~anndata.AnnData.obsp`, dtype `float`)\n the graph that was used for clustering.\n\n **Q** - `float` (:attr:`~anndata.AnnData.uns`, dtype `float`)\n the modularity score for communities on graph.\n\n Example\n -------\n >>> from anndata import AnnData\n >>> import scanpy as sc\n >>> import scanpy.external as sce\n >>> import numpy as np\n >>> import pandas as pd\n\n With annotated data as input:\n\n >>> adata = sc.datasets.pbmc3k()\n >>> sc.pp.normalize_per_cell(adata)\n\n Then do PCA:\n\n >>> sc.pp.pca(adata, n_comps=100)\n\n Compute phenograph clusters:\n\n **Louvain** community detection\n\n >>> sce.tl.phenograph(adata, clustering_algo=\"louvain\", k=30)\n\n **Leiden** community detection\n\n >>> sce.tl.phenograph(adata, clustering_algo=\"leiden\", k=30)\n\n Return only `Graph` object\n\n >>> sce.tl.phenograph(adata, clustering_algo=None, k=30)\n\n Now to show phenograph on tSNE (for example):\n\n Compute tSNE:\n\n >>> sc.tl.tsne(adata, random_state=7)\n\n Plot phenograph clusters on tSNE:\n\n >>> sc.pl.tsne(\n ... adata, color = [\"pheno_louvain\", \"pheno_leiden\"], s = 100,\n ... palette = sc.pl.palettes.vega_20_scanpy, legend_fontsize = 10\n ... )\n\n Cluster and cluster centroids for input Numpy ndarray\n\n >>> df = np.random.rand(1000, 40)\n >>> dframe = pd.DataFrame(df)\n >>> dframe.index, dframe.columns = (map(str, dframe.index), map(str, dframe.columns))\n >>> adata = AnnData(dframe)\n >>> sc.pp.pca(adata, n_comps=20)\n >>> sce.tl.phenograph(adata, clustering_algo=\"leiden\", k=50)\n >>> sc.tl.tsne(adata, random_state=1)\n >>> sc.pl.tsne(\n ... adata, color=['pheno_leiden'], s=100,\n ... palette=sc.pl.palettes.vega_20_scanpy, legend_fontsize=10\n ... )\n \"\"\"\n start = logg.info(\"PhenoGraph clustering\")\n\n try:\n import phenograph\n\n assert phenograph.__version__ >= \"1.5.3\"\n except (ImportError, AssertionError, AttributeError):\n raise ImportError(\n \"please install the latest release of phenograph:\\n\\t\"\n \"pip install -U PhenoGraph\"\n )\n\n if isinstance(data, AnnData):\n adata = data\n try:\n data = data.obsm[\"X_pca\"]\n except KeyError:\n raise KeyError(\"Please run `sc.pp.pca` on `data` and try again!\")\n else:\n adata = None\n copy = True\n\n comm_key = (\n f\"pheno_{clustering_algo}\" if clustering_algo in [\"louvain\", \"leiden\"] else \"\"\n )\n ig_key = \"pheno_{}_ig\".format(\"jaccard\" if jaccard else \"gaussian\")\n q_key = \"pheno_{}_q\".format(\"jaccard\" if jaccard else \"gaussian\")\n\n communities, graph, Q = phenograph.cluster(\n data=data,\n clustering_algo=clustering_algo,\n k=k,\n directed=directed,\n prune=prune,\n min_cluster_size=min_cluster_size,\n jaccard=jaccard,\n primary_metric=primary_metric,\n n_jobs=n_jobs,\n q_tol=q_tol,\n louvain_time_limit=louvain_time_limit,\n nn_method=nn_method,\n partition_type=partition_type,\n resolution_parameter=resolution_parameter,\n n_iterations=n_iterations,\n use_weights=use_weights,\n seed=seed,\n **kargs,\n )\n\n logg.info(\" finished\", time=start)\n\n if copy:\n return communities, graph, Q\n\n if adata is not None:\n adata.obsp[ig_key] = graph.tocsr()\n if comm_key:\n adata.obs[comm_key] = pd.Categorical(communities)\n if Q:\n adata.uns[q_key] = Q\n\n\n\"\"\"\\\nCalculate scores based on relative expression change of maker pairs\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nfrom packaging.version import Version\n\nfrom ..._settings import settings\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from collections.abc import Collection, Mapping\n from typing import Union\n\n import pandas as pd\n from anndata import AnnData\n\n Genes = Collection[Union[str, int, bool]]\n\n\n@doctest_needs(\"pypairs\")\ndef sandbag(\n adata: AnnData,\n annotation: Mapping[str, Genes] | None = None,\n *,\n fraction: float = 0.65,\n filter_genes: Genes | None = None,\n filter_samples: Genes | None = None,\n) -> dict[str, list[tuple[str, str]]]:\n \"\"\"\\\n Calculate marker pairs of genes :cite:p:`Scialdone2015,Fechtner2018`.\n\n Calculates the pairs of genes serving as marker pairs for each phase,\n based on a matrix of gene counts and an annotation of known phases.\n\n This reproduces the approach of :cite:t:`Scialdone2015` in the implementation of\n :cite:t:`Fechtner2018`.\n\n More information and bug reports `here\n `__.\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n annotation\n Mapping from category to genes, e.g. `{'phase': [Gene1, ...]}`.\n Defaults to ``data.vars['category']``.\n fraction\n Fraction of cells per category where marker criteria must be satisfied.\n filter_genes\n Genes for sampling the reference set. Defaults to all genes.\n filter_samples\n Cells for sampling the reference set. Defaults to all samples.\n\n Returns\n -------\n A dict mapping from category to lists of marker pairs, e.g.:\n `{'Category_1': [(Gene_1, Gene_2), ...], ...}`.\n\n Examples\n --------\n >>> from scanpy.external.tl import sandbag\n >>> from pypairs import datasets\n >>> adata = datasets.leng15()\n >>> marker_pairs = sandbag(adata, fraction=0.5)\n \"\"\"\n _check_import()\n from pypairs import settings as pp_settings\n from pypairs.pairs import sandbag\n\n pp_settings.verbosity = settings.verbosity\n pp_settings.n_jobs = settings.n_jobs\n pp_settings.writedir = settings.writedir\n pp_settings.cachedir = settings.cachedir\n pp_settings.logfile = settings.logfile\n\n return sandbag(\n data=adata,\n annotation=annotation,\n fraction=fraction,\n filter_genes=filter_genes,\n filter_samples=filter_samples,\n )\n\n\ndef cyclone(\n adata: AnnData,\n marker_pairs: Mapping[str, Collection[tuple[str, str]]] | None = None,\n *,\n iterations: int = 1000,\n min_iter: int = 100,\n min_pairs: int = 50,\n) -> pd.DataFrame:\n \"\"\"\\\n Assigns scores and predicted class to observations :cite:p:`Scialdone2015` :cite:p:`Fechtner2018`.\n\n Calculates scores for each observation and each phase and assigns prediction\n based on marker pairs indentified by :func:`~scanpy.external.tl.sandbag`.\n\n This reproduces the approach of :cite:t:`Scialdone2015` in the implementation of\n :cite:t:`Fechtner2018`.\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n marker_pairs\n Mapping of categories to lists of marker pairs.\n See :func:`~scanpy.external.tl.sandbag` output.\n iterations\n An integer scalar specifying the number of\n iterations for random sampling to obtain a cycle score.\n min_iter\n An integer scalar specifying the minimum number of iterations\n for score estimation.\n min_pairs\n An integer scalar specifying the minimum number of pairs\n for score estimation.\n\n Returns\n -------\n A :class:`~pandas.DataFrame` with samples as index and categories as columns\n with scores for each category for each sample and a additional column with\n the name of the max scoring category for each sample.\n\n If `marker_pairs` contains only the cell cycle categories G1, S and G2M an\n additional column `pypairs_cc_prediction` will be added.\n Where category S is assigned to samples where G1 and G2M score are < 0.5.\n \"\"\"\n _check_import()\n from pypairs import settings as pp_settings\n from pypairs.pairs import cyclone\n\n pp_settings.verbosity = settings.verbosity\n pp_settings.n_jobs = settings.n_jobs\n pp_settings.writedir = settings.writedir\n pp_settings.cachedir = settings.cachedir\n pp_settings.logfile = settings.logfile\n\n return cyclone(\n data=adata,\n marker_pairs=marker_pairs,\n iterations=iterations,\n min_iter=min_iter,\n min_pairs=min_pairs,\n )\n\n\ndef _check_import():\n try:\n import pypairs\n except ImportError:\n raise ImportError(\"You need to install the package `pypairs`.\")\n\n min_version = Version(\"3.0.9\")\n if Version(pypairs.__version__) < min_version:\n raise ImportError(f\"Please only use `pypairs` >= {min_version}\")\n\n\n\"\"\"\\\nHarmony time series for data visualization with augmented affinity matrix at\ndiscrete time points\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\n\nfrom ... import logging as logg\nfrom ..._compat import old_positionals\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n\n@old_positionals(\"n_neighbors\", \"n_components\", \"n_jobs\", \"copy\")\n@doctest_needs(\"harmony\")\ndef harmony_timeseries(\n adata: AnnData,\n tp: str,\n *,\n n_neighbors: int = 30,\n n_components: int | None = 1000,\n n_jobs: int = -2,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Harmony time series for data visualization with augmented affinity matrix\n at discrete time points :cite:p:`Nowotschin2019`.\n\n Harmony time series is a framework for data visualization, trajectory\n detection and interpretation for scRNA-seq data measured at discrete\n time points. Harmony constructs an augmented affinity matrix by augmenting\n the kNN graph affinity matrix with mutually nearest neighbors between\n successive time points. This augmented affinity matrix forms the basis for\n generated a force directed layout for visualization and also serves as input\n for computing the diffusion operator which can be used for trajectory\n detection using Palantir_.\n\n .. _Palantir: https://github.com/dpeerlab/Palantir\n\n .. note::\n More information and bug reports `here\n `__.\n\n Parameters\n ----------\n adata\n Annotated data matrix of shape n_obs `×` n_vars. Rows correspond to\n cells and columns to genes. Rows represent two or more time points,\n where replicates of the same time point are consecutive in order.\n tp\n key name of observation annotation `.obs` representing time points. Time\n points should be categorical of `dtype=category`. The unique categories for\n the categorical will be used as the time points to construct the timepoint\n connections.\n n_neighbors\n Number of nearest neighbors for graph construction.\n n_components\n Minimum number of principal components to use. Specify `None` to use\n pre-computed components. The higher the value the better to capture 85% of the\n variance.\n n_jobs\n Nearest Neighbors will be computed in parallel using n_jobs.\n copy\n Return a copy instead of writing to `adata`.\n\n Returns\n -------\n Depending on `copy`, returns or updates `.obsm`, `.obsp` and `.uns` with the following:\n\n **X_harmony** - :class:`~numpy.ndarray` (:attr:`~anndata.AnnData.obsm`, dtype `float`)\n force directed layout\n **harmony_aff** - :class:`~scipy.sparse.spmatrix` (:attr:`~anndata.AnnData.obsp`, dtype `float`)\n affinity matrix\n **harmony_aff_aug** - :class:`~scipy.sparse.spmatrix` (:attr:`~anndata.AnnData.obsp`, dtype `float`)\n augmented affinity matrix\n **harmony_timepoint_var** - `str` (:attr:`~anndata.AnnData.uns`)\n The name of the variable passed as `tp`\n **harmony_timepoint_connections** - :class:`~numpy.ndarray` (:attr:`~anndata.AnnData.uns`, dtype `str`)\n The links between time points\n\n Example\n -------\n\n >>> from itertools import product\n >>> import pandas as pd\n >>> from anndata import AnnData\n >>> import scanpy as sc\n >>> import scanpy.external as sce\n\n **Load** `AnnData`\n\n A sample with real data is available here_.\n\n .. _here: https://github.com/dpeerlab/Harmony/tree/master/data\n\n Random data sets of three time points with two replicates each:\n\n >>> adata_ref = sc.datasets.pbmc3k()\n >>> start = [596, 615, 1682, 1663, 1409, 1432]\n >>> adata = AnnData.concatenate(\n ... *(adata_ref[i : i + 1000] for i in start),\n ... join=\"outer\",\n ... batch_key=\"sample\",\n ... batch_categories=[f\"sa{i}_Rep{j}\" for i, j in product((1, 2, 3), (1, 2))],\n ... )\n >>> time_points = adata.obs[\"sample\"].str.split(\"_\", expand=True)[0]\n >>> adata.obs[\"time_points\"] = pd.Categorical(\n ... time_points, categories=['sa1', 'sa2', 'sa3']\n ... )\n\n Normalize and filter for highly expressed genes\n\n >>> sc.pp.normalize_total(adata, target_sum=10000)\n >>> sc.pp.log1p(adata)\n >>> sc.pp.highly_variable_genes(adata, n_top_genes=1000, subset=True)\n\n Run harmony_timeseries\n\n >>> sce.tl.harmony_timeseries(adata, tp=\"time_points\", n_components=500)\n\n Plot time points:\n\n >>> sce.pl.harmony_timeseries(adata)\n\n For further demonstration of Harmony visualizations please follow the notebook\n `Harmony_sample_notebook.ipynb\n `_.\n It provides a comprehensive guide to draw *gene expression trends*,\n amongst other things.\n \"\"\"\n\n try:\n import harmony\n except ImportError:\n raise ImportError(\"\\nplease install harmony:\\n\\n\\tpip install harmonyTS\")\n\n adata = adata.copy() if copy else adata\n logg.info(\"Harmony augmented affinity matrix\")\n\n if adata.obs[tp].dtype.name != \"category\":\n raise ValueError(f\"{tp!r} column does not contain Categorical data\")\n timepoints = adata.obs[tp].cat.categories.tolist()\n timepoint_connections = pd.DataFrame(np.array([timepoints[:-1], timepoints[1:]]).T)\n\n # compute the augmented and non-augmented affinity matrices\n aug_aff, aff = harmony.core.augmented_affinity_matrix(\n data_df=adata.to_df(),\n timepoints=adata.obs[tp],\n timepoint_connections=timepoint_connections,\n n_neighbors=n_neighbors,\n n_jobs=n_jobs,\n pc_components=n_components,\n )\n\n # Force directed layouts\n layout = harmony.plot.force_directed_layout(aug_aff, adata.obs.index)\n\n adata.obsm[\"X_harmony\"] = np.asarray(layout)\n adata.obsp[\"harmony_aff\"] = aff\n adata.obsp[\"harmony_aff_aug\"] = aug_aff\n adata.uns[\"harmony_timepoint_var\"] = tp\n adata.uns[\"harmony_timepoint_connections\"] = np.asarray(timepoint_connections)\n\n return adata if copy else None\n\n\n\"\"\"\\\nEmbed high-dimensional data using PHATE\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nfrom ... import logging as logg\nfrom ..._compat import old_positionals\nfrom ..._settings import settings\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from anndata import AnnData\n\n from ..._utils import AnyRandom\n\n\n@old_positionals(\n \"k\",\n \"a\",\n \"n_landmark\",\n \"t\",\n \"gamma\",\n \"n_pca\",\n \"knn_dist\",\n \"mds_dist\",\n \"mds\",\n \"n_jobs\",\n \"random_state\",\n \"verbose\",\n \"copy\",\n)\n@doctest_needs(\"phate\")\ndef phate(\n adata: AnnData,\n n_components: int = 2,\n *,\n k: int = 5,\n a: int = 15,\n n_landmark: int = 2000,\n t: int | str = \"auto\",\n gamma: float = 1.0,\n n_pca: int = 100,\n knn_dist: str = \"euclidean\",\n mds_dist: str = \"euclidean\",\n mds: Literal[\"classic\", \"metric\", \"nonmetric\"] = \"metric\",\n n_jobs: int | None = None,\n random_state: AnyRandom = None,\n verbose: bool | int | None = None,\n copy: bool = False,\n **kwargs,\n) -> AnnData | None:\n \"\"\"\\\n PHATE :cite:p:`Moon2019`.\n\n Potential of Heat-diffusion for Affinity-based Trajectory Embedding (PHATE)\n embeds high dimensional single-cell data into two or three dimensions for\n visualization of biological progressions.\n\n For more information and access to the object-oriented interface, read the\n `PHATE documentation `__. For\n tutorials, bug reports, and R/MATLAB implementations, visit the `PHATE\n GitHub page `__. For help\n using PHATE, go `here `__.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n n_components\n number of dimensions in which the data will be embedded\n k\n number of nearest neighbors on which to build kernel\n a\n sets decay rate of kernel tails.\n If None, alpha decaying kernel is not used\n n_landmark\n number of landmarks to use in fast PHATE\n t\n power to which the diffusion operator is powered\n sets the level of diffusion. If 'auto', t is selected\n according to the knee point in the Von Neumann Entropy of\n the diffusion operator\n gamma\n Informational distance constant between -1 and 1.\n `gamma=1` gives the PHATE log potential, `gamma=0` gives\n a square root potential.\n n_pca\n Number of principal components to use for calculating\n neighborhoods. For extremely large datasets, using\n n_pca < 20 allows neighborhoods to be calculated in\n log(n_samples) time.\n knn_dist\n recommended values: 'euclidean' and 'cosine'\n Any metric from `scipy.spatial.distance` can be used\n distance metric for building kNN graph\n mds_dist\n recommended values: 'euclidean' and 'cosine'\n Any metric from `scipy.spatial.distance` can be used\n distance metric for MDS\n mds\n Selects which MDS algorithm is used for dimensionality reduction.\n n_jobs\n The number of jobs to use for the computation.\n If `None`, `sc.settings.n_jobs` is used.\n If -1 all CPUs are used. If 1 is given, no parallel computing code is\n used at all, which is useful for debugging.\n For n_jobs below -1, (n_cpus + 1 + n_jobs) are used. Thus for\n n_jobs = -2, all CPUs but one are used\n random_state\n Random seed. Defaults to the global `numpy` random number generator\n verbose\n If `True` or an `int`/`Verbosity` ≥ 2/`hint`, print status messages.\n If `None`, `sc.settings.verbosity` is used.\n copy\n Return a copy instead of writing to `adata`.\n kwargs\n Additional arguments to `phate.PHATE`\n\n Returns\n -------\n Depending on `copy`, returns or updates `adata` with the following fields.\n\n **X_phate** : `np.ndarray`, (`adata.obs`, shape=[n_samples, n_components], dtype `float`)\n PHATE coordinates of data.\n\n Examples\n --------\n >>> from anndata import AnnData\n >>> import scanpy.external as sce\n >>> import phate\n >>> tree_data, tree_clusters = phate.tree.gen_dla(\n ... n_dim=100,\n ... n_branch=20,\n ... branch_length=100,\n ... )\n >>> tree_data.shape\n (2000, 100)\n >>> adata = AnnData(tree_data)\n >>> sce.tl.phate(adata, k=5, a=20, t=150)\n >>> adata.obsm['X_phate'].shape\n (2000, 2)\n >>> sce.pl.phate(adata)\n \"\"\"\n start = logg.info(\"computing PHATE\")\n adata = adata.copy() if copy else adata\n verbosity = settings.verbosity if verbose is None else verbose\n verbose = verbosity if isinstance(verbosity, bool) else verbosity >= 2\n n_jobs = settings.n_jobs if n_jobs is None else n_jobs\n try:\n import phate\n except ImportError:\n raise ImportError(\n \"You need to install the package `phate`: please run `pip install \"\n \"--user phate` in a terminal.\"\n )\n X_phate = phate.PHATE(\n n_components=n_components,\n k=k,\n a=a,\n n_landmark=n_landmark,\n t=t,\n gamma=gamma,\n n_pca=n_pca,\n knn_dist=knn_dist,\n mds_dist=mds_dist,\n mds=mds,\n n_jobs=n_jobs,\n random_state=random_state,\n verbose=verbose,\n **kwargs,\n ).fit_transform(adata)\n # update AnnData instance\n adata.obsm[\"X_phate\"] = X_phate # annotate samples with PHATE coordinates\n logg.info(\n \" finished\",\n time=start,\n deep=(\"added\\n\" \" 'X_phate', PHATE coordinates (adata.obsm)\"),\n )\n return adata if copy else None\n\n\n\"\"\"\\\nRun the Self-Assembling Manifold algorithm\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nfrom ... import logging as logg\nfrom ..._compat import old_positionals\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from anndata import AnnData\n from samalg import SAM\n\n\n@old_positionals(\n \"max_iter\",\n \"num_norm_avg\",\n \"k\",\n \"distance\",\n \"standardization\",\n \"weight_pcs\",\n \"sparse_pca\",\n \"n_pcs\",\n \"n_genes\",\n \"projection\",\n \"inplace\",\n \"verbose\",\n)\n@doctest_needs(\"samalg\")\ndef sam(\n adata: AnnData,\n *,\n max_iter: int = 10,\n num_norm_avg: int = 50,\n k: int = 20,\n distance: str = \"correlation\",\n standardization: Literal[\"Normalizer\", \"StandardScaler\", \"None\"] = \"StandardScaler\",\n weight_pcs: bool = False,\n sparse_pca: bool = False,\n n_pcs: int | None = 150,\n n_genes: int | None = 3000,\n projection: Literal[\"umap\", \"tsne\", \"None\"] = \"umap\",\n inplace: bool = True,\n verbose: bool = True,\n) -> SAM | tuple[SAM, AnnData]:\n \"\"\"\\\n Self-Assembling Manifolds single-cell RNA sequencing analysis tool :cite:p:`Tarashansky2019`.\n\n SAM iteratively rescales the input gene expression matrix to emphasize\n genes that are spatially variable along the intrinsic manifold of the data.\n It outputs the gene weights, nearest neighbor matrix, and a 2D projection.\n\n The AnnData input should contain unstandardized, non-negative values.\n Preferably, the data should be log-normalized and no genes should be filtered out.\n\n\n Parameters\n ----------\n\n k\n The number of nearest neighbors to identify for each cell.\n\n distance\n The distance metric to use when identifying nearest neighbors.\n Can be any of the distance metrics supported by\n :func:`~scipy.spatial.distance.pdist`.\n\n max_iter\n The maximum number of iterations SAM will run.\n\n projection\n If 'tsne', generates a t-SNE embedding. If 'umap', generates a UMAP\n embedding. If 'None', no embedding will be generated.\n\n standardization\n If 'Normalizer', use sklearn.preprocessing.Normalizer, which\n normalizes expression data prior to PCA such that each cell has\n unit L2 norm. If 'StandardScaler', use\n sklearn.preprocessing.StandardScaler, which normalizes expression\n data prior to PCA such that each gene has zero mean and unit\n variance. Otherwise, do not normalize the expression data. We\n recommend using 'StandardScaler' for large datasets with many\n expected cell types and 'Normalizer' otherwise. If 'None', no\n transformation is applied.\n\n num_norm_avg\n The top 'num_norm_avg' dispersions are averaged to determine the\n normalization factor when calculating the weights. This prevents\n genes with large spatial dispersions from skewing the distribution\n of weights.\n\n weight_pcs\n If True, scale the principal components by their eigenvalues. In\n datasets with many expected cell types, setting this to False might\n improve the resolution as these cell types might be encoded by lower-\n variance principal components.\n\n sparse_pca\n If True, uses an implementation of PCA that accepts sparse inputs.\n This way, we no longer need a temporary dense copy of the sparse data.\n However, this implementation is slower and so is only worth using when\n memory constraints become noticeable.\n\n n_pcs\n Determines the number of top principal components selected at each\n iteration of the SAM algorithm. If None, this number is chosen\n automatically based on the size of the dataset. If weight_pcs is\n set to True, this parameter primarily affects the runtime of the SAM\n algorithm (more PCs = longer runtime).\n\n n_genes\n Determines the number of top SAM-weighted genes to use at each iteration\n of the SAM algorithm. If None, this number is chosen automatically\n based on the size of the dataset. This parameter primarily affects\n the runtime of the SAM algorithm (more genes = longer runtime). For\n extremely homogeneous datasets, decreasing `n_genes` may improve\n clustering resolution.\n\n inplace\n Set fields in `adata` if True. Otherwise, returns a copy.\n\n verbose\n If True, displays SAM log statements.\n\n Returns\n -------\n sam_obj if inplace is True or (sam_obj,AnnData) otherwise\n\n adata - AnnData\n `.var['weights']`\n SAM weights for each gene.\n `.var['spatial_dispersions']`\n Spatial dispersions for each gene (these are used to compute the\n SAM weights)\n `.uns['sam']`\n Dictionary of SAM-specific outputs, such as the parameters\n used for preprocessing ('preprocess_args') and running\n ('run_args') SAM.\n `.uns['neighbors']`\n A dictionary with key 'connectivities' containing the kNN adjacency\n matrix output by SAM. If built-in scanpy dimensionality reduction\n methods are to be used using the SAM-output AnnData, users\n should recompute the neighbors using `.obs['X_pca']` with\n `scanpy.pp.neighbors`.\n `.obsm['X_pca']`\n The principal components output by SAM.\n `.obsm['X_umap']`\n The UMAP projection output by SAM.\n `.layers['X_disp']`\n The expression matrix used for nearest-neighbor averaging.\n `.layers['X_knn_avg']`\n The nearest-neighbor-averaged expression data used for computing the\n spatial dispersions of genes.\n\n Example\n -------\n >>> import scanpy.external as sce\n >>> import scanpy as sc\n\n *** Running SAM ***\n\n Assuming we are given an AnnData object called `adata`, we can run the SAM\n algorithm as follows:\n\n >>> sam_obj = sce.tl.sam(adata,inplace=True)\n\n The input AnnData object should contain unstandardized, non-negative\n expression values. Preferably, the data should be log-normalized and no\n genes should be filtered out.\n\n Please see the documentation for a description of all available parameters.\n\n For more detailed tutorials, please visit the original Github repository:\n https://github.com/atarashansky/self-assembling-manifold/tree/master/tutorial\n\n *** Plotting ***\n\n To visualize the output, we can use:\n\n >>> sce.pl.sam(adata,projection='X_umap')\n\n `sce.pl.sam` accepts all keyword arguments used in the\n `matplotlib.pyplot.scatter` function.\n\n *** SAMGUI ***\n\n SAM comes with the SAMGUI module, a graphical-user interface written with\n `Plotly` and `ipythonwidgets` for interactively exploring and annotating\n the scRNAseq data and running SAM.\n\n Dependencies can be installed with Anaconda by following the instructions in\n the self-assembling-manifold Github README:\n https://github.com/atarashansky/self-assembling-manifold\n\n In a Jupyter notebook, execute the following to launch the interface:\n\n >>> from samalg.gui import SAMGUI\n >>> sam_gui = SAMGUI(sam_obj) # sam_obj is your SAM object\n >>> sam_gui.SamPlot\n\n This can also be enabled in Jupyer Lab by following the instructions in the\n self-assembling-manifold README.\n\n \"\"\"\n\n try:\n from samalg import SAM\n except ImportError:\n raise ImportError(\n \"\\nplease install sam-algorithm: \\n\\n\"\n \"\\tgit clone git://github.com/atarashansky/self-assembling-manifold.git\\n\"\n \"\\tcd self-assembling-manifold\\n\"\n \"\\tpip install .\"\n )\n\n logg.info(\"Self-assembling manifold\")\n\n s = SAM(counts=adata, inplace=inplace)\n\n logg.info(\"Running SAM\")\n s.run(\n max_iter=max_iter,\n num_norm_avg=num_norm_avg,\n k=k,\n distance=distance,\n preprocessing=standardization,\n weight_PCs=weight_pcs,\n npcs=n_pcs,\n n_genes=n_genes,\n projection=projection,\n sparse_pca=sparse_pca,\n verbose=verbose,\n )\n\n s.adata.uns[\"sam\"] = {}\n for attr in [\"nnm\", \"preprocess_args\", \"run_args\", \"ranked_genes\"]:\n s.adata.uns[\"sam\"][attr] = s.adata.uns.pop(attr, None)\n\n return s if inplace else (s, s.adata)\n\n\nfrom __future__ import annotations\n\nfrom ._harmony_timeseries import harmony_timeseries\nfrom ._palantir import palantir, palantir_results\nfrom ._phate import phate\nfrom ._phenograph import phenograph\nfrom ._pypairs import cyclone, sandbag\nfrom ._sam import sam\nfrom ._trimap import trimap\nfrom ._wishbone import wishbone\n\n__all__ = [\n \"harmony_timeseries\",\n \"palantir\",\n \"palantir_results\",\n \"phate\",\n \"phenograph\",\n \"cyclone\",\n \"sandbag\",\n \"sam\",\n \"trimap\",\n \"wishbone\",\n]\n\n\n\"\"\"\\\nEmbed high-dimensional data using TriMap\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport scipy.sparse as scp\n\nfrom ... import logging as logg\nfrom ..._compat import old_positionals\nfrom ..._settings import settings\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from anndata import AnnData\n\n\n@old_positionals(\n \"n_inliers\",\n \"n_outliers\",\n \"n_random\",\n \"metric\",\n \"weight_adj\",\n \"lr\",\n \"n_iters\",\n \"verbose\",\n \"copy\",\n)\n@doctest_needs(\"trimap\")\ndef trimap(\n adata: AnnData,\n n_components: int = 2,\n *,\n n_inliers: int = 10,\n n_outliers: int = 5,\n n_random: int = 5,\n metric: Literal[\"angular\", \"euclidean\", \"hamming\", \"manhattan\"] = \"euclidean\",\n weight_adj: float = 500.0,\n lr: float = 1000.0,\n n_iters: int = 400,\n verbose: bool | int | None = None,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n TriMap: Large-scale Dimensionality Reduction Using Triplets :cite:p:`Amid2019`.\n\n TriMap is a dimensionality reduction method that uses triplet constraints\n to form a low-dimensional embedding of a set of points. The triplet\n constraints are of the form \"point i is closer to point j than point k\".\n The triplets are sampled from the high-dimensional representation of the\n points and a weighting scheme is used to reflect the importance of each\n triplet.\n\n TriMap provides a significantly better global view of the data than the\n other dimensionality reduction methods such t-SNE, LargeVis, and UMAP.\n The global structure includes relative distances of the clusters, multiple\n scales in the data, and the existence of possible outliers. We define a\n global score to quantify the quality of an embedding in reflecting the\n global structure of the data.\n\n Parameters\n ----------\n adata\n Annotated data matrix.\n n_components\n Number of dimensions of the embedding.\n n_inliers\n Number of inlier points for triplet constraints.\n n_outliers\n Number of outlier points for triplet constraints.\n n_random\n Number of random triplet constraints per point.\n metric\n Distance measure: 'angular', 'euclidean', 'hamming', 'manhattan'.\n weight_adj\n Adjusting the weights using a non-linear transformation.\n lr\n Learning rate.\n n_iters\n Number of iterations.\n verbose\n If `True`, print the progress report.\n If `None`, `sc.settings.verbosity` is used.\n copy\n Return a copy instead of writing to `adata`.\n\n Returns\n -------\n Depending on `copy`, returns or updates `adata` with the following fields.\n\n **X_trimap** : :class:`~numpy.ndarray`, (:attr:`~anndata.AnnData.obsm`, shape=(n_samples, n_components), dtype `float`)\n TriMap coordinates of data.\n\n Example\n -------\n\n >>> import scanpy as sc\n >>> import scanpy.external as sce\n >>> pbmc = sc.datasets.pbmc68k_reduced()\n >>> pbmc = sce.tl.trimap(pbmc, copy=True)\n >>> sce.pl.trimap(pbmc, color=['bulk_labels'], s=10)\n \"\"\"\n\n try:\n from trimap import TRIMAP\n except ImportError:\n raise ImportError(\"\\nplease install trimap: \\n\\n\\tsudo pip install trimap\")\n adata = adata.copy() if copy else adata\n start = logg.info(\"computing TriMap\")\n adata = adata.copy() if copy else adata\n verbosity = settings.verbosity if verbose is None else verbose\n verbose = verbosity if isinstance(verbosity, bool) else verbosity > 0\n\n if \"X_pca\" in adata.obsm:\n n_dim_pca = adata.obsm[\"X_pca\"].shape[1]\n X = adata.obsm[\"X_pca\"][:, : min(n_dim_pca, 100)]\n else:\n X = adata.X\n if scp.issparse(X):\n raise ValueError(\n \"trimap currently does not support sparse matrices. Please\"\n \"use a dense matrix or apply pca first.\"\n )\n logg.warning(\"`X_pca` not found. Run `sc.pp.pca` first for speedup.\")\n X_trimap = TRIMAP(\n n_dims=n_components,\n n_inliers=n_inliers,\n n_outliers=n_outliers,\n n_random=n_random,\n lr=lr,\n distance=metric,\n weight_adj=weight_adj,\n n_iters=n_iters,\n verbose=verbose,\n ).fit_transform(X)\n adata.obsm[\"X_trimap\"] = X_trimap\n logg.info(\n \" finished\",\n time=start,\n deep=\"added\\n 'X_trimap', TriMap coordinates (adata.obsm)\",\n )\n return adata if copy else None\n\n\nfrom __future__ import annotations\n\nfrom types import MappingProxyType\nfrom typing import TYPE_CHECKING\n\nfrom ..._compat import old_positionals\n\nif TYPE_CHECKING:\n from collections.abc import Mapping, Sequence\n from typing import Any, Literal\n\n from anndata import AnnData\n\n from ..._utils import AnyRandom\n\n _AEType = Literal[\"zinb-conddisp\", \"zinb\", \"nb-conddisp\", \"nb\"]\n\n\n@old_positionals(\n \"ae_type\",\n \"normalize_per_cell\",\n \"scale\",\n \"log1p\",\n \"hidden_size\",\n \"hidden_dropout\",\n \"batchnorm\",\n \"activation\",\n \"init\",\n \"network_kwds\",\n \"epochs\",\n \"reduce_lr\",\n \"early_stop\",\n \"batch_size\",\n \"optimizer\",\n \"random_state\",\n \"threads\",\n \"learning_rate\",\n \"verbose\",\n \"training_kwds\",\n \"return_model\",\n \"return_info\",\n \"copy\",\n)\ndef dca(\n adata: AnnData,\n mode: Literal[\"denoise\", \"latent\"] = \"denoise\",\n *,\n ae_type: _AEType = \"nb-conddisp\",\n normalize_per_cell: bool = True,\n scale: bool = True,\n log1p: bool = True,\n # network args\n hidden_size: Sequence[int] = (64, 32, 64),\n hidden_dropout: float | Sequence[float] = 0.0,\n batchnorm: bool = True,\n activation: str = \"relu\",\n init: str = \"glorot_uniform\",\n network_kwds: Mapping[str, Any] = MappingProxyType({}),\n # training args\n epochs: int = 300,\n reduce_lr: int = 10,\n early_stop: int = 15,\n batch_size: int = 32,\n optimizer: str = \"RMSprop\",\n random_state: AnyRandom = 0,\n threads: int | None = None,\n learning_rate: float | None = None,\n verbose: bool = False,\n training_kwds: Mapping[str, Any] = MappingProxyType({}),\n return_model: bool = False,\n return_info: bool = False,\n copy: bool = False,\n) -> AnnData | None:\n \"\"\"\\\n Deep count autoencoder :cite:p:`Eraslan2019`.\n\n Fits a count autoencoder to the raw count data given in the anndata object\n in order to denoise the data and to capture hidden representation of\n cells in low dimensions. Type of the autoencoder and return values are\n determined by the parameters.\n\n .. note::\n More information and bug reports `here `__.\n\n Parameters\n ----------\n adata\n An anndata file with `.raw` attribute representing raw counts.\n mode\n `denoise` overwrites `adata.X` with denoised expression values.\n In `latent` mode DCA adds `adata.obsm['X_dca']` to given adata\n object. This matrix represent latent representation of cells via DCA.\n ae_type\n Type of the autoencoder. Return values and the architecture is\n determined by the type e.g. `nb` does not provide dropout\n probabilities. Types that end with \"-conddisp\", assumes that dispersion is mean dependant.\n normalize_per_cell\n If true, library size normalization is performed using\n the `sc.pp.normalize_per_cell` function in Scanpy and saved into adata\n object. Mean layer is re-introduces library size differences by\n scaling the mean value of each cell in the output layer. See the\n manuscript for more details.\n scale\n If true, the input of the autoencoder is centered using\n `sc.pp.scale` function of Scanpy. Note that the output is kept as raw\n counts as loss functions are designed for the count data.\n log1p\n If true, the input of the autoencoder is log transformed with a\n pseudocount of one using `sc.pp.log1p` function of Scanpy.\n hidden_size\n Width of hidden layers.\n hidden_dropout\n Probability of weight dropout in the autoencoder (per layer if list\n or tuple).\n batchnorm\n If true, batch normalization is performed.\n activation\n Activation function of hidden layers.\n init\n Initialization method used to initialize weights.\n network_kwds\n Additional keyword arguments for the autoencoder.\n epochs\n Number of total epochs in training.\n reduce_lr\n Reduces learning rate if validation loss does not improve in given number of epochs.\n early_stop\n Stops training if validation loss does not improve in given number of epochs.\n batch_size\n Number of samples in the batch used for SGD.\n optimizer\n Type of optimization method used for training.\n random_state\n Seed for python, numpy and tensorflow.\n threads\n Number of threads to use in training. All cores are used by default.\n learning_rate\n Learning rate to use in the training.\n verbose\n If true, prints additional information about training and architecture.\n training_kwds\n Additional keyword arguments for the training process.\n return_model\n If true, trained autoencoder object is returned. See \"Returns\".\n return_info\n If true, all additional parameters of DCA are stored in `adata.obsm` such as dropout\n probabilities (obsm['X_dca_dropout']) and estimated dispersion values\n (obsm['X_dca_dispersion']), in case that autoencoder is of type\n zinb or zinb-conddisp.\n copy\n If true, a copy of anndata is returned.\n\n Returns\n -------\n If `copy` is true and `return_model` is false, AnnData object is returned.\n\n In \"denoise\" mode, `adata.X` is overwritten with the denoised values.\n In \"latent\" mode, latent low dimensional representation of cells are stored\n in `adata.obsm['X_dca']` and `adata.X` is not modified.\n Note that these values are not corrected for library size effects.\n\n If `return_info` is true, all estimated distribution parameters are stored\n in AnnData like this:\n\n `.obsm[\"X_dca_dropout\"]`\n The mixture coefficient (pi) of the zero component in ZINB,\n i.e. dropout probability (if `ae_type` is `zinb` or `zinb-conddisp`).\n `.obsm[\"X_dca_dispersion\"]`\n The dispersion parameter of NB.\n `.uns[\"dca_loss_history\"]`\n The loss history of the training.\n See `.history` attribute of Keras History class for mode details.\n\n Finally, the raw counts are stored in `.raw` attribute of AnnData object.\n\n If `return_model` is given, trained model is returned.\n When both `copy` and `return_model` are true,\n a tuple of anndata and model is returned in that order.\n \"\"\"\n\n try:\n from dca.api import dca\n except ImportError:\n raise ImportError(\"Please install dca package (>= 0.2.1) via `pip install dca`\")\n\n return dca(\n adata,\n mode=mode,\n ae_type=ae_type,\n normalize_per_cell=normalize_per_cell,\n scale=scale,\n log1p=log1p,\n hidden_size=hidden_size,\n hidden_dropout=hidden_dropout,\n batchnorm=batchnorm,\n activation=activation,\n init=init,\n network_kwds=network_kwds,\n epochs=epochs,\n reduce_lr=reduce_lr,\n early_stop=early_stop,\n batch_size=batch_size,\n optimizer=optimizer,\n random_state=random_state,\n threads=threads,\n learning_rate=learning_rate,\n verbose=verbose,\n training_kwds=training_kwds,\n return_model=return_model,\n return_info=return_info,\n copy=copy,\n )\n\n\n\"\"\"\\\nDenoise high-dimensional data using MAGIC\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nfrom packaging.version import Version\n\nfrom ... import logging as logg\nfrom ..._settings import settings\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from collections.abc import Sequence\n from typing import Literal\n\n from anndata import AnnData\n\n from ..._utils import AnyRandom\n\nMIN_VERSION = \"2.0\"\n\n\n@doctest_needs(\"magic\")\ndef magic(\n adata: AnnData,\n name_list: Literal[\"all_genes\", \"pca_only\"] | Sequence[str] | None = None,\n *,\n knn: int = 5,\n decay: float | None = 1,\n knn_max: int | None = None,\n t: Literal[\"auto\"] | int = 3,\n n_pca: int | None = 100,\n solver: Literal[\"exact\", \"approximate\"] = \"exact\",\n knn_dist: str = \"euclidean\",\n random_state: AnyRandom = None,\n n_jobs: int | None = None,\n verbose: bool = False,\n copy: bool | None = None,\n **kwargs,\n) -> AnnData | None:\n \"\"\"\\\n Markov Affinity-based Graph Imputation of Cells (MAGIC) API :cite:p:`vanDijk2018`.\n\n MAGIC is an algorithm for denoising and transcript recover of single cells\n applied to single-cell sequencing data. MAGIC builds a graph from the data\n and uses diffusion to smooth out noise and recover the data manifold.\n\n The algorithm implemented here has changed primarily in two ways\n compared to the algorithm described in :cite:t:`vanDijk2018`. Firstly, we use\n the adaptive kernel described in :cite:t:`Moon2019` for\n improved stability. Secondly, data diffusion is applied\n in the PCA space, rather than the data space, for speed and\n memory improvements.\n\n More information and bug reports\n `here `__. For help, visit\n .\n\n Parameters\n ----------\n adata\n An anndata file with `.raw` attribute representing raw counts.\n name_list\n Denoised genes to return. The default `'all_genes'`/`None`\n may require a large amount of memory if the input data is sparse.\n Another possibility is `'pca_only'`.\n knn\n number of nearest neighbors on which to build kernel.\n decay\n sets decay rate of kernel tails.\n If None, alpha decaying kernel is not used.\n knn_max\n maximum number of nearest neighbors with nonzero connection.\n If `None`, will be set to 3 * `knn`.\n t\n power to which the diffusion operator is powered.\n This sets the level of diffusion. If 'auto', t is selected\n according to the Procrustes disparity of the diffused data.\n n_pca\n Number of principal components to use for calculating\n neighborhoods. For extremely large datasets, using\n n_pca < 20 allows neighborhoods to be calculated in\n roughly log(n_samples) time. If `None`, no PCA is performed.\n solver\n Which solver to use. \"exact\" uses the implementation described\n in :cite:t:`vanDijk2018`. \"approximate\" uses a faster\n implementation that performs imputation in the PCA space and then\n projects back to the gene space. Note, the \"approximate\" solver may\n return negative values.\n knn_dist\n recommended values: 'euclidean', 'cosine', 'precomputed'\n Any metric from `scipy.spatial.distance` can be used\n distance metric for building kNN graph. If 'precomputed',\n `data` should be an n_samples x n_samples distance or\n affinity matrix.\n random_state\n Random seed. Defaults to the global `numpy` random number generator.\n n_jobs\n Number of threads to use in training. All cores are used by default.\n verbose\n If `True` or an integer `>= 2`, print status messages.\n If `None`, `sc.settings.verbosity` is used.\n copy\n If true, a copy of anndata is returned. If `None`, `copy` is True if\n `genes` is not `'all_genes'` or `'pca_only'`. `copy` may only be False\n if `genes` is `'all_genes'` or `'pca_only'`, as the resultant data\n will otherwise have different column names from the input data.\n kwargs\n Additional arguments to `magic.MAGIC`.\n\n Returns\n -------\n If `copy` is True, AnnData object is returned.\n\n If `subset_genes` is not `all_genes`, PCA on MAGIC values of cells are\n stored in `adata.obsm['X_magic']` and `adata.X` is not modified.\n\n The raw counts are stored in `.raw` attribute of AnnData object.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> import scanpy.external as sce\n >>> adata = sc.datasets.paul15()\n >>> sc.pp.normalize_per_cell(adata)\n >>> sc.pp.sqrt(adata) # or sc.pp.log1p(adata)\n >>> adata_magic = sce.pp.magic(adata, name_list=['Mpo', 'Klf1', 'Ifitm1'], knn=5)\n >>> adata_magic.shape\n (2730, 3)\n >>> sce.pp.magic(adata, name_list='pca_only', knn=5)\n >>> adata.obsm['X_magic'].shape\n (2730, 100)\n >>> sce.pp.magic(adata, name_list='all_genes', knn=5)\n >>> adata.X.shape\n (2730, 3451)\n \"\"\"\n\n try:\n from magic import MAGIC, __version__\n except ImportError:\n raise ImportError(\n \"Please install magic package via `pip install --user \"\n \"git+git://github.com/KrishnaswamyLab/MAGIC.git#subdirectory=python`\"\n )\n else:\n if Version(__version__) < Version(MIN_VERSION):\n raise ImportError(\n \"scanpy requires magic-impute >= \"\n f\"v{MIN_VERSION} (detected: v{__version__}). \"\n \"Please update magic package via `pip install --user \"\n \"--upgrade magic-impute`\"\n )\n\n start = logg.info(\"computing MAGIC\")\n all_or_pca = isinstance(name_list, (str, type(None)))\n if all_or_pca and name_list not in {\"all_genes\", \"pca_only\", None}:\n raise ValueError(\n \"Invalid string value for `name_list`: \"\n \"Only `'all_genes'` and `'pca_only'` are allowed.\"\n )\n if copy is None:\n copy = not all_or_pca\n elif not all_or_pca and not copy:\n raise ValueError(\n \"Can only perform MAGIC in-place with `name_list=='all_genes' or \"\n f\"`name_list=='pca_only'` (got {name_list}). Consider setting \"\n \"`copy=True`\"\n )\n adata = adata.copy() if copy else adata\n n_jobs = settings.n_jobs if n_jobs is None else n_jobs\n\n X_magic = MAGIC(\n knn=knn,\n decay=decay,\n knn_max=knn_max,\n t=t,\n n_pca=n_pca,\n solver=solver,\n knn_dist=knn_dist,\n random_state=random_state,\n n_jobs=n_jobs,\n verbose=verbose,\n **kwargs,\n ).fit_transform(adata, genes=name_list)\n logg.info(\n \" finished\",\n time=start,\n deep=(\n \"added\\n 'X_magic', PCA on MAGIC coordinates (adata.obsm)\"\n if name_list == \"pca_only\"\n else \"\"\n ),\n )\n # update AnnData instance\n if name_list == \"pca_only\":\n # special case – update adata.obsm with smoothed values\n adata.obsm[\"X_magic\"] = X_magic.X\n elif copy:\n # just return X_magic\n X_magic.raw = adata\n adata = X_magic\n else:\n # replace data with smoothed data\n adata.raw = adata\n adata.X = X_magic.X\n\n if copy:\n return adata\n\n\n\"\"\"\nUse harmony to integrate cells from different experiments.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\n\nfrom ..._compat import old_positionals\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n\n@old_positionals(\"basis\", \"adjusted_basis\")\n@doctest_needs(\"harmonypy\")\ndef harmony_integrate(\n adata: AnnData,\n key: str,\n *,\n basis: str = \"X_pca\",\n adjusted_basis: str = \"X_pca_harmony\",\n **kwargs,\n):\n \"\"\"\\\n Use harmonypy :cite:p:`Korsunsky2019` to integrate different experiments.\n\n Harmony :cite:p:`Korsunsky2019` is an algorithm for integrating single-cell\n data from multiple experiments. This function uses the python\n port of Harmony, ``harmonypy``, to integrate single-cell data\n stored in an AnnData object. As Harmony works by adjusting the\n principal components, this function should be run after performing\n PCA but before computing the neighbor graph, as illustrated in the\n example below.\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n key\n The name of the column in ``adata.obs`` that differentiates\n among experiments/batches.\n basis\n The name of the field in ``adata.obsm`` where the PCA table is\n stored. Defaults to ``'X_pca'``, which is the default for\n ``sc.pp.pca()``.\n adjusted_basis\n The name of the field in ``adata.obsm`` where the adjusted PCA\n table will be stored after running this function. Defaults to\n ``X_pca_harmony``.\n kwargs\n Any additional arguments will be passed to\n ``harmonypy.run_harmony()``.\n\n Returns\n -------\n Updates adata with the field ``adata.obsm[obsm_out_field]``,\n containing principal components adjusted by Harmony such that\n different experiments are integrated.\n\n Example\n -------\n First, load libraries and example dataset, and preprocess.\n\n >>> import scanpy as sc\n >>> import scanpy.external as sce\n >>> adata = sc.datasets.pbmc3k()\n >>> sc.pp.recipe_zheng17(adata)\n >>> sc.pp.pca(adata)\n\n We now arbitrarily assign a batch metadata variable to each cell\n for the sake of example, but during real usage there would already\n be a column in ``adata.obs`` giving the experiment each cell came\n from.\n\n >>> adata.obs['batch'] = 1350*['a'] + 1350*['b']\n\n Finally, run harmony. Afterwards, there will be a new table in\n ``adata.obsm`` containing the adjusted PC's.\n\n >>> sce.pp.harmony_integrate(adata, 'batch')\n >>> 'X_pca_harmony' in adata.obsm\n True\n \"\"\"\n try:\n import harmonypy\n except ImportError:\n raise ImportError(\"\\nplease install harmonypy:\\n\\n\\tpip install harmonypy\")\n\n X = adata.obsm[basis].astype(np.float64)\n\n harmony_out = harmonypy.run_harmony(X, adata.obs, key, **kwargs)\n\n adata.obsm[adjusted_basis] = harmony_out.Z_corr.T\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nfrom ..._compat import old_positionals\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from typing import Callable\n\n from anndata import AnnData\n from sklearn.metrics import DistanceMetric\n\n\n@old_positionals(\"batch_key\", \"use_rep\", \"approx\", \"use_annoy\", \"metric\", \"copy\")\n@doctest_needs(\"bbknn\")\ndef bbknn(\n adata: AnnData,\n *,\n batch_key: str = \"batch\",\n use_rep: str = \"X_pca\",\n approx: bool = True,\n use_annoy: bool = True,\n metric: str | Callable | DistanceMetric = \"euclidean\",\n copy: bool = False,\n neighbors_within_batch: int = 3,\n n_pcs: int = 50,\n trim: int | None = None,\n annoy_n_trees: int = 10,\n pynndescent_n_neighbors: int = 30,\n pynndescent_random_state: int = 0,\n use_faiss: bool = True,\n set_op_mix_ratio: float = 1.0,\n local_connectivity: int = 1,\n **kwargs,\n) -> AnnData | None:\n \"\"\"\\\n Batch balanced kNN :cite:p:`Polanski2019`.\n\n Batch balanced kNN alters the kNN procedure to identify each cell's top neighbours in\n each batch separately instead of the entire cell pool with no accounting for batch.\n The nearest neighbours for each batch are then merged to create a final list of\n neighbours for the cell. Aligns batches in a quick and lightweight manner.\n\n For use in the scanpy workflow as an alternative to :func:`~scanpy.pp.neighbors`.\n\n .. note::\n\n This is just a wrapper of :func:`bbknn.bbknn`: up to date docstring,\n more information and bug reports there.\n\n Params\n ------\n adata\n Needs the PCA computed and stored in `adata.obsm[\"X_pca\"]`.\n batch_key\n `adata.obs` column name discriminating between your batches.\n use_rep\n The dimensionality reduction in `.obsm` to use for neighbour detection. Defaults to PCA.\n approx\n If `True`, use approximate neighbour finding - annoy or PyNNDescent. This results\n in a quicker run time for large datasets while also potentially increasing the degree of\n batch correction.\n use_annoy\n Only used when `approx=True`. If `True`, will use annoy for neighbour finding. If\n `False`, will use pyNNDescent instead.\n metric\n What distance metric to use. The options depend on the choice of neighbour algorithm.\n\n \"euclidean\", the default, is always available.\n\n Annoy supports \"angular\", \"manhattan\" and \"hamming\".\n\n PyNNDescent supports metrics listed in `pynndescent.distances.named_distances`\n and custom functions, including compiled Numba code.\n\n >>> import pynndescent\n >>> pynndescent.distances.named_distances.keys() # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE\n dict_keys(['euclidean', 'l2', 'sqeuclidean', 'manhattan', 'taxicab', 'l1', 'chebyshev', 'linfinity',\n 'linfty', 'linf', 'minkowski', 'seuclidean', 'standardised_euclidean', 'wminkowski', ...])\n\n KDTree supports members of :class:`sklearn.neighbors.KDTree`’s ``valid_metrics`` list, or parameterised\n :class:`~sklearn.metrics.DistanceMetric` objects:\n\n >>> import sklearn.neighbors\n >>> sklearn.neighbors.KDTree.valid_metrics\n ['euclidean', 'l2', 'minkowski', 'p', 'manhattan', 'cityblock', 'l1', 'chebyshev', 'infinity']\n\n .. note:: check the relevant documentation for up-to-date lists.\n copy\n If `True`, return a copy instead of writing to the supplied adata.\n neighbors_within_batch\n How many top neighbours to report for each batch; total number of neighbours in\n the initial k-nearest-neighbours computation will be this number times the number\n of batches. This then serves as the basis for the construction of a symmetrical\n matrix of connectivities.\n n_pcs\n How many dimensions (in case of PCA, principal components) to use in the analysis.\n trim\n Trim the neighbours of each cell to these many top connectivities. May help with\n population independence and improve the tidiness of clustering. The lower the value the\n more independent the individual populations, at the cost of more conserved batch effect.\n If `None`, sets the parameter value automatically to 10 times `neighbors_within_batch`\n times the number of batches. Set to 0 to skip.\n annoy_n_trees\n Only used with annoy neighbour identification. The number of trees to construct in the\n annoy forest. More trees give higher precision when querying, at the cost of increased\n run time and resource intensity.\n pynndescent_n_neighbors\n Only used with pyNNDescent neighbour identification. The number of neighbours to include\n in the approximate neighbour graph. More neighbours give higher precision when querying,\n at the cost of increased run time and resource intensity.\n pynndescent_random_state\n Only used with pyNNDescent neighbour identification. The RNG seed to use when creating\n the graph.\n use_faiss\n If `approx=False` and the metric is \"euclidean\", use the faiss package to compute\n nearest neighbours if installed. This improves performance at a minor cost to numerical\n precision as faiss operates on float32.\n set_op_mix_ratio\n UMAP connectivity computation parameter, float between 0 and 1, controlling the\n blend between a connectivity matrix formed exclusively from mutual nearest neighbour\n pairs (0) and a union of all observed neighbour relationships with the mutual pairs\n emphasised (1)\n local_connectivity\n UMAP connectivity computation parameter, how many nearest neighbors of each cell\n are assumed to be fully connected (and given a connectivity value of 1)\n\n Returns\n -------\n The `adata` with the batch-corrected graph.\n \"\"\"\n try:\n from bbknn import bbknn\n except ImportError:\n raise ImportError(\"Please install bbknn: `pip install bbknn`.\")\n return bbknn(\n adata=adata,\n batch_key=batch_key,\n use_rep=use_rep,\n approx=approx,\n use_annoy=use_annoy,\n metric=metric,\n copy=copy,\n neighbors_within_batch=neighbors_within_batch,\n n_pcs=n_pcs,\n trim=trim,\n annoy_n_trees=annoy_n_trees,\n pynndescent_n_neighbors=pynndescent_n_neighbors,\n pynndescent_random_state=pynndescent_random_state,\n use_faiss=use_faiss,\n set_op_mix_ratio=set_op_mix_ratio,\n local_connectivity=local_connectivity,\n **kwargs,\n )\n\n\n\"\"\"\nUse Scanorama to integrate cells from different experiments.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\n\nfrom ..._compat import old_positionals\nfrom ..._utils._doctests import doctest_needs\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n\n@old_positionals(\n \"basis\", \"adjusted_basis\", \"knn\", \"sigma\", \"approx\", \"alpha\", \"batch_size\"\n)\n@doctest_needs(\"scanorama\")\ndef scanorama_integrate(\n adata: AnnData,\n key: str,\n *,\n basis: str = \"X_pca\",\n adjusted_basis: str = \"X_scanorama\",\n knn: int = 20,\n sigma: float = 15,\n approx: bool = True,\n alpha: float = 0.10,\n batch_size: int = 5000,\n **kwargs,\n) -> None:\n \"\"\"\\\n Use Scanorama :cite:p:`Hie2019` to integrate different experiments.\n\n Scanorama :cite:p:`Hie2019` is an algorithm for integrating single-cell\n data from multiple experiments stored in an AnnData object. This\n function should be run after performing PCA but before computing\n the neighbor graph, as illustrated in the example below.\n\n This uses the implementation of scanorama_ :cite:p:`Hie2019`.\n\n .. _scanorama: https://github.com/brianhie/scanorama\n\n Parameters\n ----------\n adata\n The annotated data matrix.\n key\n The name of the column in ``adata.obs`` that differentiates\n among experiments/batches. Cells from the same batch must be\n contiguously stored in ``adata``.\n basis\n The name of the field in ``adata.obsm`` where the PCA table is\n stored. Defaults to ``'X_pca'``, which is the default for\n ``sc.pp.pca()``.\n adjusted_basis\n The name of the field in ``adata.obsm`` where the integrated\n embeddings will be stored after running this function. Defaults\n to ``X_scanorama``.\n knn\n Number of nearest neighbors to use for matching.\n sigma\n Correction smoothing parameter on Gaussian kernel.\n approx\n Use approximate nearest neighbors with Python ``annoy``;\n greatly speeds up matching runtime.\n alpha\n Alignment score minimum cutoff.\n batch_size\n The batch size used in the alignment vector computation. Useful\n when integrating very large (>100k samples) datasets. Set to\n large value that runs within available memory.\n kwargs\n Any additional arguments will be passed to\n ``scanorama.assemble()``.\n\n Returns\n -------\n Updates adata with the field ``adata.obsm[adjusted_basis]``,\n containing Scanorama embeddings such that different experiments\n are integrated.\n\n Example\n -------\n First, load libraries and example dataset, and preprocess.\n\n >>> import scanpy as sc\n >>> import scanpy.external as sce\n >>> adata = sc.datasets.pbmc3k()\n >>> sc.pp.recipe_zheng17(adata)\n >>> sc.pp.pca(adata)\n\n We now arbitrarily assign a batch metadata variable to each cell\n for the sake of example, but during real usage there would already\n be a column in ``adata.obs`` giving the experiment each cell came\n from.\n\n >>> adata.obs['batch'] = 1350*['a'] + 1350*['b']\n\n Finally, run Scanorama. Afterwards, there will be a new table in\n ``adata.obsm`` containing the Scanorama embeddings.\n\n >>> sce.pp.scanorama_integrate(adata, 'batch', verbose=1)\n Processing datasets a <=> b\n >>> 'X_scanorama' in adata.obsm\n True\n \"\"\"\n try:\n import scanorama\n except ImportError:\n raise ImportError(\"\\nplease install Scanorama:\\n\\n\\tpip install scanorama\")\n\n # Get batch indices in linear time.\n curr_batch = None\n batch_names = []\n name2idx = {}\n for idx in range(adata.X.shape[0]):\n batch_name = adata.obs[key].iat[idx]\n if batch_name != curr_batch:\n curr_batch = batch_name\n if batch_name in batch_names:\n # Contiguous batches important for preserving cell order.\n raise ValueError(\"Detected non-contiguous batches.\")\n batch_names.append(batch_name) # Preserve name order.\n name2idx[batch_name] = []\n name2idx[batch_name].append(idx)\n\n # Separate batches.\n datasets_dimred = [\n adata.obsm[basis][name2idx[batch_name]] for batch_name in batch_names\n ]\n\n # Integrate.\n integrated = scanorama.assemble(\n datasets_dimred, # Assemble in low dimensional space.\n knn=knn,\n sigma=sigma,\n approx=approx,\n alpha=alpha,\n ds_names=batch_names,\n batch_size=batch_size,\n **kwargs,\n )\n\n adata.obsm[adjusted_basis] = np.concatenate(integrated)\n\n\n\"\"\"\nHashSolo script provides a probabilistic cell hashing demultiplexing method\nwhich generates a noise distribution and signal distribution for\neach hashing barcode from empirically observed counts. These distributions\nare updates from the global signal and noise barcode distributions, which\nhelps in the setting where not many cells are observed. Signal distributions\nfor a hashing barcode are estimated from samples where that hashing barcode\nhas the highest count. Noise distributions for a hashing barcode are estimated\nfrom samples where that hashing barcode is one the k-2 lowest barcodes, where\nk is the number of barcodes. A doublet should then have its two highest\nbarcode counts most likely coming from a signal distribution for those barcodes.\nA singlet should have its highest barcode from a signal distribution, and its\nsecond highest barcode from a noise distribution. A negative two highest\nbarcodes should come from noise distributions. We test each of these\nhypotheses in a bayesian fashion, and select the most probable hypothesis.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom itertools import product\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom scipy.stats import norm\n\nfrom ..._compat import old_positionals\nfrom ..._utils import check_nonnegative_integers\nfrom ..._utils._doctests import doctest_skip\n\nif TYPE_CHECKING:\n from collections.abc import Sequence\n\n from anndata import AnnData\n from numpy.typing import ArrayLike, NDArray\n\n\ndef _calculate_log_likelihoods(\n data: np.ndarray, number_of_noise_barcodes: int\n) -> tuple[NDArray[np.float64], NDArray[np.float64], dict[int, str]]:\n \"\"\"Calculate log likelihoods for each hypothesis, negative, singlet, doublet\n\n Parameters\n ----------\n data\n cells by hashing counts matrix\n number_of_noise_barcodes\n number of barcodes to used to calculated noise distribution\n\n Returns\n -------\n log_likelihoods_for_each_hypothesis\n a 2d np.array log likelihood of each hypothesis\n all_indices\n counter_to_barcode_combo\n \"\"\"\n\n def gaussian_updates(\n data: np.ndarray, mu_o: float, std_o: float\n ) -> tuple[float, float]:\n \"\"\"Update parameters of your gaussian\n https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf\n\n Parameters\n ----------\n data\n 1-d array of counts\n mu_o\n global mean for hashing count distribution\n std_o\n global std for hashing count distribution\n\n Returns\n -------\n mean\n of gaussian\n std\n of gaussian\n \"\"\"\n lam_o = 1 / (std_o**2)\n n = len(data)\n lam = 1 / np.var(data) if len(data) > 1 else lam_o\n lam_n = lam_o + n * lam\n mu_n = (\n (np.mean(data) * n * lam + mu_o * lam_o) / lam_n if len(data) > 0 else mu_o\n )\n return mu_n, (1 / (lam_n / (n + 1))) ** (1 / 2)\n\n eps = 1e-15\n # probabilites for negative, singlet, doublets\n log_likelihoods_for_each_hypothesis = np.zeros((data.shape[0], 3))\n\n all_indices = np.empty(data.shape[0])\n num_of_barcodes = data.shape[1]\n number_of_non_noise_barcodes = (\n num_of_barcodes - number_of_noise_barcodes\n if number_of_noise_barcodes is not None\n else 2\n )\n\n num_of_noise_barcodes = num_of_barcodes - number_of_non_noise_barcodes\n\n # assume log normal\n data = np.log(data + 1)\n data_arg = np.argsort(data, axis=1)\n data_sort = np.sort(data, axis=1)\n\n # global signal and noise counts useful for when we have few cells\n # barcodes with the highest number of counts are assumed to be a true signal\n # barcodes with rank < k are considered to be noise\n global_signal_counts = np.ravel(data_sort[:, -1])\n global_noise_counts = np.ravel(data_sort[:, :-number_of_non_noise_barcodes])\n global_mu_signal_o, global_sigma_signal_o = (\n np.mean(global_signal_counts),\n np.std(global_signal_counts),\n )\n global_mu_noise_o, global_sigma_noise_o = (\n np.mean(global_noise_counts),\n np.std(global_noise_counts),\n )\n\n noise_params_dict = {}\n signal_params_dict = {}\n\n # for each barcode get empirical noise and signal distribution parameterization\n for x in np.arange(num_of_barcodes):\n sample_barcodes = data[:, x]\n sample_barcodes_noise_idx = np.where(data_arg[:, :num_of_noise_barcodes] == x)[\n 0\n ]\n sample_barcodes_signal_idx = np.where(data_arg[:, -1] == x)\n\n # get noise and signal counts\n noise_counts = sample_barcodes[sample_barcodes_noise_idx]\n signal_counts = sample_barcodes[sample_barcodes_signal_idx]\n\n # get parameters of distribution, assuming lognormal do update from global values\n noise_param = gaussian_updates(\n noise_counts, global_mu_noise_o, global_sigma_noise_o\n )\n signal_param = gaussian_updates(\n signal_counts, global_mu_signal_o, global_sigma_signal_o\n )\n noise_params_dict[x] = noise_param\n signal_params_dict[x] = signal_param\n\n counter_to_barcode_combo: dict[int, str] = {}\n counter = 0\n\n # for each combination of noise and signal barcode calculate probiltiy of in silico and real cell hypotheses\n for noise_sample_idx, signal_sample_idx in product(\n np.arange(num_of_barcodes), np.arange(num_of_barcodes)\n ):\n signal_subset = data_arg[:, -1] == signal_sample_idx\n noise_subset = data_arg[:, -2] == noise_sample_idx\n subset = signal_subset & noise_subset\n if sum(subset) == 0:\n continue\n\n indices = np.where(subset)[0]\n barcode_combo = \"_\".join([str(noise_sample_idx), str(signal_sample_idx)])\n all_indices[np.where(subset)[0]] = counter\n counter_to_barcode_combo[counter] = barcode_combo\n counter += 1\n noise_params = noise_params_dict[noise_sample_idx]\n signal_params = signal_params_dict[signal_sample_idx]\n\n # calculate probabilties for each hypothesis for each cell\n data_subset = data[subset]\n log_signal_signal_probs = np.log(\n norm.pdf(\n data_subset[:, signal_sample_idx],\n *signal_params[:-2],\n loc=signal_params[-2],\n scale=signal_params[-1],\n )\n + eps\n )\n signal_noise_params = signal_params_dict[noise_sample_idx]\n log_noise_signal_probs = np.log(\n norm.pdf(\n data_subset[:, noise_sample_idx],\n loc=signal_noise_params[-2],\n scale=signal_noise_params[-1],\n )\n + eps\n )\n\n log_noise_noise_probs = np.log(\n norm.pdf(\n data_subset[:, noise_sample_idx],\n loc=noise_params[-2],\n scale=noise_params[-1],\n )\n + eps\n )\n log_signal_noise_probs = np.log(\n norm.pdf(\n data_subset[:, signal_sample_idx],\n loc=noise_params[-2],\n scale=noise_params[-1],\n )\n + eps\n )\n\n probs_of_negative = np.sum(\n [log_noise_noise_probs, log_signal_noise_probs], axis=0\n )\n probs_of_singlet = np.sum(\n [log_noise_noise_probs, log_signal_signal_probs], axis=0\n )\n probs_of_doublet = np.sum(\n [log_noise_signal_probs, log_signal_signal_probs], axis=0\n )\n log_probs_list = [probs_of_negative, probs_of_singlet, probs_of_doublet]\n\n # each cell and each hypothesis probability\n for prob_idx, log_prob in enumerate(log_probs_list):\n log_likelihoods_for_each_hypothesis[indices, prob_idx] = log_prob\n return (\n log_likelihoods_for_each_hypothesis,\n all_indices,\n counter_to_barcode_combo,\n )\n\n\ndef _calculate_bayes_rule(\n data: np.ndarray, priors: ArrayLike, number_of_noise_barcodes: int\n) -> dict[str, np.ndarray]:\n \"\"\"\n Calculate bayes rule from log likelihoods\n\n Parameters\n ----------\n data\n Anndata object filled only with hashing counts\n priors\n a list of your prior for each hypothesis\n first element is your prior for the negative hypothesis\n second element is your prior for the singlet hypothesis\n third element is your prior for the doublet hypothesis\n We use [0.01, 0.8, 0.19] by default because we assume the barcodes\n in your cell hashing matrix are those cells which have passed QC\n in the transcriptome space, e.g. UMI counts, pct mito reads, etc.\n number_of_noise_barcodes\n number of barcodes to used to calculated noise distribution\n\n Returns\n -------\n A dict of bayes key results with the following entries:\n\n `\"most_likely_hypothesis\"`\n A 1d np.array of the most likely hypothesis\n `\"probs_hypotheses\"`\n A 2d np.array probability of each hypothesis\n `\"log_likelihoods_for_each_hypothesis\"`\n A 2d np.array log likelihood of each hypothesis\n \"\"\"\n priors = np.array(priors)\n log_likelihoods_for_each_hypothesis, _, _ = _calculate_log_likelihoods(\n data, number_of_noise_barcodes\n )\n probs_hypotheses = (\n np.exp(log_likelihoods_for_each_hypothesis)\n * priors\n / np.sum(\n np.multiply(np.exp(log_likelihoods_for_each_hypothesis), priors),\n axis=1,\n )[:, None]\n )\n most_likely_hypothesis = np.argmax(probs_hypotheses, axis=1)\n return {\n \"most_likely_hypothesis\": most_likely_hypothesis,\n \"probs_hypotheses\": probs_hypotheses,\n \"log_likelihoods_for_each_hypothesis\": log_likelihoods_for_each_hypothesis,\n }\n\n\n@old_positionals(\n \"priors\", \"pre_existing_clusters\", \"number_of_noise_barcodes\", \"inplace\"\n)\n@doctest_skip(\"Illustrative but not runnable doctest code\")\ndef hashsolo(\n adata: AnnData,\n cell_hashing_columns: Sequence[str],\n *,\n priors: tuple[float, float, float] = (0.01, 0.8, 0.19),\n pre_existing_clusters: str | None = None,\n number_of_noise_barcodes: int | None = None,\n inplace: bool = True,\n) -> AnnData | None:\n \"\"\"Probabilistic demultiplexing of cell hashing data using HashSolo :cite:p:`Bernstein2020`.\n\n .. note::\n More information and bug reports `here `__.\n\n Parameters\n ----------\n adata\n The (annotated) data matrix of shape `n_obs` × `n_vars`.\n Rows correspond to cells and columns to genes.\n cell_hashing_columns\n `.obs` columns that contain cell hashing counts.\n priors\n Prior probabilities of each hypothesis, in\n the order `[negative, singlet, doublet]`. The default is set to\n `[0.01, 0.8, 0.19]` assuming barcode counts are from cells that\n have passed QC in the transcriptome space, e.g. UMI counts, pct\n mito reads, etc.\n pre_existing_clusters\n The column in `.obs` containing pre-existing cluster assignments\n (e.g. Leiden clusters or cell types, but not batch assignments).\n If provided, demultiplexing will be performed separately for each\n cluster.\n number_of_noise_barcodes\n The number of barcodes used to create the noise distribution.\n Defaults to `len(cell_hashing_columns) - 2`.\n inplace\n Whether to update `adata` in-place or return a copy.\n\n Returns\n -------\n A copy of the input `adata` if `inplace=False`, otherwise the input\n `adata`. The following fields are added:\n\n `.obs[\"most_likely_hypothesis\"]`\n Index of the most likely hypothesis, where `0` corresponds to negative,\n `1` to singlet, and `2` to doublet.\n `.obs[\"cluster_feature\"]`\n The cluster assignments used for demultiplexing.\n `.obs[\"negative_hypothesis_probability\"]`\n Probability of the negative hypothesis.\n `.obs[\"singlet_hypothesis_probability\"]`\n Probability of the singlet hypothesis.\n `.obs[\"doublet_hypothesis_probability\"]`\n Probability of the doublet hypothesis.\n `.obs[\"Classification\"]`:\n Classification of the cell, one of the barcodes in `cell_hashing_columns`,\n `\"Negative\"`, or `\"Doublet\"`.\n\n Examples\n -------\n >>> import anndata\n >>> import scanpy.external as sce\n >>> adata = anndata.read_h5ad(\"data.h5ad\")\n >>> sce.pp.hashsolo(adata, [\"Hash1\", \"Hash2\", \"Hash3\"])\n >>> adata.obs.head()\n \"\"\"\n print(\n \"Please cite HashSolo paper:\\nhttps://www.cell.com/cell-systems/fulltext/S2405-4712(20)30195-2\"\n )\n adata = adata.copy() if not inplace else adata\n data = adata.obs[cell_hashing_columns].values\n if not check_nonnegative_integers(data):\n raise ValueError(\"Cell hashing counts must be non-negative\")\n if (number_of_noise_barcodes is not None) and (\n number_of_noise_barcodes >= len(cell_hashing_columns)\n ):\n raise ValueError(\n \"number_of_noise_barcodes must be at least one less \\\n than the number of samples you have as determined by the number of \\\n cell_hashing_columns you've given as input \"\n )\n num_of_cells = adata.shape[0]\n results = pd.DataFrame(\n np.zeros((num_of_cells, 6)),\n columns=[\n \"most_likely_hypothesis\",\n \"probs_hypotheses\",\n \"cluster_feature\",\n \"negative_hypothesis_probability\",\n \"singlet_hypothesis_probability\",\n \"doublet_hypothesis_probability\",\n ],\n index=adata.obs_names,\n )\n if pre_existing_clusters is not None:\n cluster_features = pre_existing_clusters\n unique_cluster_features = np.unique(adata.obs[cluster_features])\n for cluster_feature in unique_cluster_features:\n cluster_feature_bool_vector = adata.obs[cluster_features] == cluster_feature\n posterior_dict = _calculate_bayes_rule(\n data[cluster_feature_bool_vector],\n priors,\n number_of_noise_barcodes,\n )\n results.loc[cluster_feature_bool_vector, \"most_likely_hypothesis\"] = (\n posterior_dict[\"most_likely_hypothesis\"]\n )\n results.loc[cluster_feature_bool_vector, \"cluster_feature\"] = (\n cluster_feature\n )\n results.loc[\n cluster_feature_bool_vector, \"negative_hypothesis_probability\"\n ] = posterior_dict[\"probs_hypotheses\"][:, 0]\n results.loc[\n cluster_feature_bool_vector, \"singlet_hypothesis_probability\"\n ] = posterior_dict[\"probs_hypotheses\"][:, 1]\n results.loc[\n cluster_feature_bool_vector, \"doublet_hypothesis_probability\"\n ] = posterior_dict[\"probs_hypotheses\"][:, 2]\n else:\n posterior_dict = _calculate_bayes_rule(data, priors, number_of_noise_barcodes)\n results.loc[:, \"most_likely_hypothesis\"] = posterior_dict[\n \"most_likely_hypothesis\"\n ]\n results.loc[:, \"cluster_feature\"] = 0\n results.loc[:, \"negative_hypothesis_probability\"] = posterior_dict[\n \"probs_hypotheses\"\n ][:, 0]\n results.loc[:, \"singlet_hypothesis_probability\"] = posterior_dict[\n \"probs_hypotheses\"\n ][:, 1]\n results.loc[:, \"doublet_hypothesis_probability\"] = posterior_dict[\n \"probs_hypotheses\"\n ][:, 2]\n\n adata.obs[\"most_likely_hypothesis\"] = results.loc[\n adata.obs_names, \"most_likely_hypothesis\"\n ]\n adata.obs[\"cluster_feature\"] = results.loc[adata.obs_names, \"cluster_feature\"]\n adata.obs[\"negative_hypothesis_probability\"] = results.loc[\n adata.obs_names, \"negative_hypothesis_probability\"\n ]\n adata.obs[\"singlet_hypothesis_probability\"] = results.loc[\n adata.obs_names, \"singlet_hypothesis_probability\"\n ]\n adata.obs[\"doublet_hypothesis_probability\"] = results.loc[\n adata.obs_names, \"doublet_hypothesis_probability\"\n ]\n\n adata.obs[\"Classification\"] = None\n adata.obs.loc[adata.obs[\"most_likely_hypothesis\"] == 2, \"Classification\"] = (\n \"Doublet\"\n )\n adata.obs.loc[adata.obs[\"most_likely_hypothesis\"] == 0, \"Classification\"] = (\n \"Negative\"\n )\n all_sings = adata.obs[\"most_likely_hypothesis\"] == 1\n singlet_sample_index = np.argmax(\n adata.obs.loc[all_sings, cell_hashing_columns].values, axis=1\n )\n adata.obs.loc[all_sings, \"Classification\"] = adata.obs[\n cell_hashing_columns\n ].columns[singlet_sample_index]\n\n return adata if not inplace else None\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nfrom ..._settings import settings\n\nif TYPE_CHECKING:\n from collections.abc import Collection, Sequence\n from typing import Any, Literal\n\n import numpy as np\n import pandas as pd\n from anndata import AnnData\n\n\ndef mnn_correct(\n *datas: AnnData | np.ndarray,\n var_index: Collection[str] | None = None,\n var_subset: Collection[str] | None = None,\n batch_key: str = \"batch\",\n index_unique: str = \"-\",\n batch_categories: Collection[Any] | None = None,\n k: int = 20,\n sigma: float = 1.0,\n cos_norm_in: bool = True,\n cos_norm_out: bool = True,\n svd_dim: int | None = None,\n var_adj: bool = True,\n compute_angle: bool = False,\n mnn_order: Sequence[int] | None = None,\n svd_mode: Literal[\"svd\", \"rsvd\", \"irlb\"] = \"rsvd\",\n do_concatenate: bool = True,\n save_raw: bool = False,\n n_jobs: int | None = None,\n **kwargs,\n) -> tuple[\n np.ndarray | AnnData,\n list[pd.DataFrame],\n list[tuple[float | None, int]] | None,\n]:\n \"\"\"\\\n Correct batch effects by matching mutual nearest neighbors :cite:p:`Haghverdi2018` :cite:p:`Kang2018`.\n\n This uses the implementation of mnnpy_ :cite:p:`Kang2018`.\n\n Depending on `do_concatenate`, returns matrices or `AnnData` objects in the\n original order containing corrected expression values or a concatenated\n matrix or AnnData object.\n\n Be reminded that it is not advised to use the corrected data matrices for\n differential expression testing.\n\n More information and bug reports `here `__.\n\n .. _mnnpy: https://github.com/chriscainx/mnnpy\n\n Parameters\n ----------\n datas\n Expression matrices or AnnData objects. Matrices should be shaped like\n n_obs × n_vars (n_cell × n_gene) and have consistent number of columns.\n AnnData objects should have same number of variables.\n var_index\n The index (list of str) of vars (genes). Necessary when using only a\n subset of vars to perform MNN correction, and should be supplied with\n `var_subset`. When `datas` are AnnData objects, `var_index` is ignored.\n var_subset\n The subset of vars (list of str) to be used when performing MNN\n correction. Typically, a list of highly variable genes (HVGs).\n When set to `None`, uses all vars.\n batch_key\n The `batch_key` for :meth:`~anndata.AnnData.concatenate`.\n Only valid when `do_concatenate` and supplying `AnnData` objects.\n index_unique\n The `index_unique` for :meth:`~anndata.AnnData.concatenate`.\n Only valid when `do_concatenate` and supplying `AnnData` objects.\n batch_categories\n The `batch_categories` for :meth:`~anndata.AnnData.concatenate`.\n Only valid when `do_concatenate` and supplying AnnData objects.\n k\n Number of mutual nearest neighbors.\n sigma\n The bandwidth of the Gaussian smoothing kernel used to compute the\n correction vectors. Default is 1.\n cos_norm_in\n Whether cosine normalization should be performed on the input data prior\n to calculating distances between cells.\n cos_norm_out\n Whether cosine normalization should be performed prior to computing corrected expression values.\n svd_dim\n The number of dimensions to use for summarizing biological substructure\n within each batch. If None, biological components will not be removed\n from the correction vectors.\n var_adj\n Whether to adjust variance of the correction vectors. Note this step\n takes most computing time.\n compute_angle\n Whether to compute the angle between each cell’s correction vector and\n the biological subspace of the reference batch.\n mnn_order\n The order in which batches are to be corrected. When set to None, datas\n are corrected sequentially.\n svd_mode\n `'svd'` computes SVD using a non-randomized SVD-via-ID algorithm,\n while `'rsvd'` uses a randomized version. `'irlb'` perfores\n truncated SVD by implicitly restarted Lanczos bidiagonalization\n (forked from https://github.com/airysen/irlbpy).\n do_concatenate\n Whether to concatenate the corrected matrices or AnnData objects. Default is True.\n save_raw\n Whether to save the original expression data in the\n :attr:`~anndata.AnnData.raw` attribute.\n n_jobs\n The number of jobs. When set to `None`, automatically uses\n :attr:`scanpy._settings.ScanpyConfig.n_jobs`.\n kwargs\n optional keyword arguments for irlb.\n\n Returns\n -------\n datas\n Corrected matrix/matrices or AnnData object/objects, depending on the\n input type and `do_concatenate`.\n mnn_list\n A list containing MNN pairing information as DataFrames in each iteration step.\n angle_list\n A list containing angles of each batch.\n \"\"\"\n if len(datas) < 2:\n return datas, [], []\n\n try:\n import mnnpy\n from mnnpy import mnn_correct\n except ImportError:\n raise ImportError(\n \"Please install the package mnnpy \"\n \"(https://github.com/chriscainx/mnnpy). \"\n )\n\n n_jobs = settings.n_jobs if n_jobs is None else n_jobs\n\n if n_jobs < 2:\n mnnpy.settings.normalization = \"single\"\n else:\n mnnpy.settings.normalization = \"parallel\"\n\n datas, mnn_list, angle_list = mnn_correct(\n *datas,\n var_index=var_index,\n var_subset=var_subset,\n batch_key=batch_key,\n index_unique=index_unique,\n batch_categories=batch_categories,\n k=k,\n sigma=sigma,\n cos_norm_in=cos_norm_in,\n cos_norm_out=cos_norm_out,\n svd_dim=svd_dim,\n var_adj=var_adj,\n compute_angle=compute_angle,\n mnn_order=mnn_order,\n svd_mode=svd_mode,\n do_concatenate=do_concatenate,\n save_raw=save_raw,\n n_jobs=n_jobs,\n **kwargs,\n )\n return datas, mnn_list, angle_list\n\n\nfrom __future__ import annotations\n\nfrom sklearn.utils import deprecated\n\nfrom ...preprocessing import _scrublet\nfrom ._bbknn import bbknn\nfrom ._dca import dca\nfrom ._harmony_integrate import harmony_integrate\nfrom ._hashsolo import hashsolo\nfrom ._magic import magic\nfrom ._mnn_correct import mnn_correct\nfrom ._scanorama_integrate import scanorama_integrate\n\nscrublet = deprecated(\"Import from sc.pp instead\")(_scrublet.scrublet)\nscrublet_simulate_doublets = deprecated(\"Import from sc.pp instead\")(\n _scrublet.scrublet_simulate_doublets\n)\n\n__all__ = [\n \"bbknn\",\n \"dca\",\n \"harmony_integrate\",\n \"hashsolo\",\n \"magic\",\n \"mnn_correct\",\n \"scanorama_integrate\",\n]\n\n\n\"\"\"Shared docstrings for experimental function parameters.\"\"\"\n\nfrom __future__ import annotations\n\ndoc_adata = \"\"\"\\\nadata\n The annotated data matrix of shape `n_obs` × `n_vars`.\n Rows correspond to cells and columns to genes.\n\"\"\"\n\ndoc_dist_params = \"\"\"\\\ntheta\n The negative binomial overdispersion parameter `theta` for Pearson residuals.\n Higher values correspond to less overdispersion \\\n (`var = mean + mean^2/theta`), and `theta=np.inf` corresponds to a Poisson model.\nclip\n Determines if and how residuals are clipped:\n\n * If `None`, residuals are clipped to the interval \\\n `[-sqrt(n_obs), sqrt(n_obs)]`, where `n_obs` is the number of cells in the dataset (default behavior).\n * If any scalar `c`, residuals are clipped to the interval `[-c, c]`. Set \\\n `clip=np.inf` for no clipping.\n\"\"\"\n\ndoc_check_values = \"\"\"\\\ncheck_values\n If `True`, checks if counts in selected layer are integers as expected by this\n function, and return a warning if non-integers are found. Otherwise, proceed\n without checking. Setting this to `False` can speed up code for large datasets.\n\"\"\"\n\ndoc_layer = \"\"\"\\\nlayer\n Layer to use as input instead of `X`. If `None`, `X` is used.\n\"\"\"\n\ndoc_subset = \"\"\"\\\nsubset\n Inplace subset to highly-variable genes if `True` otherwise merely indicate\n highly variable genes.\n\"\"\"\n\ndoc_genes_batch_chunk = \"\"\"\\\nn_top_genes\n Number of highly-variable genes to keep. Mandatory if `flavor='seurat_v3'` or\n `flavor='pearson_residuals'`.\nbatch_key\n If specified, highly-variable genes are selected within each batch separately\n and merged. This simple process avoids the selection of batch-specific genes\n and acts as a lightweight batch correction method. Genes are first sorted by\n how many batches they are a HVG. If `flavor='pearson_residuals'`, ties are\n broken by the median rank (across batches) based on within-batch residual\n variance.\nchunksize\n If `flavor='pearson_residuals'`, this dertermines how many genes are processed at\n once while computing the residual variance. Choosing a smaller value will reduce\n the required memory.\n\"\"\"\n\ndoc_pca_chunk = \"\"\"\\\nn_comps\n Number of principal components to compute in the PCA step.\nrandom_state\n Random seed for setting the initial states for the optimization in the PCA step.\nkwargs_pca\n Dictionary of further keyword arguments passed on to `scanpy.pp.pca()`.\n\"\"\"\n\ndoc_inplace = \"\"\"\\\ninplace\n If `True`, update `adata` with results. Otherwise, return results. See below for\n details of what is returned.\n\"\"\"\n\ndoc_copy = \"\"\"\\\ncopy\n If `True`, the function runs on a copy of the input object and returns the\n modified copy. Otherwise, the input object is modified direcly. Not compatible\n with `inplace=False`.\n\"\"\"\n\n\nfrom __future__ import annotations\n\nfrom . import pp\n\n__all__ = [\"pp\"]\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom functools import partial\nfrom math import sqrt\nfrom typing import TYPE_CHECKING\n\nimport numba as nb\nimport numpy as np\nimport pandas as pd\nimport scipy.sparse as sp_sparse\nfrom anndata import AnnData\n\nfrom scanpy import logging as logg\nfrom scanpy._settings import Verbosity, settings\nfrom scanpy._utils import _doc_params, check_nonnegative_integers, view_to_actual\nfrom scanpy.experimental._docs import (\n doc_adata,\n doc_check_values,\n doc_dist_params,\n doc_genes_batch_chunk,\n doc_inplace,\n doc_layer,\n)\nfrom scanpy.get import _get_obs_rep\nfrom scanpy.preprocessing._distributed import materialize_as_ndarray\nfrom scanpy.preprocessing._utils import _get_mean_var\n\nif TYPE_CHECKING:\n from typing import Literal\n\n from numpy.typing import NDArray\n\n\n@nb.njit(parallel=True)\ndef _calculate_res_sparse(\n indptr: NDArray[np.integer],\n index: NDArray[np.integer],\n data: NDArray[np.float64],\n *,\n sums_genes: NDArray[np.float64],\n sums_cells: NDArray[np.float64],\n sum_total: np.float64,\n clip: np.float64,\n theta: np.float64,\n n_genes: int,\n n_cells: int,\n) -> NDArray[np.float64]:\n def get_value(cell: int, sparse_idx: int, stop_idx: int) -> np.float64:\n \"\"\"\n This function navigates the sparsity of the CSC (Compressed Sparse Column) matrix,\n returning the value at the specified cell location if it exists, or zero otherwise.\n \"\"\"\n if sparse_idx < stop_idx and index[sparse_idx] == cell:\n return data[sparse_idx]\n else:\n return np.float64(0.0)\n\n def clac_clipped_res_sparse(gene: int, cell: int, value: np.float64) -> np.float64:\n mu = sums_genes[gene] * sums_cells[cell] / sum_total\n mu_sum = value - mu\n pre_res = mu_sum / sqrt(mu + mu * mu / theta)\n res = np.float64(min(max(pre_res, -clip), clip))\n return res\n\n residuals = np.zeros(n_genes, dtype=np.float64)\n for gene in nb.prange(n_genes):\n start_idx = indptr[gene]\n stop_idx = indptr[gene + 1]\n\n sparse_idx = start_idx\n var_sum = np.float64(0.0)\n sum_clipped_res = np.float64(0.0)\n for cell in range(n_cells):\n value = get_value(cell, sparse_idx, stop_idx)\n clipped_res = clac_clipped_res_sparse(gene, cell, value)\n if value > 0:\n sparse_idx += 1\n sum_clipped_res += clipped_res\n\n mean_clipped_res = sum_clipped_res / n_cells\n sparse_idx = start_idx\n for cell in range(n_cells):\n value = get_value(cell, sparse_idx, stop_idx)\n clipped_res = clac_clipped_res_sparse(gene, cell, value)\n if value > 0:\n sparse_idx += 1\n diff = clipped_res - mean_clipped_res\n var_sum += diff * diff\n\n residuals[gene] = var_sum / n_cells\n return residuals\n\n\n@nb.njit(parallel=True)\ndef _calculate_res_dense(\n matrix,\n *,\n sums_genes: NDArray[np.float64],\n sums_cells: NDArray[np.float64],\n sum_total: np.float64,\n clip: np.float64,\n theta: np.float64,\n n_genes: int,\n n_cells: int,\n) -> NDArray[np.float64]:\n def clac_clipped_res_dense(gene: int, cell: int) -> np.float64:\n mu = sums_genes[gene] * sums_cells[cell] / sum_total\n value = matrix[cell, gene]\n\n mu_sum = value - mu\n pre_res = mu_sum / sqrt(mu + mu * mu / theta)\n res = np.float64(min(max(pre_res, -clip), clip))\n return res\n\n residuals = np.zeros(n_genes, dtype=np.float64)\n\n for gene in nb.prange(n_genes):\n sum_clipped_res = np.float64(0.0)\n for cell in range(n_cells):\n sum_clipped_res += clac_clipped_res_dense(gene, cell)\n mean_clipped_res = sum_clipped_res / n_cells\n\n var_sum = np.float64(0.0)\n for cell in range(n_cells):\n clipped_res = clac_clipped_res_dense(gene, cell)\n diff = clipped_res - mean_clipped_res\n var_sum += diff * diff\n\n residuals[gene] = var_sum / n_cells\n return residuals\n\n\ndef _highly_variable_pearson_residuals(\n adata: AnnData,\n *,\n theta: float = 100,\n clip: float | None = None,\n n_top_genes: int = 1000,\n batch_key: str | None = None,\n chunksize: int = 1000,\n check_values: bool = True,\n layer: str | None = None,\n subset: bool = False,\n inplace: bool = True,\n) -> pd.DataFrame | None:\n view_to_actual(adata)\n X = _get_obs_rep(adata, layer=layer)\n computed_on = layer if layer else \"adata.X\"\n\n # Check for raw counts\n if check_values and not check_nonnegative_integers(X):\n warnings.warn(\n \"`flavor='pearson_residuals'` expects raw count data, but non-integers were found.\",\n UserWarning,\n )\n # check theta\n if theta <= 0:\n # TODO: would \"underdispersion\" with negative theta make sense?\n # then only theta=0 were undefined..\n raise ValueError(\"Pearson residuals require theta > 0\")\n # prepare clipping\n\n if batch_key is None:\n batch_info = np.zeros(adata.shape[0], dtype=int)\n else:\n batch_info = adata.obs[batch_key].values\n n_batches = len(np.unique(batch_info))\n\n # Get pearson residuals for each batch separately\n residual_gene_vars = []\n for batch in np.unique(batch_info):\n adata_subset_prefilter = adata[batch_info == batch]\n X_batch_prefilter = _get_obs_rep(adata_subset_prefilter, layer=layer)\n\n # Filter out zero genes\n with settings.verbosity.override(Verbosity.error):\n nonzero_genes = np.ravel(X_batch_prefilter.sum(axis=0)) != 0\n adata_subset = adata_subset_prefilter[:, nonzero_genes]\n X_batch = _get_obs_rep(adata_subset, layer=layer)\n\n # Prepare clipping\n if clip is None:\n n = X_batch.shape[0]\n clip = np.sqrt(n)\n if clip < 0:\n raise ValueError(\"Pearson residuals require `clip>=0` or `clip=None`.\")\n\n if sp_sparse.issparse(X_batch):\n X_batch = X_batch.tocsc()\n X_batch.eliminate_zeros()\n calculate_res = partial(\n _calculate_res_sparse,\n X_batch.indptr,\n X_batch.indices,\n X_batch.data.astype(np.float64),\n )\n else:\n X_batch = np.array(X_batch, dtype=np.float64, order=\"F\")\n calculate_res = partial(_calculate_res_dense, X_batch)\n\n sums_genes = np.array(X_batch.sum(axis=0)).ravel()\n sums_cells = np.array(X_batch.sum(axis=1)).ravel()\n sum_total = np.sum(sums_genes)\n\n residual_gene_var = calculate_res(\n sums_genes=sums_genes,\n sums_cells=sums_cells,\n sum_total=np.float64(sum_total),\n clip=np.float64(clip),\n theta=np.float64(theta),\n n_genes=X_batch.shape[1],\n n_cells=X_batch.shape[0],\n )\n\n # Add 0 values for genes that were filtered out\n unmasked_residual_gene_var = np.zeros(len(nonzero_genes))\n unmasked_residual_gene_var[nonzero_genes] = residual_gene_var\n residual_gene_vars.append(unmasked_residual_gene_var.reshape(1, -1))\n\n residual_gene_vars = np.concatenate(residual_gene_vars, axis=0)\n\n # Get rank per gene within each batch\n # argsort twice gives ranks, small rank means most variable\n ranks_residual_var = np.argsort(np.argsort(-residual_gene_vars, axis=1), axis=1)\n ranks_residual_var = ranks_residual_var.astype(np.float32)\n # count in how many batches a genes was among the n_top_genes\n highly_variable_nbatches = np.sum(\n (ranks_residual_var < n_top_genes).astype(int), axis=0\n )\n # set non-top genes within each batch to nan\n ranks_residual_var[ranks_residual_var >= n_top_genes] = np.nan\n ranks_masked_array = np.ma.masked_invalid(ranks_residual_var)\n # Median rank across batches, ignoring batches in which gene was not selected\n medianrank_residual_var = np.ma.median(ranks_masked_array, axis=0).filled(np.nan)\n\n means, variances = materialize_as_ndarray(_get_mean_var(X))\n df = pd.DataFrame.from_dict(\n dict(\n means=means,\n variances=variances,\n residual_variances=np.mean(residual_gene_vars, axis=0),\n highly_variable_rank=medianrank_residual_var,\n highly_variable_nbatches=highly_variable_nbatches.astype(np.int64),\n highly_variable_intersection=highly_variable_nbatches == n_batches,\n )\n )\n df = df.set_index(adata.var_names)\n\n # Sort genes by how often they selected as hvg within each batch and\n # break ties with median rank of residual variance across batches\n df.sort_values(\n [\"highly_variable_nbatches\", \"highly_variable_rank\"],\n ascending=[False, True],\n na_position=\"last\",\n inplace=True,\n )\n\n high_var = np.zeros(df.shape[0], dtype=bool)\n high_var[:n_top_genes] = True\n df[\"highly_variable\"] = high_var\n df = df.loc[adata.var_names, :]\n\n if inplace:\n adata.uns[\"hvg\"] = {\"flavor\": \"pearson_residuals\", \"computed_on\": computed_on}\n logg.hint(\n \"added\\n\"\n \" 'highly_variable', boolean vector (adata.var)\\n\"\n \" 'highly_variable_rank', float vector (adata.var)\\n\"\n \" 'highly_variable_nbatches', int vector (adata.var)\\n\"\n \" 'highly_variable_intersection', boolean vector (adata.var)\\n\"\n \" 'means', float vector (adata.var)\\n\"\n \" 'variances', float vector (adata.var)\\n\"\n \" 'residual_variances', float vector (adata.var)\"\n )\n adata.var[\"means\"] = df[\"means\"].values\n adata.var[\"variances\"] = df[\"variances\"].values\n adata.var[\"residual_variances\"] = df[\"residual_variances\"]\n adata.var[\"highly_variable_rank\"] = df[\"highly_variable_rank\"].values\n if batch_key is not None:\n adata.var[\"highly_variable_nbatches\"] = df[\n \"highly_variable_nbatches\"\n ].values\n adata.var[\"highly_variable_intersection\"] = df[\n \"highly_variable_intersection\"\n ].values\n adata.var[\"highly_variable\"] = df[\"highly_variable\"].values\n\n if subset:\n adata._inplace_subset_var(df[\"highly_variable\"].values)\n\n else:\n if batch_key is None:\n df = df.drop(\n [\"highly_variable_nbatches\", \"highly_variable_intersection\"], axis=1\n )\n if subset:\n df = df.iloc[df.highly_variable.values, :]\n\n return df\n\n\n@_doc_params(\n adata=doc_adata,\n dist_params=doc_dist_params,\n genes_batch_chunk=doc_genes_batch_chunk,\n check_values=doc_check_values,\n layer=doc_layer,\n inplace=doc_inplace,\n)\ndef highly_variable_genes(\n adata: AnnData,\n *,\n theta: float = 100,\n clip: float | None = None,\n n_top_genes: int | None = None,\n batch_key: str | None = None,\n chunksize: int = 1000,\n flavor: Literal[\"pearson_residuals\"] = \"pearson_residuals\",\n check_values: bool = True,\n layer: str | None = None,\n subset: bool = False,\n inplace: bool = True,\n) -> pd.DataFrame | None:\n \"\"\"\\\n Select highly variable genes using analytic Pearson residuals :cite:p:`Lause2021`.\n\n In :cite:t:`Lause2021`, Pearson residuals of a negative binomial offset model are computed\n (with overdispersion `theta` shared across genes). By default, overdispersion\n `theta=100` is used and residuals are clipped to `sqrt(n_obs)`. Finally, genes\n are ranked by residual variance.\n\n Expects raw count input.\n\n Parameters\n ----------\n {adata}\n {dist_params}\n {genes_batch_chunk}\n flavor\n Choose the flavor for identifying highly variable genes. In this experimental\n version, only 'pearson_residuals' is functional.\n {check_values}\n {layer}\n subset\n If `True`, subset the data to highly-variable genes after finding them.\n Otherwise merely indicate highly variable genes in `adata.var` (see below).\n {inplace}\n\n Returns\n -------\n If `inplace=True`, `adata.var` is updated with the following fields. Otherwise,\n returns the same fields as :class:`~pandas.DataFrame`.\n\n highly_variable : :class:`bool`\n boolean indicator of highly-variable genes.\n means : :class:`float`\n means per gene.\n variances : :class:`float`\n variance per gene.\n residual_variances : :class:`float`\n For `flavor='pearson_residuals'`, residual variance per gene. Averaged in the\n case of multiple batches.\n highly_variable_rank : :class:`float`\n For `flavor='pearson_residuals'`, rank of the gene according to residual.\n variance, median rank in the case of multiple batches.\n highly_variable_nbatches : :class:`int`\n If `batch_key` given, denotes in how many batches genes are detected as HVG.\n highly_variable_intersection : :class:`bool`\n If `batch_key` given, denotes the genes that are highly variable in all batches.\n\n Notes\n -----\n Experimental version of `sc.pp.highly_variable_genes()`\n \"\"\"\n\n logg.info(\"extracting highly variable genes\")\n\n if not isinstance(adata, AnnData):\n raise ValueError(\n \"`pp.highly_variable_genes` expects an `AnnData` argument, \"\n \"pass `inplace=False` if you want to return a `pd.DataFrame`.\"\n )\n\n if flavor == \"pearson_residuals\":\n if n_top_genes is None:\n raise ValueError(\n \"`pp.highly_variable_genes` requires the argument `n_top_genes`\"\n \" for `flavor='pearson_residuals'`\"\n )\n return _highly_variable_pearson_residuals(\n adata,\n layer=layer,\n n_top_genes=n_top_genes,\n batch_key=batch_key,\n theta=theta,\n clip=clip,\n chunksize=chunksize,\n subset=subset,\n check_values=check_values,\n inplace=inplace,\n )\n else:\n raise ValueError(\n \"This is an experimental API and only `flavor=pearson_residuals` is available.\"\n )\n\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\n\nfrom scanpy import experimental\nfrom scanpy._utils import _doc_params\nfrom scanpy.experimental._docs import (\n doc_adata,\n doc_check_values,\n doc_dist_params,\n doc_genes_batch_chunk,\n doc_inplace,\n doc_pca_chunk,\n)\nfrom scanpy.preprocessing import pca\n\nif TYPE_CHECKING:\n import pandas as pd\n from anndata import AnnData\n\n\n@_doc_params(\n adata=doc_adata,\n dist_params=doc_dist_params,\n genes_batch_chunk=doc_genes_batch_chunk,\n pca_chunk=doc_pca_chunk,\n check_values=doc_check_values,\n inplace=doc_inplace,\n)\ndef recipe_pearson_residuals(\n adata: AnnData,\n *,\n theta: float = 100,\n clip: float | None = None,\n n_top_genes: int = 1000,\n batch_key: str | None = None,\n chunksize: int = 1000,\n n_comps: int | None = 50,\n random_state: float | None = 0,\n kwargs_pca: dict = {},\n check_values: bool = True,\n inplace: bool = True,\n) -> tuple[AnnData, pd.DataFrame] | None:\n \"\"\"\\\n Full pipeline for HVG selection and normalization by analytic Pearson residuals :cite:p:`Lause2021`.\n\n Applies gene selection based on Pearson residuals. On the resulting subset,\n Pearson residual normalization and PCA are performed.\n\n Expects raw count input.\n\n Params\n ------\n {adata}\n {dist_params}\n {genes_batch_chunk}\n {pca_chunk}\n {check_values}\n {inplace}\n\n Returns\n -------\n If `inplace=False`, separately returns the gene selection results (as\n :class:`~pandas.DataFrame`) and Pearson residual-based PCA results (as\n :class:`~anndata.AnnData`). If `inplace=True`, updates `adata` with the\n following fields for gene selection results:\n\n `.var['highly_variable']` : bool\n boolean indicator of highly-variable genes.\n `.var['means']` : float\n means per gene.\n `.var['variances']` : float\n variances per gene.\n `.var['residual_variances']` : float\n Pearson residual variance per gene. Averaged in the case of multiple\n batches.\n `.var['highly_variable_rank']` : float\n Rank of the gene according to residual variance, median rank in the\n case of multiple batches.\n `.var['highly_variable_nbatches']` : int\n If batch_key is given, this denotes in how many batches genes are\n detected as HVG.\n `.var['highly_variable_intersection']` : bool\n If batch_key is given, this denotes the genes that are highly variable\n in all batches.\n\n The following fields contain Pearson residual-based PCA results and\n normalization settings:\n\n `.uns['pearson_residuals_normalization']['pearson_residuals_df']`\n The subset of highly variable genes, normalized by Pearson residuals.\n `.uns['pearson_residuals_normalization']['theta']`\n The used value of the overdisperion parameter theta.\n `.uns['pearson_residuals_normalization']['clip']`\n The used value of the clipping parameter.\n\n `.obsm['X_pca']`\n PCA representation of data after gene selection and Pearson residual\n normalization.\n `.varm['PCs']`\n The principal components containing the loadings. When `inplace=True` this\n will contain empty rows for the genes not selected during HVG selection.\n `.uns['pca']['variance_ratio']`\n Ratio of explained variance.\n `.uns['pca']['variance']`\n Explained variance, equivalent to the eigenvalues of the covariance matrix.\n \"\"\"\n\n hvg_args = dict(\n flavor=\"pearson_residuals\",\n n_top_genes=n_top_genes,\n batch_key=batch_key,\n theta=theta,\n clip=clip,\n chunksize=chunksize,\n check_values=check_values,\n )\n\n if inplace:\n experimental.pp.highly_variable_genes(adata, **hvg_args, inplace=True)\n # TODO: are these copies needed?\n adata_pca = adata[:, adata.var[\"highly_variable\"]].copy()\n else:\n hvg = experimental.pp.highly_variable_genes(adata, **hvg_args, inplace=False)\n # TODO: are these copies needed?\n adata_pca = adata[:, hvg[\"highly_variable\"]].copy()\n\n experimental.pp.normalize_pearson_residuals(\n adata_pca, theta=theta, clip=clip, check_values=check_values\n )\n pca(adata_pca, n_comps=n_comps, random_state=random_state, **kwargs_pca)\n\n if inplace:\n normalization_param = adata_pca.uns[\"pearson_residuals_normalization\"]\n normalization_dict = dict(\n **normalization_param, pearson_residuals_df=adata_pca.to_df()\n )\n\n adata.uns[\"pca\"] = adata_pca.uns[\"pca\"]\n adata.varm[\"PCs\"] = np.zeros(shape=(adata.n_vars, n_comps))\n adata.varm[\"PCs\"][adata.var[\"highly_variable\"]] = adata_pca.varm[\"PCs\"]\n adata.uns[\"pearson_residuals_normalization\"] = normalization_dict\n adata.obsm[\"X_pca\"] = adata_pca.obsm[\"X_pca\"]\n return None\n else:\n return adata_pca, hvg\n\n\nfrom __future__ import annotations\n\nfrom types import MappingProxyType\nfrom typing import TYPE_CHECKING\nfrom warnings import warn\n\nimport numpy as np\nfrom anndata import AnnData\nfrom scipy.sparse import issparse\n\nfrom ... import logging as logg\nfrom ..._utils import (\n _doc_params,\n _empty,\n check_nonnegative_integers,\n view_to_actual,\n)\nfrom ...experimental._docs import (\n doc_adata,\n doc_check_values,\n doc_copy,\n doc_dist_params,\n doc_inplace,\n doc_layer,\n doc_pca_chunk,\n)\nfrom ...get import _get_obs_rep, _set_obs_rep\nfrom ...preprocessing._docs import doc_mask_var_hvg\nfrom ...preprocessing._pca import _handle_mask_var, pca\n\nif TYPE_CHECKING:\n from collections.abc import Mapping\n from typing import Any\n\n from ..._utils import Empty\n\n\ndef _pearson_residuals(X, theta, clip, check_values, *, copy: bool = False):\n X = X.copy() if copy else X\n\n # check theta\n if theta <= 0:\n # TODO: would \"underdispersion\" with negative theta make sense?\n # then only theta=0 were undefined..\n raise ValueError(\"Pearson residuals require theta > 0\")\n # prepare clipping\n if clip is None:\n n = X.shape[0]\n clip = np.sqrt(n)\n if clip < 0:\n raise ValueError(\"Pearson residuals require `clip>=0` or `clip=None`.\")\n\n if check_values and not check_nonnegative_integers(X):\n warn(\n \"`normalize_pearson_residuals()` expects raw count data, but non-integers were found.\",\n UserWarning,\n )\n\n if issparse(X):\n sums_genes = np.sum(X, axis=0)\n sums_cells = np.sum(X, axis=1)\n sum_total = np.sum(sums_genes).squeeze()\n else:\n sums_genes = np.sum(X, axis=0, keepdims=True)\n sums_cells = np.sum(X, axis=1, keepdims=True)\n sum_total = np.sum(sums_genes)\n\n mu = np.array(sums_cells @ sums_genes / sum_total)\n diff = np.array(X - mu)\n residuals = diff / np.sqrt(mu + mu**2 / theta)\n\n # clip\n residuals = np.clip(residuals, a_min=-clip, a_max=clip)\n\n return residuals\n\n\n@_doc_params(\n adata=doc_adata,\n dist_params=doc_dist_params,\n check_values=doc_check_values,\n layer=doc_layer,\n inplace=doc_inplace,\n copy=doc_copy,\n)\ndef normalize_pearson_residuals(\n adata: AnnData,\n *,\n theta: float = 100,\n clip: float | None = None,\n check_values: bool = True,\n layer: str | None = None,\n inplace: bool = True,\n copy: bool = False,\n) -> AnnData | dict[str, np.ndarray] | None:\n \"\"\"\\\n Applies analytic Pearson residual normalization, based on :cite:t:`Lause2021`.\n\n The residuals are based on a negative binomial offset model with overdispersion\n `theta` shared across genes. By default, residuals are clipped to `sqrt(n_obs)`\n and overdispersion `theta=100` is used.\n\n Expects raw count input.\n\n Params\n ------\n {adata}\n {dist_params}\n {check_values}\n {layer}\n {inplace}\n {copy}\n\n Returns\n -------\n If `inplace=True`, `adata.X` or the selected layer in `adata.layers` is updated\n with the normalized values. `adata.uns` is updated with the following fields.\n If `inplace=False`, the same fields are returned as dictionary with the\n normalized values in `results_dict['X']`.\n\n `.uns['pearson_residuals_normalization']['theta']`\n The used value of the overdisperion parameter theta.\n `.uns['pearson_residuals_normalization']['clip']`\n The used value of the clipping parameter.\n `.uns['pearson_residuals_normalization']['computed_on']`\n The name of the layer on which the residuals were computed.\n \"\"\"\n\n if copy:\n if not inplace:\n raise ValueError(\"`copy=True` cannot be used with `inplace=False`.\")\n adata = adata.copy()\n\n view_to_actual(adata)\n X = _get_obs_rep(adata, layer=layer)\n computed_on = layer if layer else \"adata.X\"\n\n msg = f\"computing analytic Pearson residuals on {computed_on}\"\n start = logg.info(msg)\n\n residuals = _pearson_residuals(X, theta, clip, check_values, copy=not inplace)\n settings_dict = dict(theta=theta, clip=clip, computed_on=computed_on)\n\n if inplace:\n _set_obs_rep(adata, residuals, layer=layer)\n adata.uns[\"pearson_residuals_normalization\"] = settings_dict\n else:\n results_dict = dict(X=residuals, **settings_dict)\n\n logg.info(\" finished ({time_passed})\", time=start)\n\n if copy:\n return adata\n elif not inplace:\n return results_dict\n\n\n@_doc_params(\n adata=doc_adata,\n dist_params=doc_dist_params,\n pca_chunk=doc_pca_chunk,\n mask_var_hvg=doc_mask_var_hvg,\n check_values=doc_check_values,\n inplace=doc_inplace,\n)\ndef normalize_pearson_residuals_pca(\n adata: AnnData,\n *,\n theta: float = 100,\n clip: float | None = None,\n n_comps: int | None = 50,\n random_state: float = 0,\n kwargs_pca: Mapping[str, Any] = MappingProxyType({}),\n mask_var: np.ndarray | str | None | Empty = _empty,\n use_highly_variable: bool | None = None,\n check_values: bool = True,\n inplace: bool = True,\n) -> AnnData | None:\n \"\"\"\\\n Applies analytic Pearson residual normalization and PCA, based on :cite:t:`Lause2021`.\n\n The residuals are based on a negative binomial offset model with overdispersion\n `theta` shared across genes. By default, residuals are clipped to `sqrt(n_obs)`,\n overdispersion `theta=100` is used, and PCA is run with 50 components.\n\n Operates on the subset of highly variable genes in `adata.var['highly_variable']`\n by default. Expects raw count input.\n\n Params\n ------\n {adata}\n {dist_params}\n {pca_chunk}\n {mask_var_hvg}\n {check_values}\n {inplace}\n\n Returns\n -------\n If `inplace=False`, returns the Pearson residual-based PCA results (as :class:`~anndata.AnnData`\n object). If `inplace=True`, updates `adata` with the following fields:\n\n `.uns['pearson_residuals_normalization']['pearson_residuals_df']`\n The subset of highly variable genes, normalized by Pearson residuals.\n `.uns['pearson_residuals_normalization']['theta']`\n The used value of the overdisperion parameter theta.\n `.uns['pearson_residuals_normalization']['clip']`\n The used value of the clipping parameter.\n\n `.obsm['X_pca']`\n PCA representation of data after gene selection (if applicable) and Pearson\n residual normalization.\n `.varm['PCs']`\n The principal components containing the loadings. When `inplace=True` and\n `use_highly_variable=True`, this will contain empty rows for the genes not\n selected.\n `.uns['pca']['variance_ratio']`\n Ratio of explained variance.\n `.uns['pca']['variance']`\n Explained variance, equivalent to the eigenvalues of the covariance matrix.\n \"\"\"\n\n # Unify new mask argument and deprecated use_highly_varible argument\n _, mask_var = _handle_mask_var(adata, mask_var, use_highly_variable)\n del use_highly_variable\n\n if mask_var is not None:\n adata_sub = adata[:, mask_var].copy()\n adata_pca = AnnData(\n adata_sub.X.copy(), obs=adata_sub.obs[[]], var=adata_sub.var[[]]\n )\n else:\n adata_pca = AnnData(adata.X.copy(), obs=adata.obs[[]], var=adata.var[[]])\n\n normalize_pearson_residuals(\n adata_pca, theta=theta, clip=clip, check_values=check_values\n )\n pca(adata_pca, n_comps=n_comps, random_state=random_state, **kwargs_pca)\n n_comps = adata_pca.obsm[\"X_pca\"].shape[1] # might be None\n\n if inplace:\n norm_settings = adata_pca.uns[\"pearson_residuals_normalization\"]\n norm_dict = dict(**norm_settings, pearson_residuals_df=adata_pca.to_df())\n if mask_var is not None:\n adata.varm[\"PCs\"] = np.zeros(shape=(adata.n_vars, n_comps))\n adata.varm[\"PCs\"][mask_var] = adata_pca.varm[\"PCs\"]\n else:\n adata.varm[\"PCs\"] = adata_pca.varm[\"PCs\"]\n adata.uns[\"pca\"] = adata_pca.uns[\"pca\"]\n adata.uns[\"pearson_residuals_normalization\"] = norm_dict\n adata.obsm[\"X_pca\"] = adata_pca.obsm[\"X_pca\"]\n return None\n else:\n return adata_pca\n\n\nfrom __future__ import annotations\n\nfrom scanpy.experimental.pp._highly_variable_genes import highly_variable_genes\nfrom scanpy.experimental.pp._normalization import (\n normalize_pearson_residuals,\n normalize_pearson_residuals_pca,\n)\nfrom scanpy.experimental.pp._recipes import recipe_pearson_residuals\n\n__all__ = [\n \"highly_variable_genes\",\n \"normalize_pearson_residuals\",\n \"normalize_pearson_residuals_pca\",\n \"recipe_pearson_residuals\",\n]\n\n\n# Biomart queries\nfrom __future__ import annotations\n\nfrom ._queries import (\n biomart_annotations,\n enrich, # gprofiler queries\n gene_coordinates,\n mitochondrial_genes,\n)\n\n__all__ = [\n \"biomart_annotations\",\n \"enrich\",\n \"gene_coordinates\",\n \"mitochondrial_genes\",\n]\n\n\nfrom __future__ import annotations\n\nfrom collections.abc import Iterable\nfrom functools import singledispatch\nfrom types import MappingProxyType\nfrom typing import TYPE_CHECKING\n\nfrom anndata import AnnData\n\nfrom .._utils import _doc_params\nfrom .._utils._doctests import doctest_needs\nfrom ..get import rank_genes_groups_df\n\nif TYPE_CHECKING:\n from collections.abc import Mapping\n from typing import Any\n\n import pandas as pd\n\n_doc_org = \"\"\"\\\norg\n Organism to query. Must be an organism in ensembl biomart. \"hsapiens\",\n \"mmusculus\", \"drerio\", etc.\\\n\"\"\"\n\n_doc_host = \"\"\"\\\nhost\n A valid BioMart host URL. Alternative values include archive urls (like\n \"grch37.ensembl.org\") or regional mirrors (like \"useast.ensembl.org\").\\\n\"\"\"\n\n_doc_use_cache = \"\"\"\\\nuse_cache\n Whether pybiomart should use a cache for requests. Will create a\n `.pybiomart.sqlite` file in current directory if used.\\\n\"\"\"\n\n\n@_doc_params(doc_org=_doc_org, doc_host=_doc_host, doc_use_cache=_doc_use_cache)\ndef simple_query(\n org: str,\n attrs: Iterable[str] | str,\n *,\n filters: dict[str, Any] | None = None,\n host: str = \"www.ensembl.org\",\n use_cache: bool = False,\n) -> pd.DataFrame:\n \"\"\"\\\n A simple interface to biomart.\n\n Params\n ------\n {doc_org}\n attrs\n What you want returned.\n filters\n What you want to pick out.\n {doc_host}\n {doc_use_cache}\n \"\"\"\n if isinstance(attrs, str):\n attrs = [attrs]\n elif isinstance(attrs, Iterable):\n attrs = list(attrs)\n else:\n raise TypeError(f\"attrs must be of type list or str, was {type(attrs)}.\")\n try:\n from pybiomart import Server\n except ImportError:\n raise ImportError(\n \"This method requires the `pybiomart` module to be installed.\"\n )\n server = Server(host, use_cache=use_cache)\n dataset = server.marts[\"ENSEMBL_MART_ENSEMBL\"].datasets[f\"{org}_gene_ensembl\"]\n res = dataset.query(attributes=attrs, filters=filters, use_attr_names=True)\n return res\n\n\n@doctest_needs(\"pybiomart\")\n@_doc_params(doc_org=_doc_org, doc_host=_doc_host, doc_use_cache=_doc_use_cache)\ndef biomart_annotations(\n org: str,\n attrs: Iterable[str],\n *,\n host: str = \"www.ensembl.org\",\n use_cache: bool = False,\n) -> pd.DataFrame:\n \"\"\"\\\n Retrieve gene annotations from ensembl biomart.\n\n Parameters\n ----------\n {doc_org}\n attrs\n Attributes to query biomart for.\n {doc_host}\n {doc_use_cache}\n\n Returns\n -------\n Dataframe containing annotations.\n\n Examples\n --------\n Retrieve genes coordinates and chromosomes\n\n >>> import scanpy as sc\n >>> annot = sc.queries.biomart_annotations(\n ... \"hsapiens\",\n ... [\"ensembl_gene_id\", \"start_position\", \"end_position\", \"chromosome_name\"],\n ... ).set_index(\"ensembl_gene_id\")\n >>> adata.var[annot.columns] = annot\n \"\"\"\n return simple_query(org=org, attrs=attrs, host=host, use_cache=use_cache)\n\n\n@doctest_needs(\"pybiomart\")\n@_doc_params(doc_org=_doc_org, doc_host=_doc_host, doc_use_cache=_doc_use_cache)\ndef gene_coordinates(\n org: str,\n gene_name: str,\n *,\n gene_attr: str = \"external_gene_name\",\n chr_exclude: Iterable[str] = (),\n host: str = \"www.ensembl.org\",\n use_cache: bool = False,\n) -> pd.DataFrame:\n \"\"\"\\\n Retrieve gene coordinates for specific organism through BioMart.\n\n Parameters\n ----------\n {doc_org}\n gene_name\n The gene symbol (e.g. \"hgnc_symbol\" for human) for which to retrieve\n coordinates.\n gene_attr\n The biomart attribute the gene symbol should show up for.\n chr_exclude\n A list of chromosomes to exclude from query.\n {doc_host}\n {doc_use_cache}\n\n Returns\n -------\n Dataframe containing gene coordinates for the specified gene symbol.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> sc.queries.gene_coordinates(\"hsapiens\", \"MT-TF\")\n \"\"\"\n res = simple_query(\n org=org,\n attrs=[\"chromosome_name\", \"start_position\", \"end_position\"],\n filters={gene_attr: gene_name},\n host=host,\n use_cache=use_cache,\n )\n return res[~res[\"chromosome_name\"].isin(chr_exclude)]\n\n\n@doctest_needs(\"pybiomart\")\n@_doc_params(doc_org=_doc_org, doc_host=_doc_host, doc_use_cache=_doc_use_cache)\ndef mitochondrial_genes(\n org: str,\n *,\n attrname: str = \"external_gene_name\",\n host: str = \"www.ensembl.org\",\n use_cache: bool = False,\n chromosome: str = \"MT\",\n) -> pd.DataFrame:\n \"\"\"\\\n Mitochondrial gene symbols for specific organism through BioMart.\n\n Parameters\n ----------\n {doc_org}\n attrname\n Biomart attribute field to return. Possible values include\n \"external_gene_name\", \"ensembl_gene_id\", \"hgnc_symbol\", \"mgi_symbol\",\n and \"zfin_id_symbol\".\n {doc_host}\n {doc_use_cache}\n chromosome\n Mitochrondrial chromosome name used in BioMart for organism.\n\n Returns\n -------\n Dataframe containing identifiers for mitochondrial genes.\n\n Examples\n --------\n >>> import scanpy as sc\n >>> mito_gene_names = sc.queries.mitochondrial_genes(\"hsapiens\")\n >>> mito_ensembl_ids = sc.queries.mitochondrial_genes(\"hsapiens\", attrname=\"ensembl_gene_id\")\n >>> mito_gene_names_fly = sc.queries.mitochondrial_genes(\"dmelanogaster\", chromosome=\"mitochondrion_genome\")\n \"\"\"\n return simple_query(\n org,\n attrs=[attrname],\n filters={\"chromosome_name\": [chromosome]},\n host=host,\n use_cache=use_cache,\n )\n\n\n@doctest_needs(\"gprofiler\")\n@singledispatch\n@_doc_params(doc_org=_doc_org)\ndef enrich(\n container: Iterable[str] | Mapping[str, Iterable[str]],\n *,\n org: str = \"hsapiens\",\n gprofiler_kwargs: Mapping[str, Any] = MappingProxyType({}),\n) -> pd.DataFrame:\n \"\"\"\\\n Get enrichment for DE results.\n\n This is a thin convenience wrapper around the very useful gprofiler_.\n\n This method dispatches on the first argument, leading to the following two\n signatures::\n\n enrich(container, ...)\n enrich(adata: AnnData, group, key: str, ...)\n\n Where::\n\n enrich(adata, group, key, ...) = enrich(adata.uns[key][\"names\"][group], ...)\n\n .. _gprofiler: https://pypi.org/project/gprofiler-official/#description\n\n Parameters\n ----------\n container\n Contains list of genes you'd like to search. If container is a `dict` all\n enrichment queries are made at once.\n adata\n AnnData object whose group will be looked for.\n group\n The group whose genes should be used for enrichment.\n key\n Key in `uns` to find group under.\n {doc_org}\n gprofiler_kwargs\n Keyword arguments to pass to `GProfiler.profile`, see gprofiler_. Some\n useful options are `no_evidences=False` which reports gene intersections,\n `sources=['GO:BP']` which limits gene sets to only GO biological processes and\n `all_results=True` which returns all results including the non-significant ones.\n **kwargs\n All other keyword arguments are passed to `sc.get.rank_genes_groups_df`. E.g.\n pval_cutoff, log2fc_min.\n\n Returns\n -------\n Dataframe of enrichment results.\n\n Examples\n --------\n Using `sc.queries.enrich` on a list of genes:\n\n >>> import scanpy as sc\n >>> sc.queries.enrich(['KLF4', 'PAX5', 'SOX2', 'NANOG'], org=\"hsapiens\")\n >>> sc.queries.enrich({{'set1':['KLF4', 'PAX5'], 'set2':['SOX2', 'NANOG']}}, org=\"hsapiens\")\n\n Using `sc.queries.enrich` on an :class:`anndata.AnnData` object:\n\n >>> pbmcs = sc.datasets.pbmc68k_reduced()\n >>> sc.tl.rank_genes_groups(pbmcs, \"bulk_labels\")\n >>> sc.queries.enrich(pbmcs, \"CD34+\")\n \"\"\"\n try:\n from gprofiler import GProfiler\n except ImportError:\n raise ImportError(\n \"This method requires the `gprofiler-official` module to be installed.\"\n )\n gprofiler = GProfiler(user_agent=\"scanpy\", return_dataframe=True)\n gprofiler_kwargs = dict(gprofiler_kwargs)\n for k in [\"organism\"]:\n if gprofiler_kwargs.get(k) is not None:\n raise ValueError(\n f\"Argument `{k}` should be passed directly through `enrich`, \"\n \"not through `gprofiler_kwargs`\"\n )\n return gprofiler.profile(container, organism=org, **gprofiler_kwargs)\n\n\n@enrich.register(AnnData)\ndef _enrich_anndata(\n adata: AnnData,\n group: str,\n *,\n org: str | None = \"hsapiens\",\n key: str = \"rank_genes_groups\",\n pval_cutoff: float = 0.05,\n log2fc_min: float | None = None,\n log2fc_max: float | None = None,\n gene_symbols: str | None = None,\n gprofiler_kwargs: Mapping[str, Any] = MappingProxyType({}),\n) -> pd.DataFrame:\n de = rank_genes_groups_df(\n adata,\n group=group,\n key=key,\n pval_cutoff=pval_cutoff,\n log2fc_min=log2fc_min,\n log2fc_max=log2fc_max,\n gene_symbols=gene_symbols,\n )\n if gene_symbols is not None:\n gene_list = list(de[gene_symbols].dropna())\n else:\n gene_list = list(de[\"names\"].dropna())\n return enrich(gene_list, org=org, gprofiler_kwargs=gprofiler_kwargs)\n\n\n\"\"\"Moran's I global spatial autocorrelation.\"\"\"\n\nfrom __future__ import annotations\n\nfrom functools import singledispatch\nfrom typing import TYPE_CHECKING\n\nimport numba\nimport numpy as np\nfrom scipy import sparse\n\nfrom .._compat import fullname\nfrom ..get import _get_obs_rep\nfrom ._common import _check_vals, _resolve_vals\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n\n@singledispatch\ndef morans_i(\n adata: AnnData,\n *,\n vals: np.ndarray | sparse.spmatrix | None = None,\n use_graph: str | None = None,\n layer: str | None = None,\n obsm: str | None = None,\n obsp: str | None = None,\n use_raw: bool = False,\n) -> np.ndarray | float:\n r\"\"\"\n Calculate Moran’s I Global Autocorrelation Statistic.\n\n Moran’s I is a global autocorrelation statistic for some measure on a graph. It is commonly used in\n spatial data analysis to assess autocorrelation on a 2D grid. It is closely related to Geary's C,\n but not identical. More info can be found `here `_.\n\n .. math::\n\n I =\n \\frac{\n N \\sum_{i, j} w_{i, j} z_{i} z_{j}\n }{\n S_{0} \\sum_{i} z_{i}^{2}\n }\n\n Params\n ------\n adata\n vals\n Values to calculate Moran's I for. If this is two dimensional, should\n be of shape `(n_features, n_cells)`. Otherwise should be of shape\n `(n_cells,)`. This matrix can be selected from elements of the anndata\n object by using key word arguments: `layer`, `obsm`, `obsp`, or\n `use_raw`.\n use_graph\n Key to use for graph in anndata object. If not provided, default\n neighbors connectivities will be used instead.\n layer\n Key for `adata.layers` to choose `vals`.\n obsm\n Key for `adata.obsm` to choose `vals`.\n obsp\n Key for `adata.obsp` to choose `vals`.\n use_raw\n Whether to use `adata.raw.X` for `vals`.\n\n\n This function can also be called on the graph and values directly. In this case\n the signature looks like:\n\n Params\n ------\n g\n The graph\n vals\n The values\n\n\n See the examples for more info.\n\n Returns\n -------\n If vals is two dimensional, returns a 1 dimensional ndarray array. Returns\n a scalar if `vals` is 1d.\n\n\n Examples\n --------\n\n Calculate Moran’s I for each components of a dimensionality reduction:\n\n .. code:: python\n\n import scanpy as sc, numpy as np\n\n pbmc = sc.datasets.pbmc68k_processed()\n pc_c = sc.metrics.morans_i(pbmc, obsm=\"X_pca\")\n\n\n It's equivalent to call the function directly on the underlying arrays:\n\n .. code:: python\n\n alt = sc.metrics.morans_i(pbmc.obsp[\"connectivities\"], pbmc.obsm[\"X_pca\"].T)\n np.testing.assert_array_equal(pc_c, alt)\n \"\"\"\n if use_graph is None:\n # Fix for anndata<0.7\n if hasattr(adata, \"obsp\") and \"connectivities\" in adata.obsp:\n g = adata.obsp[\"connectivities\"]\n elif \"neighbors\" in adata.uns:\n g = adata.uns[\"neighbors\"][\"connectivities\"]\n else:\n raise ValueError(\"Must run neighbors first.\")\n else:\n raise NotImplementedError()\n if vals is None:\n vals = _get_obs_rep(adata, use_raw=use_raw, layer=layer, obsm=obsm, obsp=obsp).T\n return morans_i(g, vals)\n\n\n###############################################################################\n# Calculation\n###############################################################################\n# This is done in a very similar way to gearys_c. See notes there for details.\n\n\n@numba.njit(cache=True, parallel=True)\ndef _morans_i_vec(\n g_data: np.ndarray,\n g_indices: np.ndarray,\n g_indptr: np.ndarray,\n x: np.ndarray,\n) -> float:\n W = g_data.sum()\n return _morans_i_vec_W(g_data, g_indices, g_indptr, x, W)\n\n\n@numba.njit(cache=True)\ndef _morans_i_vec_W(\n g_data: np.ndarray,\n g_indices: np.ndarray,\n g_indptr: np.ndarray,\n x: np.ndarray,\n W: np.float64,\n) -> float:\n z = x - x.mean()\n z2ss = (z * z).sum()\n n = len(x)\n inum = 0.0\n\n for i in numba.prange(n):\n s = slice(g_indptr[i], g_indptr[i + 1])\n i_indices = g_indices[s]\n i_data = g_data[s]\n inum += (i_data * z[i_indices]).sum() * z[i]\n\n return len(x) / W * inum / z2ss\n\n\n@numba.njit(cache=True)\ndef _morans_i_vec_W_sparse( # noqa: PLR0917\n g_data: np.ndarray,\n g_indices: np.ndarray,\n g_indptr: np.ndarray,\n x_data: np.ndarray,\n x_indices: np.ndarray,\n n: int,\n W: np.float64,\n) -> float:\n x = np.zeros(n, dtype=x_data.dtype)\n x[x_indices] = x_data\n return _morans_i_vec_W(g_data, g_indices, g_indptr, x, W)\n\n\n@numba.njit(cache=True, parallel=True)\ndef _morans_i_mtx(\n g_data: np.ndarray,\n g_indices: np.ndarray,\n g_indptr: np.ndarray,\n X: np.ndarray,\n) -> np.ndarray:\n m, n = X.shape\n assert n == len(g_indptr) - 1\n W = g_data.sum()\n out = np.zeros(m, dtype=np.float64)\n for k in numba.prange(m):\n x = X[k, :]\n out[k] = _morans_i_vec_W(g_data, g_indices, g_indptr, x, W)\n return out\n\n\n@numba.njit(cache=True, parallel=True)\ndef _morans_i_mtx_csr( # noqa: PLR0917\n g_data: np.ndarray,\n g_indices: np.ndarray,\n g_indptr: np.ndarray,\n x_data: np.ndarray,\n x_indices: np.ndarray,\n x_indptr: np.ndarray,\n x_shape: tuple,\n) -> np.ndarray:\n m, n = x_shape\n W = g_data.sum()\n out = np.zeros(m, dtype=np.float64)\n x_data_list = np.split(x_data, x_indptr[1:-1])\n x_indices_list = np.split(x_indices, x_indptr[1:-1])\n for k in numba.prange(m):\n out[k] = _morans_i_vec_W_sparse(\n g_data,\n g_indices,\n g_indptr,\n x_data_list[k],\n x_indices_list[k],\n n,\n W,\n )\n return out\n\n\n###############################################################################\n# Interface (taken from gearys C)\n###############################################################################\n\n\n@morans_i.register(sparse.csr_matrix)\ndef _morans_i(g: sparse.csr_matrix, vals: np.ndarray | sparse.spmatrix) -> np.ndarray:\n assert g.shape[0] == g.shape[1], \"`g` should be a square adjacency matrix\"\n vals = _resolve_vals(vals)\n g_data = g.data.astype(np.float64, copy=False)\n if isinstance(vals, sparse.csr_matrix):\n assert g.shape[0] == vals.shape[1]\n new_vals, idxer, full_result = _check_vals(vals)\n result = _morans_i_mtx_csr(\n g_data,\n g.indices,\n g.indptr,\n new_vals.data.astype(np.float64, copy=False),\n new_vals.indices,\n new_vals.indptr,\n new_vals.shape,\n )\n full_result[idxer] = result\n return full_result\n elif isinstance(vals, np.ndarray) and vals.ndim == 1:\n assert g.shape[0] == vals.shape[0]\n return _morans_i_vec(g_data, g.indices, g.indptr, vals)\n elif isinstance(vals, np.ndarray) and vals.ndim == 2:\n assert g.shape[0] == vals.shape[1]\n new_vals, idxer, full_result = _check_vals(vals)\n result = _morans_i_mtx(\n g_data, g.indices, g.indptr, new_vals.astype(np.float64, copy=False)\n )\n full_result[idxer] = result\n return full_result\n else:\n msg = (\n \"Moran’s I metric not implemented for vals of type \"\n f\"{fullname(type(vals))} and ndim {vals.ndim}.\"\n )\n raise NotImplementedError(msg)\n\n\n\"\"\"\nMetrics which don't quite deserve their own file.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pandas as pd\nfrom natsort import natsorted\nfrom pandas.api.types import CategoricalDtype\n\nif TYPE_CHECKING:\n from collections.abc import Sequence\n\n\ndef confusion_matrix(\n orig: pd.Series | np.ndarray | Sequence,\n new: pd.Series | np.ndarray | Sequence,\n data: pd.DataFrame | None = None,\n *,\n normalize: bool = True,\n) -> pd.DataFrame:\n \"\"\"\\\n Given an original and new set of labels, create a labelled confusion matrix.\n\n Parameters `orig` and `new` can either be entries in data or categorical arrays\n of the same size.\n\n Params\n ------\n orig\n Original labels.\n new\n New labels.\n data\n Optional dataframe to fill entries from.\n normalize\n Should the confusion matrix be normalized?\n\n\n Examples\n --------\n\n .. plot::\n\n import scanpy as sc; import seaborn as sns\n pbmc = sc.datasets.pbmc68k_reduced()\n cmtx = sc.metrics.confusion_matrix(\"bulk_labels\", \"louvain\", pbmc.obs)\n sns.heatmap(cmtx)\n\n \"\"\"\n from sklearn.metrics import confusion_matrix as _confusion_matrix\n\n if data is not None:\n if isinstance(orig, str):\n orig = data[orig]\n if isinstance(new, str):\n new = data[new]\n\n # Coercing so I don't have to deal with it later\n orig, new = pd.Series(orig), pd.Series(new)\n assert len(orig) == len(new)\n\n unique_labels = pd.unique(np.concatenate((orig.values, new.values)))\n\n # Compute\n mtx = _confusion_matrix(orig, new, labels=unique_labels)\n if normalize:\n sums = mtx.sum(axis=1)[:, np.newaxis]\n mtx = np.divide(mtx, sums, where=sums != 0)\n\n # Label\n orig_name = \"Original labels\" if orig.name is None else orig.name\n new_name = \"New Labels\" if new.name is None else new.name\n df = pd.DataFrame(\n mtx,\n index=pd.Index(unique_labels, name=orig_name),\n columns=pd.Index(unique_labels, name=new_name),\n )\n\n # Filter\n if isinstance(orig.dtype, CategoricalDtype):\n orig_idx = pd.Series(orig).cat.categories\n else:\n orig_idx = natsorted(pd.unique(orig))\n if isinstance(new.dtype, CategoricalDtype):\n new_idx = pd.Series(new).cat.categories\n else:\n new_idx = natsorted(pd.unique(new))\n df = df.loc[np.array(orig_idx), np.array(new_idx)]\n\n return df\n\n\n\"\"\"Geary's C autocorrelation.\"\"\"\n\nfrom __future__ import annotations\n\nfrom functools import singledispatch\nfrom typing import TYPE_CHECKING\n\nimport numba\nimport numpy as np\nfrom scipy import sparse\n\nfrom .._compat import fullname\nfrom ..get import _get_obs_rep\nfrom ._common import _check_vals, _resolve_vals\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n\n@singledispatch\ndef gearys_c(\n adata: AnnData,\n *,\n vals: np.ndarray | sparse.spmatrix | None = None,\n use_graph: str | None = None,\n layer: str | None = None,\n obsm: str | None = None,\n obsp: str | None = None,\n use_raw: bool = False,\n) -> np.ndarray | float:\n r\"\"\"\n Calculate `Geary's C `_, as used\n by `VISION `_.\n\n Geary's C is a measure of autocorrelation for some measure on a graph. This\n can be to whether measures are correlated between neighboring cells. Lower\n values indicate greater correlation.\n\n .. math::\n\n C =\n \\frac{\n (N - 1)\\sum_{i,j} w_{i,j} (x_i - x_j)^2\n }{\n 2W \\sum_i (x_i - \\bar{x})^2\n }\n\n Params\n ------\n adata\n vals\n Values to calculate Geary's C for. If this is two dimensional, should\n be of shape `(n_features, n_cells)`. Otherwise should be of shape\n `(n_cells,)`. This matrix can be selected from elements of the anndata\n object by using key word arguments: `layer`, `obsm`, `obsp`, or\n `use_raw`.\n use_graph\n Key to use for graph in anndata object. If not provided, default\n neighbors connectivities will be used instead.\n layer\n Key for `adata.layers` to choose `vals`.\n obsm\n Key for `adata.obsm` to choose `vals`.\n obsp\n Key for `adata.obsp` to choose `vals`.\n use_raw\n Whether to use `adata.raw.X` for `vals`.\n\n\n This function can also be called on the graph and values directly. In this case\n the signature looks like:\n\n Params\n ------\n g\n The graph\n vals\n The values\n\n\n See the examples for more info.\n\n Returns\n -------\n If vals is two dimensional, returns a 1 dimensional ndarray array. Returns\n a scalar if `vals` is 1d.\n\n\n Examples\n --------\n\n Calculate Geary’s C for each components of a dimensionality reduction:\n\n .. code:: python\n\n import scanpy as sc, numpy as np\n\n pbmc = sc.datasets.pbmc68k_processed()\n pc_c = sc.metrics.gearys_c(pbmc, obsm=\"X_pca\")\n\n\n It's equivalent to call the function directly on the underlying arrays:\n\n .. code:: python\n\n alt = sc.metrics.gearys_c(pbmc.obsp[\"connectivities\"], pbmc.obsm[\"X_pca\"].T)\n np.testing.assert_array_equal(pc_c, alt)\n \"\"\"\n if use_graph is None:\n # Fix for anndata<0.7\n if hasattr(adata, \"obsp\") and \"connectivities\" in adata.obsp:\n g = adata.obsp[\"connectivities\"]\n elif \"neighbors\" in adata.uns:\n g = adata.uns[\"neighbors\"][\"connectivities\"]\n else:\n raise ValueError(\"Must run neighbors first.\")\n else:\n raise NotImplementedError()\n if vals is None:\n vals = _get_obs_rep(adata, use_raw=use_raw, layer=layer, obsm=obsm, obsp=obsp).T\n return gearys_c(g, vals)\n\n\n###############################################################################\n# Calculation\n###############################################################################\n# Some notes on the implementation:\n# * This could be phrased as tensor multiplication. However that does not get\n# parallelized, which boosts performance almost linearly with cores.\n# * Due to the umap setting the default threading backend, a parallel numba\n# function that calls another parallel numba function can get stuck. This\n# ends up meaning code re-use will be limited until umap 0.4.\n# See: https://github.com/lmcinnes/umap/issues/306\n# * There can be a fair amount of numerical instability here (big reductions),\n# so data is cast to float64. Removing these casts/ conversion will cause the\n# tests to fail.\n\n\n@numba.njit(cache=True, parallel=True)\ndef _gearys_c_vec(\n data: np.ndarray,\n indices: np.ndarray,\n indptr: np.ndarray,\n x: np.ndarray,\n) -> float:\n W = data.sum()\n return _gearys_c_vec_W(data, indices, indptr, x, W)\n\n\n@numba.njit(cache=True, parallel=True)\ndef _gearys_c_vec_W(\n data: np.ndarray,\n indices: np.ndarray,\n indptr: np.ndarray,\n x: np.ndarray,\n W: np.float64,\n):\n n = len(indptr) - 1\n x = x.astype(np.float64)\n x_bar = x.mean()\n\n total = 0.0\n for i in numba.prange(n):\n s = slice(indptr[i], indptr[i + 1])\n i_indices = indices[s]\n i_data = data[s]\n total += np.sum(i_data * ((x[i] - x[i_indices]) ** 2))\n\n numer = (n - 1) * total\n denom = 2 * W * ((x - x_bar) ** 2).sum()\n return numer / denom\n\n\n# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n# Inner functions (per element C)\n# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n# For calling gearys_c on collections.\n# TODO: These are faster if we can compile them in parallel mode. However,\n# `workqueue` does not allow nested functions to be parallelized.\n# Additionally, there are currently problems with numba's compiler around\n# parallelization of this code:\n# https://github.com/numba/numba/issues/6774#issuecomment-788789663\n\n\n@numba.njit(cache=True)\ndef _gearys_c_inner_sparse_x_densevec(\n g_data: np.ndarray,\n g_indices: np.ndarray,\n g_indptr: np.ndarray,\n x: np.ndarray,\n W: np.float64,\n) -> float:\n x_bar = x.mean()\n total = 0.0\n n = len(x)\n for i in numba.prange(n):\n s = slice(g_indptr[i], g_indptr[i + 1])\n i_indices = g_indices[s]\n i_data = g_data[s]\n total += np.sum(i_data * ((x[i] - x[i_indices]) ** 2))\n numer = (n - 1) * total\n denom = 2 * W * ((x - x_bar) ** 2).sum()\n return numer / denom\n\n\n@numba.njit(cache=True)\ndef _gearys_c_inner_sparse_x_sparsevec( # noqa: PLR0917\n g_data: np.ndarray,\n g_indices: np.ndarray,\n g_indptr: np.ndarray,\n x_data: np.ndarray,\n x_indices: np.ndarray,\n n: int,\n W: np.float64,\n) -> float:\n x = np.zeros(n, dtype=np.float64)\n x[x_indices] = x_data\n x_bar = np.sum(x_data) / n\n total = 0.0\n n = len(x)\n for i in numba.prange(n):\n s = slice(g_indptr[i], g_indptr[i + 1])\n i_indices = g_indices[s]\n i_data = g_data[s]\n total += np.sum(i_data * ((x[i] - x[i_indices]) ** 2))\n numer = (n - 1) * total\n # Expanded from 2 * W * ((x_k - x_k_bar) ** 2).sum(), but uses sparsity\n # to skip some calculations\n # fmt: off\n denom = (\n 2 * W\n * (\n np.sum(x_data ** 2)\n - np.sum(x_data * x_bar * 2)\n + (x_bar ** 2) * n\n )\n )\n # fmt: on\n return numer / denom\n\n\n@numba.njit(cache=True, parallel=True)\ndef _gearys_c_mtx(\n g_data: np.ndarray,\n g_indices: np.ndarray,\n g_indptr: np.ndarray,\n X: np.ndarray,\n) -> np.ndarray:\n m, n = X.shape\n assert n == len(g_indptr) - 1\n W = g_data.sum()\n out = np.zeros(m, dtype=np.float64)\n for k in numba.prange(m):\n x = X[k, :].astype(np.float64)\n out[k] = _gearys_c_inner_sparse_x_densevec(g_data, g_indices, g_indptr, x, W)\n return out\n\n\n@numba.njit(cache=True, parallel=True)\ndef _gearys_c_mtx_csr( # noqa: PLR0917\n g_data: np.ndarray,\n g_indices: np.ndarray,\n g_indptr: np.ndarray,\n x_data: np.ndarray,\n x_indices: np.ndarray,\n x_indptr: np.ndarray,\n x_shape: tuple,\n) -> np.ndarray:\n m, n = x_shape\n W = g_data.sum()\n out = np.zeros(m, dtype=np.float64)\n x_data_list = np.split(x_data, x_indptr[1:-1])\n x_indices_list = np.split(x_indices, x_indptr[1:-1])\n for k in numba.prange(m):\n out[k] = _gearys_c_inner_sparse_x_sparsevec(\n g_data,\n g_indices,\n g_indptr,\n x_data_list[k],\n x_indices_list[k],\n n,\n W,\n )\n return out\n\n\n###############################################################################\n# Interface\n###############################################################################\n\n\n@gearys_c.register(sparse.csr_matrix)\ndef _gearys_c(g: sparse.csr_matrix, vals: np.ndarray | sparse.spmatrix) -> np.ndarray:\n assert g.shape[0] == g.shape[1], \"`g` should be a square adjacency matrix\"\n vals = _resolve_vals(vals)\n g_data = g.data.astype(np.float64, copy=False)\n if isinstance(vals, sparse.csr_matrix):\n assert g.shape[0] == vals.shape[1]\n new_vals, idxer, full_result = _check_vals(vals)\n result = _gearys_c_mtx_csr(\n g_data,\n g.indices,\n g.indptr,\n new_vals.data.astype(np.float64, copy=False),\n new_vals.indices,\n new_vals.indptr,\n new_vals.shape,\n )\n full_result[idxer] = result\n return full_result\n elif isinstance(vals, np.ndarray) and vals.ndim == 1:\n assert g.shape[0] == vals.shape[0]\n return _gearys_c_vec(g_data, g.indices, g.indptr, vals)\n elif isinstance(vals, np.ndarray) and vals.ndim == 2:\n assert g.shape[0] == vals.shape[1]\n new_vals, idxer, full_result = _check_vals(vals)\n result = _gearys_c_mtx(g_data, g.indices, g.indptr, new_vals)\n full_result[idxer] = result\n return full_result\n else:\n msg = (\n \"Geary’s C metric not implemented for vals of type \"\n f\"{fullname(type(vals))} and ndim {vals.ndim}.\"\n )\n raise NotImplementedError(msg)\n\n\nfrom __future__ import annotations\n\nimport warnings\nfrom functools import singledispatch\nfrom typing import TYPE_CHECKING, TypeVar\n\nimport numpy as np\nimport pandas as pd\nfrom scipy import sparse\n\nfrom .._compat import DaskArray\n\nif TYPE_CHECKING:\n from numpy.typing import NDArray\n\n\n@singledispatch\ndef _resolve_vals(val: NDArray | sparse.spmatrix) -> NDArray | sparse.csr_matrix:\n return np.asarray(val)\n\n\n@_resolve_vals.register(np.ndarray)\n@_resolve_vals.register(sparse.csr_matrix)\n@_resolve_vals.register(DaskArray)\ndef _(val):\n return val\n\n\n@_resolve_vals.register(sparse.spmatrix)\ndef _(val):\n return sparse.csr_matrix(val)\n\n\n@_resolve_vals.register(pd.DataFrame)\n@_resolve_vals.register(pd.Series)\ndef _(val):\n return val.to_numpy()\n\n\nV = TypeVar(\"V\", np.ndarray, sparse.csr_matrix)\n\n\ndef _check_vals(\n vals: V,\n) -> tuple[V, NDArray[np.bool_] | slice, NDArray[np.float64]]:\n \"\"\"\\\n Checks that values wont cause issues in computation.\n\n Returns new set of vals, and indexer to put values back into result.\n\n For details on why this is neccesary, see:\n https://github.com/scverse/scanpy/issues/1806\n \"\"\"\n from scanpy._utils import is_constant\n\n full_result = np.empty(vals.shape[0], dtype=np.float64)\n full_result.fill(np.nan)\n idxer = ~is_constant(vals, axis=1)\n if idxer.all():\n idxer = slice(None)\n else:\n warnings.warn(\n UserWarning(\n f\"{len(idxer) - idxer.sum()} variables were constant, will return nan for these.\",\n )\n )\n return vals[idxer], idxer, full_result\n\n\nfrom __future__ import annotations\n\nfrom ._gearys_c import gearys_c\nfrom ._metrics import confusion_matrix\nfrom ._morans_i import morans_i\n\n__all__ = [\"gearys_c\", \"morans_i\", \"confusion_matrix\"]\n\n\n\n\n\"\"\"\nFunctions returning copies of datasets as cheaply as possible,\ni.e. without having to hit the disk or (in case of ``_pbmc3k_normalized``) recomputing normalization.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport warnings\nfrom functools import cache\nfrom typing import TYPE_CHECKING\n\nimport scanpy as sc\n\nif TYPE_CHECKING:\n from anndata import AnnData\n\n# Functions returning the same objects (easy to misuse)\n\n\n_pbmc3k = cache(sc.datasets.pbmc3k)\n_pbmc3k_processed = cache(sc.datasets.pbmc3k_processed)\n_pbmc68k_reduced = cache(sc.datasets.pbmc68k_reduced)\n_krumsiek11 = cache(sc.datasets.krumsiek11)\n_paul15 = cache(sc.datasets.paul15)\n\n\n# Functions returning copies\n\n\ndef pbmc3k() -> AnnData:\n return _pbmc3k().copy()\n\n\ndef pbmc3k_processed() -> AnnData:\n return _pbmc3k_processed().copy()\n\n\ndef pbmc68k_reduced() -> AnnData:\n return _pbmc68k_reduced().copy()\n\n\ndef krumsiek11() -> AnnData:\n with warnings.catch_warnings():\n warnings.filterwarnings(\n \"ignore\", \"Observation names are not unique\", module=\"anndata\"\n )\n return _krumsiek11().copy()\n\n\ndef paul15() -> AnnData:\n return _paul15().copy()\n\n\n# Derived datasets\n\n\n@cache\ndef _pbmc3k_normalized() -> AnnData:\n pbmc = pbmc3k()\n pbmc.X = pbmc.X.astype(\"float64\") # For better accuracy\n sc.pp.filter_genes(pbmc, min_counts=1)\n sc.pp.log1p(pbmc)\n sc.pp.normalize_total(pbmc)\n sc.pp.highly_variable_genes(pbmc)\n return pbmc\n\n\ndef pbmc3k_normalized() -> AnnData:\n return _pbmc3k_normalized().copy()\n\n\n\"\"\"\nThis file contains helper functions for the scanpy test suite.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport warnings\nfrom itertools import permutations\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nfrom anndata.tests.helpers import asarray, assert_equal\n\nimport scanpy as sc\n\nif TYPE_CHECKING:\n from scanpy._compat import DaskArray\n\n# TODO: Report more context on the fields being compared on error\n# TODO: Allow specifying paths to ignore on comparison\n\n###########################\n# Representation choice\n###########################\n# These functions can be used to check that functions are correctly using arugments like `layers`, `obsm`, etc.\n\n\ndef anndata_v0_8_constructor_compat(X, *args, **kwargs):\n \"\"\"Constructor for anndata that uses dtype of X for test compatibility with older versions of AnnData.\n\n Once the minimum version of AnnData is 0.9, this function can be replaced with the default constructor.\n \"\"\"\n import anndata as ad\n from packaging.version import Version\n\n if Version(ad.__version__) < Version(\"0.9\"):\n return ad.AnnData(X=X, *args, **kwargs, dtype=X.dtype)\n else:\n return ad.AnnData(X=X, *args, **kwargs)\n\n\ndef check_rep_mutation(func, X, *, fields=(\"layer\", \"obsm\"), **kwargs):\n \"\"\"Check that only the array meant to be modified is modified.\"\"\"\n adata = anndata_v0_8_constructor_compat(X.copy())\n\n for field in fields:\n sc.get._set_obs_rep(adata, X, **{field: field})\n X_array = asarray(X)\n\n adata_X = func(adata, copy=True, **kwargs)\n adatas_proc = {\n field: func(adata, copy=True, **{field: field}, **kwargs) for field in fields\n }\n\n # Modified fields\n for field in fields:\n result_array = asarray(\n sc.get._get_obs_rep(adatas_proc[field], **{field: field})\n )\n np.testing.assert_array_equal(asarray(adata_X.X), result_array)\n\n # Unmodified fields\n for field in fields:\n np.testing.assert_array_equal(X_array, asarray(adatas_proc[field].X))\n np.testing.assert_array_equal(\n X_array, asarray(sc.get._get_obs_rep(adata_X, **{field: field}))\n )\n for field_a, field_b in permutations(fields, 2):\n result_array = asarray(\n sc.get._get_obs_rep(adatas_proc[field_a], **{field_b: field_b})\n )\n np.testing.assert_array_equal(X_array, result_array)\n\n\ndef check_rep_results(func, X, *, fields=[\"layer\", \"obsm\"], **kwargs):\n \"\"\"Checks that the results of a computation add values/ mutate the anndata object in a consistent way.\"\"\"\n # Gen data\n empty_X = np.zeros(shape=X.shape, dtype=X.dtype)\n adata = sc.AnnData(\n X=empty_X.copy(),\n layers={\"layer\": empty_X.copy()},\n obsm={\"obsm\": empty_X.copy()},\n )\n\n adata_X = adata.copy()\n adata_X.X = X.copy()\n\n adatas_proc = {}\n for field in fields:\n cur = adata.copy()\n sc.get._set_obs_rep(cur, X.copy(), **{field: field})\n adatas_proc[field] = cur\n\n # Apply function\n func(adata_X, **kwargs)\n for field in fields:\n func(adatas_proc[field], **{field: field}, **kwargs)\n\n # Reset X\n adata_X.X = empty_X.copy()\n for field in fields:\n sc.get._set_obs_rep(adatas_proc[field], empty_X.copy(), **{field: field})\n\n for field_a, field_b in permutations(fields, 2):\n assert_equal(adatas_proc[field_a], adatas_proc[field_b])\n for field in fields:\n assert_equal(adata_X, adatas_proc[field])\n\n\ndef _check_check_values_warnings(function, adata, expected_warning, kwargs={}):\n \"\"\"\n Runs `function` on `adata` with provided arguments `kwargs` twice:\n once with `check_values=True` and once with `check_values=False`.\n Checks that the `expected_warning` is only raised whtn `check_values=True`.\n \"\"\"\n\n # expecting 0 no-int warnings\n with warnings.catch_warnings(record=True) as record:\n function(adata.copy(), **kwargs, check_values=False)\n warning_msgs = [w.message.args[0] for w in record]\n assert expected_warning not in warning_msgs\n\n # expecting 1 no-int warning\n with warnings.catch_warnings(record=True) as record:\n function(adata.copy(), **kwargs, check_values=True)\n warning_msgs = [w.message.args[0] for w in record]\n assert expected_warning in warning_msgs\n\n\n# Delayed imports for case where we aren't using dask\ndef as_dense_dask_array(*args, **kwargs) -> DaskArray:\n from anndata.tests.helpers import as_dense_dask_array\n\n return as_dense_dask_array(*args, **kwargs)\n\n\ndef as_sparse_dask_array(*args, **kwargs) -> DaskArray:\n from anndata.tests.helpers import as_sparse_dask_array\n\n return as_sparse_dask_array(*args, **kwargs)\n\n\n\"\"\"Like fixtures, but more flexible\"\"\"\n\nfrom __future__ import annotations\n\nfrom typing import TYPE_CHECKING\n\nimport pytest\nfrom anndata.tests.helpers import asarray\nfrom scipy import sparse\n\nfrom .._helpers import (\n as_dense_dask_array,\n as_sparse_dask_array,\n)\nfrom .._pytest.marks import needs\n\nif TYPE_CHECKING:\n from collections.abc import Iterable\n from typing import Literal\n\n from _pytest.mark.structures import ParameterSet\n\n\ndef param_with(\n at: ParameterSet,\n *,\n marks: Iterable[pytest.Mark | pytest.MarkDecorator] = (),\n id: str | None = None,\n) -> ParameterSet:\n return pytest.param(*at.values, marks=[*at.marks, *marks], id=id or at.id)\n\n\nMAP_ARRAY_TYPES: dict[\n tuple[Literal[\"mem\", \"dask\"], Literal[\"dense\", \"sparse\"]],\n tuple[ParameterSet, ...],\n] = {\n (\"mem\", \"dense\"): (pytest.param(asarray, id=\"numpy_ndarray\"),),\n (\"mem\", \"sparse\"): (\n pytest.param(sparse.csr_matrix, id=\"scipy_csr\"),\n pytest.param(sparse.csc_matrix, id=\"scipy_csc\"),\n ),\n (\"dask\", \"dense\"): (\n pytest.param(\n as_dense_dask_array,\n marks=[needs.dask, pytest.mark.anndata_dask_support],\n id=\"dask_array_dense\",\n ),\n ),\n (\"dask\", \"sparse\"): (\n pytest.param(\n as_sparse_dask_array,\n marks=[needs.dask, pytest.mark.anndata_dask_support],\n id=\"dask_array_sparse\",\n ),\n # probably not necessary to also do csc\n ),\n}\n\nARRAY_TYPES_MEM = tuple(\n at for (strg, _), ats in MAP_ARRAY_TYPES.items() if strg == \"mem\" for at in ats\n)\nARRAY_TYPES_DASK = tuple(\n at for (strg, _), ats in MAP_ARRAY_TYPES.items() if strg == \"dask\" for at in ats\n)\n\nARRAY_TYPES_DENSE = tuple(\n at for (_, spsty), ats in MAP_ARRAY_TYPES.items() if spsty == \"dense\" for at in ats\n)\nARRAY_TYPES_SPARSE = tuple(\n at for (_, spsty), ats in MAP_ARRAY_TYPES.items() if \"sparse\" in spsty for at in ats\n)\nARRAY_TYPES_SPARSE_DASK_UNSUPPORTED = tuple(\n (\n param_with(at, marks=[pytest.mark.xfail(reason=\"sparse-in-dask not supported\")])\n if attrs[0] == \"dask\" and \"sparse\" in attrs[1]\n else at\n )\n for attrs, ats in MAP_ARRAY_TYPES.items()\n for at in ats\n)\n\nARRAY_TYPES = tuple(at for ats in MAP_ARRAY_TYPES.values() for at in ats)\n\n\nfrom __future__ import annotations\n\nimport sys\nfrom enum import Enum, auto\nfrom importlib.util import find_spec\nfrom typing import TYPE_CHECKING\n\nimport pytest\nfrom packaging.version import Version\n\nif TYPE_CHECKING:\n from collections.abc import Callable\n\n\nSKIP_EXTRA: dict[str, Callable[[], str | None]] = {}\n\n\ndef _skip_if_skmisc_too_old() -> str | None:\n import numpy as np\n import skmisc\n\n if Version(skmisc.__version__) <= Version(\"0.3.1\") and Version(\n np.__version__\n ) >= Version(\"2\"):\n return \"scikit-misc≤0.3.1 requires numpy<2\"\n return None\n\n\nSKIP_EXTRA[\"skmisc\"] = _skip_if_skmisc_too_old\n\n\ndef _next_val(name: str, start: int, count: int, last_values: list[str]) -> str:\n \"\"\"Distribution name for matching modules\"\"\"\n return name.replace(\"_\", \"-\")\n\n\nclass QuietMarkDecorator(pytest.MarkDecorator):\n def __init__(self, mark: pytest.Mark) -> None:\n super().__init__(mark, _ispytest=True)\n\n\nclass needs(QuietMarkDecorator, Enum):\n \"\"\"\n Pytest skip marker evaluated at module import.\n\n This allows us to see the amount of skipped tests at the start of a test run.\n :func:`pytest.importorskip` skips tests after they started running.\n \"\"\"\n\n # _generate_next_value_ needs to come before members, also it’s finnicky:\n # https://github.com/python/mypy/issues/7591#issuecomment-652800625\n _generate_next_value_ = (\n staticmethod(_next_val) if sys.version_info >= (3, 10) else _next_val\n )\n\n mod: str\n\n dask = auto()\n dask_ml = auto()\n fa2 = auto()\n gprofiler = \"gprofiler-official\"\n leidenalg = auto()\n louvain = auto()\n openpyxl = auto()\n igraph = auto()\n pybiomart = auto()\n skimage = \"scikit-image\"\n skmisc = \"scikit-misc\"\n zarr = auto()\n zappy = auto()\n # external\n bbknn = auto()\n harmony = \"harmonyTS\"\n harmonypy = auto()\n magic = \"magic-impute\"\n palantir = auto()\n phate = auto()\n phenograph = auto()\n pypairs = auto()\n samalg = \"sam-algorithm\"\n scanorama = auto()\n trimap = auto()\n wishbone = \"wishbone-dev\"\n\n def __init__(self, mod: str) -> None:\n self.mod = mod\n reason = self.skip_reason\n dec = pytest.mark.skipif(bool(reason), reason=reason or \"\")\n super().__init__(dec.mark)\n\n @property\n def skip_reason(self) -> str | None:\n if find_spec(self._name_):\n if skip_extra := SKIP_EXTRA.get(self._name_):\n return skip_extra()\n return None\n reason = f\"needs module `{self._name_}`\"\n if self._name_.casefold() != self.mod.casefold().replace(\"-\", \"_\"):\n reason = f\"{reason} (`pip install {self.mod}`)\"\n return reason\n\n\n\"\"\"A private pytest plugin\"\"\"\n\nfrom __future__ import annotations\n\nimport os\nimport sys\nfrom typing import TYPE_CHECKING\n\nimport pytest\n\nfrom .fixtures import * # noqa: F403\nfrom .marks import needs\n\nif TYPE_CHECKING:\n from collections.abc import Generator, Iterable\n\n\n# Defining it here because it’s autouse.\n@pytest.fixture(autouse=True)\ndef _global_test_context(\n request: pytest.FixtureRequest,\n cache: pytest.Cache,\n tmp_path_factory: pytest.TempPathFactory,\n) -> Generator[None, None, None]:\n \"\"\"Switch to agg backend, reset settings, and close all figures at teardown.\"\"\"\n # make sure seaborn is imported and did its thing\n import seaborn as sns # noqa: F401\n from matplotlib import pyplot as plt\n from matplotlib.testing import setup\n\n import scanpy as sc\n\n setup()\n sc.settings.logfile = sys.stderr\n sc.settings.verbosity = \"hint\"\n sc.settings.autoshow = True\n # create directory for debug data\n cache.mkdir(\"debug\")\n # reuse data files between test runs (unless overwritten in the test)\n sc.settings.datasetdir = cache.mkdir(\"scanpy-data\")\n # create new writedir for each test run\n sc.settings.writedir = tmp_path_factory.mktemp(\"scanpy_write\")\n\n if isinstance(request.node, pytest.DoctestItem):\n _modify_doctests(request)\n\n yield\n\n plt.close(\"all\")\n\n\n@pytest.fixture(autouse=True, scope=\"session\")\ndef max_threads() -> Generator[int, None, None]:\n \"\"\"Limit number of threads used per worker when using pytest-xdist.\n\n Prevents oversubscription of the CPU when multiple tests with parallel code are\n running at once.\n \"\"\"\n if (n_workers := os.environ.get(\"PYTEST_XDIST_WORKER_COUNT\")) is not None:\n import threadpoolctl\n\n n_cpus = os.cpu_count() or 1\n n_workers = int(n_workers)\n max_threads = max(n_cpus // n_workers, 1)\n\n with threadpoolctl.threadpool_limits(limits=max_threads):\n yield max_threads\n else:\n yield 0\n\n\ndef pytest_addoption(parser: pytest.Parser) -> None:\n parser.addoption(\n \"--internet-tests\",\n action=\"store_true\",\n default=False,\n help=(\n \"Run tests that retrieve stuff from the internet. \"\n \"This increases test time.\"\n ),\n )\n\n\ndef pytest_collection_modifyitems(\n config: pytest.Config, items: Iterable[pytest.Item]\n) -> None:\n import pytest\n\n run_internet = config.getoption(\"--internet-tests\")\n skip_internet = pytest.mark.skip(reason=\"need --internet-tests option to run\")\n for item in items:\n # All tests marked with `pytest.mark.internet` get skipped unless\n # `--run-internet` passed\n if not run_internet and (\"internet\" in item.keywords):\n item.add_marker(skip_internet)\n\n\ndef _modify_doctests(request: pytest.FixtureRequest) -> None:\n from scanpy._utils import _import_name\n\n assert isinstance(request.node, pytest.DoctestItem)\n\n request.getfixturevalue(\"_doctest_env\")\n\n func = _import_name(request.node.name)\n needs_mod: str | None\n skip_reason: str | None\n if (\n (needs_mod := getattr(func, \"_doctest_needs\", None))\n and (skip_reason := needs[needs_mod].skip_reason)\n ) or (skip_reason := getattr(func, \"_doctest_skip_reason\", None)):\n pytest.skip(reason=skip_reason)\n if getattr(func, \"_doctest_internet\", False) and not request.config.getoption(\n \"--internet-tests\"\n ):\n pytest.skip(reason=\"need --internet-tests option to run\")\n\n\ndef pytest_itemcollected(item: pytest.Item) -> None:\n # Dask AnnData tests require anndata > 0.10\n import anndata\n from packaging.version import Version\n\n requires_anndata_dask_support = (\n len([mark for mark in item.iter_markers(name=\"anndata_dask_support\")]) > 0\n )\n\n if requires_anndata_dask_support and Version(anndata.__version__) < Version(\"0.10\"):\n item.add_marker(\n pytest.mark.skip(reason=\"dask support requires anndata version > 0.10\")\n )\n\n\nassert (\n \"scanpy\" not in sys.modules\n), \"scanpy is already imported, this will mess up test coverage\"\n\n\n\"\"\"Fixtures for parametrized datasets.\"\"\"\n\nfrom __future__ import annotations\n\nfrom itertools import product\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pytest\nfrom anndata import AnnData, read_h5ad\nfrom anndata import __version__ as anndata_version\nfrom packaging.version import Version\nfrom scipy import sparse\n\nif Version(anndata_version) >= Version(\"0.10.0\"):\n from anndata._core.sparse_dataset import (\n BaseCompressedSparseDataset as SparseDataset,\n )\n from anndata.experimental import sparse_dataset\n\n def make_sparse(x):\n return sparse_dataset(x)\nelse:\n from anndata._core.sparse_dataset import SparseDataset\n\n def make_sparse(x):\n return SparseDataset(x)\n\n\nif TYPE_CHECKING:\n from collections.abc import Callable\n\n from numpy.typing import DTypeLike\n\n\n@pytest.fixture(\n scope=\"session\",\n params=list(\n product([sparse.csr_matrix.toarray, sparse.csr_matrix], [\"float32\", \"int64\"])\n ),\n ids=lambda x: f\"{x[0].__name__}-{x[1]}\",\n)\ndef pbmc3ks_parametrized_session(request) -> dict[bool, AnnData]:\n from ..._helpers.data import pbmc3k\n\n sparsity_func, dtype = request.param\n return {\n small: _prepare_pbmc_testdata(pbmc3k(), sparsity_func, dtype, small=small)\n for small in [True, False]\n }\n\n\n@pytest.fixture\ndef pbmc3k_parametrized(pbmc3ks_parametrized_session) -> Callable[[], AnnData]:\n return pbmc3ks_parametrized_session[False].copy\n\n\n@pytest.fixture\ndef pbmc3k_parametrized_small(pbmc3ks_parametrized_session) -> Callable[[], AnnData]:\n return pbmc3ks_parametrized_session[True].copy\n\n\n@pytest.fixture(\n scope=\"session\",\n params=[np.random.randn, lambda *x: sparse.random(*x, format=\"csr\")],\n ids=[\"sparse\", \"dense\"],\n)\n# worker_id for xdist since we don't want to override open files\ndef backed_adata(\n request: pytest.FixtureRequest,\n tmp_path_factory: pytest.TempPathFactory,\n worker_id: str = \"serial\",\n) -> AnnData:\n tmp_path = tmp_path_factory.mktemp(\"backed_adata\")\n rand_func = request.param\n tmp_path = tmp_path / f\"test_{rand_func.__name__}_{worker_id}.h5ad\"\n X = rand_func(200, 10).astype(np.float32)\n cat = np.random.randint(0, 3, (X.shape[0],)).ravel()\n adata = AnnData(X, obs={\"cat\": cat})\n adata.obs[\"percent_mito\"] = np.random.rand(X.shape[0])\n adata.obs[\"n_counts\"] = X.sum(axis=1)\n adata.obs[\"cat\"] = adata.obs[\"cat\"].astype(\"category\")\n adata.layers[\"X_copy\"] = adata.X[...]\n adata.write_h5ad(tmp_path)\n adata = read_h5ad(tmp_path, backed=\"r\")\n adata.layers[\"X_copy\"] = (\n make_sparse(adata.file[\"X\"])\n if isinstance(adata.X, SparseDataset)\n else adata.file[\"X\"]\n )\n return adata\n\n\ndef _prepare_pbmc_testdata(\n adata: AnnData,\n sparsity_func: Callable[\n [np.ndarray | sparse.spmatrix], np.ndarray | sparse.spmatrix\n ],\n dtype: DTypeLike,\n *,\n small: bool,\n) -> AnnData:\n \"\"\"Prepares 3k PBMC dataset with batch key `batch` and defined datatype/sparsity.\n\n Params\n ------\n sparsity_func\n sparsity function applied to adata.X (e.g. csr_matrix.toarray for dense or csr_matrix for sparse)\n dtype\n numpy dtype applied to adata.X (e.g. 'float32' or 'int64')\n small\n False (default) returns full data, True returns small subset of the data.\n \"\"\"\n import scanpy as sc\n\n if small:\n adata = adata[:1000, :500].copy()\n sc.pp.filter_cells(adata, min_genes=1)\n np.random.seed(42)\n adata.obs[\"batch\"] = np.random.randint(0, 3, size=adata.shape[0])\n sc.pp.filter_genes(adata, min_cells=1)\n adata.X = sparsity_func(adata.X.astype(dtype))\n return adata\n\n\n\"\"\"This file contains some common fixtures for use in tests.\n\nThis is kept seperate from the helpers file because it relies on pytest.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport warnings\nfrom typing import TYPE_CHECKING\n\nimport numpy as np\nimport pytest\n\nfrom .data import (\n backed_adata,\n pbmc3k_parametrized,\n pbmc3k_parametrized_small,\n pbmc3ks_parametrized_session,\n)\n\nif TYPE_CHECKING:\n from collections.abc import Generator\n from pathlib import Path\n\n__all__ = [\n \"float_dtype\",\n \"_doctest_env\",\n \"pbmc3ks_parametrized_session\",\n \"pbmc3k_parametrized\",\n \"pbmc3k_parametrized_small\",\n \"backed_adata\",\n]\n\n\n@pytest.fixture(params=[np.float64, np.float32])\ndef float_dtype(request):\n return request.param\n\n\n@pytest.fixture\ndef _doctest_env(cache: pytest.Cache, tmp_path: Path) -> Generator[None, None, None]:\n from scanpy._compat import chdir\n\n showwarning_orig = warnings.showwarning\n\n def showwarning(message, category, filename, lineno, file=None, line=None): # noqa: PLR0917\n if file is None:\n if line is None:\n import linecache\n\n line = linecache.getline(filename, lineno)\n line = line.strip()\n print(f\"{category.__name__}: {message}\\n {line}\")\n else:\n showwarning_orig(message, category, filename, lineno, file, line)\n\n # make errors visible and the rest ignored\n warnings.filters = [\n (\"default\", *rest) for action, *rest in warnings.filters if action == \"error\"\n ] + [(\"ignore\", None, Warning, None, 0)]\n\n warnings.showwarning = showwarning\n with chdir(tmp_path):\n yield\n warnings.showwarning = showwarning_orig","difficulty":"easy","domain":"Code Repository Understanding","length":"long","question":"In the case you got a collection of 4 single cell sequencing dataset from 4 experiments (10x) from national genome database whereof the development of megakaryocytes in mouse. There are in total 4 days’ experiment (Day0, Day3, Day5, Day7). What are the required steps (functions) to clusters evolvement during the development? (Hint: the technical drawback may lead to that two cells are in one droplet which should have been separated)","sub_domain":"Code repo QA"}
Source: https://huggingface.co/datasets/zai-org/LongBench-v2
initial import
Posting: /agents
GET /api/v1/write?intent=publish&task_id=54cef6f7-f84e-5bb3-8876-6e72cff6c368&body={url_encoded_text}&agent_name={optional_name}&nonce={optional_random_id}