UNAGI walkthrough

A walkthrough of UNAGI pipeline, from loading the example data, to train UNAGI model, perform in-silico perturbation to results visualization.

Install UNAGI from pypi

The installation takes around 5 minutes, however, it depends on your download speed.

!pip install scUNAGI

Requirement already satisfied: scUNAGI in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (0.4.0)
Requirement already satisfied: recommonmark in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.7.1)
Requirement already satisfied: h5py>=3.9.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (3.11.0)
Requirement already satisfied: idna>=3.4 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (3.7)
Requirement already satisfied: igraph>=0.10.7 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.10.8)
Requirement already satisfied: networkx>=3.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (3.2.1)
Requirement already satisfied: numba>=0.57.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.60.0)
Requirement already satisfied: numpy==1.24.4 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (1.24.4)
Requirement already satisfied: pandas==1.5.3 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (1.5.3)
Requirement already satisfied: parso>=0.8.3 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.8.4)
Requirement already satisfied: pillow>=9.4.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (10.4.0)
Requirement already satisfied: pip>=23.2.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (24.0)
Requirement already satisfied: psutil>=5.9.5 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (6.0.0)
Requirement already satisfied: ptyprocess>=0.7.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.7.0)
Requirement already satisfied: pure-eval>=0.2.2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.2.2)
Requirement already satisfied: pycparser>=2.21 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (2.22)
Requirement already satisfied: pygments>=2.16.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (2.18.0)
Requirement already satisfied: pynndescent>=0.5.10 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.5.13)
Requirement already satisfied: pyopenssl>=23.2.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (24.1.0)
Requirement already satisfied: pyparsing>=3.1.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (3.1.2)
Requirement already satisfied: pysocks>=1.7.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (1.7.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (2.9.0.post0)
Requirement already satisfied: python-igraph>=0.10.8 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.10.8)
Requirement already satisfied: pytz>=2023.3.post1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (2024.1)
Requirement already satisfied: pyzmq>=25.1.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (26.0.3)
Requirement already satisfied: lightning==2.0.8 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (2.0.8)
Requirement already satisfied: numpyro==0.13.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.13.0)
Requirement already satisfied: requests>=2.31.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (2.32.3)
Requirement already satisfied: scanpy>=1.9.5 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (1.10.2)
Requirement already satisfied: jax==0.4.20 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.4.20)
Requirement already satisfied: jaxlib==0.4.20 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.4.20)
Requirement already satisfied: scikit-learn>=1.3.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (1.5.1)
Requirement already satisfied: scipy>=1.11.4 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (1.13.1)
Requirement already satisfied: seaborn>=0.12.2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.13.2)
Requirement already satisfied: setuptools>=68.0.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (69.5.1)
Requirement already satisfied: six>=1.16.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (1.16.0)
Requirement already satisfied: stack-data>=0.6.2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.6.3)
Requirement already satisfied: statsmodels>=0.14.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.14.2)
Requirement already satisfied: stdlib-list>=0.8.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.10.0)
Requirement already satisfied: sympy>=1.11.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (1.12.1)
Requirement already satisfied: tbb>=2021.8.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (2021.13.0)
Requirement already satisfied: texttable>=1.6.7 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (1.7.0)
Requirement already satisfied: threadpoolctl>=3.2.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (3.5.0)
Requirement already satisfied: torchaudio>=2.0.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (2.3.1)
Requirement already satisfied: torchvision>=0.15.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.18.1)
Requirement already satisfied: tornado>=6.3.3 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (6.4.1)
Requirement already satisfied: tqdm>=4.66.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (4.66.4)
Requirement already satisfied: traitlets>=5.9.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (5.14.3)
Requirement already satisfied: typing-extensions>=4.7.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (4.12.2)
Requirement already satisfied: tzdata>=2023c in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (2024.1)
Requirement already satisfied: umap-learn>=0.5.3 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.5.6)
Requirement already satisfied: wcwidth>=0.2.6 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.2.13)
Requirement already satisfied: wheel>=0.38.4 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.43.0)
Requirement already satisfied: pyro-ppl>=1.8.6 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (1.9.1)
Requirement already satisfied: scvi-tools>=1.0.3 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (1.1.5)
Requirement already satisfied: leidenalg>=0.10.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scUNAGI) (0.10.2)
Requirement already satisfied: ml-dtypes>=0.2.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from jax==0.4.20->scUNAGI) (0.4.0)
Requirement already satisfied: opt-einsum in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from jax==0.4.20->scUNAGI) (3.3.0)
Requirement already satisfied: importlib-metadata>=4.6 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from jax==0.4.20->scUNAGI) (8.0.0)
Requirement already satisfied: Jinja2<5.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (3.1.4)
Requirement already satisfied: PyYAML<8.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (6.0.1)
Requirement already satisfied: arrow<3.0,>=1.2.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (1.3.0)
Requirement already satisfied: backoff<4.0,>=2.2.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (2.2.1)
Requirement already satisfied: beautifulsoup4<6.0,>=4.8.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (4.12.3)
Requirement already satisfied: click<10.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (8.1.7)
Requirement already satisfied: croniter<1.5.0,>=1.3.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (1.4.1)
Requirement already satisfied: dateutils<2.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (0.6.12)
Requirement already satisfied: deepdiff<8.0,>=5.7.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (7.0.1)
Requirement already satisfied: fastapi<2.0,>=0.92.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (0.111.0)
Requirement already satisfied: fsspec<2025.0,>=2022.5.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (2024.6.1)
Requirement already satisfied: inquirer<5.0,>=2.10.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (3.3.0)
Requirement already satisfied: lightning-cloud>=0.5.37 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (0.5.70)
Requirement already satisfied: lightning-utilities<2.0,>=0.7.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (0.11.3.post0)
Requirement already satisfied: packaging in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (24.1)
Requirement already satisfied: pydantic<2.2.0,>=1.7.4 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (2.1.1)
Requirement already satisfied: python-multipart<2.0,>=0.0.5 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (0.0.9)
Requirement already satisfied: rich<15.0,>=12.3.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (13.7.1)
Requirement already satisfied: starlette in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (0.37.2)
Requirement already satisfied: starsessions<2.0,>=1.2.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (1.3.0)
Requirement already satisfied: torch<4.0,>=1.11.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (2.3.1)
Requirement already satisfied: torchmetrics<2.0,>=0.7.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (1.4.0.post0)
Requirement already satisfied: urllib3<4.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (1.26.19)
Requirement already satisfied: uvicorn<2.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (0.30.1)
Requirement already satisfied: websocket-client<3.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (1.8.0)
Requirement already satisfied: websockets<13.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (12.0)
Requirement already satisfied: pytorch-lightning in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning==2.0.8->scUNAGI) (2.3.2)
Requirement already satisfied: multipledispatch in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from numpyro==0.13.0->scUNAGI) (1.0.0)
Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from numba>=0.57.1->scUNAGI) (0.43.0)
Requirement already satisfied: joblib>=0.11 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from pynndescent>=0.5.10->scUNAGI) (1.4.2)
Requirement already satisfied: cryptography<43,>=41.0.5 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from pyopenssl>=23.2.0->scUNAGI) (42.0.8)
Requirement already satisfied: pyro-api>=0.1.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from pyro-ppl>=1.8.6->scUNAGI) (0.1.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from requests>=2.31.0->scUNAGI) (3.3.2)
Requirement already satisfied: certifi>=2017.4.17 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from requests>=2.31.0->scUNAGI) (2024.7.4)
Requirement already satisfied: anndata>=0.8 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scanpy>=1.9.5->scUNAGI) (0.10.8)
Requirement already satisfied: get-annotations in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scanpy>=1.9.5->scUNAGI) (0.1.2)
Requirement already satisfied: legacy-api-wrap>=1.4 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scanpy>=1.9.5->scUNAGI) (1.4)
Requirement already satisfied: matplotlib>=3.6 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scanpy>=1.9.5->scUNAGI) (3.9.1)
Requirement already satisfied: natsort in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scanpy>=1.9.5->scUNAGI) (8.4.0)
Requirement already satisfied: patsy in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scanpy>=1.9.5->scUNAGI) (0.5.6)
Requirement already satisfied: session-info in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scanpy>=1.9.5->scUNAGI) (1.0.0)
Requirement already satisfied: docrep>=0.3.2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scvi-tools>=1.0.3->scUNAGI) (0.3.2)
Requirement already satisfied: flax in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scvi-tools>=1.0.3->scUNAGI) (0.8.4)
Requirement already satisfied: ml-collections>=0.1.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scvi-tools>=1.0.3->scUNAGI) (0.1.1)
Requirement already satisfied: mudata>=0.1.2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scvi-tools>=1.0.3->scUNAGI) (0.2.4)
Requirement already satisfied: optax in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from scvi-tools>=1.0.3->scUNAGI) (0.2.2)
Requirement already satisfied: executing>=1.2.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from stack-data>=0.6.2->scUNAGI) (2.0.1)
Requirement already satisfied: asttokens>=2.1.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from stack-data>=0.6.2->scUNAGI) (2.4.1)
Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sympy>=1.11.1->scUNAGI) (1.3.0)
Requirement already satisfied: filelock in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (3.15.4)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (12.1.105)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (12.1.105)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (12.1.105)
Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (8.9.2.26)
Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (12.1.3.1)
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (11.0.2.54)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (10.3.2.106)
Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (11.4.5.107)
Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (12.1.0.106)
Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (2.20.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (12.1.105)
Requirement already satisfied: triton==2.3.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (2.3.1)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch<4.0,>=1.11.0->lightning==2.0.8->scUNAGI) (12.5.82)
Requirement already satisfied: commonmark>=0.8.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from recommonmark->scUNAGI) (0.9.1)
Requirement already satisfied: docutils>=0.11 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from recommonmark->scUNAGI) (0.21.2)
Requirement already satisfied: sphinx>=1.3.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from recommonmark->scUNAGI) (7.3.7)
Requirement already satisfied: array-api-compat!=1.5,>1.4 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from anndata>=0.8->scanpy>=1.9.5->scUNAGI) (1.7.1)
Requirement already satisfied: exceptiongroup in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from anndata>=0.8->scanpy>=1.9.5->scUNAGI) (1.2.1)
Requirement already satisfied: types-python-dateutil>=2.8.10 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from arrow<3.0,>=1.2.0->lightning==2.0.8->scUNAGI) (2.9.0.20240316)
Requirement already satisfied: soupsieve>1.2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from beautifulsoup4<6.0,>=4.8.0->lightning==2.0.8->scUNAGI) (2.5)
Requirement already satisfied: cffi>=1.12 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from cryptography<43,>=41.0.5->pyopenssl>=23.2.0->scUNAGI) (1.16.0)
Requirement already satisfied: ordered-set<4.2.0,>=4.1.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from deepdiff<8.0,>=5.7.0->lightning==2.0.8->scUNAGI) (4.1.0)
Requirement already satisfied: fastapi-cli>=0.0.2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (0.0.4)
Requirement already satisfied: httpx>=0.23.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (0.27.0)
Requirement already satisfied: ujson!=4.0.2,!=4.1.0,!=4.2.0,!=4.3.0,!=5.0.0,!=5.1.0,>=4.0.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (5.10.0)
Requirement already satisfied: orjson>=3.2.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (3.10.6)
Requirement already satisfied: email_validator>=2.0.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (2.2.0)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from fsspec[http]<2025.0,>2021.06.0->lightning==2.0.8->scUNAGI) (3.9.5)
Requirement already satisfied: zipp>=0.5 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from importlib-metadata>=4.6->jax==0.4.20->scUNAGI) (3.19.2)
Requirement already satisfied: blessed>=1.19.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from inquirer<5.0,>=2.10.0->lightning==2.0.8->scUNAGI) (1.20.0)
Requirement already satisfied: editor>=1.6.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from inquirer<5.0,>=2.10.0->lightning==2.0.8->scUNAGI) (1.6.6)
Requirement already satisfied: readchar>=3.0.6 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from inquirer<5.0,>=2.10.0->lightning==2.0.8->scUNAGI) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from Jinja2<5.0->lightning==2.0.8->scUNAGI) (2.1.5)
Requirement already satisfied: pyjwt in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning-cloud>=0.5.37->lightning==2.0.8->scUNAGI) (2.8.0)
Requirement already satisfied: boto3 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning-cloud>=0.5.37->lightning==2.0.8->scUNAGI) (1.34.140)
Requirement already satisfied: protobuf in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from lightning-cloud>=0.5.37->lightning==2.0.8->scUNAGI) (5.27.2)
Requirement already satisfied: contourpy>=1.0.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from matplotlib>=3.6->scanpy>=1.9.5->scUNAGI) (1.2.1)
Requirement already satisfied: cycler>=0.10 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from matplotlib>=3.6->scanpy>=1.9.5->scUNAGI) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from matplotlib>=3.6->scanpy>=1.9.5->scUNAGI) (4.53.1)
Requirement already satisfied: kiwisolver>=1.3.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from matplotlib>=3.6->scanpy>=1.9.5->scUNAGI) (1.4.5)
Requirement already satisfied: importlib-resources>=3.2.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from matplotlib>=3.6->scanpy>=1.9.5->scUNAGI) (6.4.0)
Requirement already satisfied: absl-py in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from ml-collections>=0.1.1->scvi-tools>=1.0.3->scUNAGI) (2.1.0)
Requirement already satisfied: contextlib2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from ml-collections>=0.1.1->scvi-tools>=1.0.3->scUNAGI) (21.6.0)
Requirement already satisfied: annotated-types>=0.4.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from pydantic<2.2.0,>=1.7.4->lightning==2.0.8->scUNAGI) (0.7.0)
Requirement already satisfied: pydantic-core==2.4.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from pydantic<2.2.0,>=1.7.4->lightning==2.0.8->scUNAGI) (2.4.0)
Requirement already satisfied: markdown-it-py>=2.2.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from rich<15.0,>=12.3.0->lightning==2.0.8->scUNAGI) (3.0.0)
Requirement already satisfied: sphinxcontrib-applehelp in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sphinx>=1.3.1->recommonmark->scUNAGI) (1.0.8)
Requirement already satisfied: sphinxcontrib-devhelp in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sphinx>=1.3.1->recommonmark->scUNAGI) (1.0.6)
Requirement already satisfied: sphinxcontrib-jsmath in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sphinx>=1.3.1->recommonmark->scUNAGI) (1.0.1)
Requirement already satisfied: sphinxcontrib-htmlhelp>=2.0.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sphinx>=1.3.1->recommonmark->scUNAGI) (2.0.5)
Requirement already satisfied: sphinxcontrib-serializinghtml>=1.1.9 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sphinx>=1.3.1->recommonmark->scUNAGI) (1.1.10)
Requirement already satisfied: sphinxcontrib-qthelp in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sphinx>=1.3.1->recommonmark->scUNAGI) (1.0.7)
Requirement already satisfied: snowballstemmer>=2.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sphinx>=1.3.1->recommonmark->scUNAGI) (2.2.0)
Requirement already satisfied: babel>=2.9 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sphinx>=1.3.1->recommonmark->scUNAGI) (2.15.0)
Requirement already satisfied: alabaster~=0.7.14 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sphinx>=1.3.1->recommonmark->scUNAGI) (0.7.16)
Requirement already satisfied: imagesize>=1.3 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sphinx>=1.3.1->recommonmark->scUNAGI) (1.4.1)
Requirement already satisfied: tomli>=2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from sphinx>=1.3.1->recommonmark->scUNAGI) (2.0.1)
Requirement already satisfied: anyio<5,>=3.4.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from starlette->lightning==2.0.8->scUNAGI) (4.4.0)
Requirement already satisfied: itsdangerous<3.0.0,>=2.0.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from starsessions<2.0,>=1.2.1->lightning==2.0.8->scUNAGI) (2.2.0)
Requirement already satisfied: h11>=0.8 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from uvicorn<2.0->lightning==2.0.8->scUNAGI) (0.14.0)
Requirement already satisfied: msgpack in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from flax->scvi-tools>=1.0.3->scUNAGI) (1.0.8)
Requirement already satisfied: orbax-checkpoint in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from flax->scvi-tools>=1.0.3->scUNAGI) (0.5.16)
Requirement already satisfied: tensorstore in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from flax->scvi-tools>=1.0.3->scUNAGI) (0.1.63)
Requirement already satisfied: chex>=0.1.86 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from optax->scvi-tools>=1.0.3->scUNAGI) (0.1.86)
Requirement already satisfied: aiosignal>=1.1.2 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>2021.06.0->lightning==2.0.8->scUNAGI) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>2021.06.0->lightning==2.0.8->scUNAGI) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>2021.06.0->lightning==2.0.8->scUNAGI) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>2021.06.0->lightning==2.0.8->scUNAGI) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>2021.06.0->lightning==2.0.8->scUNAGI) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>2021.06.0->lightning==2.0.8->scUNAGI) (4.0.3)
Requirement already satisfied: sniffio>=1.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from anyio<5,>=3.4.0->starlette->lightning==2.0.8->scUNAGI) (1.3.1)
Requirement already satisfied: toolz>=0.9.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from chex>=0.1.86->optax->scvi-tools>=1.0.3->scUNAGI) (0.12.1)
Requirement already satisfied: runs in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from editor>=1.6.0->inquirer<5.0,>=2.10.0->lightning==2.0.8->scUNAGI) (1.2.2)
Requirement already satisfied: xmod in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from editor>=1.6.0->inquirer<5.0,>=2.10.0->lightning==2.0.8->scUNAGI) (1.8.1)
Requirement already satisfied: dnspython>=2.0.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from email_validator>=2.0.0->fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (2.6.1)
Requirement already satisfied: typer>=0.12.3 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from fastapi-cli>=0.0.2->fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (0.12.3)
Requirement already satisfied: httpcore==1.* in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from httpx>=0.23.0->fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (1.0.5)
Requirement already satisfied: mdurl~=0.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from markdown-it-py>=2.2.0->rich<15.0,>=12.3.0->lightning==2.0.8->scUNAGI) (0.1.2)
Requirement already satisfied: httptools>=0.5.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from uvicorn[standard]>=0.12.0->fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (0.6.1)
Requirement already satisfied: python-dotenv>=0.13 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from uvicorn[standard]>=0.12.0->fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (1.0.1)
Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from uvicorn[standard]>=0.12.0->fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (0.19.0)
Requirement already satisfied: watchfiles>=0.13 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from uvicorn[standard]>=0.12.0->fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (0.22.0)
Requirement already satisfied: botocore<1.35.0,>=1.34.140 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from boto3->lightning-cloud>=0.5.37->lightning==2.0.8->scUNAGI) (1.34.140)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from boto3->lightning-cloud>=0.5.37->lightning==2.0.8->scUNAGI) (1.0.1)
Requirement already satisfied: s3transfer<0.11.0,>=0.10.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from boto3->lightning-cloud>=0.5.37->lightning==2.0.8->scUNAGI) (0.10.2)
Requirement already satisfied: etils[epath,epy] in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from orbax-checkpoint->flax->scvi-tools>=1.0.3->scUNAGI) (1.5.2)
Requirement already satisfied: nest_asyncio in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from orbax-checkpoint->flax->scvi-tools>=1.0.3->scUNAGI) (1.6.0)
Requirement already satisfied: shellingham>=1.3.0 in /mnt/md2/test_yumin/miniconda/envs/test_unagi/lib/python3.9/site-packages (from typer>=0.12.3->fastapi-cli>=0.0.2->fastapi<2.0,>=0.92.0->lightning==2.0.8->scUNAGI) (1.5.4)

Use the example dataset

!mkdir data
!cp -r ../../../UNAGI/data/example data/
!git clone https://github.com/phoenixding/idrem.git

Cloning into 'idrem'...
remote: Enumerating objects: 473, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 473 (delta 0), reused 1 (delta 0), pack-reused 469
Receiving objects: 100% (473/473), 121.70 MiB | 6.54 MiB/s, done.
Resolving deltas: 100% (139/139), done.

Load UNAGI

import warnings
warnings.filterwarnings('ignore')
from UNAGI import UNAGI
unagi = UNAGI()

load the dataset

The example dataset contains 4 stages, and the attribute to indicate the stage id is ‘stage’, the cell-type annotation attribute is ‘name.simple’

unagi.setup_data('./data/example',total_stage=4,stage_key='stage')

Cell graphs found, skipping cell graph construction!

Configure the model architecture of UNAGI and training hyper-parameters

unagi.setup_training('example',dist='ziln',device='cuda:0',GPU=True,epoch_iter=3,epoch_initial=2,max_iter=3,BATCHSIZE=512)
unagi.run_UNAGI(idrem_dir = './idrem')

Species: Running on Human data
...
0
vae_loss 2209.888913574219
dis_loss 0.3502234868705273
adversarial_loss 2.601238065958023
[epoch 000]  average training loss: 2209.8889
CPO parameters are not set up, using default parameters
anchor_neighbors: 15, max_neighbors: 35, min_neighbors: 10, resolution_min: 0.8, resolution_max: 1.5
top gene
done
top gene
done
top gene
done
top gene
done
IDREM parameters are not set up, using default parameters
Minimum_Absolute_Log_Ratio_Expression: 0.5, Convergence_Likelihood: 0.001, Minimum_Standard_Deviation: 0.5
b''
[[[2], [0], [1, 4], [5, 7]], [[3], [2, 5], [2, 5, 6], [1, 2, 4, 6]], [[1], [3], [0, 3], [0, 3]], [[4], [4], [7], [8]], [[11], [9], [9, 10, 11], [10, 11]], [[8], [10], [8], [9]]]
['2', '0', '1n4', '5n7']
['3', '2n5', '2n5n6', '1n2n4n6']
['1', '3', '0n3', '0n3']
['4', '4', '7', '8']
['11', '9', '9n10n11', '10n11']
['8', '10', '8', '9']
['2-0-1n4-5n7.txt', '8-10-8-9.txt', '4-4-7-8.txt', '3-2n5-2n5n6-1n2n4n6.txt', '11-9-9n10n11-10n11.txt', '1-3-0n3-0n3.txt']

b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\nwriting Json..\nTime: 20380ms\n'
b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\nwriting Json..\nTime: 29339ms\n'

b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\nwriting Json..\nTime: 29083ms\n'
b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n17\nwriting Json..\nTime: 30040ms\n'

b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\nwriting Json..\nTime: 28805ms\n'
b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\nwriting Json..\nTime: 31519ms\n'
/mnt/md2/test_yumin/UNAGI/docs/notebooks/examples
b''
b''

b''
idrem Done
getting TFs from  1-3-0n3-0n3.txt_viz
getting TFs from  2-0-1n4-5n7.txt_viz
getting TFs from  8-10-8-9.txt_viz
getting TFs from  4-4-7-8.txt_viz
getting TFs from  11-9-9n10n11-10n11.txt_viz
getting TFs from  3-2n5-2n5n6-1n2n4n6.txt_viz
getting Target genes from  1-3-0n3-0n3.txt_viz
getting Target genes from  2-0-1n4-5n7.txt_viz
getting Target genes from  8-10-8-9.txt_viz
getting Target genes from  4-4-7-8.txt_viz
getting Target genes from  11-9-9n10n11-10n11.txt_viz
getting Target genes from  3-2n5-2n5n6-1n2n4n6.txt_viz
Species: Running on Human data
...
load last iteration model.....
0
vae_loss 1893.5693901705108
dis_loss 0.1039636106044054
adversarial_loss 3.603592357635498
[epoch 000]  average training loss: 1893.5694
CPO parameters are not set up, using default parameters
anchor_neighbors: 15, max_neighbors: 35, min_neighbors: 10, resolution_min: 0.8, resolution_max: 1.5
top gene
done
top gene
done
top gene
done
top gene
done
IDREM parameters are not set up, using default parameters
Minimum_Absolute_Log_Ratio_Expression: 0.5, Convergence_Likelihood: 0.001, Minimum_Standard_Deviation: 0.5
b''
[[[10], [0, 4], [2, 4, 6], [0, 1, 4, 10, 12]], [[5], [1, 5], [0, 5], [3, 9]], [[2], [2, 13], [1, 9], [2]], [[7], [6], [7], [5, 7]], [[8], [7], [3, 11], [6, 14]], [[9], [10], [8], [8]], [[12], [11], [10, 12], [11, 15]], [[13], [14], [14], [13]]]
['10', '0n4', '2n4n6', '0n1n4n10n12']
['5', '1n5', '0n5', '3n9']
['2', '2n13', '1n9', '2']
['7', '6', '7', '5n7']
['8', '7', '3n11', '6n14']
['9', '10', '8', '8']
['12', '11', '10n12', '11n15']
['13', '14', '14', '13']
['9-10-8-8.txt', '2-2n13-1n9-2.txt', '13-14-14-13.txt', '5-1n5-0n5-3n9.txt', '12-11-10n12-11n15.txt', '10-0n4-2n4n6-0n1n4n10n12.txt', '8-7-3n11-6n14.txt', '7-6-7-5n7.txt']

b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\nwriting Json..\nTime: 26233ms\n'
b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\nwriting Json..\nTime: 29479ms\n'

b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\nwriting Json..\nTime: 25504ms\n'
b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\nwriting Json..\nTime: 30492ms\n'

b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\nwriting Json..\nTime: 28373ms\n'
b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\nwriting Json..\nTime: 28926ms\n'

b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\nwriting Json..\nTime: 27441ms\n'
b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\nwriting Json..\nTime: 28206ms\n'
/mnt/md2/test_yumin/UNAGI/docs/notebooks/examples
b''

b''
b''
idrem Done
getting TFs from  2-2n13-1n9-2.txt_viz
getting TFs from  5-1n5-0n5-3n9.txt_viz
getting TFs from  12-11-10n12-11n15.txt_viz
getting TFs from  9-10-8-8.txt_viz
getting TFs from  7-6-7-5n7.txt_viz
getting TFs from  13-14-14-13.txt_viz
getting TFs from  10-0n4-2n4n6-0n1n4n10n12.txt_viz
getting TFs from  8-7-3n11-6n14.txt_viz
getting Target genes from  2-2n13-1n9-2.txt_viz
getting Target genes from  5-1n5-0n5-3n9.txt_viz
getting Target genes from  12-11-10n12-11n15.txt_viz
getting Target genes from  9-10-8-8.txt_viz
getting Target genes from  7-6-7-5n7.txt_viz
getting Target genes from  13-14-14-13.txt_viz
getting Target genes from  10-0n4-2n4n6-0n1n4n10n12.txt_viz
getting Target genes from  8-7-3n11-6n14.txt_viz

import warnings
warnings.filterwarnings('ignore')
from UNAGI import UNAGI
unagi = UNAGI()
unagi.analyse_UNAGI('./data/example/1/stagedata/dataset.h5ad',iteration=1,progressionmarker_background_sampling_times=1,run_pertubration=False)

calculate hierarchical markers.....
hierarchical static markers done
calculateDataPathwayOverlapGene done
calculateTopPathwayGeneRanking done
getting Target genes from  2-2n13-1n9-2.txt_viz
getting Target genes from  5-1n5-0n5-3n9.txt_viz
getting Target genes from  12-11-10n12-11n15.txt_viz
getting Target genes from  9-10-8-8.txt_viz
getting Target genes from  7-6-7-5n7.txt_viz
getting Target genes from  13-14-14-13.txt_viz
getting Target genes from  10-0n4-2n4n6-0n1n4n10n12.txt_viz
getting Target genes from  8-7-3n11-6n14.txt_viz
Dynamic markers discovery.....done....
The analysis has been done, please check the outputs!

Load the post-analysis data

import pickle
import scanpy as sc
from UNAGI import plotting
adata = sc.read_h5ad('./example_1/dataset.h5ad')
adata.uns = pickle.load(open('./example_1/attribute.pkl', 'rb'))

Plot the composition of cells in individual stages

plotting.cell_type_composition(adata,'ident','stage',dpi=80)

Plot the UMAPS of cell embeddings for individual stages

plotting.plot_stages_latent_representation(adata,'ident',stage_key='stage',dpi=80)

6750
ARI:  0.593386226864957
NMIs:  0.599135261528311
silhouette score:  0.12659452391931225
3152
ARI:  0.6095139890374347
NMIs:  0.6661252476745311
silhouette score:  0.1784075426358319
4195
ARI:  0.6988464913878341
NMIs:  0.7117183636069638
silhouette score:  0.15824279407608866
13550
ARI:  0.6717429532801696
NMIs:  0.7298776099660566
silhouette score:  0.15755916147747387

ARIs:  [0.6433724151425988]
NMI:  [0.6767141206939656]
silhouette score:  0.15520100552717667

Plot dotplots of cell type markers

plotted_genes = ['IGF1', 'NLGN1', 'ROBO2', 'SLIT2', 'EGFEM1P', 'LINC02388', 'DACH2', 'FGF14', 'ITGBL1', 'MMRN1', 'CCL21', 'ADARB2', 'LRRTM4', 'ZNF385D', 'PRUNE2', 'LDB2', 'AQP1', 'FLT1', 'SLCO2A1', 'EPAS1', 'PCDH17']
for i in adata.obs['stage'].unique():
    stageadata = adata[adata.obs['stage']==i]
    sc.pl.dotplot(stageadata, groupby='ident', var_names=plotted_genes,vmax=2,swap_axes=True)

Plot increasing gene markers and decreasing gene markers from a specific track

Note:

• Please check the ./example_1/idrem to find the identified tracks’ names.

• Replace the target_track = '10-0n4-2n4n6-0n1n4n10n12' with your identified track.

• Some tracks might not have increasing or decreasing genes.

import matplotlib.pyplot as plt
import pickle
import pandas as pd
import os
import json
import numpy as np
import scanpy as sc
from cycler import cycler
def readIdremJson(path, filename):
    # print('getting Target genes from ', filename)

    path = os.path.join(path,filename,'DREM.json')
    f=open(path,"r")
    lf=f.readlines()
    f.close()
    lf="".join(lf)
    lf=lf[5:-2]+']'
    tt=json.loads(lf,strict=False)
    return tt
def readIdremJson(path, filename):
    # print('getting Target genes from ', filename)

    path = os.path.join(path,filename,'DREM.json')
    f=open(path,"r")
    lf=f.readlines()
    f.close()
    lf="".join(lf)
    lf=lf[5:-2]+']'
    tt=json.loads(lf,strict=False)
    return tt
def getvalueofMarkers(idrem,filename, gene):
    tt = readIdremJson(idrem,filename)
    temp = np.array(tt[8])
    idrem_genes = np.array(temp[1:,0].tolist())
    tendency = temp[1:,4].astype(float)* temp[1:,3].astype(float) * temp[1:,2].astype(float) * temp[1:,1].astype(float)
    tendency[tendency <0] = 0
    index = [i for i, x in enumerate(tendency) if x <= 0]
    genenames = temp[1:,0].tolist()
    gene_idx = genenames.index(gene)
    change = temp[1:,4].astype(float) - temp[1:,1].astype(float)
    stage0 = temp[1:,1].astype(float)
    stage1 = temp[1:,2].astype(float)-stage0
    stage2 = temp[1:,3].astype(float)-stage0
    stage3 = temp[1:,4].astype(float)-stage0
    # stage0 = 
    #return [stage1[gene_idx],stage2[gene_idx],stage3[gene_idx]]
    return [0,stage1[gene_idx],stage2[gene_idx],stage3[gene_idx] ]#[change[gene_idx]]
def getvaluesFromIDREM(path,genes,tracks,target_track):
    out = {}
    for gene in genes:
        out[gene] = []
        filenames = tracks
        for each in filenames:
            if each == target_track:
                name = each+'.txt_viz'
                if each[0] != '.':
                    each = each.split('.')[0]#.split('-')[-1].split('n')
                    out[gene]+=getvalueofMarkers(path,name,gene.split('\\')[0])
    return out



######
pm = adata.uns['progressionMarkers']
target_track = '10-0n4-2n4n6-0n1n4n10n12' # use a track from ./example_1/idrem folder, use, x-xnx-x-x format
increasing = []
decreasing = []
genes_increasing = []
genes_decreasing = []
plt.rcParams['axes.prop_cycle'] = plt.cycler("color", plt.cm.tab20.colors)
for each in pm.keys():
    if each == target_track:
        
        tt = readIdremJson('./example_1/idrem',each+'.txt_viz')
        tt = readIdremJson('./example_1/idrem',each+'.txt_viz')
        scope = []
        for i, gene in enumerate(tt[0][6]['genesInNode']):
            if gene:
                scope.append(tt[3][i])
        track = pm[each]
        df = pd.DataFrame.from_dict(track['increasing'])
        print(scope)
        temp = list(df['gene'].values)
        go = []
        for gene in scope:
            if gene in list(df['gene'].values):
                go.append(gene)
        # go = np.array(go)
        df = df.loc[df['gene'].isin(go)]
      
        df.sort_values(by=['rank'], inplace=True)
        increasing.append(df.values[:10])
        genes_increasing+=list(df['gene'].values[:10]+'\\'+str(each))
        df = pd.DataFrame.from_dict(track['decreasing'])
        scope = []
        for i, gene in enumerate(tt[0][8]['genesInNode']):
            if gene:
                scope.append(tt[3][i])
        
        go = []
        for gene in scope:
            if gene in list(df['gene'].values):
                go.append(gene)
        # go = np.array(go)
        df = df.loc[df['gene'].isin(go)]
        df.sort_values(by=['rank'], inplace=True)
        decreasing.append(df.values[:10])
        genes_decreasing+=list(df['gene'].values[:10]+'\\'+str(each))

out_increasing = getvaluesFromIDREM('./example_1/idrem',genes_increasing,list(pm.keys()),target_track=target_track)
out_decreasing = getvaluesFromIDREM('./example_1/idrem',genes_decreasing,list(pm.keys()),target_track=target_track)

import matplotlib.pyplot as plt
fig, ax = plt.subplots(1,1,figsize=(4,4),dpi=80)

gene_names = list(out_increasing.keys())
increasing_gene_names = [x.split('\\')[0] for x in gene_names]
ax.tick_params(top=True, bottom=False,
                   labeltop=True, labelbottom=False)

    # Rotate the tick labels and set their alignment.
plt.setp(ax.get_yticklabels(), rotation=-30, ha="right",rotation_mode="anchor")
plt.setp(ax.get_xticklabels(), rotation=-45, ha="right",rotation_mode="anchor")

im0 = ax.plot(np.array([0,1,2,3]),np.array(list(out_increasing.values())+list(out_decreasing.values())).T)
gene_names = list(out_decreasing.keys())
gene_names = [x.split('\\')[0] for x in gene_names]
ax.legend(increasing_gene_names+gene_names,loc=2,fontsize='xx-small',bbox_to_anchor=(1.08, 1))
ax.tick_params(top=False, bottom=True,
                   labeltop=False, labelbottom=True)
ax.set_xticks(np.array([0,1,2,3])+0.25,('Control','Stage 1', 'Stage 2','Stage 3'))
ax.set_ylim(-1.5,1)

ax.set_title('Top Dynamic Genes in the FibAlv-4')
ax.set_ylabel('Expression Change (Log2FC)')
ax.set_xlabel('Disease Progression')

# plt.savefig('genes_in_fib4.pdf',bbox_inches='tight', pad_inches=0)
plt.show()

['SCGB1A1', 'CPA6', 'TMEM132D', 'AC112770.1', 'DLGAP1', 'CDH2', 'PLD5', 'FRMD5', 'LINC02388', 'SYT14', 'MIR924HG', 'ROBO2', 'STAP1', 'SLC30A8', 'TRIM5', 'GPRC5D-AS1', 'MUC5AC', 'AC083837.1', 'BPIFA1', 'CACNA1A', 'DGKI', 'AC109466.1', 'FGF13', 'HMGA2', 'MYH11', 'ENPP1', 'AC012078.2', 'AC009570.2', 'EFNA5', 'NLGN1', 'LINC02147', 'MIR646HG', 'IGLC7', 'AREG', 'ZFPM2-AS1', 'IGF2BP3', 'AC073114.1', 'AC113137.1', 'COL11A1', 'AC011287.1', 'BPIFB2', 'EPHB2', 'GRM5', 'PLA2G5', 'KCNQ3', 'GRIP1', 'KRT17', 'ELAPOR2', 'HS3ST2', 'STK32A', 'AC044893.1', 'BMPR1B', 'AC013652.1', 'SAA1', 'AC078923.1', 'DCC', 'LINC01088', 'TNIK', 'MIR31HG', 'SERPINB11', 'GREM1', 'NREP', 'ATP8A2', 'ADAMTS6', 'C8ORF37-AS1', 'RIMS2', 'NKAIN3', 'LINC02257', 'FABP6', 'MDM2', 'MSMB', 'LINC00540', 'AC092691.1', 'THSD4', 'TFF3', 'PSD3', 'RGS7', 'MAP2', 'LINC00511', 'MIR325HG', 'AGR2', 'ENPP2', 'WDFY4', 'ZNF736P9Y', 'SYNPO2', 'LEKR1', 'IGFBP5', 'FHL5', 'PTPN13', 'KCNB2', 'TNIP3', 'DACH2', 'MIR503HG', 'CELF2', 'DEPP1', 'PLAT', 'CLNK', 'SLC25A48', 'ASTN2', 'AC093791.2', 'LINC02315', 'BGN', 'DPP10', 'MANCR', 'ADGRB3', 'ROR1-AS1', 'INPP4B', 'ARHGAP42', 'ABI3BP', 'COL12A1', 'VWA3A', 'LRRTM3', 'LINC01768', 'CSMD1', 'ARHGEF26', 'PURPL', 'GCLC', 'SCN7A', 'CHI3L1', 'CAPN13', 'CCL2', 'MYO1D', 'AL008633.1', 'SFTA1P', 'MIR222HG', 'LINC02511', 'WFDC2', 'VCAM1', 'LMCD1', 'FGF1', 'ALOX12P2', 'PTCH2', 'BMP1', 'LCN2', 'NETO1', 'ZMAT4', 'CREB5', 'AL589693.1', 'MUC4', 'LINC00640', 'WAKMAR2', 'AC092131.1', 'LINC02015', 'BRINP3', 'FHOD3', 'PLEKHS1', 'CTXND1', 'IGFL4', 'SULT1E1', 'EDIL3', 'LINC01099', 'AK8', 'CXCL14', 'ANO4', 'COL7A1', 'AP001831.1', 'LHFPL3', 'LDLRAD1', 'EYA1', 'ISLR', 'AUXG01000058.1', 'PROM1', 'CCDC3', 'AC008591.1', 'LNCAROD', 'TNS4', 'TNFRSF11B', 'ATP10A', 'FRMPD4', 'PIGR', 'S100A2', 'NPSR1-AS1', 'PGBD5', 'AC008050.1', 'KSR1', 'POU2AF1', 'SERPINB13', 'BMPER', 'PTPRD', 'VSNL1', 'TMPRSS4', 'KCNK1', 'CCDC162P', 'EYA2', 'NPFFR2', 'ACSS3', 'LINC00607', 'SVEP1', 'LINC00603', 'RUFY3', 'AL162726.3', 'CLDN1', 'PTGIS', 'KLB', 'CNN1', 'CNTN6', 'DES', 'AC007563.2', 'TAGLN', 'LYPD6B', 'ARHGAP6', 'PMEPA1', 'SLC24A3', 'CYP2F1', 'HULC', 'NPNT', 'AC020637.1', 'NOS1AP', 'MMP1', 'FAM189A2', 'HOMER1', 'IL1RAPL1', 'AC243829.2', 'MX2', 'LONRF2', 'SLC35F1', 'MCTP2', 'AL359546.1', 'RAB31', 'BBOX1-AS1', 'SHISA9', 'ADAM23', 'DISC1FP1', 'GRIK4', 'ADAMTS5', 'CTNNA3', 'LINC01239', 'SYNDIG1', 'AF165147.1', 'AC026704.1', 'AL355304.1', 'GPR132', 'SCHLAP1', 'LINC02694', 'AL353660.1', 'FRMD4B', 'MOK', 'MIR4300HG', 'LINC02798', 'AC006041.2', 'AC139720.1', 'AC114689.3', 'MFAP5', 'CCDC146', 'ABHD2', 'NCALD', 'LGR6', 'AL033504.1', 'GRIN2A', 'AC004160.1', 'LHFPL6', 'CNTN1', 'LINC01605', 'TRHDE', 'SPOCK1', 'VNN1', 'DMD', 'AMPH', 'COL10A1', 'AC109927.1', 'CCDC148', 'DGKB', 'IGSF21', 'LINC02408', 'KANK4', 'MIR205HG', 'AL035401.1', 'AC007529.2', 'BIRC3', 'KC6', 'ZG16B', 'LGR4', 'CPE', 'AC106793.1', 'DSG3', 'SPARC', 'AL390334.1', 'OPCML', 'SV2B', 'AC011029.1', 'SMC5-AS1', 'RGS7BP', 'SGO1-AS1', 'SLCO5A1', 'ADCY2', 'AL139383.1', 'C1QTNF7-AS1', 'SLC10A6', 'P2RY14', 'SKAP1', 'AL157911.1', 'AC007785.1', 'GRIA1', 'AC077690.1', 'AC073332.1', 'AP005212.4', 'LINC01655', 'CCDC200', 'LRRN1', 'GFRA1', 'AC005537.1', 'SRGAP3', 'MMP7', 'STK32B', 'RAB7B', 'PCSK5', 'CDH3', 'CXCL1', 'AC018742.1', 'ADAMTSL3', 'SNCAIP', 'CXCL12', 'FMO1', 'ADCY5', 'STS', 'ALDH1A2', 'CFAP20DC', 'MIR4713HG', 'TSPAN2', 'EDIL3-DT', 'OXR1', 'PRUNE2', 'RASGRF2', 'CENPP', 'AC113414.1', 'GYG2P1', 'LINC01950', 'AL512380.2', 'LINC02384', 'KIAA1549L', 'EFHC2', 'DZIP1L', 'LEF1', 'LIX1-AS1', 'RNF213-AS1', 'LINC01374', 'LINC01725', 'AKR1C1', 'ANKRD36BP2', 'LINC01031', 'LBH', 'IL2RA', 'BTBD11', 'SAMD5', 'TLN2', 'MEGF6', 'CROCC', 'PRICKLE1', 'ENOX1', 'SOD2', 'PKIA-AS1', 'AL096854.1', 'SEMA3A', 'CILP', 'AP000676.5', 'PLAU', 'BLK', 'AL139220.2', 'CCNJL', 'LINC01340', 'SLC25A21', 'ENPEP', 'AC108519.1', 'CRLF1', 'DMGDH', 'MAILR', 'TTC25', 'ANKS1B', 'MARCHF3', 'LINC01117', 'EDA', 'ASTN1', 'GPR156', 'IRF4', 'EXT1', 'ARRDC3-AS1', 'SMAD3', 'FAXC', 'MAMDC2-AS1', 'TENT5C', 'LDLRAD3', 'LINC02109', 'CNTNAP2', 'AC079382.2', 'PLA2G2A', 'AC005999.2', 'NT5DC3', 'AC105411.1', 'SPP1', 'IGLL5', 'IGFL2-AS1', 'DPYD-AS1', 'MMP11', 'ARNTL2', 'CDH12', 'EGFR', 'AC104574.2', 'AL160272.2', 'WASHC1', 'LINC01151', 'LINC01812', 'OLFML3', 'ANKRD36C', 'AC012494.1', 'IGFBP2', 'MFGE8', 'PFKP', 'AC004817.5', 'AC104248.1', 'LOXL4']

Plot dendrogram of hierarchical clusterings

plotting.plot_hc_dendrogram(adata,'stage','ident',dpi=80)

Plot hierarchical static markers heatmap

plotting.hierarchical_static_markers_heatmap(adata,stage=0,cluster=3,level=1,n_genes=20,stage_key='stage',celltype_key='ident')

WARNING: `var_names` is a dictionary. This will reset the current value of `var_group_labels` and `var_group_positions`.

Perturbation using customized pathway database

!cp ../../../UNAGI/data/gesa_pathways.npy ./data/gesa_pathways.npy
import numpy as np
built_in_pathway_data = np.load('./data/gesa_pathways.npy',allow_pickle=True).item()

#The keys are the pathway names
print(list(built_in_pathway_data.keys())[:10]) # show first 10 pathways
#The values are the gene sets
print(built_in_pathway_data['BIOCARTA_GRANULOCYTES_PATHWAY'])
customized_pathway_database = {}
customized_pathway_database['Pathway_A'] = ['COL6A3', 'MET', 'COL7A1', 'MMP1', 'COL11A1', 'COL1A2', 'COL5A2', 'COL4A3', 'COL12A1', 'COL10A1', 'COL5A1', 'COL3A1', 'COL4A4', 'COL14A1', 'COL8A1', 'MMP9', 'COL4A1', 'MMP7', 'COL15A1', 'COL1A1', 'COL17A1', 'COL4A6']
customized_pathway_database['Pathway_B'] = ['MAP2','THBS1',]
np.save('./example_1/customized_pathway_database.npy',customized_pathway_database)
from UNAGI import UNAGI
import warnings
warnings.filterwarnings("ignore")
unagi = UNAGI()
data_path = './example_1/dataset.h5ad'
iteration = 1 #which iteration of the model to use
change_level = 0.5 #reduce the expression to 50% of the original value
customized_pathway = './example_1/customized_pathway_database.npy'
results = unagi.customize_pathway_perturbation(data_path,iteration,customized_pathway,change_level,target_dir='./example_1',device='cuda:0')

['BIOCARTA_GRANULOCYTES_PATHWAY', 'BIOCARTA_LYM_PATHWAY', 'BIOCARTA_BLYMPHOCYTE_PATHWAY', 'BIOCARTA_CARM_ER_PATHWAY', 'BIOCARTA_LAIR_PATHWAY', 'BIOCARTA_VDR_PATHWAY', 'BIOCARTA_MTA3_PATHWAY', 'BIOCARTA_GABA_PATHWAY', 'BIOCARTA_EGFR_SMRTE_PATHWAY', 'BIOCARTA_MONOCYTE_PATHWAY']
['CXCL8', 'IFNG', 'IL1A', 'CSF3', 'SELP', 'ITGAM', 'ITGAL', 'TNF', 'ITGB2', 'PECAM1', 'ICAM2', 'C5', 'SELPLG', 'ICAM1', 'SELL']
calculateDataPathwayOverlapGene done
Start perturbation....
track: 2
finsihed
time: 2.9918649196624756
track: 3
finsihed
time: 3.068023443222046
track: 9
finsihed
time: 2.962941884994507
track: 11
finsihed
time: 2.8387045860290527
track: 15
finsihed
time: 2.922428846359253
track: 8
finsihed
time: 3.162583589553833
track: 5
finsihed
time: 2.9377224445343018
track: 7
finsihed
time: 2.924626111984253
track: 13
finsihed
time: 2.9527530670166016
track: 0
finsihed
time: 3.053570508956909
track: 1
finsihed
time: 2.8400957584381104
track: 4
finsihed
time: 2.870522975921631
track: 10
finsihed
time: 2.876889705657959
track: 12
finsihed
time: 2.902686595916748
track: 6
finsihed
time: 2.980454444885254
track: 14
finsihed
time: 2.9713222980499268
random background done
Finish results analysis

from UNAGI.perturbations import get_top_pathways
get_top_pathways(results, change_level, top_n=10)

	pathways	perturbation score	pval_adjusted	regulated genes	idrem_suggestion
0	Pathway_A	0.997498	0.000007	[COL6A3, MET, COL7A1, MMP1, COL11A1, COL1A2, C...	[COL6A3:-, MET:-, COL7A1:-, MMP1:-, COL11A1:-,...

Customized drug perturbation

import numpy as np
customized_drug_database = {}
customized_drug_database['Drug_A'] = ['TACR1:-', 'MAPK4:+', 'DUSP10:-', 'EGFR:+']
customized_drug_database['Drug_B'] = ['C3:-', 'KDR:+', 'INSR:-','PLA2G2A:-']
customized_drug_database['Drug_C'] = ['COL1A1:-', 'ROBO2:+']
np.save('./example_1/customized_drug_database.npy',customized_drug_database)

from UNAGI import UNAGI
import warnings
warnings.filterwarnings("ignore")
unagi = UNAGI()
data_path = './example_1/dataset.h5ad'
iteration = 1 #which iteration of the model to use
change_level = 0.5 #reduce the expression to 50% of the original value
customized_drug = './example_1/customized_drug_database.npy'
results = unagi.customize_drug_perturbation(data_path,iteration,customized_drug,change_level,target_dir='./example_1/',device='cuda:0')

Start perturbation....
2
3
9
11
15
8
5
7
13
0
1
4
10
12
6
14
drug perturabtion done
random background done
Finish results analysis

from UNAGI.perturbations import get_top_compounds
get_top_compounds(results, change_level, top_n=10)

	compound	perturbation score	pval_adjusted	drug_regulation	idrem_suggestion
0	Drug_A	0.881716	2.189923e-09	[TACR1:-, MAPK4:+, DUSP10:-, EGFR:+]	[TACR1:-, MAPK4:+, DUSP10:-, EGFR:-]