[BUG]Leiden

[shanggutianzhen]

Describe the bug
Hello, I encountered this problem when performing the Leiden clustering.

Steps/Code to reproduce bug

IndexError Traceback (most recent call last)
Cell In[11], line 1
----> 1 rsc.tl.leiden(adata, resolution=0.8)
229 @functools.wraps(fn)
File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/rapids_singlecell/tools/_clustering.py:137, in leiden(adata, resolution, random_state, restrict_to, key_added, adjacency, n_iterations, use_weights, neighbors_key, obsp, copy)
128 leiden_parts, _ = culeiden(
129 g,
130 resolution=resolution,
131 random_state=random_state,
132 max_iter=n_iterations,
133 )
135 # Format output
136 groups = (
--> 137 leiden_parts.to_pandas().sort_values("vertex")[["partition"]].to_numpy().ravel()
138 )
139 if restrict_to is not None:
140 if key_added == "leiden":

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/cudf/utils/performance_tracking.py:51, in _performance_tracking..wrapper(*args, **kwargs)
43 if nvtx.enabled():
44 stack.enter_context(
45 nvtx.annotate(
46 message=func.qualname,
(...)
49 )
50 )
---> 51 return func(*args, **kwargs)

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/cudf/core/dataframe.py:5482, in DataFrame.to_pandas(self, nullable, arrow_type)
5406 """
5407 Convert to a Pandas DataFrame.
5408
(...)
5478 dtype: object
5479 """
5480 out_index = self.index.to_pandas()
5481 out_data = {
-> 5482 i: col.to_pandas(nullable=nullable, arrow_type=arrow_type)
5483 for i, col in enumerate(self._columns)
5484 }
5486 out_df = pd.DataFrame(out_data, index=out_index)
5487 out_df.columns = self._data.to_pandas_index()

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/cudf/core/column/numerical.py:711, in NumericalColumn.to_pandas(self, nullable, arrow_type)
709 return pd.Index(pandas_array, copy=False)
710 elif self.dtype.kind in set("iuf") and not self.has_nulls():
--> 711 return pd.Index(self.values_host, copy=False)
712 else:
713 return super().to_pandas(nullable=nullable, arrow_type=arrow_type)

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/cudf/core/column/column.py:220, in ColumnBase.values_host(self)
217 raise ValueError("Column must have no nulls.")
219 with acquire_spill_lock():
--> 220 return self.data_array_view(mode="read").copy_to_host()

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/cudf/core/column/column.py:135, in ColumnBase.data_array_view(self, mode)
133 else:
134 obj = None
--> 135 return cuda.as_cuda_array(obj).view(self.dtype)

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/api.py:76, in as_cuda_array(obj, sync)
74 raise TypeError("obj doesn't implement the cuda array interface.")
75 else:
---> 76 return from_cuda_array_interface(obj.cuda_array_interface,
77 owner=obj, sync=sync)

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/devices.py:231, in require_context.._require_cuda_co@Fast ntext(*args, **kws)
230 def _require_cuda_context(*args, **kws):
--> 231 with _runtime.ensure_context():
232 return fn(*args, **kws)

File ~/miniconda3/envs/RSC/lib/python3.12/contextlib.py:137, in _GeneratorContextManager.enter(self)
135 del self.args, self.kwds, self.func
136 try:
--> 137 return next(self.gen)
138 except StopIteration:
139 raise RuntimeError("generator didn't yield") from None

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/devices.py:123, in _Runtime.ensure_context(self)
121 with driver.get_active_context():
122 oldctx = self._get_attached_context()
--> 123 newctx = self.get_or_create_context(None)
124 self._set_attached_context(newctx)
125 try:

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/devices.py:138, in _Runtime.get_or_create_context(self, devnum)
136 attached_ctx = self._get_attached_context()
137 if attached_ctx is None:
--> 138 return self._get_or_create_context_uncached(devnum)
139 else:
140 return attached_ctx

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/devices.py:158, in _Runtime._get_or_create_context_uncached(self, devnum)
155 return self._activate_context_for(0)
156 else:
157 # Get primary context for the active device
--> 158 ctx = self.gpus[ac.devnum].get_primary_context()
159 # Is active context the primary context?
160 if USE_NV_BINDING:

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/devices.py:40, in _DeviceList.getitem(self, devnum)
36 def getitem(self, devnum):
37 '''
38 Returns the context manager for device devnum.
39 '''
---> 40 return self.lst[devnum]

IndexError: list index out of range

[Intron7]

@shanggutianzhen Thanks for filing this bug report and for the traceback. To help us reproduce and fix it, could you please fill out the fields in the issue template (environment, exact steps to reproduce, expected vs actual behavior, etc.)?

The template is there so we don’t miss important details, and it really speeds up debugging.

[shanggutianzhen]

environment
Python=3.12.9
📦 当前进程已加载的包及其版本：

IPython : 8.32.0
affine : 2.4.0
anndata : 0.11.3
asciitree : 0.3.3
asttokens : 3.0.0
bcrypt : 4.3.0
brotli : 1.1.0
cachetools : 5.5.1
cellcharter : 0.3.3
certifi : 2025.1.31
cffi : 1.17.1
charset_normalizer : 3.4.1
chex : 0.1.88
click : 8.1.8
cloudpickle : 3.1.1
colorama : 0.4.6
comm : 0.2.2
cryptography : 44.0.2
cudf : 24.12.0
cugraph : 24.12.0
cuml : 24.12.0
cupy : 13.3.0
cycler : 0.12.1
cytoolz : 1.0.1
dask : 2024.11.2
dask_cuda : 24.12.0
dask_cudf : 24.12.0
dask_expr : 1.1.19
dask_image : 2024.5.3
datashader : 0.17.0
datetime : 5.5
debugpy : 1.8.12
decorator : 5.1.1
defusedxml : 0.7.1
deprecated : 1.2.18
dill : 0.3.9
distributed : 2024.11.2
docrep : 0.3.2
etils : 1.12.0
executing : 2.1.0
fastrlock : 0.8.3
filelock : 3.17.0
flax : 0.10.3
flowio : 1.3.0
fsspec : 2023.6.0
geopandas : 1.0.1
h5py : 3.13.0
huggingface_hub : 0.30.1
idna : 3.10
igraph : 0.11.8
imagecodecs : 2024.9.22
imageio : 2.37.0
ipykernel : 6.29.5
ipywidgets : 8.1.5
jax : 0.5.0
jax_cuda12_plugin : 0.5.0
jaxlib : 0.5.0
jedi : 0.19.2
jinja2 : 3.0.3
joblib : 1.4.2
jupyter_client : 8.6.3
jupyter_core : 5.7.2
kiwisolver : 1.4.8
lazy_loader : 0.4
legacy_api_wrap : 1.4.1
leidenalg : 0.10.2
lightning : 2.5.0.post0
lightning_utilities : 0.12.0
llvmlite : 0.43.0
locket : 1.0.0
louvain : 0.8.2
lz4 : 4.3.3
markupsafe : 3.0.2
matplotlib : 3.6.3
matplotlib_inline : 0.1.7
matplotlib_scalebar : 0.9.0
ml_collections : 1.0.0
ml_dtypes : 0.5.1
more_itertools : 10.6.0
mpmath : 1.3.0
msgpack : 1.1.0
mudata : 0.3.1
multipledispatch : 0.6.0
multiscale_spatial_image : 1.0.1
natsort : 8.4.0
networkx : 3.4.2
numba : 0.60.0
numcodecs : 0.15.1
numpy : 1.26.4
numpyro : 0.17.0
nvtx : 0.2.10
ome_zarr : 0.10.3
opt_einsum : 3.4.0
optax : 0.2.4
packaging : 24.2
pandas : 2.2.3
param : 2.2.0
paramiko : 3.5.1
parso : 0.8.4
patsy : 1.0.1
pickleshare : 0.7.5
pims : 0.7
platformdirs : 4.3.6
plotly : 6.0.0
pooch : 1.8.2
prompt_toolkit : 3.0.50
psutil : 6.1.1
pure_eval : 0.2.3
pyarrow : 17.0.0
pycparser : 2.22
pyct : 0.5.0
pygments : 2.19.1
pylibcudf : 24.12.0
pylibcugraph : 24.12.0
pylibraft : 24.12.0
pynndescent : 0.5.13
pynvml : 11.4.1
pyparsing : 3.2.1
pyproj : 3.7.1
pytorch_lightning : 2.5.0.post0
pytz : 2024.1
raft_dask : 24.12.0
rapids_dask_dependency : 24.12.0a0
rapids_singlecell : 0.11.1
rasterio : 1.4.3
readfcs : 2.0.1
regex : 2024.11.6
requests : 2.32.3
rich : 13.9.4
rmm : 24.12.1
safetensors : 0.5.3
scanpy : 1.11.0
scipy : 1.13.1
seaborn : 0.13.2
setuptools : 75.8.0
shapely : 2.0.7
simplejson : 3.20.1
six : 1.17.0
sknw : 0.15
slicerator : 1.1.0
sortedcontainers : 2.4.0
sparse : 0.15.5
spatial_image : 1.1.0
spatialdata : 0.2.5
spatialdata_io : 0.1.5
spatialdata_plot : 0.2.7
squidpy : 1.6.2
stack_data : 0.6.3
statsmodels : 0.14.5
sympy : 1.13.1
tblib : 3.0.0
texttable : 1.7.0
threadpoolctl : 3.5.0
tifffile : 2024.12.12
tokenizers : 0.21.1
toolz : 1.0.0
torch : 2.6.0
torchgmm : 0.1.2
torchmetrics : 1.6.1
tornado : 6.4.2
tqdm : 4.67.1
traitlets : 5.14.3
transformers : 4.50.3
treelite : 4.3.0
triton : 3.2.0
typing_extensions : 4.12.2
urllib3 : 1.26.20
validators : 0.34.0
wcwidth : 0.2.13
wrapt : 1.17.2
xarray : 2024.7.0
xarray_dataclasses : 1.9.1
xarray_schema : 0.0.3
zarr : 2.18.4
zict : 3.0.0
zstandard : 0.19.0

exact steps to reproduce
####注释####
####首先做细胞注释，Dotplot####
adata=sc.read_h5ad("data/SCM/scRNA_qc_filtered.h5ad")
adata.raw=adata.copy()
rsc.get.anndata_to_GPU(adata)
adata.layers["counts"] = adata.X.copy()
rsc.pp.normalize_total(adata, target_sum=1e4)
rsc.pp.log1p(adata)
adata.layers['logcounts'] = adata.X.copy()
rsc.pp.highly_variable_genes(adata, n_top_genes=3000, flavor="seurat_v3", batch_key="sample", layer="counts")
adata = adata[:, adata.var["highly_variable"]]
rsc.get.anndata_to_CPU(adata, layer="counts") # 将 'counts' 层的数据转换回 CPU (可选)
rsc.get.anndata_to_CPU(adata, layer="logcounts")
scvi.settings.seed = 39
scvi.model.SCVI.setup_anndata(adata, layer="counts", batch_key="sample")
model = scvi.model.SCVI(adata, n_layers=3, n_latent=30, gene_likelihood="nb")
model.train(batch_size=512)
path = os.path.join("Test", "test")
model.save(path, overwrite=True)
SCVI_LATENT_KEY = "X_scVI"
adata.obsm[SCVI_LATENT_KEY] = model.get_latent_representation()
rsc.pp.neighbors(adata, use_rep=SCVI_LATENT_KEY)
rsc.tl.leiden(adata,resolution=0.5)

expected behavior

Output the relevant clustering results

actual behavior

IndexError Traceback (most recent call last)
Cell In[2], line 23
21 adata.obsm[SCVI_LATENT_KEY] = model.get_latent_representation()
22 rsc.pp.neighbors(adata, use_rep=SCVI_LATENT_KEY)
---> 23 rsc.tl.leiden(adata,resolution=0.5)

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/rapids_singlecell/tools/_clustering.py:137, in leiden(adata, resolution, random_state, restrict_to, key_added, adjacency, n_iterations, use_weights, neighbors_key, obsp, copy)
128 leiden_parts, _ = culeiden(
129 g,
130 resolution=resolution,
131 random_state=random_state,
132 max_iter=n_iterations,
133 )
135 # Format output
136 groups = (
--> 137 leiden_parts.to_pandas().sort_values("vertex")[["partition"]].to_numpy().ravel()
138 )
139 if restrict_to is not None:
140 if key_added == "leiden":

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/cudf/utils/performance_tracking.py:51, in _performance_tracking..wrapper(*args, **kwargs)
43 if nvtx.enabled():
44 stack.enter_context(
45 nvtx.annotate(
46 message=func.qualname,
(...)
49 )
50 )
---> 51 return func(*args, **kwargs)

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/cudf/core/dataframe.py:5482, in DataFrame.to_pandas(self, nullable, arrow_type)
5406 """
5407 Convert to a Pandas DataFrame.
5408
(...)
5478 dtype: object
5479 """
5480 out_index = self.index.to_pandas()
5481 out_data = {
-> 5482 i: col.to_pandas(nullable=nullable, arrow_type=arrow_type)
5483 for i, col in enumerate(self._columns)
5484 }
5486 out_df = pd.DataFrame(out_data, index=out_index)
5487 out_df.columns = self._data.to_pandas_index()

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/cudf/core/column/numerical.py:711, in NumericalColumn.to_pandas(self, nullable, arrow_type)
709 return pd.Index(pandas_array, copy=False)
710 elif self.dtype.kind in set("iuf") and not self.has_nulls():
--> 711 return pd.Index(self.values_host, copy=False)
712 else:
713 return super().to_pandas(nullable=nullable, arrow_type=arrow_type)

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/cudf/core/column/column.py:220, in ColumnBase.values_host(self)
217 raise ValueError("Column must have no nulls.")
219 with acquire_spill_lock():
--> 220 return self.data_array_view(mode="read").copy_to_host()

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/cudf/core/column/column.py:135, in ColumnBase.data_array_view(self, mode)
133 else:
134 obj = None
--> 135 return cuda.as_cuda_array(obj).view(self.dtype)

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/api.py:76, in as_cuda_array(obj, sync)
74 raise TypeError("obj doesn't implement the cuda array interface.")
75 else:
---> 76 return from_cuda_array_interface(obj.cuda_array_interface,
77 owner=obj, sync=sync)

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/devices.py:231, in require_context.._require_cuda_context(*args, **kws)
229 @functools.wraps(fn)
230 def _require_cuda_context(*args, **kws):
--> 231 with _runtime.ensure_context():
232 return fn(*args, **kws)

File ~/miniconda3/envs/RSC/lib/python3.12/contextlib.py:137, in _GeneratorContextManager.enter(self)
135 del self.args, self.kwds, self.func
136 try:
--> 137 return next(self.gen)
138 except StopIteration:
139 raise RuntimeError("generator didn't yield") from None

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/devices.py:123, in _Runtime.ensure_context(self)
121 with driver.get_active_context():
122 oldctx = self._get_attached_context()
--> 123 newctx = self.get_or_create_context(None)
124 self._set_attached_context(newctx)
125 try:

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/devices.py:138, in _Runtime.get_or_create_context(self, devnum)
136 attached_ctx = self._get_attached_context()
137 if attached_ctx is None:
--> 138 return self._get_or_create_context_uncached(devnum)
139 else:
140 return attached_ctx

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/devices.py:158, in _Runtime._get_or_create_context_uncached(self, devnum)
155 return self._activate_context_for(0)
156 else:
157 # Get primary context for the active device
--> 158 ctx = self.gpus[ac.devnum].get_primary_context()
159 # Is active context the primary context?
160 if USE_NV_BINDING:

File ~/miniconda3/envs/RSC/lib/python3.12/site-packages/numba_cuda/numba/cuda/cudadrv/devices.py:40, in _DeviceList.getitem(self, devnum)
36 def getitem(self, devnum):
37 '''
38 Returns the context manager for device devnum.
39 '''
---> 40 return self.lst[devnum]

IndexError: list index out of range

[Intron7]

I see that your rapids-singlecell and rapids versions are very outdated. Can you please confirm that the bug exists with 0.14.1

[shanggutianzhen]

I'm very sorry. After the upgrade, the problem still occurred in this area. The current error message is

---

TypeError Traceback (most recent call last)
Cell In[2], line 23
21 adata.obsm[SCVI_LATENT_KEY] = model.get_latent_representation()
22 rsc.pp.neighbors(adata, use_rep=SCVI_LATENT_KEY)
---> 23 rsc.tl.leiden(adata,resolution=0.5)

File ~/miniconda3/envs/RSC_Update20260310/lib/python3.12/site-packages/rapids_singlecell/tools/_clustering.py:226, in leiden(adata, resolution, random_state, theta, restrict_to, key_added, adjacency, n_iterations, use_weights, neighbors_key, obsp, dtype, use_dask, copy)
223 else:
224 from cugraph import leiden as culeiden
--> 226 g = _create_graph(adjacency, dtype, use_weights=use_weights)
227 # Cluster
228 if isinstance(resolution, float | int):

File ~/miniconda3/envs/RSC_Update20260310/lib/python3.12/site-packages/rapids_singlecell/tools/_clustering.py:49, in _create_graph(adjacency, dtype, use_weights)
47 warnings.simplefilter("ignore")
48 if use_weights:
---> 49 g.from_cudf_edgelist(
50 df,
51 source="source",
52 destination="destination",
53 weight="weights",
54 vertices=vertices,
55 )
56 else:
57 g.from_cudf_edgelist(
58 df,
59 source="source",
60 destination="destination",
61 vertices=vertices,
62 )

TypeError: Graph.from_cudf_edgelist() got an unexpected keyword argument 'vertices'

[Intron7]

Did you also update the whole rapids stack eg cugraph or just rsc

[shanggutianzhen]

In fact, I only updated the RSC.
Because when executing the following command, the download would fail, causing my entire system to crash.
pip install --upgrade rapids-singlecell==0.14.1 --upgrade-strategy eager

[Intron7]

Please create a new updated environment.

[shanggutianzhen]

Oh, I understand what you mean. So, should I create a new environment and re-assemble RSC along with its supporting components? Or, in this current situation, could you specify some packages and their versions for me, so that I can make them run on the upgraded RSC?

[shanggutianzhen]

Thank you for your suggestion. This indeed solved my problem.