Finding the Optimal Number of Clusters (k) for Your Dataset

Step 1 : “First, I loaded the single-cell dataset into an AnnData object using Scanpy.”

“Next, I ran a preprocessing pipeline — normalization, log-transformation, and selection of ~4,000 highly variable genes.”

**what it does: **“This step reduces noise and focuses the analysis on the most informative genes, making clustering more meaningful.”

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Step 2 : I scaled the data, applied PCA for dimensionality reduction, and visualized variance explained along with batch effects.

what it does: This helps capture the main biological signals while reducing noise, ensuring clusters are formed in a meaningful low-dimensional space.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Step 3: I built a neighborhood graph, applied UMAP for visualization, and performed Leiden clustering across multiple resolutions.

What it does :“Exploring different resolutions helps reveal both broad cell types and finer subpopulations, giving flexibility in choosing the optimal k.”

“Here’s how clustering granularity changes with the resolution parameter in Leiden — from broad groups at low values to finer subpopulations at higher ones.” “This visual comparison highlights why there’s no single ‘right’ k — the choice depends on the biological question we want to answer.

“If resolution 0.5 is chosen as the best, the variable adata.obs["leiden_res_0.50"].value_counts() contains the distribution of cells across each cluster."

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -