Graph-based clustering via ant colony optimization (ACO): finding semantic cores and multilevel hierarchical clustering
I'm developing a clustering method: the data becomes a k-nearest-neighbor graph, ants deposit pheromone on its edges with no optimization objective, and clusters emerge by cutting the weak edges. The mechanism itself isn't new - similar solutions appear earlier in the literature - but whether it works as a standalone clustering tool, and for which data, is a question that's been debated for over twenty years, and I'm interested in working out my own answer to it. I've been running the project since December 2025; the practical goal is to reach quality comparable to other clustering algorithms (Leiden, for instance), not to beat them.
The first stage, intelliant-core, was the prototype from my thesis: cutting the raw Conceptual Captions dataset 92x down to ~35k dense semantic cores. I relabeled that set and fine-tuned a model with LoRA on it; the dataset and the repo with the labeling and training are in open access.
The second stage, intelliant, is the current work: one pipeline covering graph construction, the pheromone field, and clustering, with the intermediate results inspectable for analysis. Right now I'm calibrating and tuning parameters on synthetic and training datasets, and next comes comparing intelliant against other clustering algorithms on real data. It's CPU-only for now (Numba compiled to C); a GPU version is further down the roadmap.
The third stage - multilevel hierarchical clustering, the core task of the research - drops the separate noise-absorption step and hands the whole process to the ants. Each semantic core found becomes a dense absorbing point, a "black hole" by analogy, which should in theory keep neighboring clusters from merging at the next graph-coarsening level.
- intelliant-core(Open in a new tab)pypi.org/project/intelliant-core/
- wcag-caption-research(Open in a new tab)github.com/yourdisenchantment/wcag-caption-research
- Dataset(Open in a new tab)doi.org/10.6084/m9.figshare.32258007
- intelliant(Open in a new tab)github.com/yourdisenchantment/intelliant
