A Constructive Heuristic for Pattern Assignment in an Ensemble of Attractor Neural Networks to Increase Storage Capacity

Abstract

This work explores strategies to optimize pattern subset assignments to an ensemble of attractor network modules. For exploiting ensemble modules with diluted connectivity, the assignment process aims to minimize pattern collisions (correlation) and reduce cross-talk noise in each module. Considering the combinatorial complexity of the assignment problem, heuristic strategies are indispensable for finding near-optimal solutions. First, the proposed constructive assignment heuristic is based on minimizing the similarity between the subsets of patterns assigned to the ensemble modules. For this heuristic greedy assignment, three similarity metrics (cosine, Jaccard, and Hamming) are employed. Random assignment is evaluated as a baseline, while an upper limit of the solution is established using a network learning/retrieval dynamic with a sample of possible pattern assignments. Additionally, the constructive greedy assignment based on similarity is compared with a population-based genetic algorithm. The genetic algorithm outperforms the greedy strategy when the network ensemble contains a small number of modules due to its ability to explore a broader solution space. On the other hand, the greedy constructive approach demonstrates better performance when the network ensemble contains larger numbers of modules, where the genetic algorithm struggles with maintaining subsets with minimum overlap. To validate these strategies, three datasets are tested: a Random Patterns dataset, a fingerprint dataset (extracted from the FVC2004 database), and the Digital Retinal Images for Vessel Extraction (DRIVE) dataset. This work demonstrates that a correct assignment of patterns to the modules of an Ensemble of Attractor Neural Networks significantly improves their ability to retrieve the assigned patterns, with performance varying across datasets and strategies.