MDEGL: Evolution of Neural Networks via Neighborhood-Based Mutation and Subpopulations

TL;DR

Optimization of Neural Network (NN) architectures has long been a manual "dark art." This paper presents MDEGL, a hybrid evolutionary algorithm that merges Differential Evolution with Global and Local Neighborhoods (DEGL) and the Simple Subpopulation Scheme (SSS). By balancing exploration and exploitation through topological neighborhoods and fostering diversity via subpopulations, MDEGL automatically evolves high-performing NNs that outperform traditional training algorithms like Backpropagation and Scaled Conjugate Gradient.

The Motivation: Escaping the Local Optima Trap

The search space for an optimal neural network is notoriously "multimodal"—it contains many peaks and valleys where an optimization algorithm can get stuck. Standard Differential Evolution (DE) strategies often lean too heavily on the "best" individual, leading to premature convergence.

The authors recognized that to find a near-optimal NN, an algorithm needs:

1. Exploitation: Refining solutions near the best-known global point.
2. Exploration: Searching local neighborhoods to discover hidden "peaks."
3. Diversity: Maintaining different "species" of solutions to prevent the entire population from collapsing into a single, potentially sub-optimal, region.

Methodology: The Architecture of MDEGL

1. The Core Infrastructure: DEGL

Unlike standard DE, which uses a global best vector for mutation, DEGL employs a ring topology. Each individual interacts with a neighborhood of radius $k$.

• Local Mutant ($L_{i,G}$): Based on the best vector in the local neighborhood.
• Global Mutant ($g_{i,G}$): Based on the best vector in the entire population.

These are combined using a weight factor $\omega$: ${\vec{v}}_{i,G}=\omega \cdot {\vec{g}}_{i,G}+(1-\omega )\cdot {\vec{L}}_{i,G}$ This $\omega$ parameter is the "throttle" that balances global and local search.

2. Genetic Encoding of a Neural Network

The individual in MDEGL is a continuous vector that maps to specific NN properties. This includes:

• Learning Algorithm: Selection between BP, RPROP, LM, or SCG.
• Topology: Number of layers (1-3) and neurons (up to 10/layer).
• Weights: Transferred directly into the network.

3. Diversity through SSS

The Simple Subpopulation Scheme (SSS) acts as a multimodal stabilizer. It assigns "labels" to individuals, effectively creating species. Fitness is adjusted by the size of the subpopulation: $f^{\prime }(i)=f(i)imes||S_j||$ This discourages "crowding," forcing the algorithm to explore different areas of the search space.

Experimental Results and SOTA Comparison

The authors tested MDEGL against standard algorithms and a major hybrid baseline, EAPSONN (which combines GA, PSO, and Evolution Strategies).

Key Benchmarks (PROBEN1)

• Card Dataset: MDEGL achieved a mean test error of 0.0434, vastly superior to standard BP (0.2900).
• Glass Dataset: MDEGL's error (0.2484) was nearly 60% lower than the second-best classical algorithm.

When pitted against the complex EAPSONN method, MDEGL proved to be statistically equivalent or superior in 5 out of 6 datasets, demonstrating that a well-designed DE-variant can match or beat more complex multi-algorithm ensembles.

Deep Insight: Why Does It Work?

The success of MDEGL lies in the dynamic adaptivity of $\omega$. By testing linear, exponential, and self-adaptive scaling for $\omega$, the authors found that allowing the algorithm to shift from local exploration to global exploitation (or vice versa) is critical. The inclusion of SSS ensures that even as the algorithm converges, it keeps "scouts" in other regions, preventing the common pitfall of evolving a population of identical, mediocre networks.

Conclusion & Future Outlook

MDEGL represents a significant step in Neuroevolution. It successfully automates the design of shallow NNs for classification tasks. Future Work:

1. Ensemble Evolution: Using the final diverse sub-populations to create a committee/ensemble of NNs.
2. Meta-Learning: Using meta-learning to decide which evolutionary operators to use based on the dataset characteristics.

For researchers in AutoML and Evolutionary Computation, MDEGL offers a powerful template for balancing search dynamics in high-dimensional, multimodal spaces.