MetaDE and AutoPSO: A Two-Level Architecture for Method Evolution

Method evolution(Meta-Evolution) elevates the method that generates candidates into the object being optimized: algorithms no longer just find solutions—they must also improve how they find solutions.MetaDE and AutoPSO are two representative implementations of this idea, and both share the same skeleton—an inner–outer two-level architecture: the outer level searches thealgorithm space, while the inner level uses the candidate algorithm to solve thetarget taskThe inner-level performance is fed back as a fitness signal to the outer level, driving the algorithm's continuous self-improvement and forming a closed loop.

General Abstraction of the Two-Level Architecture

Outer level (algorithm space search): Searches hyperparameters, strategies, and component combinations that define the algorithm itself. Its search space is not the solution space but the space of algorithm configurations and structures.
Inner level (target task solving): Uses the candidate algorithms dispatched by the outer level to actually solve the task, producing measurable performance that serves as the evaluation basis for that candidate.
Signal loop: Inner-level performance is aggregated into fitness and fed back to the outer level, which uses it to eliminate poor configurations and retain and mutate high-quality ones. Once this loop is established, solving and improving how to solve happen simultaneously.

MetaDE: Handing DE's Configuration to the Outer-Level Evolution

MetaDE keeps differential evolution as the inner-level solver and hands DE's own configuration to the outer-level search: the choice of mutation strategy, the crossover rate, and the scaling factor are no longer fixed constants but objects of evolution, adaptively adjusted to the problem and optimization stage. It answers the question "how should this configuration be set at the current task stage?"—this is method evolution at the configuration level.

AutoPSO: From Tuning Parameters to Modifying Algorithm Structure

AutoPSO treats PSO's core parameters and learning mechanisms ascomposable components—different combinations of inertia weight, learning factors, and individual/global learning mechanisms are automatically searched and composed by the outer level to produce PSO variants tailored to different problems. Compared with merely tuning parameters, it transforms "choosing an algorithm" into "searching for algorithm structure," achieving a leap from parameter adaptation to mechanism self-generation. The two complement each other: MetaDE focuses on automatic tuning at the configuration and hyperparameter level, while AutoPSO focuses on automatic composition at the structure and component level, together covering the evolution range from parameter adaptation to mechanism self-generation.

Cost Structure and Engineering Dependencies

The key cost of two-level optimization is the multiplicative effect: each outer-level evaluation of a candidate algorithm typically requires multiple inner runs (repeated evaluation due to randomness, joint multi-task validation, etc.), significantly amplifying the computational budget. Therefore, trade-offs such asbudget allocation,approximate evaluation,early stoppingorparallelizationare necessary. In engineering terms, efficient execution relies on tensorized parallelism, runtime scheduling, and algorithm composability; otherwise the scale of the outer-level search will be dragged down by repeated inner-level evaluations.

Mindset Shift

This architecture shifts the researcher's role from "manually designing a single algorithm" to "designing a space in which algorithms can evolve, along with an evaluation mechanism": what is delivered is no longer a fixed algorithm but a search space and a set of feedback rules that let algorithms continuously improve themselves.

MetaDE Inner–Outer Two-Level Architecture

The diagram shows MetaDE's inner–outer closed loop: the outer level searches the algorithm configuration space for mutation strategies, crossover rates, and scaling factors; the inner level uses these configurations to run differential evolution on the target task, then aggregates inner-level performance into fitness fed back to the outer level, driving configuration elimination, retention, and mutation.

Question 1

In the two-level architecture represented by MetaDE and AutoPSO, what is the direct object of the outer-level search?

The sampling distribution of training data

The set of candidate solutions for the target task

The physical topology of the GPU cluster

The algorithm's own configuration, structure, and component combinations

Question 2

In MetaDE, which of the following sets of objects does the outer level primarily evolve?

The learning rate schedule of gradient descent

PSO's velocity update formula and learning factors

DE's mutation strategy, crossover rate, and scaling factor

The number of neural network layers and attention heads

Question 3

Regarding the division of labor between AutoPSO and MetaDE, which of the following descriptions is the most accurate?

AutoPSO focuses on automatic composition of parameters and learning mechanisms, turning "choosing an algorithm" into "searching for algorithm structure"; MetaDE focuses on configuration-level evolution

AutoPSO only tunes hyperparameters, and MetaDE only modifies training data

AutoPSO works without inner-level evaluation

The two are completely equivalent, just implemented in different languages

Question 4

Regarding the cost structure of two-level optimization, which of the following statements is correct?

Using a GPU alone can completely eliminate the extra overhead

Each outer-level evaluation of a candidate algorithm often requires multiple inner runs, significantly amplifying the budget and requiring budget allocation, approximate evaluation, or early stopping

One outer-level evaluation of a candidate algorithm requires only one inner run, with overhead comparable to a single level

Inner-level runs consume no computational budget; only the outer level does

Question 5

After adopting the method-evolution two-level architecture, what is the cognitive shift in the researcher's role?

Completely abandoning algorithm design and relying solely on random search

Shifting from manually designing a single algorithm to designing a space in which algorithms can evolve, along with an evaluation mechanism

Shifting from writing code to only doing data labeling

Shifting from modeling to focusing only on hardware selection

Case Study: Building a Two-Level Evolution Service for Continuously Changing Tasks

Read the scenario and answer the two questions, explaining your design and trade-offs.

A team maintains a combinatorial optimization solving service running on a GPU cluster, with task types changing every quarter. Previously, engineers manually fixed the configurations of differential evolution and particle swarm optimization, but after the task distribution changed, these fixed configurations quickly became ineffective. The team plans to introduce method evolution: the outer level searches the algorithm space, the inner level uses candidate algorithms to solve the day's task, and they plan to rely on tensorized parallelism and runtime scheduling. The project manager is concerned about runaway evaluation budgets and requires an architecture description and cost-control plan first.

1. Please explain what the inner and outer levels are each responsible for in this system, and how signals flow between them.

Answer:
The outer level is responsible for searching for candidate algorithms in the algorithm space: it can evolve DE's mutation strategy, crossover rate, and scaling factor (the MetaDE path), or automatically compose PSO's inertia weight, learning factors, and learning mechanisms (the AutoPSO path). The inner level runs and evaluates each candidate algorithm on the day's target task, producing comparable performance results; inner-level performance is aggregated into fitness and fed back to the outer level, which uses it to eliminate poor candidates and retain and mutate high-quality configurations, iterating continuously. This forms a closed loop of "the outer level searches the algorithm space, the inner level solves the target task, and performance feedback drives improvement."

2. Facing the budget pressure of "one outer-level evaluation requiring multiple inner runs," please provide at least two feasible trade-off measures and explain the cost or risk of each.

Answer:
The optional measures and their corresponding costs are as follows: ① Budget allocation — set separate compute quotas for the outer and inner levels and limit the number of inner repetitions; the cost is increased evaluation variance and possible misselection. ② Approximate evaluation — estimate fitness using data subsets or short runs; the cost is high signal noise and possible underestimation of the best candidate. ③ Early stopping — prematurely terminate inner runs for obviously poor candidates; the cost is possibly eliminating slow-burn configurations that perform well later. ④ Tensorized parallelism and runtime scheduling — batch concurrent inner runs, significantly reducing wall-clock time but not the total compute, and limited by memory and scheduling overhead. A good answer should explain the correspondence between each measure and its cost or risk, and reflect the multiplicative estimate of "outer-level search scale × number of inner runs per evaluation."