The Expansion of Evolutionary Objects: From Numerical Parameters to Software Systems

In the early practice of evolutionary computation, the objects being "evolved" were almost alwaysnumerical parameters—learning rate, batch size, regularization strength, weight initialization. The search space was just a set of numbers, and evaluation meant one training run plus one validation run. As compute scale grew and large model capabilities matured, the objects that can be evolved expanded markedly: programs, model architectures, algorithmic mechanisms, and even an entire block ofsoftware engineering state(configurations, dependency versions, interface contracts, deployment and runtime strategies).

This expansion path can be represented as a clear arrow chain: numerical parameters → programs and model architectures → software engineering state and system-level candidates. The further to the right the level is, the stronger the expressiveness and the more types of change it can cover—making system-level evolution possible.

Three Levels of Evolutionary Objects, Each with Different Costs

Numerical parameter level: A candidate is just a set of numbers—cheap to generate and quick to evaluate—but it can only answer "how well it is tuned," and cannot supply a mechanism that was missing to begin with.
Program and model architecture level: Candidates are network architectures, operator compositions, prompt workflows, or code patches; changes alter "what the system can do," not just "how accurately it does it."
Software engineering state level: Candidates are configurations, dependency versions, interface contracts, and deployment strategies, directly determining whether the system can run stably and be reproduced.

Shifting Focus: From "How to Generate Candidates" to "How to Execute and Evaluate"

When candidates go from numbers to programs, "generating candidates" is no longer the bottleneck: large models can mass-produce structures, mechanisms, and patches. What truly determines whether evolution is effective isthe execution and evaluation mechanism—whoever can actually run the candidates and, within acceptable cost, judge whether they are worth keeping.

The stronger the expressiveness, the heavier the evaluation burden. A numerical parameter only needs one training run to be scored; a code patch, however, requires sandbox isolation, regression testing, and benchmark comparison before it can answer three questions: Is it truly effective? Does it introduce risk? Can the results be reproduced? Sandboxes, regression tests, and benchmarks are precisely the engineering devices used to constrain cost, risk, and reproducibility—and they allow changes to be reliably kept or eliminated.

Engineering Trade-offs: More Candidates Does Not Mean Better Evolution

In team practice, a common path is to first tune learning rates and hyperparameters, then gradually expand to searching network architectures, operator compositions, prompt workflows, or code patches. Each expansion of a level requires answering in step: How much compute and time will evaluation cost? Will failed candidates contaminate the existing system? Can the same results be reproduced next time? Only by balancing expressiveness, evaluation cost, reproducibility, and safety does system-level evolution truly become feasible.

Common Misconception

Equating "more candidates" directly with "more effective evolution." Increasing the number of candidates only widens the search space; only when the execution and evaluation mechanisms are sufficient to screen out invalid changes will more candidates translate into a better system.

Three-Level Expansion of Evolutionary Objects

The arrow chain in the figure goes from numerical parameters to programs and model architectures, then to software engineering state and system-level candidates: the further right, the more objects can be changed and the stronger the expressiveness. But the cost rises accordingly—numerical parameters only need one training run to score, whereas code patches require sandbox isolation, regression testing, and benchmark comparison to judge effectiveness, risk, and reproducibility.

Question 1

Compared with early evolutionary practice, what is the most essential change described on this page regarding the "expansion of evolutionary objects"?

Extending from single-machine training to multi-machine distributed training

Extending from only tuning numerical parameters to programs, model architectures, algorithmic mechanisms, and software engineering state

Extending from manually setting evaluation metrics to automatically generating evaluation metrics

Extending from supervised learning to reinforcement learning

Question 2

Why is "more candidates" not directly equivalent to "more effective evolution"?

Because candidates generated by large models are always lower in quality than human-designed ones

Because only when the execution and evaluation mechanisms can screen out invalid changes do more candidates translate into a better system

Because as the number of candidates grows, the speed of a single training run inevitably decreases

Because candidates cannot be stored once their number exceeds a certain threshold

Question 3

When the evolutionary object changes from numerical parameters to code patches, why does the evaluation stage need sandboxes, regression tests, and benchmarks?

To reduce the model's parameter size

To increase the diversity of generated candidates

To allow candidates to copy and merge with each other

To judge, within controllable cost, whether a candidate is effective, whether it introduces risk, and whether results can be reproduced

Question 4

Regarding the shifting focus between "generation" and "evaluation," which of the following descriptions matches the views on this page?

Since evaluation is more critical, there is no need to keep improving generation capability

The importance of generation and evaluation has never changed

As large model capabilities strengthen, the key question shifts from "how to generate candidates" to "how to execute and evaluate them and retain effective changes"

Generating candidates has become the hardest part, and evaluation is now essentially automated

Question 5

After a team expands the evolutionary object from hyperparameters to prompt workflows, what should be filled in the most?

Execution and evaluation devices such as sandboxes, regression tests, and benchmarks, along with careful trade-offs among cost, risk, and reproducibility

Invest more compute and keep generating candidates in bulk

A more sophisticated visualization interface

Compress the number of candidates per round down to 1

Case Study: Expanding the Evolutionary Object from Hyperparameters to Code Patches

Read the scenario and answer the questions below.

A team uses an evolutionary system to optimize a recommendation service. In the first phase, only the learning rate and batch size are evolved: candidates are a set of numbers, scored with one training run, and hundreds of candidates can be evaluated per day. In the second phase, the evolutionary object is extended to prompt workflows and code patches for the ranking module: candidates are directly runnable changes that may affect live interfaces, dependency versions, and service response latency. The team quickly finds that the speed of candidate production rises sharply, but the proportion actually adopted into the next round declines.

1. Why can the adoption rate decline even as candidates are produced faster? Explain in terms of the shifting focus between "generation" and "evaluation."

Answer:
Because generating candidates is no longer the tightest link—large models can mass-produce structures, mechanisms, and code patches, so the numbers grow rapidly by nature. The real bottleneck has shifted to execution and evaluation: candidates are programs, not numbers, and must first be run in a controlled environment, then pass regression tests and benchmark comparisons, before it is possible to judge whether they are effective, safe, and reproducible. If the evaluation devices are not upgraded in step with the object level, a large number of candidates cannot be reliably judged and can only pile up in the queue, so the adoption rate declines.

2. If you were in charge of the second phase, what engineering devices would you add to the evaluation mechanism? Explain what type of risk each of them constrains.

Answer:
Key points: ① Sandboxed isolated execution—constrains the risk of "failed candidates contaminating the existing system, dependencies, or live environment"; ② Regression testing—constrains the risk of "changes breaking existing functionality"; ③ Benchmark metric comparison—constrains the misjudgment risk of "looking better while actually offering no improvement"; ④ Cost budgets and timeout controls—constrains the cost risk of "runaway evaluation overhead"; ⑤ Seed and result records—constrains the risk of "results not being reproducible." Overall, it is a trade-off among expressiveness, evaluation cost, reproducibility, and safety, rather than simply pursuing a larger number of candidates.