Core Methods of Batched Tensor Computation

In the tensorized foundation of EvoX,batched tensor computationsits at the algorithmic expression layer: it uniformly maps the candidates, evaluations, and states within an evolutionary loop intotensors with a batch dimension, which are then executed by GPU-friendly batched operators and compiled computation graphs. The goal is crystal clear — transform "serial evolution, one individual at a time" into "parallel evolution of the entire population," so that compute power is truly applied to the search itself rather than consumed by scheduling and launch overhead.

1. Expressing the Evolutionary Process as Tensors

By convention, the batch dimension N represents the number of candidates (individuals), so the three core elements of evolution can be directly expressed as tensors:

Candidate tensor: the population X is organized as [N × D], where D is the number of decision variables per candidate.
Objective tensor: objectives and constraints F are organized as [N × M], where M is the number of objective/constraint terms; in the multi-objective case, each objective corresponds to a different column of F.
State variables: generation count, current best values, step sizes, etc. are maintained in scalar or small-tensor buffers, rather than recorded per individual using Python objects.

In this way, crossover, mutation, selection, and evaluation are all batch-processed along the N dimension, and per-individual branches and control flow no longer appear in the evolutionary loop.

2. Batched Operators: From N Single-Candidate Calls to a Single Batched Operation

Three minimal comparable expressions outline the basic form of batched operators:

Mutation: X′ = X + σ·randn(X), where the entire batch of candidates each receives a random perturbation in a single batched operation, with σ controlling the perturbation strength.
Selection: idx = topk(F, k), which selects the indices of the top k candidates from the fitness values of N candidates in a single operation.
Evaluation: Y = f_θ(X), replacing N single-candidate calls with a single batched forward pass or batched simulation.

The key is not any particular operator, but that all candidates share a single tensor computation graph: one batched call can slash interpreter overhead and kernel launch overhead, allowing GPU/multi-core compute power to truly serve the evolutionary search. Constraint handling and out-of-bounds resolution can likewise be accomplished within the same batched operation using mask tensors, without branching separately for individual candidates.

3. Computation Graph Optimization and Throughput Metrics

Writing multi-generation evolution as a compilable dataflow graph can squeeze out even more performance:Operator Fusionreduces the number of kernel launches,Intra-batch Parallelismenables all candidates to execute simultaneously,Asynchronous Executionallows computation and data transfer to overlap,Memory Reusereduces intermediate tensor allocations. Accordingly, performance metrics should also shift from "per-step latency" tocandidates/sec or generations/secthroughput — under batched execution, a single call processes the entire batch of candidates, and only overall throughput truly reflects search efficiency.

Common Pitfall

Don't wrap batched operators inside per-individual loops — that reintroduces interpreter overhead and kernel launch overhead, and the batch dimension loses its parallel significance. The test is simple: within a single evolutionary step, there should be scheduling only at the "batch" level, never at the "individual" level.

Concept Map: Core Methods of Batched Tensor Computation

The concept map expresses the evolutionary process as tensors with a batch dimension: population X is N×D, objectives and constraints F are N×M, and crossover, mutation, selection, and evaluation are all batch-processed along the N dimension. Mutation, topk selection, and batched forward evaluation each require only a single batched call, and performance metrics shift from per-step latency to throughput in candidates/sec.

Question 1

If a population contains N candidates, each with D decision variables and M objective terms, what are the shapes of the population tensor X and the objective tensor F, respectively?

X is [N × D], F is [N × M]

X is [D × N], F is [M × N]

X is [N × M], F is [D × M]

X is [N × N], F is [M × M]

Question 2

When rewriting a per-individual Python loop as batched tensor operations, where does the most direct performance gain come from?

Eliminating per-candidate interpreter overhead and kernel launch overhead

Reducing the computational complexity of the objective function itself

Improving the numerical precision of individual candidate evaluation

Reducing the total number of candidates that need to be evaluated

Question 3

The mutation operator is written as X′ = X + σ·randn(X). What is its correct meaning?

σ must decrease to zero over generations, otherwise evolution won't converge

Adding a random perturbation to each candidate in the entire batch within a single batched operation, with σ controlling the perturbation strength

Every candidate must use exactly the same random numbers to ensure fairness

This expression applies only to a single candidate and cannot be batched

Question 4

What is the semantics of the selection operator idx = topk(F, k) in tensorized evolution?

Compress the candidate count to k, then randomly reinitialize the population

Independently select the top k decision variables for each candidate

Take the top k terms along the objective dimension M as retained objectives

Select the indices of the top k candidates from the fitness values of N candidates in a single batch

Question 5

Tensorized evolution shifts performance metrics from "per-step latency" to "candidates/sec (or generations/sec)." What is the most reasonable explanation?

Because latency metrics cannot be measured on GPUs

Throughput and latency are numerically identical, just in different units

Under batched execution, a single call processes the entire batch of candidates, so only overall throughput reflects search efficiency

Per-step latency no longer has any reference value

Case Study: Migrating a Per-Individual Evolutionary Loop to Batched Tensor Execution

Read the scenario and answer the questions below.

A team uses EvoX to optimize a supercritical airfoil, needing to simultaneously balance the critical buffet lift coefficient and the cruise lift-to-drag ratio. The current implementation calls high-fidelity simulation for each individual, resulting in very low GPU utilization — only a small number of generations can be advanced in a whole day. The team plans to migrate to batched tensor computation: the population is denoted as X:[N × D], objectives as F:[N × M], evaluations are provided in batches by a surrogate model fθ, and the multi-generation evolution is to be compiled into an executable dataflow graph.

1. Explain how to organize candidates, objectives, and states as tensors, and how to batch-process mutation, crossover, selection, and evaluation along the batch dimension N.

Answer:
The batch dimension N carries the candidates: X:[N × D] is randomly initialized, where D is the number of airfoil parameters; F:[N × M] is filled in one shot by the batched evaluation Y = fθ(X), where M is the number of objective terms such as lift and drag; state variables such as generation count and current best values are maintained in scalar or small-tensor buffers. Mutation is written as X′ = X + σ·randn(X); crossover recombines pairwise along the batch with masks handling boundaries; selection uses idx = topk(F, k) to obtain the retained candidate indices in one shot. The key is to eliminate per-individual Python control flow so all candidates share a single tensor computation graph, with N always as the outermost batch dimension.

2. What optimizations can be applied to the computation graph to improve throughput? And how should trade-offs be made when GPU memory is insufficient?

Answer:
Four types of optimization are possible: operator fusion reduces kernel launch counts; intra-batch parallelism enables all candidates to execute simultaneously; asynchronous execution overlaps computation with data transfer; and memory reuse reduces intermediate tensor allocations. Metrics shift accordingly from per-step latency to candidates/sec or generations/sec, because a single batched call carries the entire batch of candidates. When GPU memory is insufficient, you can split the batch into several chunks executed sequentially (chunking / mini-batching), reduce numerical precision, or shorten the lifespan of intermediate tensors; however, you must ensure that all candidates within the same generation use a consistent evaluation standard, so that splitting into batches doesn't cause candidates to lose fair comparison.