5

Tensorized Organization of Population, Evaluation, and State

AI999第4课：现代计算底座与张量化并行演化
00:00

In the engineering implementation of evolutionary algorithms,tensorized organizationmaps the questions of “what the population is, how it is evaluated, and how far it has progressed” from scattered objects and lists into uniform, regular tensors. At its core, it establishes a unified data contract for the population, evaluation, and state, giving batched computation, device residency, and subsequent parallel scheduling a shared memory semantics.

Tensorized Organization of Population, Evaluation, and StateUnifying candidate solutions, fitness evaluations, and algorithm state into tensors that can be computed in whole batchesTraditional Serial OrganizationObjects and lists stored dispersedlyCandidate ListList / collection of objectsIndividual EvaluationLooping calls to the evaluation functionScattered StateScalars / global variablesPer-individual loopFrequent host–device round tripsTensorizationtensorized organizationPopulation · Evaluation · State unified into regular tensors, forming a unified data contractNDPopulation Tensor[N, D]N is the batch dimension, D is the decision dimensionBatched mutation and crossoverDone in one passNMKEvaluation Matrix[N, M] or [N, M+K]M objectives, K constraintsVectorized evaluation functionReturns results for the whole batchState Tensor ContainerScalars or small tensorsGeneration, best solution, RNG stateResident on device uniformlyCheckpointable and reproducibleExample: N = 1024, D = 50, M = 2 · Population [1024, 50], Evaluation [1024, 2]Batch dimension outermost: operators vectorize across the entire N dimensionVariable-length individuals: padding + masking, or bucketingUnified state: reproducible, traceable, batch-updatable

Population Tensorization: Encoded as [N, D]

Organize N candidate solutions along the batch dimension into an encoding tensor of shape [N, D]. Continuous variables, discrete variables, or multi-segment encodings can be expressed as field tensors or grouped blocks while maintaining a uniform data type and device placement.

  • Batch dimension outermost: N is typically placed as the outermost dimension so that the same operator executes in a vectorized manner over the entire batch.
  • Field blocking: If an individual consists of multiple parameter segments, it can be split into [N, D1] + [N, D2] + … while keeping contiguous storage.
  • Variable-length handling: Variable-length individuals are handled with padding plus masking or bucketing to prevent heterogeneous structures from breaking batched computation.

Matrix Representation of Evaluation Results: [N, M] and [N, K]

A single batched evaluation produces an [N, M] fitness matrix; multiple objectives, multiple constraints, and additional metrics can be placed side by side as [N, M+K] or stored in separate tables, avoiding per-individual loops that call the evaluation function.

Unified Management of State Tensors

Generation number, RNG state, strategy hyperparameters, best solution, history cache, ranking or sampling state, and more also go into tensors or a structured state container, ensuring reproducibility, traceability, and batch updates.

Example: N=1024, D=50, M=2
The population is represented as a [1024, 50] tensor, evaluation values as a [1024, 2] tensor, and constraints as a [1024, K] tensor. The generation number generation, best solution best_x, and RNG state rng_state reside uniformly on the device as state tensors. Each generation's update can be completed over the same batch dimension as a whole, rather than through per-individual Python loops.

Benefits and Boundaries of the Organization

Unified tensors reduce host–device round trips, lower loop overhead, and provide a data foundation for subsequent operator parallelism; however, they require handling mask validity, padding waste, multi-objective scaling growth, and state synchronization costs.