Class Activity: Designing a Multi-Objective Parameter Optimization Evaluation Plan

This page is aclass activity design: it turns thelift–dragpair of conflicting objectives into a reproducible, comparable multi-objective parameter optimization evaluation plan deliverable to expert decision-making. The point is not to have the model give a direct answer, but to design an evaluation mechanism that can be repeatedly tested — who takes part in the evaluation, which metrics are used, under what constraints and budget, and by what rules candidates are retained.

The activity simulates a closed loop:generate candidates → enter evaluation → compare benefits, costs, and constraints based on feedback → retain effective solutions → record and proceed to the next round. You need to propose a plan for a parameterized object (for example, airfoil parameters) and make the lift–drag trade-off explicitly visible in it — in a word, turn "conflict" into "executable review rules."

Metrics, Constraints, and Budget

Lift objectivedefined as maximizing the lift coefficient or lift performance;drag objectivedefined as minimizing the drag coefficient or drag performance. These two are the original conflicting objectives and must each be listed independently.
Composite metrics: metrics such as the lift-to-drag ratio can be used for explanation and presentation, but they cannot replace the original conflicting objectives — otherwise the trade-off would be prematurely "flattened" by a single ratio.
Constraint tiering: parameter bounds and geometric and engineering constraints (such as relative thickness and pitching moment range) are usually set as hard constraints, and any violation leads to elimination; performance preferences and cost preferences are set as soft preferences for ranking.
Budget declaration: specify the number of simulation or surrogate model calls, time, and compute limits, and distinguish the quota for "surrogate evaluation" and "high-fidelity verification"; otherwise different plans cannot be compared fairly.

Evaluation Process and Candidate Generation

The evaluation closed loop is organized into six steps:parameter representation → candidate generation → evaluation computation → metric aggregation → solution comparison → record archiving. The candidate generation rules determine the coverage of the exploration: initial sampling spreads out globally, perturbation, crossover, and mutation handle local refinement, and preference guidance intensifies coverage in regions experts care about. Candidates are required to be distributed across different lift-drag trade-off regions, avoiding collapse of an entire generation into a single direction of "high lift" or "low drag."

Selection Rules and Output Candidate Set

Selection may use Pareto non-dominance, weighted sum, threshold filtering, reference points, or preference ranking. Whichever is chosen, the result should not be "keeping only the one with the highest total score": the output should be a set of candidates covering different trade-offs, accompanied by metric evidence, constraint satisfaction, differences among candidates, and uncertainty ranges, so that domain experts can judge whether the trade-offs are reasonable.

Common Pitfall

Combining lift and drag into a single total score and then taking the maximum may look simplifying, but in fact it uses the choice of weights to make the trade-off on the experts' behalf in advance and also loses candidate diversity. The correct approach is to retain multiple candidate groups and leave the decision of "which one to choose" to after the evaluation interface.

Deliverables

A one-page evaluation plan template: metrics table (including original objectives and auxiliary metrics), flowchart (six-step closed loop), candidate generation rules, selection rules, output candidate set, and risk and uncertainty notes. Also record parameters, metric definitions, evaluation budget, random seeds, selection criteria, and elimination rationale, so that the next round of exploration can inherit them directly.

Class Activity Evaluation Plan Template

The figure is a one-page evaluation plan template for the class activity: centered on the two independent conflicting objectives of lift and drag, it lists the metrics table, six-step evaluation closed loop, candidate generation rules, selection rules, and output candidate set. It also requires declaring the evaluation budget and recording random seeds and elimination rationale, making the plan comparable, reproducible, and ready for expert decision-making.

Question 1

In this activity's evaluation plan, how should lift and drag be defined as optimization objectives?

Optimize only lift and leave drag for discussion when interpreting results

Combine them into a single lift-to-drag ratio score and directly maximize that ratio

Have experts specify one of the objectives on the fly after each round of evaluation

Set them as separate objectives: maximize lift coefficient and minimize drag coefficient

Question 2

Regarding how constraints and evaluation budget should be written, which of the following best meets the activity's requirements for the plan?

Automatically relax hard constraints when the budget is tight so the optimization process can finish

Write only the constraints and let the executor decide the budget on the spot

Treat the evaluation budget as an objective to be minimized and compare it alongside lift and drag

Set thickness and pitching moment range as hard constraints, with violation leading to elimination, and also specify the number of surrogate or simulation calls and the compute limit

Question 3

To make candidates cover different lift-drag trade-off regions, what should candidate generation rules most include?

Sample densely only in the high-lift region to ensure the output solutions look "good"

Generate completely at random with no preferences or parameter bounds

Use initial sampling to spread globally, then combine perturbation, crossover mutation, and preference guidance so candidates are distributed across multiple trade-off regions

Generate only one candidate per round and then fine-tune it step by step

Question 4

After the first round of evaluation ends, what should the solution comparison step most appropriately output?

Sort all candidate scores from high to low into a long list

Keep only the single solution with the highest weighted sum score

A set of non-dominated candidates covering different lift-drag trade-offs, along with the differences among them and the supporting rationale

Randomly draw one candidate to go directly into verification

Question 5

To enable the next round of exploration to inherit this round's results, what should least be missing from the archived records?

Record only the shape of the overall score curve

Record only the parameter values of the final selected solution

Parameter representation and bounds, metric definitions, evaluation budget, random seeds, selection criteria, and reasons for elimination

Record only the prediction accuracy of the surrogate model

Case: Review of a Multi-Objective Evaluation Plan for a Supercritical Airfoil

Read the scenario and answer the questions.

The team designed a multi-objective optimization evaluation plan for a supercritical airfoil: the objectives are to maximize the lift coefficient and minimize the drag coefficient; the hard constraints are that the relative thickness is not below the lower limit and the pitching moment coefficient falls within the allowable range; surrogate models replace most high-fidelity simulations, with the total evaluation budget set to N calls. After the first round, 40 candidate groups were obtained. The team ranked them by weighted sum and kept only the top-scoring airfoil for verification.

1. What problems does this plan have in terms of "candidate retention and diversity"? How would you modify the selection rules?

Answer:
The problem is that weighted-sum ranking keeps only a single highest score, which is equivalent to resolving the conflict between lift and drag in advance using a preselected set of weights. Pareto diversity is thereby lost, experts cannot see the trade-off range, and the basis for the weights themselves is not explained. A better direction: switch to Pareto non-dominance selection and retain several candidates on the front that cover different lift-drag trade-off regions as the candidate set; if weighted sum must be used, provide multiple weight sets or reference points to form a candidate set, record the basis for each weight set, and explain how diversity is maintained.

2. To ensure the second round of exploration can inherit the first round's results, what additional records and interface elements should the plan include?

Answer:
For records: parameter representation and bounds, metric definitions and units, the division between hard constraints and soft preferences, evaluation budget and actual number of calls, surrogate model version, random seeds, selection criteria, and reasons for elimination. For interfaces: each candidate should be accompanied by metric evidence (lift and drag values), constraint satisfaction, the surrogate model's uncertainty range, and an explanation of the differences among candidate groups, and experts should be provided with an interface for judging trade-offs (reference points or preference ranking), so the next round can adjust the sampling strategy and budget allocation based on these records.