What is CMA-ES and why does MatCraft use it?

CMA-ES (Covariance Matrix Adaptation Evolution Strategy) is a derivative-free optimization algorithm that excels at solving non-convex, noisy, and multi-modal optimization problems. It is the default optimizer in MatCraft and is particularly well-suited for materials optimization.

How CMA-ES Works

CMA-ES maintains a multivariate normal distribution over the search space and iteratively updates it to concentrate probability mass around promising regions:

1. Sample: Generate a population of candidate solutions from the current distribution.
2. Evaluate: Score each candidate using the surrogate model (or a physics evaluator).
3. Select: Rank candidates by fitness and select the top performers.
4. Update: Adapt the mean and covariance matrix of the distribution based on the selected candidates. The covariance matrix captures correlations between parameters, enabling the algorithm to learn the "shape" of the fitness landscape.

Why CMA-ES for Materials

• No gradients required: Materials objectives are often non-differentiable (e.g., outputs from simulations or physical experiments). CMA-ES works purely from function evaluations.
• Handles correlations: Material components are frequently correlated (e.g., increasing one filler reduces the fraction available for another). The covariance matrix naturally captures these relationships.
• Noise-tolerant: Experimental measurements are inherently noisy. CMA-ES uses population-based sampling that smooths out noise.
• Self-adapting step size: The algorithm automatically adjusts its exploration radius — searching broadly early on and narrowing as it converges.

Configuration in MatCraft

optimizer:
  type: cma-es
  sigma0: 0.3          # Initial step size (fraction of parameter range)
  population_size: 20  # Candidates per generation (default: 4 + 3*ln(n))
  max_generations: 100 # Safety limit per iteration

sigma0 is the most important hyperparameter. A value of 0.3 means the initial search covers roughly 30% of each parameter's range. Start with 0.3 and decrease to 0.1 if you have a good prior on where the optimum lies.

Comparison to Alternatives

CMA-ES outperforms random search, grid search, and simple genetic algorithms on problems with 3-50 parameters. For very high-dimensional problems (>100 parameters), consider the separable CMA-ES variant (type: sep-cma-es) which scales more efficiently.