matlab-optimize-gpu-codegen skill

Optimize MATLAB for GPU Code Generation

Iteratively optimize a MATLAB design file for GPU Coder: compile, benchmark, apply structural optimizations, profile with gpuPerformanceAnalyzer, fix diagnostics, and verify numerical equivalence at every step.

When to Use

• User has a MATLAB function and wants faster GPU MEX or CUDA code
• User mentions GPU Coder, codegen, CUDA, gpuPerformanceAnalyzer
• User asks to profile generated GPU/CUDA code or GPU MEX (profiling generated GPU code is the entry point to this skill's diagnostic-fix workflow)
• User wants to reduce GPU memory, improve kernel parallelism, or fix Performance Analyzer diagnostics
• User has a .m design file and representative inputs

When NOT to Use

• Workflows with no codegen
• coder.gpuConfig("exe") — standalone executables cannot be benchmarked or equivalence-checked from MATLAB. Suggest the user switch to mex or lib/dll and regenerate exe from the final optimized source.
• Simulink GPU code generation
• Writing new MATLAB functions from scratch (this skill optimizes existing code)
• Optimizing helper functions called by the design file — this skill optimizes the main design file only
• Hardware setup or CUDA toolkit installation
• General MATLAB performance tuning without GPU involvement
• The user's prompt does not mention GPU, codegen, CUDA, MEX, or profiling. Activation must be driven by the user's prompt alone — do not infer GPU intent from filenames or function contents. If unsure, ask before activating.

Workflow

Setup — Create Session Directory

All codegen artifacts, profiling outputs, and optimized versions go in a single temp directory for the entire session. pwd must be sessionDir for every codegen/benchmark/PA call — otherwise compiled MEX and SIL binaries land in the user's working directory.

Two helpers this skill calls — benchmarkMex and extractDiagnostics — live in the scripts/ subfolder of this skill (the folder containing this SKILL.md). A MATLAB function is only callable by name when its folder is on the path, so add scripts/ to the path in Setup and remove it in the cleanup. Then call the helpers by bare name (benchmarkMex(...), extractDiagnostics(...)).

% TEMPLATE — not executable
sessionDir = fullfile(tempdir, "gpu_opt_" + string(datetime("now", Format="yyyyMMdd_HHmmss")));
mkdir(sessionDir);
scriptsDir = fullfile("<skill_dir>", "scripts");   % this skill's scripts/ folder (absolute)
addedDir = fileparts(which("<designFile>"));
addpath(scriptsDir, addedDir);
oldDir = cd(sessionDir);
cleanupCd = onCleanup(@() (cd(oldDir), rmpath(scriptsDir), rmpath(addedDir)));   %#ok<NASGU>

Write all artifacts to sessionDir only — never to the user's working directory.

Step 1 — Codegen on Original

Run codegen on the unmodified design file to discover what actually fails. Do NOT guess which functions are unsupported — let the compiler tell you.

1a. Pick the codegen config. Use the config the user provides. If none specified, default to MEX:

cfg = coder.gpuConfig("mex");  % default — replace if user specifies a config

Supported targets: mex, lib, dll. The exe target is not supported by this skill, so Steps 2–5 cannot benchmark or verify equivalence. If the user provides coder.gpuConfig("exe"), stop and ask them to either:

• switch to mex for the optimization workflow (recommended — fastest iteration), or
• switch to lib/dll if they need a deployable artifact (the skill will enable SIL to benchmark via a generated MEX).

Once optimization is complete, the user can regenerate with exe from the final optimized source.

1b. For lib/dll configs: enable SIL. This is mandatory — without SIL there is no callable MEX, so Steps 2–5 cannot benchmark. Set this before calling codegen:

% TEMPLATE — not executable
if ~isa(cfg, 'coder.MexCodeConfig')
    cfg.VerificationMode = 'SIL';   % required for benchmarking lib/dll targets
end

If SIL fails or is unavailable (e.g., Embedded Coder license missing), report this and stop — do not silently skip benchmarking.

1c. Resolve inputs. If the user provided concrete input values, use them as-is. If the user provided only types/sizes (e.g., "two double vectors of size 1024x1"), synthesize inputs matching the spec — record exactly what you generated (type, size, location, generator) so the Final Report can list it. A reasonable default is randn with a fixed rng seed for floats, randi for integers, rand > 0.5 for logicals; keep inputs on the CPU unless the user said otherwise or PA later flags UseGpuInput. If the user gave neither values nor types/sizes, ask for representative sizes and types — codegen -args needs a concrete signature and the input shape drives which optimizations win.

1d. Run codegen:

% TEMPLATE — not executable
codegen -config cfg <designFile> -args {<inputs>}

If codegen fails, read the errors and fix only what is reported as unsupported. Save the fixed file as <designFile>_v1.m in sessionDir.

v1 must successfully codegen. Re-run codegen until it passes. This produces the baseline MEX: <designFile>_v1_mex (for lib/dll configs, the SIL-generated MEX has the same name and is callable identically).

Step 2 — Baseline Benchmark

Use the MEX generated in Step 1 directly — do not re-codegen. Benchmark with the convergence-based helper:

% TEMPLATE — not executable
baseline = benchmarkMex("<designFile>_v1_mex", {<inputs>});
baselineTime = baseline.MedianTime;
fprintf("Baseline: %.4f ms\n", baselineTime*1000);

Rules:

• Always use benchmarkMex (or gputimeit) — never use tic/toc for GPU timing (GPU ops are async)
• benchmarkMex handles warmup and convergence automatically
• Record baselineTime — all improvements are measured against this

Step 3 — Structural Optimization Loop

Apply optimizations iteratively. Each iteration:

1. Create <designFile>_v<N>.m in sessionDir with the next optimization
2. Verify numerical equivalence against the original (multi-input — see Step 3b)
3. Run codegen to sessionDir — if it fails, fix or revert
4. Benchmark the new MEX with benchmarkMex — compare against best so far
5. Keep the fastest passing version as the current best

MATLAB-level restructuring patterns (rewrites that improve codegen regardless of which GPU primitive you eventually choose):

Pattern in Source	Optimization	Effect on Generated CUDA
Array-valued expression inside a loop that does not depend on the loop variable (e.g., w = weights / sum(weights) recomputed every iteration)	Hoist the expression above the loop	Eliminates redundant per-thread computation. GPU Coder cannot always prove loop-invariance for array expressions — it inlines them into the kernel, so every thread recomputes the same work.
Implicit expansion could replace an explicit loop (e.g., for i=1:N, out(i,:) = A(i,:) + B; end where B is a row vector)	Replace the loop with out = A + B using implicit expansion	GPU Coder generates a single fused kernel for implicit expansion. The explicit loop may also parallelize, but implicit expansion produces cleaner kernels with no loop overhead.
Divergent branching inside a parallelizable loop (e.g., if x(i) > 0, a = f(x(i)); else, a = g(x(i)); end where the condition varies unpredictably across elements)	Where both branches are cheap, compute both and select with a mask (e.g., a = mask.*f(x) + (~mask).*g(x))	Divergent if/else causes warp divergence — threads in the same warp serialize across branches. A branchless mask keeps every thread on the same instruction path, restoring full warp throughput.

The table above covers MATLAB-level restructuring only; it does not enumerate GPU Coder primitives. Before committing to any optimization, read references/gpu-codegen-functions.md end-to-end — it documents kernel pragmas, parallel reductions and scans, atomics, stencils, and memory placement. The right primitive depends on the loop's data-flow shape (where the result lives, how dependencies chain, whether iterations collide); the closest-looking table row is often not the right answer. Match semantics to the loop, not surface appearance.

Exit criteria for this loop:

• Performance target met (if user specified one)
• No more structural optimizations the agent can identify
• Maximum 5 structural iterations (Step 5's diagnostic-fix loop has its own separate cap of 5 — the two are independent)

Step 3b — Verify Numerical Equivalence

Run both original and optimized on up to 5 input sets. Use the user's original inputs as the baseline, then generate variants matching the same types and sizes. Match the generator to the input type (randn for float, randi for integer, rand > 0.5 for logical) and keep values in the type's valid range.

% TEMPLATE — not executable — float case; adapt generator to the input type
% Input sets — variants are built from in1, the first input resolved in Step 1c
in1 = <user_or_synthesized_input_from_step_1c>;     % original example
rng(42); in3 = randn(size(in1), 'like', in1);       % random
rng(99); in4 = randn(size(in1), 'like', in1) * 1e6; % large magnitude

Include an edge-case input only where it is valid for this function — an all-zeros set exercises little and is degenerate if the function divides by something derived from the input. Pick a variant that meaningfully stresses the function instead.

Compare each output independently:

% TEMPLATE — not executable
outOrig = <designFile>(testInput);
outOpt  = <designFile>_v<N>(testInput);

% Gather gpuArray outputs to CPU for comparison
if isa(outOpt, 'gpuArray'), outOpt = gather(outOpt); end

% If outputs can contain NaN, first check NaN positions match (see NaN rule
% below) — max() ignores NaN, so a corrupted NaN would otherwise pass silently.
denom = max(abs(outOrig(:)));
if denom == 0
    relErr = max(abs(outOpt(:)));  % absolute error when expected is zero
else
    relErr = max(abs(outOrig(:) - outOpt(:))) / denom;
end
fprintf("relErr = %.2e\n", relErr);

Tolerance by output type:

Output Type	Tolerance
double	1e-6
single	1e-3
half	1e-2
Integer / logical	Exact (isequal)

Rules:

• Always compare against the original unmodified function, not the previous version
• If the function has multiple outputs, check each with its own type-appropriate tolerance
• If output types differ between original and optimized, that's a bug — fix it
• Normalize relative error by max(abs(expected)), not element-wise
• Handle NaN: check NaN positions match (isequal(isnan(a), isnan(b))), then compare non-NaN elements
• gpuArray wrapping of inputs is allowed (e.g., after UseGpuInput fix) — it doesn't change the function contract

Step 4 — Profile with gpuPerformanceAnalyzer

Always run this step unless the user specified a performance target and it was met in Step 3. PA reveals issues (memory copies, low parallelism) that benchmarks alone cannot detect.

Run PA on the current best version to find remaining bottlenecks. Pass the same cfg from Step 1 so PA profiles the same target the user is optimizing for:

% TEMPLATE — not executable
paDir = fullfile(sessionDir, "pa_v<N>");
mkdir(paDir);

gpuPerformanceAnalyzer("<designFile>_v<N>", {<inputs>}, ...
    Config=cfg, ...
    OutFolder=paDir, ...
    LaunchReport=false);

Key NVPs:

• Config=cfg — reuse the codegen config from Step 1 (user-provided or the default coder.gpuConfig("mex"))
• LaunchReport=false — suppresses GUI report
• OutFolder — subdirectory within session dir

Extract diagnostics from the mldatx report:

% TEMPLATE — not executable
mldatxFile = fullfile(paDir, "html", "gpuProfiler.mldatx");
results = extractDiagnostics(mldatxFile);

If results.numDiagnostics == 0, the profile is clean — skip to the Final Report.

The results struct contains:

• results.numDiagnostics — count of issues found
• results.diagnostics(i).id — namespaced ID (e.g., "gpucoder:diagnostic:UseGpuInput")
• results.diagnostics(i).message — human-readable description
• results.diagnostics(i).file — path to the generated .cu file
• results.diagnostics(i).line — line number in the generated code

Step 5 — Fix Diagnostics

Read results.diagnostics and apply targeted fixes. Each fix iteration:

1. Create <designFile>_v<N+1>.m in sessionDir
2. Run codegen — if it fails, fix or revert
3. Verify equivalence against the original (same procedure as Step 3b — same inputs, same tolerances, compare against the unmodified function)
4. Benchmark with benchmarkMex — compare against current best

Diagnostic → Fix Mapping:

Diagnostic ID	Fix
gpucoder:diagnostic:NoKernelFunPragma	The named function has no kernel pragma, so no kernels are generated. Add coder.gpu.kernelfun to the named function.
gpucoder:diagnostic:LargeLocalMemoryUsagePerThread	The kernel uses a large amount of per-thread local memory. Simplify expressions inside the loop body: split compound expressions into fewer steps, extract large temporary arrays outside the loop, or reduce the number of intermediate variables to lower per-thread register/local memory pressure.
gpucoder:diagnostic:UseGpuInput	The flagged input is first used on the GPU, forcing a CPU→GPU memory copy. Convert only the flagged input to gpuArray before passing to codegen/PA. Do NOT speculatively convert other inputs.
gpucoder:diagnostic:KernelDiagnosticsNotEnoughParallelism	The kernel launches too few threads — each thread performs extensive work, so parallelism is low. Increase the number of independent iterations: reshape the computation so the parallelized loop covers more elements, split serial multi-step work into separate vectorized operations, or tile the data into more independent chunks.
gpucoder:diagnostic:LongCPULoopHasLargeMemcpy	A CPU-bound loop forces GPU-CPU synchronization on each pass. Restructure so GPU results are not read back to CPU within the loop. Keep intermediates on GPU until the loop completes. If the loop body launches kernels, parallelize the outer loop or merge it with the inner GPU work.
gpucoder:diagnostic:LongRunningLoopLargeIteration	A high-iteration loop dominates runtime and is not parallelized. Vectorize the loop body or restructure so iterations are independent (no loop-carried dependencies). If truly independent, coder.gpu.kernelfun or coder.gpu.kernel will map it to a kernel.
gpucoder:diagnostic:KernelLaunchOverheadLargeInLoop	Parallelized inner loops launch many trivial kernels with high overhead. Parallelize the outer (parent) loop instead of the inner loops. This fuses many small kernel launches into fewer, larger kernels. Use coder.gpu.kernel on the outer loop if the compiler won't auto-parallelize it.
gpucoder:diagnostic:MemoryDiagnosticsRepeatedMemoryCopyInsideLoop	A variable is copied between CPU and GPU every iteration, forcing synchronization. Move the variable's allocation and initialization outside the loop. If the variable is written on GPU and read on CPU (or vice versa) each iteration, restructure so it stays on one device for the entire loop duration.
gpucoder:diagnostic:LongRunningLoopUnknownReason	A CPU-bound loop contributes significant time for an undetermined reason. Investigate the loop body for serial dependencies. If independent: vectorize or add coder.gpu.kernelfun. If dependencies exist: check whether the dependent part can be separated from an independent part. If the loop calls functions that launch kernels internally, that prevents outer-loop parallelization — inline or restructure those calls.

Iteration cap: Stop after 5 diagnostic-fix rounds (separate from Step 3's structural-iteration cap). Report remaining issues if any cannot be resolved.

Step 6 — Final Report

Present the best result:

# GPU Codegen Optimization Results

## Files
- Original: <designFile>.m
- Best version: <designFile>_v<N>.m
- Session directory: <sessionDir>

## Inputs Used
| Arg | Type | Size | Location | Source |
|-----|------|------|----------|--------|
| in1 | double | 1024x1 | CPU | user-provided / synthesized via `randn(1024,1)` with `rng(0)` |
| ... | ... | ... | ... | ... |

State "user-provided" when the user gave concrete values, or describe the
generator (function + seed) when inputs were synthesized. Reported timings
below are reproducible only with these inputs.

## Performance
| Version | Time (ms) | vs Baseline |
|---------|-----------|-------------|
| Baseline | X.XX | — |
| v2  | X.XX | Y% faster |
| v<N> | X.XX | Z% faster |

## Diagnostics Resolved
| Diagnostic | Status |
|---|---|
| <issue> | RESOLVED |
| <issue> | OPEN (reason) |

## Changes Applied
1. <change description>
2. <change description>

## Numerical Verification
All versions verified equivalent (max relErr below the type-appropriate
tolerance) across N test inputs.

The final deliverable is <designFile>_v<N>.m — the best-performing version that passes equivalence checks.

Conventions

• Never overwrite the original design file — always create _v<N> variants
• All artifacts go in a single session tempdir, never in the user's working directory
• Use benchmarkMex (or gputimeit) for all timing — never use tic/toc for GPU code
• Run codegen first to discover errors — never guess function support
• Verify equivalence with multiple diverse inputs after every change
• Use user-provided codegen config; default to coder.gpuConfig("mex") if unspecified
• Iterate: optimize → benchmark → repeat — stopping after one pass leaves performance on the table
• Only convert inputs to gpuArray where profiling shows a CPU→GPU copy — do not speculatively convert all inputs

---

Copyright 2026 The MathWorks, Inc.

---