BER sweeps and coding gain

What you will accomplish

A bit-error-rate curve you can defend, and a coding gain figure measured rather than quoted.

Most of this guide is about knowing which points in a sweep are measurements and which are artifacts of when the simulator gave up.

Prerequisites

Installation complete.

Steps

Sweep an uncoded chain


orbitforge waveform ber \
  --chain uncoded:qpsk \
  --esn0-db 0:2:14 \
  --json ber-qpsk.json


Chain uncoded:qpsk (rate 1.0000, 2 bits/symbol, seed 42)
 Es/N0 dB  Eb/N0 dB         bits     errors     frames          BER       theory
     0.00     -3.01        64000      10272         64    1.6050e-1     1.587e-1
     4.00      0.99        64000       3647         64    5.6984e-2     5.650e-2
     8.00      4.99        64000        402         64    6.2813e-3     6.004e-3
    10.00      6.99       128000        108        128    8.4375e-4     7.827e-4
    12.00      8.99      3072000        102       3072    3.3203e-5     3.430e-5
    14.00     10.99     10000000          6      10000    6.0000e-7     2.695e-7

Read the `frames` column before the `BER` column

Which points are measurements

A point stops when it reaches --min-errors (default 100) or --max-frames (default 10000), whichever comes first. The frames column tells you which.

Es/N0	Errors	Frames	Stopped because
0 to 8	402+	64	Hit 100 errors fast
12	102	3072	Hit 100 errors
14	6	10000	Hit the frame cap

The 14 dB point collected 6 errors, not 100. It is not a converged measurement; it is where the simulator ran out of budget.

Its BER reads 6.000e-7 against a theory of 2.695e-7, a factor of 2.2. That gap invites you to conclude something is wrong with the simulator.

Nothing is wrong with the simulator. Export the JSON and the answer is there:


{
  "esn0_db": 14.0,
  "bit_errors": 6,
  "ber": 6e-7,
  "ber_ci_low": 2.7498540988410946e-7,
  "ber_ci_high": 1.3091598636424195e-6,
  "theory_ber": 2.695148083031963e-7
}

The Wilson 95 percent interval spans a factor of 4.8. The console table does not print it. Always write the JSON and look at ber_ci_low and ber_ci_high before believing a point.

Settling it directly, at 1000000 frames and a 400-error target:


orbitforge waveform ber --chain uncoded:qpsk --esn0-db 14:1:14 \
  --max-frames 1000000 --min-errors 400 --json ber14.json


    14.00     10.99   1000000000        263    1000000    2.6300e-7     2.695e-7

263 errors gives 2.630e-7 against 2.695e-7, a ratio of 0.976, with theory comfortably inside the interval. The 2.2x discrepancy was the sample size, and that one point cost 99 seconds against 1.2 seconds for the entire original sweep.

Zero errors does not mean zero BER

Coded chains reach the floor much sooner:


orbitforge waveform ber --chain conv:qpsk --esn0-db 0:2:12 --json conv.json


     4.00      4.03      5760000        102       5760    1.7708e-5            -
     6.00      6.03     10000000          0      10000     0.0000e0            -
     8.00      8.03     10000000          0      10000     0.0000e0            -
    10.00     10.03     10000000          0      10000     0.0000e0            -
    12.00     12.03     10000000          0      10000     0.0000e0            -

Four points report 0.0000e0, and the JSON gives all four the same upper bound, 3.841e-7, because all four observed zero errors in the same 10^7 bits.

The run therefore contains no evidence that 12 dB is better than 6 dB. The only defensible statement is “BER below 3.8e-7 at 95 percent confidence” for each, which is a statement about the budget, not the waveform.

Note also that theory is - for every coded chain. There is no closed form, so the cross-check available on the uncoded curve is gone precisely where the sampling is hardest.

The interval is too narrow for coded chains

Two runs of the same RS point at the same Es/N0, differing only in the sweep they were part of:

Run	Bit errors	Frame errors	BER	95% interval
`8:0.5:10`	102	6	1.787e-5	[1.472e-5, 2.169e-5]
`8.5:0.1:9.0`	109	7	3.819e-5	[3.166e-5, 4.606e-5]

The two intervals are disjoint, yet both estimate the same quantity.

The interval is computed as wilson_interval(bit_errors, bits), which treats every bit as an independent trial. For an uncoded chain that is exactly right. For a block code it is not.

RS(255,223) corrects up to 16 symbol errors per codeword. Below that, zero bit errors escape; above it, the decoder fails and releases a burst. Here each failed frame produced about 16 bit errors, so ~100 bit errors represent only 6 or 7 independent events.

The honest uncertainty scales with the number of frame errors, roughly 1 over the square root of 6, near 40 percent, not the 10 percent the bit-level interval implies.

For coded chains, judge convergence on frame_errors and fer, and treat the bit-level interval as optimistic.

Reproducibility keys on the whole sweep, not the seed

The same Es/N0 and the same seed give different results depending on which other points were in the sweep:

Sweep	Point index of 14 dB	Errors	BER
`0:2:14`	7	6	6.0e-7
`10:2:14`	2	2	2.0e-7
`14:1:14`	0	1	1.0e-7

Each is perfectly repeatable. Re-running any of them gives the identical number.

This is by design, not a defect. Per-frame streams are derived from (seed, point_index, frame_index) with a SplitMix64 mix, so results are independent of thread count and scheduling.

The consequence is that point_index is the position in the sweep, not the Es/N0 value. Reproducing a result requires recording the entire --esn0-db specification alongside the seed.

The practical rule: never compare a point lifted from one sweep against the same Es/N0 in another and attribute the difference to anything but sampling.

Measuring coding gain

Coding gain must be read on Eb/N0, energy per information bit, not Es/N0. The tool prints both, and the offset between them is the code rate.

Chain	Rate	Eb/N0 at BER 1e-5	Gain vs uncoded	Bandwidth cost
Uncoded QPSK (theory)	1.0000	9.59 dB	reference	1.00x
RS(255,223)	0.8745	about 6.15 dB	about 3.4 dB	1.14x
Conv K=7 r=1/2	0.4970	about 4.20 dB	about 5.4 dB	2.01x

Compared at equal Es/N0 the convolutional code looks far better than this, because rate 1/2 spends twice the symbols for the same information. That comparison charges the code nothing for the bandwidth it consumes.

On Eb/N0 the two codes are much closer, and the choice becomes what it really is: 5.4 dB for double the bandwidth, or 3.4 dB for 14 percent more.

Both coded figures are interpolated between measured points and carry the frame-clustering uncertainty described above. Treat them as good to a few tenths of a dB, and re-measure at the specific rate you intend to fly rather than quoting this table.

Cost, and where to spend it

Run	Wall clock
8-point uncoded sweep, defaults	1.2 s
One converged point at 1e-7	99 s

Sweep cheaply to find the waterfall, then spend frames only on the two or three points you intend to quote. Raising --max-frames across a whole sweep wastes almost all of it on points that already reached 100 errors in 64 frames.

Checklist

Did you check frames against --max-frames for every point you quote?
Did you export JSON and read the confidence interval?
For coded chains, did you judge convergence on frame errors rather than bit errors?
Are you reporting zero-error points as an upper bound rather than as zero?
Is coding gain quoted on Eb/N0, with the bandwidth cost stated?
Did you record the full sweep specification, not just the seed?

Next steps

waveform ber reference for the full chain syntax.
Measured MODCOD thresholds for turning this into a link budget input.

Question? Give us feedbackDocuments Varaha Constellation Designer main (pre-release)