Methodology for empirical evaluation of the avalanche effect, dissimilarity metrics, and ranking of key candidates in a training SPN network

BY KEYHUNTER 10.09.2026

Methodology for empirical evaluation of the avalanche effect, dissimilarity metrics, and ranking of key candidates in a training SPN network

Theoretical and practical article based on  notebook materials AI_Accelerated_Cryptanalysis_Safe_Empirical_Evaluation.ipynb

Annotation

This paper documents a computational experiment from  a Google Colab  notebook designed to securely and reproducibly study diffusion in symmetric encryption, the Hamming distance, the iterative 3-summary descriptor, and key candidate ranking procedures. The experiment uses only a custom-built 16-bit substitution-permutation network (SPN). The small block and key sizes are chosen for clarity and full controllability of the computations; the design is not a cryptographic standard and is strictly not intended for data security. The central methodological result is that avalanche measurements and ranking metrics alone do not constitute proof of security or vulnerability: scientifically sound inference requires a clear threat model, the absence of information leakage, independent training and test data, comparison with a random baseline, and reproducible experimental parameters.

1. Context, subject and purpose

This notebook  is intended as a practical accompaniment to the discussion of AI-accelerated keyspace analysis: instead of treating incorrect decryptions as completely useless, it is proposed to check whether they bear statistical features suitable for ranking candidates. In Gideon Samid’s original paper, this idea is described as a possible “de-flattening” of the probability curve in the space of untested keys. This is a research hypothesis, not a proven universal method for breaking modern cryptosystems.

The aim of the work  is to demonstrate a correct minimal protocol for experimental verification: to construct a transparent toy model, verify the reversibility of the implementation, measure local diffusion when flipping key bits, calculate compact features of difference, and evaluate the quality of the sorting procedure for key candidates relative to random order.

Ethical and technical boundaries.  The  notebook  does not contain code for brute-forcing  AES keys , ChaCha20 keys,  Bitcoin keys , cryptowallets, passwords, real intercepted messages, or any unauthorized systems. All logic applies only to the local 16-bit training system.

2. Theoretical foundations

2.1 Uniform candidate space

In the absence of discriminatory information and with  m  candidates excluded from a space of size  n,  the uniform search prior model assigns the same probability to each remaining candidate:

Pr(K i  | candidate not excluded) = 1 / (n − m).

A claim of useful ranking is meaningful only when the scoring function demonstrates statistically significant superiority over this neutral model on an independent test sample. In  the notebook,  this principle is reflected through the top-1, top-5, average rank, and Mean Reciprocal Rank (MRR).

2.2. Avalanche effect

For a pair of key bit i and ciphertext bit j, the program estimates the frequency of change of the j-th output bit after inverting the i-th  key bit . For N trials, the estimate is:

p̂ i,j  = (1 / N) · Σ t=1 N  𝟙[bit j (E(P t ,K t )) ≠ bit j (E(P t ,K t  ⊕ 2 i ))].

A p̂ i,j approaching  0.5 means that in a given sample and for a given measurement, the output bit changed in approximately half of the observations. This is consistent with good local diffusion, but does not prove cryptographic security. The strong avalanche criterion refers to stronger requirements on the distribution of dependencies, rather than a single average value.

2.3 Hamming distance and 3-summary

The Hamming distance between two binary words x and y of the same length is the number of distinct positions:

d H (x,y) = wt(x ⊕ y),

where wt is the Hamming weight. The iterative 3-summary descriptor splits the bit string into non-overlapping triplets and replaces each triplet with its majority bit. This operation is repeated until the length allows for further triplets or until a specified depth limit is reached. This is an illustrative multi-scale feature, not a standardized cryptographic metric.

3. Runtime in  Google Colab

  1. Open  Google Colab  and select  File → Upload  notebook .
  2. Please upload the correct file  AI_Accelerated_Cryptanalysis_Safe_Empirical_Evaluation.ipynb.
  3. Run cells strictly from top to bottom:  Runtime → Run all  or using the run button to the left of each cell.
  4. Do not change  SEEDif exact reproduction of the observed sequence of pseudo-random trials is required.
  5. To investigate statistical stability, increase  N_AVALANCHE, documenting the new value and run time.

The notebook  uses NumPy for generating and vector arrays, Pandas for tabular reporting, and Matplotlib for visualization. These packages are typically available in the standard Colab environment.

4. Step-by-step analysis of cells

Markdown Cell: Research Purpose

The first text box defines the scientific boundaries: avalanche behavior, candidate statistics, distances, and the benchmark are analyzed. It explicitly prohibits interpreting the experiment as an attack on  AES , ChaCha20,  Bitcoin  , or current protocols. This is an important condition for the correct publication interpretation of the results.

Markdown Cell: Execution Order

The second text cell specifies that the blocks should be run from top to bottom and specifies the role of the seed: repeatability is necessary so that another researcher can obtain the same test generation sequence.

Cell 1: Imports and Reproducibility

import random
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

SEED = 260508690
rng = np.random.default_rng(SEED)
random.seed(SEED)
print('Reproducibility seed:', SEED)

Here, libraries are imported and two pseudorandom number generators are created: the modern NumPy Generator and the standard Python module’s generator. A fixed seed  260508690 ensures deterministic results within the specified library versions. This is not a cryptographic entropy source: it is intentionally chosen for reproducibility.

Markdown Cell: Defining a Tutorial Cipher

The cell warns that the model has a 16-bit block and a 16-bit key. Therefore, its keyspace contains only 2 x 16  = 65,536 values ​​and is fundamentally unsuitable for information security. Its sole purpose is to facilitate research demonstrations.

Cell 2: Implementation of a reversible SPN

The code defines a 4-bit S-box permutation  SBOXand then computes its inverse table  INV_SBOX. The array  PERM describes a bijective permutation of 16 positions and  INV_PERM is constructed as its inverse. The constant  MASK16 limits the arithmetic to 16 bits and  ROUNDS = 4 determines the number of rounds.

def toy_encrypt(plaintext, key):
    state = plaintext & MASK16
    for round_index in range(ROUNDS):
        state ^= round_key(key, round_index)
        state = substitute_nibbles(state, SBOX)
        state = permute_bits(state, PERM)
    return state ^ round_key(key, ROUNDS)


def toy_decrypt(ciphertext, key):
    state = (ciphertext ^ round_key(key, ROUNDS)) & MASK16
    for round_index in reversed(range(ROUNDS)):
        state = permute_bits(state, INV_PERM)
        state = substitute_nibbles(state, INV_SBOX)
        state ^= round_key(key, round_index)
    return state & MASK16

In the forward direction, each round sequentially performs XOR with the round key, a nonlinear substitution over 4-bit tetrads, and a bit permutation. Decryption applies the operations in reverse order, using the inverse S-box and permutation. The function  round_key generates a training round key by cyclically shifting 16 bits and XORing with a deterministic constant. This is a transparent demonstration key schedule, not a proven key unwrapping scheme.

Cell 3: Check for reversibility

for _ in range(1000):
    plaintext = int(rng.integers(0, 2**16))
    key = int(rng.integers(0, 2**16))
    assert toy_decrypt(toy_encrypt(plaintext, key), key) == plaintext
print('Inverse-property test passed for 1,000 independent trials.')

The cell runs 1,000 independent tests. In each iteration, plaintext and a key are generated, and then their identity is verified:

D K (E K (P)) = P.

The operator  assert will immediately terminate with an error if the property is violated even once. A successful message only proves the absence of an error on the 1,000 pairs tested; a formal proof of the reversibility of all 232 message  -key pairs is not provided.

Markdown Cell: Setting Up an Avalanche Experiment

The text clarifies the operational definition: the effect of changing a single  key bit  on a single  ciphertext bit is examined . This is not the same as a full evaluation of the SAC for all inputs and all internal components, so the output is limited to the specified measurement.

Cell 4: 16×16 avalanche matrix

N_AVALANCHE = 2000
avalanche = np.zeros((16, 16), dtype=float)

for key_bit in range(16):
    output_flips = np.zeros(16, dtype=np.int64)
    for _ in range(N_AVALANCHE):
        plaintext = int(rng.integers(0, 2**16))
        key = int(rng.integers(0, 2**16))
        difference = toy_encrypt(plaintext, key) ^ toy_encrypt(plaintext, key ^ (1 << key_bit))
        for output_bit in range(16):
            output_flips[output_bit] += (difference >> output_bit) & 1
    avalanche[key_bit] = output_flips / N_AVALANCHE

For each of the 16 key bits, 2,000 trials are performed. The program encrypts the same plaintext with the original key K and with K ⊕ 2 i . XORing the two results selects the changed output bits. After normalization, a 16 × 16 probability matrix is ​​obtained. The pandas table reports the mean, minimum, and maximum per row, and the heatmap displays all the relationships “input key bit → output ciphertext bit.”

How to interpret the output:  A value of approximately 0.5 in a heatmap cell indicates approximately half of the switchings in the sample. A low mean absolute deviation  mean(abs(avalanche - 0.5)) indicates behavior closer to 0.5 in this configuration, but does not measure resistance to linear, differential, algebraic, side-channel, or other cryptanalytic methods.

Markdown Cell: Explaining Avalanche Results

This cell correctly captures three limitations: the probability of 0.5 is selective, the biases are toy-implementation specific, and increasing the number of trials improves the accuracy of the estimate but does not turn it into a proof of safety.

Cell 5: Hamming distance and iterative 3-summary

def hamming_distance(left, right):
    return (int(left) ^ int(right)).bit_count()


def summary3(bit_string):
    groups = [bit_string[index:index + 3] for index in range(0, len(bit_string) - 2, 3)]
    return ''.join('1' if group.count('1') >= 2 else '0' for group in groups)


def iterative_summary3(bit_string, max_depth=5):
    levels = [bit_string]
    current = bit_string
    for _ in range(max_depth):
        if len(current) < 3:
            break
        current = summary3(current)
        levels.append(current)
    return levels

hamming_distance Implements the formula using XOR and built-in one-bit counting  bit_count(). The function  bits16 displays the value as a 16-bit string.  summary3 It forms groups of length 3, ignoring remainders of length 1 or 2, and outputs “1” for two or three ones, otherwise “0.” The function  iterative_summary3 preserves the original string and all subsequent reduced levels.

The demonstration uses A = 43,690 =  1010101010101010 and B = 43,691 =  1010101010101011. Their Hamming distance is 1, since only the least significant bit is changed. The 3-summary sequences show how the local binary structure aggregates at increasingly coarser scales.

Markdown Cell: Controlled Ranking

The notebook  clearly formulates the critical limitation of the following test: the scorer receives a known synthetic plaintext. Therefore, this is not a cryptanalytic attack, but a test of the correctness of the ranking code and metrics. This section is especially important for preventing false positives: knowledge of the plaintext makes identifying the correct key trivial.

Cell 6: Key Candidate Benchmark

N_TESTS = 250
CANDIDATES_PER_TEST = 64
ranks = []

for _ in range(N_TESTS):
    plaintext = int(rng.integers(0, 2**16))
    true_key = int(rng.integers(0, 2**16))
    ciphertext = toy_encrypt(plaintext, true_key)
    # Form 63 unique wrong keys and add the true key.
    ...
    candidate_plaintext = toy_decrypt(ciphertext, candidate_key)
    score = -hamming_distance(candidate_plaintext, plaintext)
    ...

For each of the 250 tests, a random triple (plaintext, true_key, ciphertext) is generated, then a set of 64 keys is formed: one correct and 63 different invalid ones. Each key decrypts the ciphertext. The scoring function is equal to the negative Hamming distance between the candidate_plaintext and the original plaintext:

score(K′) = −d H (D K′ (C), P).

The closer the candidate decryption is to the known P, the higher the score. For the correct key, D K (C) = P and score = 0—the maximum possible value. Hence, the expected rank = 1. This is  a sanity check : it confirms that candidate generation, sorting, and metric calculation are working, but it does not demonstrate the intelligence of the model or reveal the key without a known message.

MetricsMeaningExpected in the current benchmark
Mean rankThe middle position of the correct keyClose to 1
Median rankMedian position1
Top-1 successThe proportion of cases where the correct key is firstClose to 1.0
Top 5 successesPercentage of top five finishesClose to 1.0
MRRAverage of values ​​1/rankClose to 1.0

For a random ranking of 64 equally likely candidates, the top-1 baseline is 1/64 = 0.015625, the top-5 baseline is 5/64 = 0.078125. The expected MRR is calculated by the program as the average of 1/r values ​​for r from 1 to 64. Comparison with these baselines is mandatory in any nontrivial experiment.

5. Achievements, evidence and limitations

Results achieved

  • A reproducible training SPN with inverse transformation is implemented.
  • An empirical test of the equality D K (E K (P)) = P was carried out on 1,000 random pairs.
  • A protocol for evaluating 256 local “key bit – ciphertext bit” dependencies was constructed for 2,000 trials for each matrix row.
  • Hamming distance and iterative 3-summary descriptor are implemented.
  • Implemented rank, top-k and MRR with explicit random baselines.

What is the evidence?

Successful execution of cell 3 provides computational evidence of the correctness of reversibility on the tested pairs. The table and heatmap of cell 4 provide observed estimates of switching frequencies for the given seed, N, number of rounds, S-box, and permutation. Cell 6 proves the correctness of the sorting logic under the intentionally provided plaintext.

What is not proven

  • The toy cipher has not been proven to be secure because the avalanche metric does not cover the full range of attacks.
  • AES , ChaCha20, NIST PQC,  Bitcoin  , or any real-world systems have not been proven vulnerable  .
  • No AI training shown: the current scorer is a deterministic function with known plaintext, not a neural network or hidden statistical distinguisher.
  • No attack has been demonstrated under ciphertext-only or realistic known-plaintext conditions.

6. Recommendations for research development

  1. First, formulate a threat model: ciphertext-only, known-plaintext, or chosen-plaintext; do not mix them.
  2. Exclude plaintext from features and scoring if the goal is to test non-trivial ranking.
  3. Before launch, fix the model architecture, feature set, data partitioning, seed, metrics, and success criteria.
  4. Separate training, validation, and test sets into independent keys and messages.
  5. Add permutation testing and bootstrap confidence intervals rather than relying on a single run.
  6. Compare the result with several baselines: random order, simple statistical features and feature ablations.
  7. Use published reduced-round challenge sets only with explicit permission and strictly specify the number of rounds.
  8. Do not transfer the results of the toy model to modern primitives without a full-fledged independent  cryptanalysis .

7. Conclusion

The Google Colab  notebook reviewed here   represents a neat educational framework for empirical cryptographic methodology. Its main value lies not in “breaking” itself, but in the reproducible separation of concepts: algorithm reversibility, diffusion, bit metrics, ranking quality, and correct comparison with the null hypothesis. The strong score in the final ranking benchmark is explained by access to the reference plaintext and is therefore expected; this is a verification of the implementation, not evidence of AI cryptanalysis. Any further scientific claim must be supported by independent test data, leak control, statistical analysis, and a correctly defined adversary model.

References — recommended materials

  1. Samid, G.  AI-Accelerated Brute Force Cryptanalysis . arXiv:2605.08690, 2026. The original conceptual paper to which  the notebook is thematically related .
  2. Shannon, C.E.  Communication Theory of Secrecy Systems . Bell System Technical Journal, 28(4), 656–715, 1949. A fundamental source on secrecy theory, key uncertainty, and the information approach to cryptography.
  3. Webster, A. F.; Tavares, S. E.  On the Design of S-Boxes . Advances in Cryptology — CRYPTO ’85, 1986. A classic work related to avalanche ideas and S-box design.
  4. Daemen, J.; Rijmen, V.  The Design of Rijndael:  AES  — The Advanced Encryption Standard . Springer, 2002. A detailed engineering and cryptographic treatment of SPN structures and  AES .
  5. Matsui, M.  Linear Cryptanalysis Method for DES Cipher . EUROCRYPT ’93, 1994. Basic work on linear cryptanalysis and distinguishers.
  6. Biham, E.; Shamir, A.  Differential Cryptanalysis of DES-like Cryptosystems . Journal of Cryptology, 4, 3–72, 1991. The definitive source on differential cryptanalysis.
  7. National Institute of Standards and Technology.  FIPS PUB 197: Advanced Encryption Standard ( AES ) . 2001, with subsequent updates.  AES normative specification .
  8. Dworkin, M.  NIST SP 800-38A: Recommendation for Block Cipher Modes of Operation — Methods and Techniques . NIST, 2001. Describes ECB, CBC, CFB, OFB, and CTR and is important for understanding that the security of a block cipher also depends on the mode of operation.
  9. Katz, J.; Lindell, Y.  Introduction to Modern Cryptography . CRC Press, 3rd ed. A modern textbook on security definitions, computational indistinguishability, and proof models.
  10. Menezes, A. J.; van Oorschot, P. C.; Vanstone, S. A.  Handbook of Applied Cryptography . CRC Press. The classic reference on symmetric cryptography, probabilistic models, and cryptanalysis.