Back to AI Research

AI Research

Applying JEPA-Style Predictive Learning to JA4-Deri... | AI Research

Key Takeaways

  • Applying JEPA-Style Predictive Learning to JA4-Derived Network Fingerprints explores whether a predictive learning technique, originally designed for images...
  • I-JEPA and V-JEPA learn by matching latent predictions to target encoder outputs rather than regenerating the original input, and this has worked well for images and video.
  • We explore whether the same objective works for compact network fingerprints.
  • We built JA4-JEPA, a Transformer-based model trained on JA4, JA4H, JA4S, and JA4X subfields drawn from JA4DB and CIC-IDS- 2017.
  • The training data combines roughly 397K samples from both sources, though no single sample contains all four view families.
Paper AbstractExpand

I-JEPA and V-JEPA learn by matching latent predictions to target encoder outputs rather than regenerating the original input, and this has worked well for images and video. We explore whether the same objective works for compact network fingerprints. We built JA4-JEPA, a Transformer-based model trained on JA4, JA4H, JA4S, and JA4X subfields drawn from JA4DB and CIC-IDS- 2017. The training data combines roughly 397K samples from both sources, though no single sample contains all four view families. We evaluated the learned representations with a frozen kNN probe on protocol-family classification across TLS, DNS, and SSH. On 39,416 heldout samples the model achieved a cosine similarity of 0.9899 and a kNN accuracy of 0.9220. These results indicate that JEPA-style predictive learning can produce useful embeddings from JA4-derived fingerprints, even with incomplete view overlap across sources. Keywords: JA4, network fingerprinting, JEPA, predictive representation learning, self-supervised learning

Applying JEPA-Style Predictive Learning to JA4-Derived Network Fingerprints explores whether a predictive learning technique, originally designed for images and video, can effectively create useful representations from compact network traffic fingerprints. By using a Transformer-based model called JA4-JEPA, the researchers aim to move beyond traditional network fingerprinting—which typically treats signatures as static lookup keys—toward a more flexible, learned representation that can generalize across different network analysis tasks.

How the Approach Works

The core of the method is the Joint-Embedding Predictive Architecture (JEPA). Instead of trying to reconstruct raw data, the model uses a "context encoder" to process visible parts of a network fingerprint and a "predictor" to guess the latent representation of hidden or missing parts. These predictions are then matched against a target encoder that uses exponential moving averages to update its weights. The model was trained on 397,000 samples containing various JA4+ subfields (JA4, JA4H, JA4S, and JA4X). Because no single sample contains all four types of data, the model learns to bridge these different views, using "PAD" tokens for missing data and "MASK" tokens for hidden information.

Key Findings

The researchers evaluated the model by freezing its learned embeddings and testing them on a protocol-family classification task (distinguishing between TLS, DNS, and SSH). The model achieved a high cosine similarity of 0.9899 and a kNN classification accuracy of 0.9220. Furthermore, in a separate benchmark on a production pilot corpus of 2.1 million fingerprint pairs, the model’s "prediction energy" signal proved to be the only method capable of effectively detecting two different types of synthetic anomalies. Unlike traditional methods, the model’s scoring speed remains constant regardless of the size of the dataset, making it computationally efficient for large-scale use.

Limitations and Future Directions

While the results are promising, the authors note several constraints. The current evaluation is limited to coarse protocol-family classification, which does not address more complex tasks like application identification or malicious activity detection. Additionally, because the training data lacked samples with full overlap across all four fingerprint families, the model relies heavily on the JA4 subfield as a bridge between views. The researchers also observed that the embedding space does not form cleanly separated global clusters, meaning the model is better suited for local neighborhood-based tasks than for global clustering. Future work will focus on testing the model with richer labels and refining its ability to provide calibrated anomaly detection.

Comments (0)

No comments yet

Be the first to share your thoughts!