FreqCondNorm: Towards Cross-domain Predictive Maint... | AI Research

FreqCondNorm: Towards Cross-domain Predictive Maintenance through a Frequency-Conditioned Transformer Foundation Model
Predictive maintenance—the practice of monitoring industrial machinery to predict failures—often struggles when models are moved from one machine to another. A major hurdle is that different sensors record data at vastly different speeds, ranging from 1 Hz (slow cycle data) to 100 kHz (fast vibration data). Standard AI models typically assume a uniform sampling rate, which limits their ability to learn from diverse datasets. This paper introduces FreqCondNorm, a new architecture designed to unify these heterogeneous signals into a single model, allowing it to learn from multiple industrial sources simultaneously. The same large language models question is explored in LimiX-2, which adds a research perspective.

Unifying Diverse Frequencies

The core innovation is a specialized normalization layer called FreqCondNorm. In standard Transformer models, a component called "LayerNorm" helps stabilize training by normalizing data. However, this component treats all data the same, regardless of the sampling rate. FreqCondNorm replaces this with a system that adjusts its internal parameters based on the specific sampling frequency of the input signal. By using a small neural network to process the log-scale sampling rate, the model can adapt its behavior to handle signals that differ by five orders of magnitude, effectively "normalizing" the difference between slow engine cycles and high-speed bearing vibrations.

Training and Performance

The researchers pretrained their model on five public industrial datasets using a combination of masked auto-encoding (learning to reconstruct missing parts of a signal) and contrastive learning (learning to recognize that different windows of data come from the same machine). When tested on fault diagnosis, the model showed strong performance, achieving 99.2% accuracy on the CWRU dataset—a significant improvement over traditional CNN baselines. Most notably, the model demonstrated an 82.1% zero-shot accuracy on the MFPT dataset, meaning it could correctly diagnose faults in a machine it had never seen before during training. The same ai evaluation question is explored in On the Regularization Landscape for the..., which adds a research perspective.

The Limits of Current Progress

While the model excels at fault diagnosis, the researchers provide an honest assessment of its limitations, particularly regarding "Remaining Useful Life" (RUL) prediction. The model did not outperform standard baselines when tasked with predicting how much time a machine has left before failure. The authors attribute this to a mismatch between their pretraining objectives and the requirements of RUL prediction. They suggest that while their current approach is excellent at identifying static fault types, it does not yet capture the long-term degradation trends necessary for life-expectancy forecasting.

A Realistic Scope

The authors emphasize that while their model is "foundation-model-style," it is not a general-purpose foundation model in the massive sense. The pretraining corpus consists of five datasets, which is modest compared to the scale of modern large-scale AI. They note that their success is currently limited to bearing-related data; for instance, the model struggled to transfer knowledge to gearbox data because the corpus lacked sufficient variety in equipment types. This transparency serves as a guide for future research, highlighting the need for more diverse datasets and refined training objectives to move beyond fault diagnosis into broader industrial applications. The same large language models question is explored in ActMap, which adds a research perspective. as detailed in the full paper on Arxiv