A two-stage monitoring scheme for multiple high-dimensional data streams

Abstract

With the rapid development of data acquisition and analysis technology, the online monitoring of data streams is particularly important in modern industries. This study proposes an efficient online monitoring scheme for multiple high-dimensional data streams. We first construct a point-wise process monitoring control chart based on statistical extreme value theory. Then we use the clustering method to roughly obtain reliable data streams and suspected abnormal data streams, within the sliding window range, further identify abnormal data streams one by one over time points, and estimate the potential change points of these abnormal streams. Finally, we conduct simulations and case studies on this method, and compare it with existing competitive methods to demonstrate its effectiveness.

Keywords:

• High-dimensional data streams
• multiple testing
• online monitoring
• statistical process control

Acknowledgments

The authors are grateful to the editor, the associate editor and two anonymous referees for their comments that have greatly improved this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by National Nature Science Foundation of China [12071233, 11926347, 11971197], Natural Science Foundation of the Higher Education Institutions of Jiangsu of China [21KJD110003], Jiangsu Government Scholarship Program [2019-43], Qinglan Project of Jiangsu Province of China [2022], Huai'an City Science and Technology Project (HAB202357).