Visual Structural Assessment and Anomaly Detection for High-Velocity Data Streams.

08:00 EDT 19th March 2020 | BioPortfolio

Summary of "Visual Structural Assessment and Anomaly Detection for High-Velocity Data Streams."

The widespread use of Internet-of-Things (IoT) technologies, smartphones, and social media services generates huge amounts of data streaming at high velocity. Automatic interpretation of these rapidly arriving data streams is required for the timely detection of interesting events that usually emerge in the form of clusters. This article proposes a new relative of the visual assessment of the cluster tendency (VAT) model, which produces a record of structural evolution in the data stream by building a cluster heat map of the entire processing history in the stream. The existing VAT-based algorithms for streaming data, called inc-VAT/inc-iVAT and dec-VAT/dec-iVAT, are not suitable for high-velocity and high-volume streaming data because of high memory requirements and slower processing speed as the accumulated data increases. The scalable iVAT (siVAT) algorithm can handle big batch data, but for streaming data, it needs to be (re)applied everytime a new datapoint arrives, which is not feasible due to the associated computation complexities. To address this problem, we propose an incremental siVAT algorithm, called inc-siVAT, which deals with the streaming data in chunks. It first extracts a small size smart sample using an intelligent sampling scheme, called maximin random sampling (MMRS), then incrementally updates the smart sample points on the fly, using our novel incremental MMRS (inc-MMRS) algorithm, to reflect changes in the data stream after each chunk is processed, and finally, produces an incrementally built iVAT image of the updated smart sample, using the inc-VAT/inc-iVAT and dec-VAT/dec-iVAT algorithms. These images can be used to visualize the evolving cluster structure and for anomaly detection in streaming data. Our method is illustrated with one synthetic and four real datasets, two of which evolve significantly over time. Our numerical experiments demonstrate the algorithm's ability to successfully identify anomalies and visualize changing cluster structure in streaming data.


Journal Details

This article was published in the following journal.

Name: IEEE transactions on cybernetics
ISSN: 2168-2275


DeepDyve research library

PubMed Articles [40904 Associated PubMed Articles listed on BioPortfolio]

Online and Unsupervised Anomaly Detection for Streaming Data Using an Array of Sliding Windows and PDDs.

In this article, we propose an online and unsupervised anomaly detection algorithm for streaming data using an array of sliding windows and the probability density-based descriptors (PDDs) (based on t...

Utility of Follow-Up Standard Sonography for Fetal Anomaly Detection: Follow-Up Sonography for Fetal Anomaly Detection.

In 2014, the National Institute of Child Health and Human Development Fetal Imaging Workshop consensus recommended that sonograms be offered routinely to all pregnant women. In the absence of another ...

The Perky effect revisited: Imagery hinders perception at high levels, but aids it at low.

Visual, although not auditory, imagery typically interferes with visual acuity: the so-called Perky effect. However, visual images can facilitate detection of visual Gabor patterns. Here we report tha...

Unsupervised Anomaly Detection With LSTM Neural Networks.

We investigate anomaly detection in an unsupervised framework and introduce long short-term memory (LSTM) neural network-based algorithms. In particular, given variable length data sequences, we first...

Accuracy and Reliability of Onset Detection Algorithms in Gait Initiation for Healthy Controls and Participants With Parkinson's Disease.

Accurate and reliable detection of the onset of gait initiation is essential for the correct assessment of gait. Thus, this study was aimed at evaluation of the reliability and accuracy of 3 different...

Clinical Trials [16636 Associated Clinical Trials listed on BioPortfolio]

Surgery and Transcatheter Intervention for Structural Heart Diseases

This project aimed to optimize the therapeutic strategy for structural heart disease by choosing optimal treatment, such as,surgical treatment,interventional and surgery combined with inte...

The Relationship Between Structure and Visual Function in Myopia Foveoschisis

Macular foveoschisis (MF) is a common complication of pathological myopia, which is characterized by choroid-retinal degeneration and vision impairment. Although pars plana vitrectomy (PPV...

Assessment of Thoracic Echography for Pleuroparenchymatous Anomaly Diagnosis Complicating Bronchiolitis: BronchioIUS

Assessment of thoracic echography for pleuroparenchymatous anomaly diagnosis complicating bronchiolitis. Assessment has performed by radiographic exam.

The Relationship Between Gait Velocity and Gait Parameters

Previous reports have the evidence implicating gait velocity may affect kinematic parameters. However, how the gait velocity will influence kinematic parameters remain unclear. The purpose...

Is There a Benefit of Whole Body Computed Tomography (WBCT) for Patients With Only High Velocity Road Traffic Collision (RTC) Vittel Criteria?

For victims of high velocity RTC, with no other Vittel criteria of gravity, normal clinical examination of the thorax, abdomen and pelvis and Glasgow Coma Scale (GCS) score of 15 : - S...

Medical and Biotech [MESH] Definitions

The visual display of data in a man-machine system. An example is when data is called from the computer and transmitted to a CATHODE RAY TUBE DISPLAY or LIQUID CRYSTAL display.

Measurement of the index of refraction (the ratio of the velocity of light or other radiation in the first of two media to its velocity in the second as it passes from one into the other).

The science concerned with the benefit and risk of drugs used in populations and the analysis of the outcomes of drug therapies. Pharmacoepidemiologic data come from both clinical trials and epidemiological studies with emphasis on methods for the detection and evaluation of drug-related adverse effects, assessment of risk vs benefit ratios in drug therapy, patterns of drug utilization, the cost-effectiveness of specific drugs, methodology of postmarketing surveillance, and the relation between pharmacoepidemiology and the formulation and interpretation of regulatory guidelines. (Pharmacoepidemiol Drug Saf 1992;1(1); J Pharmacoepidemiol 1990;1(1))

High-amplitude compression waves, across which density, pressure, and particle velocity change drastically. The mechanical force from these shock waves can be used for mechanically disrupting tissues and deposits.

Echocardiography applying the Doppler effect, with velocity detection combined with range discrimination. Short bursts of ultrasound are transmitted at regular intervals and the echoes are demodulated as they return.

Quick Search

DeepDyve research library

Searches Linking to this Article