Deakin University
Browse

File(s) under permanent embargo

Adaptive load shedding for mining frequent patterns from data streams

journal contribution
posted on 2006-01-01, 00:00 authored by X Dang, W Ng, Kok-Leong Ong
Most algorithms that focus on discovering frequent patterns from data streams assumed that the machinery is capable of managing all the incoming transactions without any delay; or without the need to drop transactions. However, this assumption is often impractical due to the inherent characteristics of data stream environments. Especially under high load conditions, there is often a shortage of system resources to process the incoming transactions. This causes unwanted latencies that in turn, affects the applicability of the data mining models produced – which often has a small window of opportunity. We propose a load shedding algorithm to address this issue. The algorithm adaptively detects overload situations and drops transactions from data streams using a probabilistic model. We tested our algorithm on both synthetic and real-life datasets to verify the feasibility of our algorithm.

History

Journal

Lecture notes in computer science

Volume

4081

Pagination

342 - 351

Publisher

Spinger-Verlag

Location

Berlin, Germany

ISSN

0302-9743

eISSN

1611-3349

Language

eng

Notes

Book title : 'Data Warehousing and Knowledge Discovery'

Publication classification

C1 Refereed article in a scholarly journal

Copyright notice

2006, Springer-Verlag Berlin Heidelberg

Usage metrics

    Research Publications

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC