This paper discusses several issues of evaluation and comparison of anomaly detection algorithms, namely lack of publicly available implementations and annotated data sets. Another problem of many methods is a detection delay caused by operating on data binned to a long time intervals. The paper presents a library under development which aims to tackle the comparison and evaluation issues. Further, the paper proposes a novel anomaly detection approach that can contribute to anomaly detection in real-time.