AIOps is the use of artificial intelligence to make IT operations management simple, accelerate the time to solve IT operations problems by automating their resolution. This post lists the most popular AIOps open source tools.
The ranking is based on the number of stars received on Github for each repository.
We have noticed that the majority of the open source AIOps projects, whether listed here or not, are using Python. The fact that Python is the first programming language used in machine learning on Github, explains what we found.
Let's get to the point.
Seldon core converts machine learning models (e.g. Pytorch, Tensorflow, H2o) or language wrappers (Python, Java, etc.) into production REST/GRPC microservices. It handles scaling to thousands of production machine learning models and provides advanced machine learning capabilities out of the box including:
Loglizer provides a toolkit that implements a number of machine learning based log analysis techniques for automated anomaly detection. The log analysis framework for anomaly detection usually comprises these 4 components:
AIOpsTools is a toolkit for Python developers who want to use existing features to build AIOps applications. Aiopstools realizes some Ops scenes by using artificial intelligence. You can import modules easily to achieve functions. This toolkit provides these 4 main capabilities:
This tool has no/incomplete English documentation; we hope to see the English version soon!
Log anomaly detector (LAD) is an open source project code named "Project Scorpio". It can connect to streaming sources and produce predictions of abnormal log lines. Internally it uses unsupervised machine learning. LAD developers incorporated a number of machine learning models to achieve this result. In addition, it involves a human in the loop feedback system.
This project contains the following 3 components:
Log3C is a general framework that identifies service system problems from system logs. It utilizes both system logs and system KPI metrics to promptly and precisely identify impactful system problems. Log3C involves four main steps:
WhyLogs Library is an implementation of WhyLogs Java Library. It is an open source tool providing statistical logging library and enabling data science and machine learning teams to effortlessly profile ML/AI pipelines and applications, producing log files that can be used for monitoring, alerts, analytics, and error analysis.
These are the 5 main features WhyLogs offer:
You can find some WhyLogs examples here.
Jumbune, an open source Big Data APM, provides deep analytics on Big Data to improve the performance of Data operations on public clouds: AWS, Azure, Google Cloud Platform, and In-premise data centers.
It comprises various modules, namely, Hadoop Job Flow Analyzer, HDFS Data Validator, and Hadoop Job Profiler and Cluster Monitoring.
Jumbune architecture can be classified into the following major blocks:
A study by Gartner shows that only five percent of big companies combine big data and machine learning. But by the year 2020, it will be about two-fifths. Another study conducted by Research and Markets found that AIOps is witnessing an impressive growth worldwide, and the market would reach $9.907 billion by 2023. According to OpsRamp, a primary service-centric AIOps platform provider revealed that AIOps tools generate value for 87% of organizations.
The AIOps market is projected to register an upward trend at a CAGR of 27% during the forecast period (2019-2024).
According to MarketsandMarkets, the global market for AIOps platforms will grow from $2.55 billion in 2018 to $11.02 billion by 2023 (average annual growth of 34%). Both industry giants and niche specialized companies compete on it - so far, everyone has enough space. However, the mergers and acquisitions have already begun, and not all will survive until 2023 - at least as independent companies.
The greatest promise of AIOps lies in its ability to automatically detect, analyze, and even fix IT issues in real-time.