Data is diverse, shouldn’t data management be too?
“Big Data” seems to be the buzzword of the moment, used to describe any data-driven application. Any and all practices dealing with large quantities of data receive the label, leaving the term on the borderline of triteness. If you work in a technical field you probably hear it on the daily.
I’d argue that using the term big data in such an all-encompassing way is reductive, and ultimately does the tech community a disservice. Because understanding the variety of ways in which data is handled is key to understanding modern data management- a complex subject in its own right.
In our particular line of work at imVision we deal predominantly with mobile networks. Specifically, combing through the incredibly dense network architecture of mobile networks in real-time to discover anomalous behavior. This requires a unique layered approach to data management- here’s why.
A bottom-up approach
In order to make decisions in environments which process enormous amounts of data, humans need to see that data aggregated. We often rely on KPIs and statistics, because we simply can’t sort through each and every piece of data. But when data is aggregated, valuable information is lost, including low granularity information like advanced cyber attacks or service degradation.
Obviously that information is a necessity, and in these cases we cannot rely on aggregated data and KPIs to make time sensitive decisions. So at imVision we turn to a more in-depth approach, analyzing the raw data itself, and performing analysis on multiple axis:content, context, and meaning for a higher degree of accuracy.
In short, we employ a rich set of anomaly detectors (algorithms) to identify abnormalities in the control messages between network functions. Each data unit (event) is analyzed by a few detectors. This includes an analysis of the content of the event and an analysis of events within the context of the flow.
This means fully understanding what's happening, and analyzing if each action is logical or allowed. For example, in the case of mobile networks, the “content” would be the information encoded within the messages that we are monitoring. Is it legal in terms of structure, fields, etc? The “context” asks if this message between element A and B is legal/allowed. And the “meaning” asks if it make sense that Element A would initiate such a request?
Often, in-depth analysis of an isolated case can tell us more about a situation, what caused it and how to fix it - than any statistical information we review. This requires a robust engine, built-in knowledge and the expertise to reach detailed conclusions.
imVision’s Anomaly Management Platform (AMP) is a self-learning anomaly detection and analytics engine, tailored to networks, which provides operational and security teams with proactive network management and prevention. AMP scans network traffic and identifies anomalies from the expected behavior in real time and automatically tracks their root cause, to understand which anomalies require attention and which are legitimate deviations from the expected performance.
Want to learn more about how data analytics is put into action for mobile networks operation and cybersecurity? Check out our resources page for in-depth reports. Or contact us with questions on our main page!