Why High-Speed Networks Are Bringing Packet and Flow-based Analytics Together
By Jay Botelho, Director of Engineering at Savvius, Inc., a LiveAction Company
According to Cisco, broadband speeds will nearly double by 2021, and more powerful networks mean a host of benefits for businesses. However, extracting insight from the flood of information traveling through these high-speed networks is a constant challenge for the IT and networking teams responsible for managing network performance and reliability. As businesses move applications to the cloud, virtualize parts of their network and pursue other digital transformation projects, IT and network engineers need actionable, real-time visibility into network and application performance issues to ensure strong performance across the board.
Complicating matters, NetOps teams are now expected to proactively identify problems before they impact the organization, and solve them as quickly as possible to minimize the effects on the end user. To do this, they need flow-based monitoring, rapid root-cause analysis and integrated packet-level forensics in a single solution, so teams can quickly identify latency, communication and capacity issues without the need to try to cobble together insight from different sources and tools.
In the past, packet analysis has been the go-to tool for troubleshooting specific problems after they have already been identified through other means. IT teams use statistical summaries and aggregated data to identify issues, then use packet analysis to investigate in more detail. However, packet data has not historically scaled for high-speed real-time monitoring, leading to monitoring solutions that use NetFlow and other techniques that are less detailed, but require less processing power. Yet network performance monitoring and diagnostic (NPMD) tools have gotten more powerful along with the networks they monitor. This has led to a new breed of solution (such as Savvius Spotlight) that simultaneously provides the precision of packet-based analytics with the speed of flow-based monitoring.
To better understand why that's happening, it's useful to consider the four steps needed to get actionable information from a modern network. First, the data must be acquired from wire data, a datacenter, the cloud or the edge. Next, the network and data must be monitored for end-user experience in true real time. Third, the team must be ready to investigate a problem or issue, from traffic to trace files. And finally, a certain level of the packet data must be retained so teams can troubleshoot.
Given that context, the new breed of NPM/APM analytics solutions that combine packet information and flow-based monitoring offer six major advantages.
1. Easily track problems to the application or the network - When problems emerge, you need to solve them fast. Understanding if this is an application or a network issue - and having the packet data to back up that claim - is critical to eliminating debates and war room discussions. For example, see at a glance which transactions on the networks are experiencing the worst network and the worst application latency, from network-wide down to an individual server. When you see application latency that is outside of the norm, a single click can provide the actual packet data comprising the network transaction. Often application errors can quickly be identified in the packet payload data. This is the best data possible for determining the root cause of the problem.
2. Gain visibility into business-critical applications - For many companies, the business is the application(s) they run. For example, an online retailer is defined by the performance of the web servers and associated web applications driving the storefront. For an airline it might be reservations, schedules and routing, etc. For these businesses, visibility into key performance indicators for these specific applications, including network and application latency and transaction quality for each and every transaction, drives real-time response that keeps the company running at its maximum potential. If those applications go down, the business will lose revenue every minute until they are restored, so maintaining application resiliency is absolutely critical.
3. Decrease the time to solve network issues - Every second counts when the network or an application has a problem. Having the ability to navigate network traffic fluidly in real-time, and then immediately click through to specific packet data, dramatically speeds resolution time. Plus, the less time the network team or IT spends troubleshooting, the more time they can spend on projects to improve the network, like cloud migrations or building data warehouses.
4. Reduce tool sprawl - IT teams are sick of adding more and more tools. The new NPM/APM solutions consolidate key functionality and offer flexible new dashboards that allow teams to monitor the information that matters most to their organization and team. For example, monitoring key applications like Office365, WebEx and Salesforce in a single dashboard that includes application performance, network performance, transaction quality and VoIP quality; metrics that in the past required several solutions from several vendors to achieve the same level of visibility.
5. Get quick access to network problem spots - NetOps needs information on the worst-performing parts of their network. Many products only calculate and display averages of network metrics like latency, utilization and VoIP quality, which can obscure problems that only affect a small number of flows. For example, a typical network will have thousands if not tens of thousands of HTTP flows at any given moment. If only a few exhibit poor network latency, yet the dashboard shows the average network latency across all HTTP connections (which many dashboards do), the few flows exhibiting poor latency will be masked by the good performance of all of the others. This example illustrates the absolute need to be able to pick the worst flows, out of millions, at any given moment to make sure critical issues are not overlooked.
6. Monitor SaaS SLAs - Whether you're offering a service or using one, being able to validate the agreement is critical, especially if problems arise or customers complain. Packet data doesn't lie, which means if you have that information, you have what's needed to ensure SLAs are being delivered. With a NPMD solution combining flow and packet data, you can set your SLA thresholds for network latency, application latency, transaction quality and VoIP quality, and let the software constantly monitor millions of flows - alerting you when even a single flow exceeds your SLAs.
Modern businesses suffer significant costs from any amount of network or application downtime, so avoiding or at least minimizing these issues is a top NetOps priority. The powerful new analytics tools available today use both flow- and packet-based analytics to help IT anticipate network and application performance problems and react in real time. This makes network problems much more visible, even as the speed and volume of network traffic increases. In many regards, the utopian vision of network continuity all day, every day, across the entire network, is finally here.