Page 1 of 2
In the field of log and IT data, San Francisco-based Splunk is king. In fact, no one has stepped to the plate to rival its unique search engine.
Splunk Professional Server 2.1, unveiled this week, marks another milestone in error log collection and detection. The Splunk search engine now can scale across multiple data centers and geographies, and it provides unlimited indexing by clustering multiple servers.
The software also comes with new C++, SOAP and REST APIs, including command-line APIs, to automate search on the command line with scripting technologies.
Essentially, the Splunk engine enables operators, administrators and developers find out what's going on in an IT infrastructure. Because Splunk is relatively new, most companies aren't aware of the technology and -- with so much data generated in real time from complex systems -- are still using simple tracking solutions to pinpoint system failures.
Today, that process is done manually, whereby operators must retrace all connected transactions by crawling through many log files. And the tracing process is slow, even when following application server and packaged middleware logs, since every piece of technology generates its own format.
Different versions of the same technology also sometimes generate different log formats, making the process even more difficult.
The amount of data exacerbates this problem. One server can generate more than 100 Mbytes of log data a day and 10 times that amount when working with a multitiered system. So in one day, an enterprise may generate a terabyte of log data.
Operators typically have to collect feeds from SNMP traps, ports, JMS messages, audit logs inside database tables, and application server logs. Even with the most skilled operators, a manual approach can take hours to figure out.
Another approach is using parsing technologies to trace information. However, home-grown systems that parse log files don't scale up well.
To add to the frustration, once a fault has been pinned down to an application or a set of modules, developers often must get involved in the identification process, because application-specific log data is almost always written only for developers to find specific errors. Most application-specific logs aren't standardized because of the variety of products and technologies in each system.
1
|
2
|
Next >>
