eDirectory Performance Tools
Novell Cool Solutions: Feature
By Juli Kerr
Digg This -
Posted: 8 May 2002
"The software hit me with so much great information after we rolled it out in our test that I couldn't wait to get the results from the rest of the environment."
-- IT Manager for 300-server NetWare network
Thousands of eDirectory users around the world run their directories without a concern for what's really going on inside them. And, when they finally get smart and decide to look into it -- it's those users who often discover what they didn't know was hurting them.
Case in point.
The IT manager for a U.S. county government office called on NetPro's professional services team to assist with an install of both DS Expert and DS Analyzer. At the time of the request, the manager was most interested in using DS Expert to conduct real-time directory health checks and DS Analyzer to drill down on the origin of any issues uncovered by DS Expert. With a 300-plus server network, a mix of 20% NetWare 4 and 80% NetWare 5, and a slew of multiple DS versions, the IT manager rightfully determined that he should take a close look at his directory.
He wasted no time doing it. While users typically experiment with new products in a test environment, this IT manager elected roll both DS Expert and DS Analyzer out in production to all of his NetWare 5 boxes -- or 80% of his servers -- almost immediately. "I couldn't come up with a good reason to wait for deployment," the IT manager said. "The software hit me with so much great information after we rolled it out in our test that I couldn't wait to get the results from the rest of the environment."
Indeed, the results knocked him off his feet. He started by reviewing different replica rings and comparing synchronization times of rings of a similar size through DS Expert, NetPro's real-time eDirectory monitoring and alerting software. This exploration revealed that every server was completing the synchronization within a replica ring in one to two seconds except for one. One server, called server1, held a read/write that showed a value of more than 1200 seconds. And, DS Expert notified the manager with an alert indicating that the synchronization was taking longer than the default setting of 20 minutes.
Uncovering the problem in the first place would have been extremely difficult without DS Expert, but to get to the bottom of the issue once it was highlighted was even more challenging. Especially if the only troubleshooting tools were manual, server-level tools. To pinpoint the reason for the server1 traffic spike, the typical administrator would employ conventional eDirectory troubleshooting tools. The exercise would go something like this:
- Run DSTrace and filtering for skulker and errors on all servers in the replica ring.
- Ascertain that server1's participation in the replica ring is extraordinary.
- Conduct extensive local troubleshooting of the box manually.
- Turn to packet analysis software and hardware and set filters to view the DS traffic.
- Pour over the data output by the analyzing mechanism to determine the origins of the traffic.
- Attempt to drill down through data points to determine the root cause of the spike.
- With assistance from Novell Technical Support, eventually uncover the workstation that is the culprit.
A Job for DS Analyzer
Instead, this IT manager deployed the solution that first identified the problem -- the eDirectory Performance Pack. This time, though, he turned to DS Analyzer, NetPro's eDirectory troubleshooting and tuning solution. Acting much like a sniffer on the wire, DS Analyzer collected all of the incoming DS packets and enabled the IT manager to drill down on the problem, analyzing the specific containers, servers, replicas, and clients that were involved with the directory traffic. DS Analyzer differentiates 13 types of DS traffic, including synchronization, bindery requests, backlinking, time, schema, etc.
Because the DS Analyzer agents had been installed on server1 earlier that week, the IT manager had four days of data stored and could graph the partition. (DS Analyzer draws as many graph lines as there are servers holding copies of the replica.) "I immediately saw a distinct difference in the graph line for Server1," he said. "Server1's synch time was continually much higher than the other four servers holding copies of the replica. In fact, from 7 a.m. until 4 p.m. on Friday and Monday, there were in excess of 1 million requests every five minutes!" The plot thickened with the IT manager used DS Analyzer to graph the 4 p.m. hour. The graph took a distinct turn down to zero, and during the weekend, it was constantly near zero. DS Analyzer showed the IT manager the number of requests, the amount of packets, and the number of bytes for this time period to establish some level of consistency. Then, he selected Server1 and drilled down with DS Analyzer on the exact DS category -- in this case Access traffic -- at 9 a.m. Almost all of the traffic was Access traffic without any resolve name traffic to go along with it. "We thought that was strange because, in most instances DS Access and Resolve name traffic should peak and valley at about the same time," says the IT manager.
To get to the real root of the issue, the manager took the 9 a.m. data point for Access, drilled down on the clients involved with this traffic, and received a list of the top 10 clients generating the Access traffic at that time. "The top client generating almost a million requests of Access traffic at 9 a.m. on Server1 was an IP address!" said the IT manager in amazement. The IP address didn't have a server name or a login name associated with it, but when he investigated the IP address, he discovered that the traffic was coming from an NT workstation that had a static IP address assigned. "No kidding," the IT manager said. "A workstation was generating this astronomical number of requests!"
Once he located the machine physically and pulled it off the wire, DS Analyzer immediately started graphing a downward line heading towards zero on that client. In addition, DS Expert showed Server1 completing its replica synchronization in two seconds. "To get to the bottom of this problem, it took me less than an hour," the manager said. "It was amazing!"
What's even more amazing is that the IT manager had no idea there was even a problem -- until he loaded DS Expert and DS Analyzer. "DS Expert identified the replication slowdown and prompted us to put DS Analyzer on the job to get to the bottom of the issue -- before it began affecting users," he related. "If that's not a good enough reason to 'take the top off the directory black box,' I don't know what is!"
For more information about NetPro's eDirectory Performance Pack, please visit http://www.netpro.com/products.
Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com