Troubleshooting 101

 Spent 2 ½ days trying to figure out why a customer's  VPN solution was not preforming the way the vendor had promised.  Never mind the fact that the networking team had no say in the installation or operation of the solution. Spent one whole day just tracking down the laundry list of issues the customer was having and any troubleshooting steps the vendor and customer had done. The second morning we traced the path checking routing and firewall ACLs. No issues, so I asked the customer if I could have a client to generate traffic on demand (and also check it’s setting). The customer was glad for any help and brought me a client. Checked firewall and IP setting, no issues. The third morning the vendor was a little upset that I was looking at the client, they were sure it was the network. After some back and forth, their tech had to go to lunch, and created an account on the server so I could do some tracert, ping, ect. 10 minutes late I had it figured out.

So, 1^st thing I did was a tracert to a connected client, and what do I see……………….

 I checked the server’s IP settings and the subnet mask was incorrect, as soon as it was corrected ALL the issues the customer had cleared up, even some they had but on the back burner.

The clients have a /21 bit mask, and the server was using a /24, the vendor’s tech swore up and down he had not changed it, and that it had been working up to that point. He even wrote an email saying “How strange it was that it stopped working now” I wrote back it was strange it had worked at all.

General troubleshooting steps

1.     Define the problem.

2.     Gather detailed information.

3.     Consider probable cause for the failure.

4.     Devise a plan to solve the problem.

5.     Implement the plan.

6.     Observe the results of the implementation.

7.     Repeat the process if the plan does not resolve the problem.

8.     Document the changes made to solve the problem.