Knightmare: A DevOps Cautionary Tale

This article is an awesome example of why automation is so important for businesses. I know that I personally will do my best to introduce more automation and configuration management tools into the environments I work on. This article is a reminder that without it, terrible things can happen!

Until next time!

Doug Seven

I was speaking at a conference last year on the topics of DevOps, Configuration as Code, and Continuous Delivery and used the following story to demonstrate the importance making deployments fully automated and repeatable as part of a DevOps/Continuous Delivery initiative. Since that conference I have been asked by several people to share the story through my blog. This story is true – this really happened. This is my telling of the story based on what I have read (I was not involved in this).

This is the story of how a company with nearly $400 million in assets went bankrupt in 45-minutes because of a failed deployment.

Background

Knight Capital Group is an American global financial services firm engaging in market making, electronic execution, and institutional sales and trading. In 2012 Knight was the largest trader in US equities with market share of around 17% on each the…

View original post 1,369 more words

Advertisements

ISAKMP (IKE Phase 1) Status Messages

Recently, I was involved in troubleshooting some VPN issues between two sites. The initial errors we were receiving were not helping us to understand what the problem was. After running the command show crypto isakmp sa we worked out that we weren’t getting the state MM_ACTIVE. Instead were were stuck with one side at MM_WAIT_MSG2 and the other side being stuck at MM_WAIT_MSG3. This was very confusing! Especially when no configuration was changed recently on either side and the VPN was working fine before the incident. We then found this page on the tunnelsup.com website, that clearly explained the IKE Phase 1 stages ISAKMP (IKE Phase 1) status messages MM_WAIT_MSG# – TunnelsUP. This is a great reference for anyone troubleshooting VPN’s and while it did not help us exactly in resolving the issue, it certainly was a great reference that pointed us in the right direction to getting to the root cause.

VPN-Phase1

Until next time!

Atlassian HipChat Now Free!

Today on the train trip into work, I was reading through my normal news feeds and was excited to read the title “HipChat is now free for unlimited users” and so I had to click through to find out more. Atlassian had posted on their HipChat Blog this fantastic announcement. In summary, they are basically offering a free “Basic” version which will allow Group Chat and one to one messaging. There will also be a “Plus” version which will include video calling and unlimited searchable message history.

Check it out for yourselves! https://blog.hipchat.com/2014/05/27/hipchat-is-now-free-for-unlimited-users/

Until next time!

Rookie Mistake: Don’t Forget to Register NPS Server in AD

I have been playing around with NPS for a while but today was the first time I had ever set up an Network Policy Service Server on a Member Server in a domain. I then wanted to use it as the Primary Server for Authentication for one of our RADIUS clients but couldn’t figure out why I wasn’t getting Authenticated, I had set it up exactly the same as my other NPS Servers. Checking the Event Viewer I noticed that I was receiving the Event ID 4402 reporting that “There is no domain controller available for domain %1”. A lot of posts out on the Internet were leading me in the direction of DNS issues or Firewall issues but none of these were the issue. I then found this post by Geoff @ UVM and it became very clear what I had forgot to do! I forgot to Register the NPS Server in Active Directory. As soon as I had done that, everything started working as expected!

To Register the NPS Server in Active Directory:

  1. Open the NPS management console
  2. Right-click on NPS (Local)
  3. Click on Register server in Active Directory

I won’t forget that one again!

Until next time!