IT Is Power. Try Living Without A Single Breadboard For A Day.

Don MacVittie

Subscribe to Don MacVittie: eMailAlertsEmail Alerts
Get Don MacVittie: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

Related Topics: Cloud Computing, Big Data on Ulitzer, OpenStack Journal

Blog Post

Adding Value, Not Complexity By @DMacVittie | @CloudExpo [#Cloud]

Web scale infrastructure demands comprehensive management solutions... Or, why I went to StackIQ.

Note: This blog is cross-posted from, as will others until I get my connections set up correctly.

For regular readers of the StackIQ blog, Hi! I'm Don MacVittie, the new Senior Solutions Architect here at StackIQ.

For regular readers of my aggregated blog, if you have not, meet StackIQ - Web Scale Infrastructure Management vendor that will knock your socks off.

Introductions completed, let's move on to the topic at hand, shall we?

OpenStack and Hadoop are both amazingly powerful platforms for those who need (and recognize that they need) them. We all know what private clouds and Big Data are, so I won't waste your time explaining them. I will point out for those who haven't had the pleasure of installing them that they are terribly difficult to install and manage. Not because they're poorly designed and written, and not because Open Source doesn't care about usability, but because they're that complex. To illustrate this point, I like to point people to this picture from the OpenStack documentation that shows a simplified depiction of the architecture:

I call this simplified because there are optional parts that you can also install and configure, and it doesn't show the construction of the physical environment used to support this system. Each of the boxes in the above diagram is a massive system in its own right - networking, RBAC, ISO management, virtualization engine... The list goes on. Hadoop has a similar architecture, and similar challenges.

As I learned how to get OpenStack up and running in a variety of environments - from beefy laptops for demos to massive datacenters - I came to the conclusion that whomever automated this process won.

I stand by that judgment. The complexity required to serve up functional systems that interoperate with the rest of the network or to allow querying of huge amounts of data in a reasonable amount of time is massive, and even little things like single server misconfigurations or failed drives can put the entire integrated system off-line or in degraded performance mode (lose the disk on your keystone server, and you'll see what I mean).

While I have not seen a truly comparable diagram out there for No SQL big data deployments, Hadoop does have an even more simplified version:

The thing is, they both serve the business by massively improving agility - Big Data driving business intelligence agility, OpenStack driving server/app provisioning agility. But in the arena of installation and ongoing management, they themselves are not very agile, and cost IT man-hours keeping these critical systems up to date and functioning properly.

That's why I came to StackIQ, because the products answer these vexing problems. And they do it well. My second week here at StackIQ, I sat down with the product and installed OpenStack - in about two hours. Without tutelage. If you know OpenStack, you know what that means. It can take two hours to install and configure virtual networking, let alone the entire architecture. I'm just starting a clean Hadoop install as I write this. Interestingly, I've not used full stack automation to rapidly deploy hadoop before, so I'm intrigued to see how well this goes. My suspicion, based on the OpenStack install, is that it will go well, even though I haven't done it before. StackIQ makes things that easy.

As I wend my way through the various features, facets, and astounding extras of the product, I'll be here updating you. I tend to enjoy blogging about occasional extraneous things too, so you are certainly welcome to read those as we go also.

Until then, I'll be in the lab, on the phone, checking email... Did I mention in the lab?

More Stories By Don MacVittie

Don MacVittie is founder of Ingrained Technology, A technical advocacy and software development consultancy. He has experience in application development, architecture, infrastructure, technical writing,DevOps, and IT management. MacVittie holds a B.S. in Computer Science from Northern Michigan University, and an M.S. in Computer Science from Nova Southeastern University.