IT Is Power. Try Living Without A Single Breadboard For A Day.

Don MacVittie

Subscribe to Don MacVittie: eMailAlertsEmail Alerts
Get Don MacVittie: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

Blog Feed Post

Isn’t it Time We Got Past Server Automation?

We have been using scripting tools and even bare metal install tools to automate the installation process of datacenter servers for a very long time. If you trace server automation back to the advent of tools like Chef and Puppet, we’ve been at it for years. If you trace it back to the astoundingly complex perl and shell scripts that have been created, we’ve been at it for decades.

And they’ve served us pretty darned well. They’ll continue to serve us pretty well too, in the right role at the right time. At the right time is critical, because while many of us (myself included) have written complex installation scripts, it was a rare organization that included ongoing maintenance or monitoring in the equation for initial rollout. To a certain extent, that makes a lot of sense. We don’t need monitoring or management if the system doesn’t get deployed; fight tomorrow’s fires tomorrow. But it does add the concern that the longer-term issues of maintenance and monitoring are often tacked on to systems as an after-thought.

There are plenty of examples of this out there, probably some in your own organization. I’ll save my example for a little later when there is more context to the discussion.

Complexity is King

The complexity of modern systems requires much more than “I need this script to get my app installed.” Taken as a list, IT has to consider

  1. Hardware installation.
  2. Hardware config (RAID Cards, specialized drivers, etc)
  3. OS Install
  4. OS Update
  5. OS Configuration (partitioning before install, networking after, on-node firewall, etc)
  6. Application prerequisite installation.
  7. Application prerequisite configuration (think SSL keys as an example, but there are many more – $JAVA_HOME is another example)
  8. Application installation.
  9. Application configuration.

And modern systems are more inter-related, meaning that this process does not occur on a single machine, but a collection of them, and final application configuration causes changes to the overall network of machines. This is particularly true for clustered systems like Big Data and OpenStack, but increasingly true for data center web applications also.

In the case of Big Data and OpenStack installations, the complexity and inter-relationships are pretty heavy, and thus time intensive. Scripting works very well to configure an app on a server, or even configure an app on a server and set up dependencies like database access. But it starts to fall short even on initial configuration of complex, multi-node systems. And that doesn’t even discuss monitoring and maintenance.

Operations teams have a ton of important work to get done, and speed of delivery is increasingly becoming a mantra. So we are faced with the conundrum of “Deliver it faster, but these complex systems require more machines to be correctly configured.” Meanwhile that other important work is still brewing up a storm in the background, costing lines of business opportunities, and perhaps even causing a higher rate of failure as less staff are available for maintenance.

It’s Time to Evolve. Apps, not servers.

Let’s be honest. The tools that serve us well for some parts of configuration are difficult to tie together and make work for large multi-node systems. They’re difficult (but plenty of shops have struggled through it) to make work for total install of individual systems, modern complex applications crossing a collection of interdependent servers compound those difficulties.

Users and lines of business don’t care about how many servers you had to set up. They really don’t. They care about the status of their application. Is it accessible, and is it performing? Those are the questions they want answers to, and neither of those say “server”, they both imply “application”.

And my example referenced above? Upgrading from a major release of OpenStack to a new one. Say Icehouse to Juno. There are changes in most of the underlying subsystems, often extreme ones as the OpenStack platform matures (OpenStack networking over previous releases springs to mind for extreme changes), and those changes need to be automated as much as possible, while still leaving IT in control of what is installed and how.

It is time to evolve what we are doing, and start utilizing something more complete. If you were to add “full stack” in front of server automation, it would be closer to what we need. We need full stack solutions that automate all of the steps outlined above, and the management of the resulting systems.

The difference between a long list that says “perform step one. Validate. Perform step two…” and a short sentence that says “Select machines to be part of this system, select which services run where, install all servers at once, from bare metal (or bare virtual metal) to functional application”. Is massive. It takes us from weeks or months to develop the systems and scripts to put the application online to hours or days. Add in the ability to expand and contract the node count dynamically, and the ability to monitor and maintain the nodes, and you have Warehouse Grade Automation.

This means more precious operations time can be spent making certain the systems are running at peak performance, or working on projects that will increase corporate revenues, while ensuring consistency across these complex systems.

PalletLoadOur Warehouse Grade Automation solution is StackIQ Boss, with StackIQ Pallets to offer a variety of Big Data, HPC, and OpenStack distributions, and finally, StackIQ Wires for those who wish to integrate corporate management platforms or create custom Pallets to install their own applications from bare-metal to functional.

But even if StackIQ isn’t on your radar, think about the change. Are we deploying individual nodes or entire software solutions? To me, it seems the answer must be entire software solutions, and those solutions should have a single install path that accommodates existing tools while offering the ability to do full-stack install and management out-of-the-box.

Because we can’t keep doing the same thing with a new generation of applications. We’ve tried that before, and eventually things must change to acknowledge the new reality.

Read the original blog entry...

More Stories By Don MacVittie

Don MacVittie is founder of Ingrained Technology, A technical advocacy and software development consultancy. He has experience in application development, architecture, infrastructure, technical writing,DevOps, and IT management. MacVittie holds a B.S. in Computer Science from Northern Michigan University, and an M.S. in Computer Science from Nova Southeastern University.