Background
Prior to DevOps application development, teams were in charge of gathering business requirements for a software program and writing code. Then a separate QA team tests the program in an isolated development environment, if requirements were met, and releases the code for operations to deploy.
The problem with this paradigm is that when the teams work separately:
Developers were often unaware of QA and Operation roadblocks that prevent the program from working as anticipated.QA and Operation were typically working across many features and have little context of the business purpose and value of the software.Each group has opposing goals that can lead to inefficiency and finger pointing when something goes wrong.
DevOps addresses these challenges by establishing collaborative cross-functional teams that share responsibility for maintaining the system that runs the software and preparing the software to run on that system with increased quality feedback and automation issues.
What is DevOps ?
DevOps is a cross department integration between Development (Dev), the department that creating the code, and Operations, the department using that code (Ops).
DevOps is lean thinking combined with agile philosophy. This is a cultural change that is possible due to tools that automate code development and review processes through continuous integration, while allowing for version control.
DevOps - Goals
Improve collaboration between all stakeholders from planning through delivery and automation of the delivery process in order to:
Improve deployment frequency
Achieve faster time to market
Lower failure rate of new releases
Shorten lead time between fixes
Improve mean time to recovery
According to the 2015 State of DevOps Report, “high-performing IT organizations deploy 30 x more frequently with 200 x shorter lead times; they have 60 x fewer failures and recover 168 x faster.”
DevOps - How it Started
2007 - Patrick Debois, a tester, who was frustrated with the time spent on working with Dev and Ops. Continuous integration was moving Dev closer to deployment, but there was still nothing out there that fully crossed the Dev and Ops activities.
2008 - Andrew Shafer posted an idea for an agile infrastructure “birds of a feather” session at the Agile 2008 Conference. Patrick tracked down Andrew and they decided to start a Google group named "Agile System Administration".
2009 - John Allspaw, senior vice president of technical operations, and Paul Hammond, director of engineering both from Flickr company, gave a presentation at the O’Reilly Velocity Conference in San Jose, “10+ Deploys per Day: Dev and Ops Cooperation at Flickr.” The presentation laid the groundwork for how Dev and Ops can effectively work together to improve software deployment.
Patrick that watched the presentation was inspired to start his own conference, DevOpsDays. The conference brought together an energetic group of forward-thinking minds trying to improve software deployment. This group of people kept the conversation going on Twitter with the hashtag #DevOpsDays. name changed to #DevOps.
2010 - More DevOpsDays were hosted in different countries and cities around the world. The face-to-face meetings ignited more people to get energized about DevOps.
2011 - Movement began to go mainstream, catching the attention of analysts like Cameron Haight from Gartner. Big vendors started to market DevOps.
2012 - DevOps was quickly turning into a buzzword and DevOpsDays continued to grow.
2013 - The public thirst for DevOps information inspired several authors to write books on the topic. Such as The Phoenix Project by Gene Kim, Kevin Behr and George Spafford.
2014 - Large companies such as Target and LEGO became some of the first companies to bring DevOps into the enterprise.
DevOps - How It Works and Maturity phases
The software team meets prior to starting a new software project. The team includes developers, testers, operations and support professionals. This team plans how to create working software that is ready for deployment.
Each day new code is deployed as the developers complete it. Automated testing ensures the code is ready to be deployed. After the code passes all the automated testing it is deployed to a small number of users. The new code is monitored for a short period to ensure there are no unforeseen problems. Once it is monitoring and stable, the new code is reproduced to the remaining users. Most of the steps after planning and development are done automatically.
There are several phases to DevOps maturity:
Waterfall Development - Before continuous integration, development teams would write a bunch of code for three to four months. Then those teams would merge their code in order to release it. The different versions of code would be so different and have so many changes that the actual integration step could take months.
Continuous Integration - integrating new developed code with the main body of code that is to be released. It is an agile engineering practice originating from the Extreme Programming methodology. Continuous integration is often the first step down the path toward DevOps maturity. The continuous integration process from a DevOps perspective involves checking your code in, compiling it into usable code and running some basic validation testing.
Continuous Delivery - an extension of continuous integration. It sits on top of continuous integration. When executing continuous delivery, you add additional automation and testing so that you don’t just merge the code with the main code line frequently, but you get the code nearly ready to deploy with almost no human intervention. It’s the practice of having the code base continuously in a ready-to-deploy state. Teams that utilize continuous delivery don’t deploy untested code.
4. Continuous Deployment - is the most advanced evolution of continuous delivery. It’s the practice of deploying all the way into production without any human intervention. The code release typically only goes to a small percentage of users and there’s an automated feedback loop that monitors quality and usage before the code is propagated further. There are a very small number of companies that are truly practicing continuous deployment. Netflix, Amazon and Google are popular examples of companies doing continuous deployment.
DevOps - Values: Culture and Tools
DevOps focuses on establishing a collaborative culture and improving efficiency through automation with DevOps tools. While some organizations and people tend to value one more than the other, the reality is it takes a combination of both culture and tools to be successful.
DevOps Culture - DevOps culture is characterized by increased collaboration, shared responsibility, improving quality, valuing feedback and increasing automation. Many of the DevOps values are agile values as DevOps is an extension of agile. Agile methods are a more holistic way of delivering software. Agile development teams measure progress in terms of working software. Product owners, developers, testers and UX people work closely together with the same goals. DevOps is just adding the operations’ mindset and maybe a team member with some of those responsibilities into the agile team. Before DevOps, progress was measured in terms of working software, with DevOps progress it is measured in terms of working software in the customer’s hands. To achieve this, Dev and Ops must collaborate with one another, share responsibility for maintaining the system that runs the software, and prepare the software to run on the system with increased quality feedback and delivery automation.
DevOps Tools - consist of configuration management, test and build systems, application deployment, version control and monitoring tools. Continuous integration, continuous delivery and continuous deployment require different tools. While all three practices can use the same tools, you will need more tools as you progress through the delivery chain.
Source Code Repository - A source code repository is a place where developers check in and change code. The source code repository manages the various versions of code that are checked in, so developers don’t write over each other’s work. Source control has probably been around for forty years, but it’s a major component of continuous integration (Git, Subversion, Cloudforce and TFS).
Build Server - an automation tool that compiles the code in the source code repository into executable code base (Jenkins, SonarQube and Artifactory).
Configuration Management - defines the configuration of a server or an environment. Popular configuration management tools are Puppet and Chef.
Virtual Infrastructure - provided by cloud vendors that sell infrastructure or platform as a service (PaaS). These infrastructures have APIs to allow you to programmatically create new machines with configuration management tools such as Puppet and Chef (Amazon Web Services and Microsoft Azure are examples of virtual infrastructures). There are also private clouds for example, VMware has vCloud. Private virtual infrastructures enable you to run a cloud on top of the hardware in your data center. Virtual infrastructures combined with automation tools to empower organizations practicing DevOps with the ability to configure a server without any fingers on the keyboard. If you want to test your brand-new code, you can automatically send it to your cloud infrastructure, build the environment and then run all of the tests without human intervention.
Test Automation - DevOps testing focuses on automated testing within your build pipeline to ensure that by the time that you have a deploy-able build, you are confident it is ready to be deployed. You can’t get to the point of continuous delivery where you’re fairly confident without any human intervention that your code is deploy-able without an extensive automated testing strategy (Selenium and Water).
Pipeline Orchestration - A pipeline is like a manufacturing assembly line that happens from the time a developer says, “I think I’m done,” all the way to the time that the code gets deployed in the production or a late-stage pre-production environment.
Unifying Enterprise Software Development and Delivery - unifies agile application life-cycle management and DevOps, providing a full picture of your entire software delivery pipeline in a single platform. An enterprise continuous delivery solution for automating, orchestrating, and visualizing the flow of change throughout the software delivery cycle.