Current IT practices, to support effective and efficient software development has seen a great paradigm shift. Now monolithic architecture is being replaced by Microservice architecture. Developer, QA and Operation team are no more working in their relative siloes. They are being bridged with the implementation of Dev+Operation amalgamation which we call, DevOps.
Now tech driven businesses need quick development and faster release cycle if they want to become more customer oriented to get quick Idea validation, all this led to the birth of CI/CD practices which is at the core of DevOps culture.
CI is straightforward and stands for continuous integration, a practice that focuses on making preparing a release easier.
Continuous delivery is an extension of continuous integration to make sure that you can release new changes to your customers quickly in a sustainable way.
Earlier the entire SDLC was linear and sequential which led to the longer release cycle of the product, but with this fast changing market dynamics aggressive competition and ever changing customer needs, companies can’t afford to cling on to same old development practices. They have to be more closer to their customer and need to constantly innovate to keep them engaged.
DevOps offered this solution and is been widely adopted by IT driven companies to improve their processes & practices for rapid delivery.
Let’s try to understand What is DevOps all about?
DevOps Definition:
I very often quote w.r.t to DevOps that
Any organization, will not be able to scale and sustain until they learn to collaborate, unite & automate all their development & operation processes using best DevOps practices . It’s a culture to bond every single individual with a common goal to deliver results which matters.
It’s time to make all your team members irrespective of their department to work closely with each other adopting tools and practices for effective and efficient delivery of software products.
DevOps is a set of practices that automates the processes between software development and IT teams, in order that they can build, test, and release software faster and more reliably. The concept of DevOps is founded on building a culture of collaboration between teams that historically functioned in relative siloes. The promised benefits include increased trust, faster software releases, ability to solve critical issues quickly, and better manage unplanned work.
Amazon defines DevOps as:
The combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market.
Microsoft puts DevOps in more simplified manner:
“DevOps is the union of People, Process, and Products to enable continuous delivery of value to our end customer.”
Which I feel is the best way to explain DevOps. It is a culture, is a philosophy to put people first in an organization, giving an congenial environment to allow them to thrive, so that irrespective of their department, they can collaborate & communicate well through a well defined process, to deliver results. To make the process efficient, allowing teams to work effectively there are many DevOps tools which work as an enabler of DevOps culture.
How DevOps Works?
As discussed above DevOps doesn’t have any fixed set of rules and practices, but it is more like a culture within any organization where teams from various departments having varied skill-sets come together to deliver an intended outcome. So how does this actually work, well let me explain it briefly through a diagram below:
So here Developers, QA, & Operation team use CI/CD practices to achieve the desired goals for the customer. Developer writes code and commits them to the source control tool like GitHub. The code gets pulled using CI tools by DevOps engineer to manage test automation and deployment to the production or staging test servers is handled through CD tools.
People having both development and operations skill sets work together and use various tools for CI-CD and Monitoring to respond quickly to customer's need and fix issues and bugs.
DevOps Tools At Various Stages Of DevOps Cycle:
As you can see below:
There are multiple tools available for DevOps engineer to avail and achieve the desired outcome at every stage of the DevOps life Cycle.
Planning:
You can use Jira or Azure DevOps Board to manage and plan your work in an Agile way.
Development:
For code management, Git is the number 1 tool to manage your Code version History, branches, Push and Pull mechanism as a distributed way. You can also use Microsoft TFVC (Team Foundation Version Control) which is a Centralized version control system.
Testing:
To automate your testing you can rely on Selenium, JUnit and Apache JMeter.
Integration:
Jenkins is one of the most popular CI tool which enables integration of Developer process with operation process seamlessly.
Travis & Bamboo are other tools for CI.
Deployment & Configuration Management
Docker is one of the most popular & widely used tools for continuous deployment. It is also known as a Software containerization tool.
Other deployment & configuration management tools are Kubernetes, Chef, Ansible, and Puppet.
Kubernetes is an open-source container management (orchestration) tool. Its container management responsibilities include container deployment, scaling & descaling of containers & container load balancing.
Monitoring:
Once your product is deployed in the right place, continuous monitoring becomes vital. Nagios, Splunk or New Relics are some of the widely used tools to support continuous monitoring
We will cover about CI/CD tools in length & breadth in the next article
“DevOps CI/CD tools for agile enterprise development “ stay tuned…
DevOps Best Practices:
As discussed at the start of the article that for an IT driven organization to become more customer oriented they need to transform themselves from traditional software development practices to agile methodology of releasing products for their customers. Let us try to understand what are those best DevOps practices they need to adopt:
- Continuous Integration
- Continuous Delivery
- Microservices
- Infrastructure as Code
- Monitoring and Logging
- Communication and Collaboration
Let me explain them in brief
1. Continuous Integration:
Amazon defines CI as:
A DevOps software development practice where developers regularly merge their code changes into a central repository, after which automated builds and tests are run. Continuous integration most often refers to the build or integration stage of the software release process and entails both an automation component (e.g. a CI or build service) and a cultural component (e.g. learning to integrate frequently). The key goals of continuous integration are to find and address bugs quicker, improve software quality, and reduce the time it takes to validate and release new software updates.
2. Continuous Delivery:
Continuous delivery is a software development practice where any code changes done by developer are automatically prepared for a release to production.
Continuous delivery expands upon continuous integration by deploying all code changes to a testing environment and/or a production environment after the build stage.
3. Microservices: An Architecture For Agile Development
It’s a new software design approach where you build a single app as a set of small services/modules. As compared to Monolithic architecture where all the front-end and back-end code base along with database are all deployed in one go all pointing to the same server address, microservice architecture based app are broken down into services where each of them runs in its own process and communicates with other services through a well-defined interface using a lightweight mechanism, typically an HTTP-based application programming interface (API).
As per Amazon:
Microservices are built around business capabilities; each service is scoped to a single purpose. You can use different frameworks or programming languages to write microservices and deploy them independently, as a single service, or as a group of services.
4. Infrastructure As A code: IaC
Is the process of managing and provisioning computer data centers through machine-readable definition files(code base), rather than physical hardware configuration or interactive configuration tools.
Amazon defines IaC :
As a practice in which infrastructure is provisioned and managed using code and software development techniques, such as version control and continuous integration. The cloud’s API-driven model enables developers and system administrators to interact with the infrastructure programmatically, and at scale, instead of needing to manually set up and configure resources.
So developers can use code-based tools to interface with infrastructure to treat them more like an app. This enables infrastructure and servers to be swiftly deployed using standardized patterns, updated with the latest patches and versions, or duplicated in repeatable ways.
Traditionally, server (lifecycle) automation and configuration management tools were used to accomplish IaC. Now enterprises are also using continuous configuration automation tools or stand-alone IaC frameworks, such as Microsoft’s PowerShell DSC or AWS CloudFormation
5. Monitoring and Logging:
Companies monitor metrics and logs to see how their application and infrastructure is performing. APM(Application performance management) strives to detect and diagnose complex application performance problems to maintain an expected level of service. APM is the translation of IT metrics into meaningful business metrics.
By capturing, categorizing, and then analyzing data and logs generated by applications and infrastructure, organizations understand how changes or updates impact users, shedding insights into the root causes of problems or unexpected changes.
As per wiki:
Two sets of performance metrics are closely monitored. The first set of performance metrics defines the performance experienced by end users of the application. One example of performance is average response times under peak load. The components of the set include load and response times.
The load is the volume of transactions processed by the application, e.g., transactions per second (tps), requests per second, pages per second. Without being loaded by computer-based demands for searches, calculations, transmissions, etc., most applications are fast enough, which is why programmers may not catch performance problems during development.The response times are the times required for an application to respond to a user’s actions at such a load.5: Communication and Collaboration:
I feel:
DevOps in an organization to succeed as a practice, requires 2 basic pillars, communication & collaboration, to work very effectively & efficiently. Without this feeling and understanding the importance of closely knit team work, it will be very difficult to adopt DevOps best practices .
Increased communication and collaboration in an organization is one of the key cultural aspects of DevOps. With this culture in place, teams come together with the right kind of attitude and motivation to set strong cultural norms around information sharing and facilitating communication through the communication tools & apps, allowing all parts of the organization to align more closely with common goals and objectives.
Why DevOps? What Are Its Benefits?
To understand the value addition DevOps and how it has been adopted by companies:
Let’s see the below infographics brought to you by veritis.
Above given infographics clearly lay down following key benefits of DevOps practice:
Speed:
DevOps facilitate organization to move at high velocity so you can innovate for customers faster, adapt to changing markets better, and grow more efficient at driving business results.
Quick Delivery:
With CI/CD based DevOps culture the application release cycle has been shortened, allowing quicker customer feedback and meaningful innovation to flourish within an organization. The quicker you can release new features and fix bugs, the faster you can respond to your customers’ needs and build competitive advantage.
Reliability:
DevOps empowers you to continuously improve upon your software quality through practices like continuous integration and continuous delivery to test that each change is functional and safe. This leads to the development of reliable & tested application & robust infra. DevOps continuous Monitoring and logging practices can help you to be well informed of your performance in real-time.
Culture:
DevOps fosters a great work culture to build more effective teams under its cultural model, which emphasizes values such as ownership and accountability.
Security:
By adopting DevOps model organization can use infrastructure as code and policy as code, to define and track compliances at scale, without sacrificing security. They can quickly progress while retaining control and preserving compliance.
DevOps Challenges:
Implementing DevOps culture in an organization is not easy. There is no standard set of rules to do so, it is more a game of changing the mindset of individuals and team of individuals. It’s like asking people to get out of their comfort zone.
“It may look difficult at the start when you try to bring any considerable change in the organization, but with great willingness changes do happen bringing progressive results. ”
So let’s see what are some of the common challenges In adopting DevOps as a culture.
1. Dev Vs Ops Mindset:
Since ages developer and operation team has been working in isolation with totally different set of task to perform. So they have been fine tuned to think and act differently. Devs trying to innovate and make changes as quickly as possible and operations trying to maintain 100% service levels. Their objectives and priorities were different, so if we have to bring DevOps as a cultural practice in organization, that very mindset they are two isolated stream has to be eclipsed
DevOps practice is all about integrating teams together and breaking down silos within IT organizations. So integrating them as a unified unit to target common objective is the first hurdle that any company needs to overcome as it adopts DevOps practices.
2. Moving From Legacy Infrastructure To Microservice Architecture
Legacy infrastructure has been existent in the companies for years, but if they have to innovate fast they have to shed this approach and adopt more scalable IT architecture of Microservices. Using Infrastructure-as-code together with microservices is yet another step towards a future of continuous innovation.
However, there is a high entry barrier to change to microservice driven system. Adopting Microservice architecture requires best DevOps practices along with CI/CD implementation to be in place. It brings enormous workloads and operational challenges for an organization along with increased cost factor.
Yes, It can be painful to shift to this modern approach of developing and deploying solutions, but once it is adopted it can transform your organization to become super efficient and scalable.
3. Conflict Of Dev and Ops Toolset:
Devs team has totally different objective and metrics to measure and so they may need a toolset which may not be required for Ops team. So it is imperative that both teams are brought together to understand where they both can collaborate and integrate tools which makes sense to both of them and to unify the objective and metrics they can monitor.
Some teams may be reluctant to part with legacy tools that are not only technologically inferior but that also slow down the entire infrastructure due to compatibility issues. So make sure, the tools that are being implemented are aligned well to the company’s product vision.
Summary:
No matter how different we are as an individual when it comes to company, we have to set aside our differences and work as a unit to achieve the big picture which is to care for the customer and solve their problems . If we can imbibe this philosophy in our organization’s work culture, DevOps will become invaluable practice valuing people over process and profit.
What’s Next?
We will look into CI-CD practices, tools and implementation strategy.
Will try to understand why they are important and how they are changing the fate of the companies in a bigger way.
Till then stay tuned……&