NetDevOps on Azure

This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.

Introduction

With every company, across every industry, digitally transforming, tons of modern applications are built at an unprecedented pace and speed, and all rely on the underlying network infrastructure.

Networking enables application components to communicate with each other, its dependencies, other applications (usually through APIs) and its consumers.

To gain abstraction and velocity, more and more organizations are moving from traditional datacentre networks into cloud networks, adopting a wide variety of cloud networking services. NetDevOps comes to the rescue, as an agile approach to help you accelerate your Azure networking deployments and operations.

But before delving into this approach, let’s briefly describe what is in its genesis – DevOps - and understand some of its benefits.

DevOps

DevOps is, first and foremost, a culture with a foundation in the Agile Manifesto that focuses on collaboration and communication between teams responsible for software development and IT operations. It combines development (Dev) and operations (Ops) to unite people, process, and technology in application planning, development, delivery and operations, to enable continuous delivery of value to your end users.

The goal of DevOps is to improve effectiveness, add velocity and shorten the software development lifecycle, by using Infrastructure-as-Code (IaC), automation and rigorous testing, Continuous Integration (CI) and Continuous Delivery (CD), as well other tools and practices to streamline and integrate the different stages of the process, from writing code to deploying and managing applications and their underlying infrastructure.

andrefgp_1-1675251706991.png

 

Figure 1 - Diagram illustrating the phases of the DevOps application lifestyle.

 

When implemented effectively, organizations adopting DevOps can reap a variety of benefits, including, but not limited to:  

  • Increased communication, collaboration and trust.
  • Faster delivery of new software features and updates.
  • Greater flexibility and agility in responding to changing business needs.
  • Improved quality and reliability of software applications and underlying infrastructure.
  • Increased efficiency and productivity of software development and IT operations teams.
  • Accelerate time-to-resolution of issues, reducing services downtime.
  • Greater observability and control over IT infrastructure and application performance.
  • Higher predictability of infrastructure and application behavior in production.

For more curious readers, feel free to read about how Microsoft plans, develops, delivers and operates reliable software and systems with DevOps.

 

NetDevOps

Unsurprisingly, NetDevOps is the combination of Networking with DevOps (Networking + DevOps), bringing DevOps principles and practices to the deployment, management and operation of network services.

Traditionally, network infrastructure deployment and operations are carried over manually and sequentially by networking teams, making it slow and prone to various risks and issues (fat fingers, repetitive tasks, misalignment with the development team’s objectives,  etc.).

What’s in it for me?

As a cloud/network engineer or architect not following NetDevOps principles, you have probably experienced one (or many) of the following unfortunate events:

  • A network change breaking the whole thing (services going down and a lot of messy troubleshooting trying to get them back up).
  • You’re asked to implement a change but the lack of current setup’s knowledge makes you afraid to take the leap.
  • Lacking permissions to deploy or update parts of the network.
  • Deploying the same infrastructure over and over, performing the same set of repetitive tasks whenever a new project takes off.
  • Lacking visibility and holistic control over the network.

As an application developer on Azure not following NetDevOps principles, you have probably experienced one (or many) of the following unfortunate events:

  • Long waiting for a simple network change, such as adding a firewall/NSG rule, or a more complex deployment, like an ExpressRoute connection.
  • Network engineers or architects are not available to attend your needs.
  • Bad application performance due to unknown/unplanned network events or bottlenecks.

NetDevOps allows you to deploy, maintain and operate networking services reliably and quickly to keep up with the demanding business requirements and the high velocity of application development. Its benefits include the ability to deliver new network services and updates more quickly and reliably, as well as improved visibility and holistic control over the network.

It goes without saying that, very much like DevOps, successfully adopting NetDevOps will require processes and tools adherence, but mostly a “culture of change” in your organization. Let’s dive right into the main principles and best-practices around NetDevOps.

 

NetDevOps Principles, Tools and Best-Practices

Hereby, you’ll find some of the most important principles, tools and best-practices your organization can leverage when adopting and implementing NetDevOps.

 

  1. Culture - Promote a culture of complete ownership, accountability and responsibility among teams. Set up continuous feedback systems that will help improve processes and escalate issues.
  2. Network segmentation - Favour dedicated Virtual Networks (VNets) (per workload, per environment) over very large VNets, so that changes made to one environment do not impact others. The idea is to build modular architectures that scale, rather than a “monolithic” architecture that gets too complex quickly. Avoid creating subnets too small to account for future growth of your workloads though - Logically segment subnets. Azure Cloud Adoption Framework Enterprise Scale Landing Zones has these principles in mind. You may want to read further about it here.

  3. Open communications - NetDevOps incentivizes constant and open communications across teams. Conway’s Law reminds us that “Organizations, who design systems, are constrained to produce designs which are copies of the communication structures of these organizations.”
  4. Implement end-to-end automation - Automation increases velocity, reduces overhead and the chance of human errors, and streamlines manual tasks. Automate end-to-end using IaC and CI/CD. Also, leverage Azure Policies for configuration automation and corporate guardrails enforcement post network infrastructure deployments.

    1. Use Infrastructure-as-Code (IaC) to define and deploy your network infrastructure - IaC is a significant component of NetDevOps. It avoids manual configuration and enforces consistency by representing desired environment states via well-documented code. Network infrastructure deployments and configurations with IaC are predictable, repeatable and consistent. To make changes, the team edits the source, not the target. The infrastructure code itself can be validated and tested to prevent common deployment issues. Additionally, writing your network as code enables version control and lets you track changes. It also lets you build up a centrally managed portfolio of solution templates that can be reused by different application teams to provision their own network infrastructure. Note that centrally managed templates can be hardened and approved by a cloud operations or security team to meet internal standards and guardrails. Azure provides native support for IaC via the Azure Resource Manager model. Teams can define declarative ARM templates that specify the infrastructure required to deploy solutions. ARM templates can be developed in JSON or Bicep, the later having a simpler syntax when compared to the former. Third-party platforms like TerraformAnsibleChef, and Pulumi also support IaC to manage automated infrastructure. Refer to this article for further information on repeatable infrastructure deployment.

    2. Continuous Integration (CI) and Continuous Delivery (CD) – Use CI/CD pipelines to accelerate the whole process from code development to deploying in production. Feel free to use Azure Pipelines or GitHub Actions to build your CI/CD pipelines. As an example, see how to Deploy to Azure with IaC and GitHub Actions. Pipelines automate several stages, from code validation to security/reliability checks and testing, from staging to production. Useful tests in the context of NetDevOps include (some may require manual intervention and/or approval gates):
      • Validate templates against recommended practices.
      • Leverage Defender for DevOps (Preview) to scan your IaC code to detect misconfigurations, security vulnerabilities and mitigate risks prior to deployment in production environments.
      • Validate configuration against enterprise policies/guardrails. Some examples include:
        • Check if firewall/NSG rules are over permissive (e.g. RDP/SSH ports open to the internet).
        • Check if public access is disabled by default in every subnet.
        • Ensure too large address ranges are not used (avoid IP space waste).
        • Look for address range overlaps.
        • Check if every network appliance has redundancy built-in.
        • Check if network watcher / NSG flow logs are enabled by default in every VNet/subnet.
        • Check if VNet peerings are connecting VNets from different environments (prod and dev, e.g).
      • After deployment to a dev environment, you may want to ensure that the changes you made or the infrastructure you deployed work as intended. Simply use ping/traceroute tests (make sure to allow ICMP in your firewall) or leverage Azure Network Watcher tools to validate intended connectivity and routing. You may also want to perform some load and performance testing at this stage. Finally, you must check the holistic health of your setup and rollback in case of any issues.
  1. Observability – Knowing what goes in and out of your network is essential to maintaining your security posture and to assessing your network performance in every environment. You should capture all inbound and outbound traffic and perform near real-time analysis on that traffic to detect threats, defects and mitigate network vulnerabilities and/or performance issues. Azure Network Watcher provides an entire suite of tools to visualize, monitor, diagnose, and troubleshoot network issues across Azure and Hybrid cloud environments. Please refer to this article for further design consideration and recommendations when it comes to planning for traffic inspection.

With NetDevOps, you shift left the important operations of modern network infrastructure to early in the development cycle, detecting vulnerabilities and risks before deployment, adding governance policy checks as part of your pipeline, incorporating reliability into de design and adding continuous monitoring right into code. This means less issues post-deployment, which results in less downtime and more reliable and secure networks.

 

andrefgp_2-1675251776952.png

Figure 2 - Shifting left the operations of modern infrastructure

 

Conclusion

NetDevOps brings the DevOps principles and tools into the networking realm. It  can be leveraged to accelerate your Azure networking deployments and operations, letting you deliver applications at a higher speed in the cloud. Your teams will spend less time implementing and troubleshooting your networks and more time developing innovative applications, bringing more value to the organization.

We have seen that, to successfully adopt and implement NetDevOps, organizations must go through a culture change and adopt principles and tools such as IaC, automation, CI/CD and continuous monitoring.

Start simple and build up. Foster constant communication and collaboration. Stay patient, curious and open-minded throughout your NetDevOps journey!

Thanks for reading! Feel free to leave your comments below. 

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.