Recently I’ve been pondering the idea of cloud-like method of consumption of traditional (physical) networks. My main premise for this was that users of a network don’t have to wait hours or days for their services to be provisioned when all that’s required is a simple change of an access port. Let me reinforce it by an example. In a typical data center network, the configuration of the core (fabric) is fairly static, while the config at the edge can change constantly as servers get add...| networkop
Every time when I get bored from my day job I tend to find some small interesting project that I can do that can give me an instant sense of accomplishment and as the result lift my spirits and improve motivation. So this time I remembered when someone once asked me if they could use Terraform to control their physical network devices and I had to explain how this is the wrong tool for the job.| networkop
In the previous post I’ve demonstrated a special-purpose CNI plugin for network simulations inside kubernetes called meshnet. I’ve shown how relatively easy it is to build a simple 3-node topology spread across multiple kubernetes nodes. However, when it comes to real-life large-scale topology simulations, using meshnet “as is” becomes problematic due to the following reasons: Uploading topology information into etcd requires a lot of manual effort. Any customisation like startup conf...| networkop
Building virtualised network topologies has been one of the best ways to learn new technologies and to test new designs before implementing them on a production network. There are plenty of tools that can help build arbitrary network topologies, some with an interactive GUI (e.g. GNS3 or EVE-NG/Unetlab) and some “headless”, with text-based configuration files (e.g. vrnetlab or topology-converter). All of these tools work by spinning up multiple instances of virtual devices and interconnec...| networkop
Virtual networking has been one of the hottest areas of research and development in recent years. Kubernetes alone has, at the time of writing, 20 different networking plugins, some of which can be combined to build even more plugins. However, if we dig a bit deeper, most of these plugins and solutions are built out of two very simple constructs: a virtual switch - anything from a linux bridge through VPP and IOVisor to OVS ACL/NAT - most commonly implemented as iptables, with anything from n...| networkop
UPDATE: It appears that the issue described in this post has been fixed or at least it doesn’t manifest itself the same way it did back in 2018. Therefore, please treat this article purely as a historical record. Whether we like it or not, the era of DevOps is upon us, fellow network engineers, and with it come opportunities to approach and solve common networking problems in new, innovative ways.| networkop
Continuing on the trend started in my previous post about OpenDaylight, I’ll move on to the next open-source product that uses BGP VPNs for optimal North-South traffic forwarding. OpenContrail is one of the most popular SDN solutions for OpenStack. It was one of the first hybrid SDN solutions, offering both pure overlay and overlay/underlay integration. It is the default SDN platform of choice for Mirantis Cloud Platform, it has multiple large-scale deployments in companies like Workday and...| networkop
For the last 5 years OpenStack has been the training ground for a lot of emerging DC SDN solutions. OpenStack integration use case was one of the most compelling and easiest to implement thanks to the limited and suboptimal implementation of the native networking stack. Today, in 2017, features like L2 population, local ARP responder, L2 gateway integration, distributed routing and service function chaining have all become available in vanilla OpenStack and don’t require a proprietary SDN c...| networkop
In the ongoing hysteria surrounding all things SDN, one important thing gets often overlooked. You don’t build SDN for its own sake. SDN is just a little cog in a big machine called “cloud”. To take it even further, I would argue that the best SDN solution is the one that you don’t know even exists. Despite what the big vendors tell you, operators are not supposed to interact with SDN interface, be it GUI or CLI.| networkop
SFC is another SDN feature that for a long time only used to be available in proprietary SDN solutions and that has recently become available in vanilla OpenStack. It serves as another proof that proprietary SDN solutions are losing the competitive edge, especially for Telco SDN/NFV use cases. Hopefully, by the end of this series of posts I’ll manage do demonstrate how to build a complete open-source solution that has feature parity (in terms of major networking features) with all the major...| networkop
For quite a long time installation and deployment have been deemed as major barriers for OpenStack adoption. The classic “install everything manually” approach could only work in small production or lab environments and the ever increasing number of project under the “Big Tent” made service-by-service installation infeasible. This led to the rise of automated installers that over time evolved from a simple collection of scripts to container management systems.| networkop
A few weeks ago I bought myself a new Dell XPS-13 and decided for the n-th time to go all-in Linux, that is to have Linux as the main and only laptop OS. Since most of my Linux experience is with Fedora-family distros, I quickly installed Fedora-25 and embarked on a long and painful journey of getting out of my Windows comfort zone and re-establishing it in Linux. One of the most important aspects for me, as a network engineer, is to have a streamlined process of accessing network devices.| networkop
The idea of using Ansible for configuration changes and state verification is not new. However the approach I’m going to demonstrate in this post, using YANG and NETCONF, will have a few notable differences: I will not use any templates and absolutely no XML/JSON for device config generation All changes will be pushed through a single, vendor and model-independent Ansible module State verification will be done with no pattern-matching or screen-scraping All configuration and operational sta...| networkop
XML, just like many more structured data formats, was not designed to be human-friendly. That’s why many network engineers lose interest in YANG as soon as the conversation gets to the XML part. JSON is a much more human-readable alternative, however very few devices support RESTCONF, and the ones that do may have buggy implementations. At the same time, a lot of network engineers have happily embraced Ansible, which extensively uses YAML.| networkop
In the previous posts about NETCONF and RESTCONF I’ve demonstrated how to interact with Cisco IOS XE device directly from the Linux shell of my development VM. This approach works fine in some cases, e.g. whenever I setup a new DC fabric, I would make calls directly to the devices I’m configuring. However, it becomes impractical in the Ops world where change is constant and involves a large number of devices.| networkop
In the previous post I have demonstrated how to make changes to interface configuration of Cisco IOS XE device using the standard IETF model. In this post I’ll show how to use Cisco’s native YANG model to modify static IP routes. To make things even more interesting I’ll use RESTCONF, an HTTP-based sibling of NETCONF. RESTCONF primer RESTCONF is a very close functional equivalent of NETCONF. Instead of SSH, RESTCONF relies on HTTP to interact with configuration data and operational stat...| networkop
To kick things off I will show how to use ncclient and pyang to configure interfaces on Cisco IOS XE device. In order to make sure everyone is on the same page and to provide some reference points for the remaining parts of the post, I would first need to cover some basic theory about NETCONF, XML and YANG. NETCONF primer NETCONF is a network management protocol that runs over a secure transport (SSH, TLS etc.| networkop
Table of Contents OpenStack - virtual network topology OVN Northbound DB - logical network topology OVN Southbound DB - logical flows L2 datapath L3 datapath OVN Controller - OpenFlow flows Physical network - GENEVE overlay Conclusion OpenStack - virtual network topology In the previous post we have installed OpenStack and created a simple virtual topology as shown below. In OpenStack’s data model this topology consists of the following elements:| networkop
Vanilla OpenStack networking has many functional, performance and scaling limitations. Projects like L2 population, local ARP responder, L2 Gateway and DVR were conceived to address those issues. However good a job these projects do, they still remain a collection of separate projects, each with its own limitations, configuration options and sets of dependencies. That led to an effort outside of OpenStack to develop a special-purpose OVS-only SDN controller that would address those issues in ...| networkop
News about UnetLab Those who read my blog regularly know that I’m a big fan of a network simulator called UnetLab. For the last two years I’ve done all my labbing in UNL and was constantly surprised by how extensible and stable it has been. I believe that projects like this are very important to our networking community because they help train the new generation of network engineers and enable them to expand their horizons.| networkop
Table of Contents Virtual topology overview Virtual topology setup non-DVR traffic flow Enabling DVR DVR East-West traffic flow External connectivity Case 1 - Overload NAT (VM2 with no FIP) Case 2 - Static NAT (VM1 with FIP) DVR Pros and Cons To be honest I was a little hesitant to write this post because the topic of Neutron’s DVR has already been exhaustively covered by many, including Assaf Muller, Eran Gampel and in the official OpenStack networking guide.| networkop
In the last post we’ve seen how to use Chef to automate the build of a 3-node OpenStack cloud. The only thing remaining is to build an underlay network supporting communication between the nodes, which is what we’re going to do next. The build process will, again, be relatively simple and will include only a few manual steps, but before we get there let me go over some of the decisions and assumptions I’ve made in my network design.| networkop
Now that I’m finally beginning to settle down at my new place of residence I can start spending more time on research and blogging. I have left off right before I was about to start exploring the native OpenStack distributed virtual routing function. However as I’d started rebuilding my OpenStack lab from scratch I realised that I was doing a lot of repetitive tasks which can be easily automated. Couple that with the fact that I needed to learn Chef for my new work and you’ve got this b...| networkop
Since I have all my OpenStack environment running inside UNetLab, it makes it really easy for me to extend my L3 fabric with a switch from another vendor. In my previous posts I’ve used Cisco and Arista switches to build a 4-leaf 2-spine CLOS fabric. For this task I’ve decided to use a Cumulus VX switch which I’ve downloaded and imported into my lab. To simulate the baremetal server (10.| networkop
Provider vs Tenant networks Before we start, let’s recap the difference between the two major Neutron network types: Tenant networks are: provisioned by tenants used for inter-VM (east-west) communication use Neutron virtual router as their default gateway Provider networks are: provisioned by OpenStack administrator(for use by tenants) match existing physical networks can be either flat (untagged VLAN) or VLAN-based (multiple VLANs) need to be L2 adjacent to network and/or compute nodes Th...| networkop
MAC learning in a controller-less VXLAN overlay VXLAN standard does not specify any control plane protocol to exchange MAC-IP bindings between VTEPs. Instead it relies on data plane flood-and-learn behaviour, just like a normal switch. To force this behaviour in an underlay, the standard stipulates that each VXLAN network should be mapped to its own multicast address and each VTEP participating in a network should join the corresponding multicast group.| networkop
Table of Contents Intro High Level Overview Unicast frame between VM1 and VM2 Unicast frame between VM1 and External host BUM frame from VM1 for MAC address of VM2 OpenvSwitch quick intro Detailed packet flow analysis Enumerating OVS ports Unicast frame between VM1 and VM2 Unicast frame to external host (192.168.247.1) BUM frame from VM1 for MAC address of VM2 Native OpenStack SDN advantages and limitation Things to explore next C2O References Intro This is going to be quite a lengthy blogpos...| networkop
OpenStack network requirements Depending on the number of deployed components, OpenStack physical network requirements could be different. In our case we’re not going to deploy any storage solution and simply use the ephemeral storage, i.e. hard disk that’s a part of a virtual machine. However, even in minimal installations, there are a number of networks that should be considered individually due to different connectivity requirements: Server OOB management network - this is usually a de...| networkop
What the hell am I trying to do? I admit that running OpenStack on anything other than baremetal is nonsense. So why would anyone want to run it with two layers of virtualisation underneath? My goal is to explore some of the new SDN/NFV technologies without leaving the confines of my home area network and/or racking up a triple-digit electricity bill. I also wanted to be able to swap underlay networks without spending hours trying to plumb together virtualized switches and servers from multip...| networkop
Current network overview Let’s start by taking a high-level look at our DC network routing topology. The core routing protocol is OSPF, it is responsible for distributing routing information between the Core and WAN layers of the network. WAN layer consists of two MPLS L3VPN services running BGP as PE-CE protocol and two DMVPN Hubs running EIGRP. All WAN layer devices perform mutual redistribution between the respective WAN protocol and OSPF.| networkop
Demo network overview The network consists of 4 nodes interconnected via point-to-point links and running EIGRP as a routing protocol. To create a local development environment you can clone my repository and reset it to work with your own Github account using git remote set-url origin https://github.com/USERNAME/OTHERREPOSITORY.git command. Local development environment contains the following files describing the modelled topology: Configuration files for each node under the ./config directo...| networkop
Packer intro Packer is a tool that can automatically create virtual machines for different hypervisors and cloud platforms. The goal is to produce identically configured VMs for either VirtualBox, VMWare, Amazon or Google clouds based on a single template file. If you’re familiar with Vagrant, then you can also use Packer to create custom Vagrant boxes. In our case, however, we’re only concerned about VMWare since it’s the only type-2 hypervisor that supports nested hardware virtualisat...| networkop
CI/CD vs ITIL How do you implement changes in your network? In today’s world there’s 95% chance that you have to write up an RFC, submit it at least a week before the planned implementation date, go through at least one CAB meeting and only then, assuming it got approved, can you implement it. But the most important question is ‘how do you test’? Do you simply content yourself with a few pings or do you make sure all main routes are in place?| networkop
Extracting Node’s UUID In the previous post we have learned how to create a Node. To perform further actions on it we need to know it’s UUID. According to HTTP specification 201 - Created response SHOULD return a Location header with resource URI, which would contain resource UUID. However, UNetLab’s implementation does not return a Location header so we need to extract that information ourselves. To do that we’ll use the previously defined .| networkop
REST SDK Design As it is with networks, design is a very crucial part of programming. I won’t pretend to be an expert in that field and merely present the way I’ve built REST SDK. Fortunately, a lot of design will mimic the objects and their relationship on the server side. I’ll slightly enhance it to improve code re-use and portability. Here are the basic objects: RestServer - implements basic application-agnostic HTTP CRUD logic UnlServer - an extension of a RestServer with specific a...| networkop
UnetLab Installation Since UNL is a separate project with its own evolving documentation I won’t try to reproduce it in my blog and I’ll simply refer all my readers to UNL download page, UNL installation instructions and UNL first boot configuration. At the time of writing UNL is distributed as an image packaged in Open Virtualization Format. I’m using VMWare Workstation as a type-2 hypervisor to import and run this image.| networkop
Management interface evolution Since the early dawn of networking, devices have been configured through VTYs. The transport has evolved from telnet to ssh but the underlying rule still maintained that network is configured manually, device-by-device by a human administrator. It’s obvious that this approach does not scale and is prone to human error, however it still remains the most prevalent method of network device configuration. The first attempt to tackle these issues has been made in 1...| networkop
In this post I will also introduce two concepts that are frequently used in software development world - DRY and “Convention over Configuration”. This post is a precursor to the upcoming FlexVPN configuration post on Packetpushers. FlexVPN network overview FlexVPN topology will consist of two FlexVPN “clouds”. Each cloud has a Hub router and multiple Spokes. Each Spoke is connected to each of the two Hubs thereby participating in both FlexVPN clouds.| networkop
This is where configuration may get particularly messy especially in presence of backdoor links and complex routing failover policies. However, as I will show, it is still possible to create a standard set of routing manipulation policies and selectively apply them to the required adjacencies to achieve the desired effect. Requirements and assumptions The new office network is designed with several layers of WAN redundancy. Primary WAN link is the preferred option to reach all other WAN desti...| networkop
Prerequisites It is assumed that by this time all detailed network design information is known including interfaces numbers, VLANs, IP addresses and LAGs. This information will be used as an input to configuration automation scripts. The inventory file is updated with a new branch-2 group [branch-2] BR2-CORE ansible_ssh_host=10.0.3.1 BR2-WAN1 ansible_ssh_host=10.0.3.2 BR2-WAN2 ansible_ssh_host=10.0.3.3 BR2-AS01 ansible_ssh_host=10.0.3.130 BR2-AS02 ansible_ssh_host=10.0.3.131 BR2-AS03 ansible_...| networkop
A lot of configuration files referenced throughout this post will be omitted for the sake of brevity, however all of them can be found in my github repository. Legacy network overview The network I’m using for demonstration is a cut-down version of a typical enterprise network. At this point of time it consists of a branch office network and a central DC network interconnected via redundant WAN links. The branch office consists of a main computer room hosting all core network devices and in...| networkop
Automating Network Configuration Automation and programmability steadily make their way into a networking domain. The idea was born in application development world where makefiles served a role of automated installation scripts. It later spread into application testing and deployment so now hardly anyone does these two things by hand. Next in line were the operating systems largely thanks to the raising popularity of PaaS solutions. Until recently network configuration has been the prerogati...| networkop
Network overview Let’s assume you’re working in a proverbial Acme Inc. It has a Data Centre hosting all centralised services and a single office branch (Branch #1). Sites are interconnected using active/backup WAN links. The company decides to expand and adds a new office in a city nearby. In additional to standard dual WAN links it’s possible to buy a cheap and high throughput backdoor link between the two branches.| networkop
Now that Ansible has done all the information gathering for us it’s time to finally make use of it. In this post I will show how to use Ansible to run traceroutes from and to the hosts defined in a test scenario and perform verification of the results of those tests. Should any of those tests fail, Ansible will provide a meaningful description of what exactly failed and why. While doing all this I’ll introduce a couple of new Ansible features like conditional looping and interactive prompts.| networkop
Ansible has a very neat feature called “fact gathering”, which collects useful information from hosts prior to executing any of the tasks and makes this information available for use within those tasks. Unfortunately, this also relies on Python being available on the remote machine which doesn’t work for Cisco IOS. In this post I’ll show how to write a simple module which will collect IP address information from remote devices and store it in global variable for future use.| networkop
Ansible is well-known for it’s low entry threshold. All what’s required to get started is just one inventory file. However Cisco IOS devices require special considerations. Passwordless SSH RSA-based authentication is still a novelty and in most cases users are authenticated based on their passwords. Another problem is the lack of Python execution environment on IOS devices, which seriously limits the choice of Ansible modules that can be used. In this post I will show how to setup Ansibl...| networkop
My development environment usually consists of a host machine running Windows and a development Linux “headless” virtual machine. I create and edit files in a Notepad++ text editor and then transfer them over to the Linux VM. Until recently I’ve been using a hypervisor-enabled “shared” folder. However, Windows file system emulators in Linux do not support symbolic links and therefore breaks a lot of applications that rely on them. This prompted me to start looking for a new way to s...| networkop
Before we proceed with TDD framework build it is important to have the development environment setup. In our case it will consist of two major components: Network Simulation Environment Ansible Development Environment To simplify things I will run both of these environments on the same Virtual Machine. For network simulation I will use UnetLab, a wonderful product developed by Andrea Dainese. Currently, UnetLab is distributed as an OVA package and is available for free download on the website.| networkop
Before we begin (optional section) Before we go on, I’d like to put a little disclaimer about terms being used in this post. TDD, and its counterpart BDD, are well-known and accepted practices in development world. Both rely on the assumption that tests will be written before the code and will drive code development. This seemingly unnatural approach became extremely popular with the advent of Agile and is still being widely used, specifically in web development.| networkop
BGP fall-over is a neat BGP convergence optimisation technique whereby BGP peering is brought down as soon as the route to neighbor disappears from a routing table. The difference between external and internal BGP is that the former usually peers over a directly-attached interface so that when the interface to neighbor is disconnected, route is withdrawn from the routing table which triggers eBGP fall-over to bring down the neighborship. iBGP, on the other hand, normally uses device loopbacks...| networkop
With the amount of configuration involved in a typical L3VPN configuration, troubleshooting process can get pretty chaotic, especially in a time-constrained environments like CCIE lab. That’s why it is extremely important to have a well-structured approach to quickly narrow down the potential problem area. I used the below algorithm while preparing for my lab exam. Like most of the networking problems, troubleshooting of L3VPNs can and must be split into two different phases - control plane...| networkop
What motivated me to write this post is a state of the IP routing of some of the enterprise networks I’ve seen. A quick show ip route command reveals a non-disentanglable mixture of dynamic and static route with multiple points of redistribution and complex, rigid filtering rules, something you’d only see in your bad dream or a CCIE-level lab. It certainly takes a good engineer to understand how it works and even that can take up to several hours.| networkop
Any network device, be it a transit router or a host, usually has multiple IP addresses assigned to its interfaces. One of the first things we learn as network engineers is how to determine which IP address is used for the locally-sourced traffic. However, the default scenario can be changed in a couple of different ways and this post is a brief documentation of the available options. The Default Scenario Whenever a local application decides to connect to a remote network endpoint, it creates...| networkop
In the previous post, I mentioned that CUE can help you work with both “industry-standard” semi-structured APIs and fully structured APIs where data is modelled using OpenAPI or JSON schema. However, there was an elephant in the room that I conveniently ignored but without which no conversation about network automation would be complete. With this post, I plan to rectify my previous omission and explain how you can use CUE to work with YANG-based APIs.| networkop
What I’ve covered in the previous blog post about CUE and Ansible were isolated use cases, disconnected islands in the sea of network automation. The idea behind that was to simplify the introduction of CUE into existing network automation workflows. However, this does not mean CUE is limited to those use cases and, in fact, CUE is most powerful when it’s used end-to-end — both to generate device configurations and to orchestrate interactions with external systems.| networkop
Hardly any conversation about network automation that happens these days can avoid the topic of automation frameworks. Amongst the few that are still actively developed, Ansible is by far the most popular choice. Ansible ecosystem has been growing rapidly over the last few years, with modules being contributed by both internal (Redhat) and external (community) developers. Having the backing of one of the largest open-source first companies has allowed Ansible to spread into all areas of infra...| networkop
In the past few years, network automation has made its way from a new and fancy way of configuring devices to a well-recognized industry practice. What started as a series of “hello world” examples has evolved into an entire discipline with books, professional certifications and dedicated career paths. It’s safe to say that today, most large-scale networks (>100 devices) are at least deployed (day 0) and sometimes managed (day 1+) using an automated workflow.| networkop
In one of his recent posts, Ivan raises a question: “I can’t grasp why Cumulus releases a Vagrant box, but not a Docker container”. Coincidentally, only a few weeks before that I had managed to create a Cumulus Linux container image. Since then, I’ve done a lot of testing and discovered limitations of the pure containerised approach and how to overcome them while still retaining the container user experience. This post is a documentation of my journey from the early days of running Cu...| networkop
eBPF has a thriving ecosystem with a plethora of educational resources both on the subject of eBPF itself and its various application, including XDP. Where it becomes confusing is when it comes to the choice of libraries and tools to interact with and orchestrate eBPF. Here you have to select between a Python-based BCC framework, C-based libbpf and a range of Go-based libraries from Dropbox, Cilium, Aqua and Calico. Another important area that is often overlooked is the “productionisation...| networkop
When using a personal VPN at home, one of the biggest problems I’ve faced was the inability to access public streaming services. I don’t care about watching Netflix from another country, I just want to be able to use my local internet connection for this kind of traffic while still encrypting everything else. This problem is commonly known in network engineering as “local internet breakout” and is often implemented at remote branch/edge sites to save costs of transporting SaaS traffic...| networkop
There comes a time in the life of every Kubernetes cluster when internal resources (pods, deployments) need to be exposed to the outside world. Doing so from a pure IP connectivity perspective is relatively easy as most of the constructs come baked-in (e.g. NodePort-type Services) or can be enabled with an off-the-shelf add-on (e.g. Ingress and LoadBalancer controllers). In this post, we’ll focus on one crucial piece of network connectivity which glues together the dynamically-allocated ext...| networkop
Every Kubernetes cluster is provisioned with a special service that provides a way for internal applications to talk to the API server. However, unlike the rest of the components that get spun up by default, you won’t find the definition of this service in any of the static manifests and this is just one of the many things that make this service unique. The Special One To make sure we’re on the same page, I’m talking about this:| networkop
Last week I posted a tweet about a Kubernetes networking puzzle. In this post, we’ll go over the details of this puzzle and uncover the true cause and motive of the misbehaving ingress. Puzzle recap Imagine you have a Kubernetes cluster with three namespaces, each with its own namespace-scoped ingress controller. You’ve created an ingress in each namespace that exposes a simple web application. You’ve checked one of them, made sure it works and moved on to other things.| networkop
Cluster API (CAPI) is a relatively new project aimed at deploying Kubernetes clusters using a declarative API (think YAML). The official documentation (a.k.a. the Cluster API book), does a very good job explaining the main concepts and goals of the project. I always find that one of the best ways to explore new technology is to see how it works locally, on my laptop, and Cluster API has a special “Docker” infrastructure provider (CAPD) specifically for that.| networkop
In September 2019 I had the honour to present at Open Networking Summit in Antwerp. My talk was about meshnet CNI plugin, k8s-topo orchestrator and how to use them for large-scale network simulations in Kubernetes. During the same conference, I attended a talk about Network Service Mesh and its new kernel-based forwarding dataplane which had a lot of similarities with the work that I’ve done for meshnet. Having had a chat with the presenters, we’ve decided that it would be interesting to ...| networkop
In the previous two posts, we’ve seen how to build a custom network API with Kubernetes CRDs and push the resulting configuration to network devices. In this post, we’ll apply the final touches by enabling oAuth2 authentication and enforcing separation between different tenants. All of these things are done while the API server processes incoming requests, so it would make sense to have a closer look at how it does that first.| networkop
In the previous post, we’ve examined the foundation of the Network-as-a-Service platform. A couple of services were used to build the configuration from data models and templates and push it to network devices using Nornir and Napalm. In this post, we’ll focus on the user-facing part of the platform. I’ll show how to expose a part of the device data model via a custom API built on top of Kubernetes and how to tie it together with the rest of the platform components.| networkop
In the previous post I’ve demonstrated how to build virtual network topologies on top of Kubernetes with the help of meshnet-cni plugin. As an example, I’ve shown topologies with 50 cEOS instances and 250 Quagga nodes. In both of these examples virtual network devices were running natively inside Docker containers, meaning they were running as (a set of) processes directly attached to the TCP/IP stack of the network namespace provided by the k8s pod.| networkop