Understanding Modern Application Architecture - Part 1
This is part 1 of a series. Here are the other parts: Understanding Modern Application Architecture - Part 2 Understanding Modern Application Architecture - Part 3 Over the past decade, there has been a change taking place in how applications are built. As applications become more expansive in capabilities and more critical to how a business operates, (or in many cases, the application is the business itself) a new style of architecture has allowed for increased scalability, portability, resiliency, and agility. To support the goals of a modern application, the surrounding infrastructure has had to evolve as well. Platforms like Kubernetes have played a big role in unlocking the potential of modern applications and is a new paradigm in itself for how infrastructure is managed and served. To help our community transition the skillset they've built to deal with monolithic applications, we've put together a series of videos to drive home concepts around modern applications. This article highlights some of the details found within the video series. In these first three videos, we breakdown the definition of a Modern Application. One might think that by name only, a modern application is simply an application that is current. But we're actually speaking in comparison to a monolithic application. Monolithic applications are made up of a single, or a just few pieces. They are rigid in how they are deployed and fragile in their dependencies. Modern applications will instead incorporate microservices. Where a monolithic application might have all functions built into one broad encompassing service, microservices will break down the service into smaller functions that can be worked on separately. A modern application will also incorporate 4 main pillars. Scalability ensures that the application can handle the needs of a growing user base, both for surges as well as long term growth. Portability ensures that the application can be transportable from its underlying environment while still maintaining all of its functionality and management plane capabilities. Resiliency ensures that failures within the system go unnoticed or pose minimal disruption to users of the application. Agility ensures that the application can accommodate for rapid changes whether that be to code or to infrastructure. There are also 6 design principles of a modern application. Being agnostic will allow the application to have freedom to run on any platform. Leveraging open source software where it makes sense can often allow you to move quickly with an application but later be able to adopt commercial versions of that software when full support is needed. Defining by code allows for more uniformity of configuration and move away rigid interfaces that require specialized knowledge. Automated CI/CD processes ensures the quick integration and deployment of code so that improvements are constantly happening while any failures are minimized and contained. Secure development ensures that application security is integrated into the development process and code is tested thoroughly before being deployed into production. Distributed Storage and Infrastructure ensures that applications are not bound by any physical limitations and components can be located where they make the most sense. These videos should help set the foundation for what a modern application is. The next videos in the series will start to define the fundamental technical components for the platforms that bring together a modern application. Continued in Part 2Understanding Modern Application Architecture - Part 3
In this last article of the series discussing Modern Application Architecture, we will be discussion manageability with respect to the traffic. As the traffic patterns grow and look quite different from monolithic applications, different approaches need to take place in order to maintain the stability of the application. Understanding Modern Application Architecture - Part 1 Understanding Modern Application Architecture - Part 2 In this next video, we discuss Service Mesh. As modern applications expand and their communications change to microservice to microservice, a service mesh can be introduced to provide control, security and visibility to that traffic. Since individual microservices can be written by different individuals or groups, the service mesh can be the intermediary that allows them to understand what is happening when one piece of code needs to speak to another piece of code. At the same time, trust and verification can happen between the microservices to ensure they are talking to what they should be talking to. In this next video we discuss Sidecar Proxies. As mentioned in the Service Mesh video, the sidecar proxy is a key piece of the mesh implementation. It is responsible for functions such as TLS termination, mutual TLS and authentication. It can also be used for tracing and other observability. This means these functions don't have to be performed by the microservice itself. In this final video, we review NGINX as a Production Grade Kubernetes Solution. While Modern Applications will adopt Open Source Solutions where possible, these applications can be mission critical ones that require the highest level of service. As mentioned in the previous videos in this series, there are a number of important pieces of a Kubernetes cluster that can be augmented, or replaced by enhanced services. NGINX can actually perform as an enhanced Ingress Controller, giving a high level of control as well as performance for inbound traffic to the cluster. NGINX with App Protect can also provide finer grain controlled web application security for the inbound web based components of the application. And finally, NGINX Service Mesh can help with the microservice to microservice control, security and visibility, offloading that function from the microservice itself. We hope that this video series has helped shed some light for those who are curious about modern application architecture. As you have questions, don't hesistate to ask in our Technical Forums!
Understanding Modern Application Architecture - Part 2
To help our Community transfer their skills to handle Modern Applications, we've released a video series to explain the major points. This article is part 2 and here are the other parts: Understanding Modern Application Architecture - Part 1 Understanding Modern Application Architecture - Part 3 This next set of videos discuss the platforms and components that make up modern applications. In this video, we review containers. These have become a key building block of microservices. They help achieve the application portability by neatly packaging up everything needing to bring up an application within a container runtime such as Docker. One great example of a container is the f5-demo-httpd container. This small lightweight container can be downloaded quickly to run a web server. It's incorporated into a lot of F5 demo environments because it is lightweight and can be customized by simply forking the repository and making your own changes. In this next video, we talk about Kubernetes (or k8s for short). While there are container runtimes like Docker that can work individually on a server, the Kubernetes project has brought the concept into a form that can be scaled out. Worker nodes, where containers are run on, can be brought together into clusters. Commands can be issued to a Master Node via YAML files and have affect across the cluster. Containers can be scheduled efficiently across a cluster which is managed as one. In this next video, we break down the Kubernetes API. The Kubernetes API is the main interface to a k8s cluster. While there are GUI solutions that can be added to a k8s cluster, they are still interfacing with the API so it is important to understand what the API is capable of and what it is doing with the cluster. The main way to issue commands to the API is through YAML files and the kubectl command. From there, the API server will interact with the other parts of the cluster to perform operations. In this next video, we discuss Securing a Kubernetes cluster. There are a number of attack vectors that need to be understood and so we review them along with some of the actions that can be taken in order to increase the security for them. In this next video, we go over Ingress Controller. An Ingres Controller is one of the main ways that traffic is brought from outside of the cluster, into a pod. This role is of particular interest to F5 customers as they can use NGINX, NGINX+ or BIG-IP to play this strategic role within a Kubernetes cluster. In this next video, we talk about Microservices. As applications are decomposed from monolithic applications to modern applications, they are broken up into microservices that carry out individual functions of an application. The microservices then communicate with each other in order to deliver the overall application. It's important to then understand this service to service communication so that you can design application services around them such as load balancing, routing, visibility and security. We hope that you've enjoyed this video series so far. In the next article, we'll be reviewing the components that aid in the management of a Kubernetes platform.Understanding Modern Application Architecture - Part 3
When cloud-native meets monolithic
According to CNCF’s Cloud Native Survey 2020 published on 17 Nov 2020, containers in production jump 300% from the first survey in 2016. Year 2020 itself increased to 92% from 84% in 2019. (https://www.cncf.io/cncf-cloud-native-survey-2020). In addition, according to F5's 2020 State of Application Services Report (https://www.f5.com/state-of-application-services-report#get-the-report), 80% of organisations are executing on digital transformation and of this organisation, there are more likely to deploy modern app architecture and app services at higher rate. So, cloud-native modern application architecture is gaining great momentum. Industries and majority organisation embracing and pivoting toward cloud-native technologies. Cloud-native provides multitude of benefits – which is not the subject of this article. F5’s BIG-IP (a.k.a classic BIG-IP) is not cloud-native. How F5’s classic BIG-IP be relevant in the cloud-native world? This article demonstrates how cloud-native meets and needs classic BIG-IP (monolithic). FYI: F5’s BIG-IP SPK (Service Proxy for Kubernetes) is BIG-IP delivered in a containerized form factor. It is cloud-native (https://www.f5.com/products/service-proxy-for-kubernetes). BIG-IP SPK will be discussed in future article. How both of them need each other? Typically, it take years for organisation who embracing cloud-native to move into a fully cloud-native technologies/infrastructure. There are use cases where modern cloud-native application needs to integrate with traditional or existing monolithic applications. Modern apps living along with traditional apps or infrastructure are common for most enterprises. F5’s classic BIG-IP can bridge those gaps. This article about use cases that we solved in one of our customer environment where how we leverage classic BIG-IP to bridge gaps between cloud-native and monolithic apps. First, let’s be clear on what cloud-native really means. To set the record straight, cloud-native doesn’t just mean running workload in the cloud, although it is partially true. There are many definition and perspective on what cloud-native really means. For the sake of this article, I will base on the official definition of cloud-native from the CNCF (Cloud Native Computing Foundation), which defines “Cloud native technologies empower organisations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil” My takeaway (characteristic of cloud-native): Scalable apps that design to run in dynamic environment (public, private and hybrid clouds). Typically delivered in the form of microservices/containers which loosely coupled systems. Easily adapted and integrated into automation system. CI/CD part of it ecosystem – frequent release, patch and updates cycle. Immutable (cattle service model) instead of mutable (Pets service model) Declarative API Kubernetes is one of the example of a cloud-naive technologies. What is the Problem Statement? Uniquely identify apps/workload/containers deployed in Kubernetes platform and apply appropriate external security control (e.g. network/application firewall) for containerized apps when it communicates with existing legacy applications deployed outside of Kubernetes (egress from Kubernetes). What are those challenges? Containers that egress off Kubernetes are by design source network address translated (SNAT) from Kubernetes nodes. External security control system such as network firewall may not be able to identify the right authorised source apps as it is hidden behind NATing. How to ensure that only authorised apps deployed in Kubernetes environment authorised to access critical legacy apps (e.g. billing or financial system) protected by network/application firewall? For multi-tenant environment with multiple namespaces in Kubernetes, how to ensure pods or namespaces have a unique identity and enforce control access to egress endpoints (outside of Kubernetes). Unique workload identity is important for end-to-end correlation, audit and traceability. How to provide end-to-end and correlated view from source to target apps. How F5 solved this with classic BIG-IP ADC and Aspen Mesh Service Mesh Architecture Overview This solution article is an extension to the original article – “Expanding Service Mesh without Envoy” published by my colleague Eric Chen. For details of that article please refer to https://aspenmesh.io/expanding-service-mesh-without-envoy/ Aspen Mesh, an innovation from F5, is an enterprise-ready service mesh built on Istio. It is tested and hardened distribution of Istio with complete support by F5. For details, please refer to https://aspenmesh.io. For the purpose of this solution, Aspen Mesh and Istio will be used interchangeably. Solution in a nutshell Each pod had its own workload identity. Part of native capabilities of Aspen Mesh (AM). The identity is in a form of client certificate managed by AM (istiod/Citadel) and generated from an organisation Intermediate CA loaded onto Istio control plane. BIG-IP on-boarded with workload identity (client certificate) and signed from the same organisation Intermediate CA (or root CA). This client certificate NOT managed by AM. F5 Virtual Server (VS) configured with client-side profile to perform mutual TLS (mTLS). F5 VS is registered onto AM. F5 VS service can be discovered from internal service registry. On egress, pod will perform mTLS with F5 VS. As F5 client certificate issues from same organisation Intermediate CA, both parties will negotiate and mutually trust each other and exchange mTLS key. An optional iRule can be implemented on BIG-IP to inspect pod identity (certificate SAN) upon successful mTLS and permit/reject request. BIG-IP implement SNAT and present a unique network identifier (e.g IP address) to network firewall. Environment BIGIP LTM (v14.x) Aspen Mesh - v1.16.x Kubernetes 1.18.x Use Case Permit microservices apps (e.g. bookinfo) to use organisation forward proxy (tinyproxy) to get to Internet which sit behind enterprise network firewall and reject all other microservices apps on the same Kubernetes cluster. Classic BIG-IP Only vs_aspenmesh-bookinfo-proxy-srv-mtls-svc configuration will be demonstrated. Similar configuration can be applied on other VS. F5 Virtual Server configuration F5's VS client profile configuration. "Client Certificate = require" require pods deployed inside AM present a valid trusted client certificate. An optional iRule to only permit pods from bookinfo namespace. Optional irule_bookinfo_spiffee to permit bookinfo apps and reject other apps. when CLIENTSSL_CLIENTCERT { set client_cert [SSL::cert 0] #log local0. "Client cert extensions - [X509::extensions $client_cert]" #Split the X509::extensions output on each newline character and log the values foreach item [split [X509::extensions [SSL::cert 0]] \n] { log local0. "$item" } if {[SSL::cert 0] ne ""} { set santemp [findstr [X509::extensions [SSL::cert 0]] "Subject Alternative Name" 43 " "] set spiffe [findstr $santemp "URI" 4] log local0. "Source SPIFFEE-->$spiffe" if { ($spiffe starts_with "spiffe://cluster.local/ns/bookinfo/") } { log local0. "Aspen Mesh mTLS: PEMITTED==>$spiffe" # Allow and SNAT from defined SNAT Pool } else { log local0. "Aspen Mesh mTLS: REJECTED==>$spiffe" reject } } } Note: As of Istio version 1.xx, client-side envoy (istio sidecar) will start a mTLS handshake with server-side BIG-IP VS (F5's client side profile). During the handshake, the client-side envoy also does a secure naming check to verify that the service account presented in the server certificate is authorised to run the target service. Then only the client-side envoy and server-side BIG-IP will establish a mTLS connection. Hence, the client certificate generated loaded onto BIG-IP have to conform to the secure naming information, which maps the server identities to the service names. For details on secure naming, please refer to https://istio.io/latest/docs/concepts/security/#secure-naming Example to generate a SPIFFE friendly certificate openssl req -new -out bookinfo.istio-spiffee-req.pem -subj "/C=AU/ST=Victoria/L=Melbourne/O=F5/OU=SE/CN=bookinfo.spiffie" -keyout bookinfo.istio-spiffee-key.pem -nodes cat > v3.ext <<-EOF authorityKeyIdentifier=keyid,issuer basicConstraints=CA:FALSE keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment extendedKeyUsage = serverAuth subjectAltName = @alt_names [alt_names] DNS.1=spiffe://cluster.local/ns/bookinfo/sa/default EOF openssl x509 -req -sha512 -days 365 \ -extfile v3.ext \ -CA ../ca1/ca-cert.pem -CAkey ../ca1/ca-key.pem -CAcreateserial \ -in bookinfo.istio-spiffee-req.pem \ -out bookinfo.istio-spiffee-cert.pem where ca1 is the intermediate CA use for Aspen Mesh. Aspen Mesh Pods and Services before registration of F5 VS $ kubectl -n bookinfo get pod,svc NAME READY STATUS RESTARTS AGE pod/details-v1-78d78fbddf-4vmdr 2/2 Running 0 4d1h pod/productpage-v1-85b9bf9cd7-f6859 2/2 Running 0 4d1h pod/ratings-v1-6c9dbf6b45-9ld6f 2/2 Running 0 4d1h pod/reviews-v1-564b97f875-bjx2r 2/2 Running 0 4d1h pod/reviews-v2-568c7c9d8f-zzn8r 2/2 Running 0 4d1h pod/reviews-v3-67b4988599-pdk25 2/2 Running 0 4d1h pod/traffic-generator-productpage-fc97f5595-pdhvv 2/2 Running 0 6d11h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/details ClusterIP 10.235.14.186 <none> 9080/TCP 6d11h service/productpage ClusterIP 10.235.37.112 <none> 9080/TCP 6d11h service/ratings ClusterIP 10.235.40.239 <none> 9080/TCP 6d11h service/reviews ClusterIP 10.235.1.21 <none> 9080/TCP 6d11h service/traffic-generator-productpage ClusterIP 10.235.17.158 <none> 80/TCP 6d11h Register bigip-proxy-svc onto Aspen Mesh $ istioctl register -n bookinfo bigip-proxy-svc 10.4.0.201 3128 --labels apps=bigip-proxy 2020-12-15T23:14:33.286854Z warn Got 'services "bigip-proxy-svc" not found' looking up svc 'bigip-proxy-svc' in namespace 'bookinfo', attempting to create it 2020-12-15T23:14:33.305890Z warn Got 'endpoints "bigip-proxy-svc" not found' looking up endpoints for 'bigip-proxy-svc' in namespace 'bookinfo', attempting to create them $ kubectl -n bookinfo get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE bigip-proxy-svc ClusterIP 10.235.45.250 <none> 3128/TCP 26s details ClusterIP 10.235.14.186 <none> 9080/TCP 6d11h productpage ClusterIP 10.235.37.112 <none> 9080/TCP 6d11h ratings ClusterIP 10.235.40.239 <none> 9080/TCP 6d11h reviews ClusterIP 10.235.1.21 <none> 9080/TCP 6d11h traffic-generator-productpage ClusterIP 10.235.17.158 <none> 80/TCP 6d11h $ kubectl -n bookinfo describe svc bigip-proxy-svc Name: bigip-proxy-svc Namespace: bookinfo Labels: apps=bigip-proxy Annotations: alpha.istio.io/kubernetes-serviceaccounts: default Selector: <none> Type: ClusterIP IP: 10.235.45.250 Port: 3128 3128/TCP TargetPort: 3128/TCP Endpoints: 10.4.0.201:3128 Session Affinity: None Events: <none> To test egress from bookinfo pod to external forward proxy (tinyproxy). Run "curl" accessing to Internet (www.f5.com) pointing to bigip-proxy-svc registered on Aspen Mesh. Example below shown executing curl binary inside "traffic-generator-productpage" pod. $ kubectl -n bookinfo exec -it $(kubectl -n bookinfo get pod -l app=traffic-generator-productpage -o jsonpath={.items..metadata.name}) -c traffic-generator -- curl -Ikx bigip-proxy-svc:3128 https://www.f5.com HTTP/1.0 200 Connection established Proxy-agent: tinyproxy/1.8.3 HTTP/1.1 200 OK Content-Type: text/html;charset=utf-8 Content-Length: 132986 Connection: keep-alive Accept-Ranges: bytes Cache-Control: no-cache="set-cookie" Content-Security-Policy: frame-ancestors 'self' *.cybersource.com *.salesforce.com *.force.com ; form-action *.cybersource.com *.salesforce.com *.force.com 'self' Date: Wed, 16 Dec 2020 06:19:48 GMT ETag: "2077a-5b68b3c0c5be0" Last-Modified: Wed, 16 Dec 2020 02:00:07 GMT Strict-Transport-Security: max-age=16070400; X-Content-Type-Options: nosniff X-Dispatcher: dispatcher1uswest2 X-Frame-Options: SAMEORIGIN X-Vhost: publish Via: 1.1 sin1-bit21, 1.1 24194e89802a1a492c5f1b22dc744e71.cloudfront.net (CloudFront) Vary: Accept-Encoding X-Cache: Hit from cloudfront X-Amz-Cf-Pop: MEL50-C2 X-Amz-Cf-Id: 7gE6sEaBP9WonZ0KjngDsr90dahHWFyDG0MwbuGn91uF7EkEJ_wdrQ== Age: 15713 Logs shown on classic BIG-IP Classic BIG-IP successfully authenticate with bookinfo with mTLS and permit access. Logs shown on forward proxy (tinyproxy). Source IP is SNATed to IP configured on classic BIG-IP. IP also allowed on network firewall. From other namespace (e.g. sm-apigw-a), try to access bigip-proxy-svc. Attempt shown rejected by classic BIG-IP. Example below shown executing curl binary in "nettools" pod. $ kubectl -n sm-apigw-a get pod NAME READY STATUS RESTARTS AGE httpbin-api-78bdd794bd-hfwkj 2/2 Running 2 22d nettools-9497dcc86-nhqmr 2/2 Running 2 22d podinfo-bbb7bf7c-j6wcs 2/2 Running 2 22d sm-apigw-a-85696f7455-rs9zh 3/3 Running 0 7d21h fbchan@logos:~/k8s-clusterX/k8s$ $ kubectl -n sm-apigw-a exec -it $(kubectl -n sm-apigw-a get pod -l app=nettools -o jsonpath={.items..metadata.name}) -c nettools -- curl -kIx bigip-proxy-svc.bookinfo.svc.cluster.local:3128 https://devcentral.f5.com curl: (56) Recv failure: Connection reset by peer command terminated with exit code 56 Classic BIG-IP Logs Classic BIG-IP reject sm-apigw-a namespace from using bigip-proxy-svc service. Summary Aspen Mesh is cloud-native Enterprise Ready Istio service mesh. Classic BIG-IP is a features rich application delivery controller (ADC). With Aspen Mesh, microservices are securely authenticated with mTLS with classic BIG-IP. Classic BIG-IP able to securely authenticate microservices apps and deliver application services based on your business and security requirement. This article addresses egress use cases. What about Ingress to Kubernetes cluster? How classic BIG-IP or cloud-native SPK coherently work together with Aspen Mesh to provides secure and consistent multi-cloud, multi-cluster application delivery services to your Kubernetes environment. This will be shared in future article. Stay tune.
NetOps Primer: What are Microservices?
Microservices are coming to a network near you. Forty-one percent (41%) of respondents to our State of Application Delivery 2018 survey told us their organizations were exploring microservices as a result of digital transformation initiatives. With them will come operational and architectural impacts, which makes them an important architectural evolution for NetOps to understand. Microservices: A Definition All too often microservices and containers are used interchangeably. It is not merely pedantry to correct this usage, as the former is an application architecture and the latter a delivery model. Containers are used to deliver a variety of infrastructure and operational services as well as microservices. They are prized for their portability; cloud agnosticism is a key characteristic of containers that makes multi-cloud an achievable goal. Nothing in the definition or guiding principles of microservices architectures makes mention of 'containers'. Containers are often used to deploy individual microservices because they are a good operational fit, but they are not required. Microservices is an architectural style of application design. It can be compared to client-server, three-tier web, and SOA. From a high-level perspective, much of application architecture is dedicated to determining where and how to execute logic and access data. For much of application architectural history, data access has driven the design of applications. Issues with access and consistency drive developers to design applications in such a way as to ensure optimal use of data stores and reliable consistency of that data across services. Business logic is distributed such that it is closest to that data, while the presentation layer is almost always distributed closest to the client. Business logic can be contained in a single "application", such as in traditional architectures, or it can be distributed. This is the case with SOA and increasingly with microservices and serverless computing. This division of business logic is the driving design principle for microservices. Microservices view an application as a collection of objects and functions that make up the "business logic". This logic enables us to 'checkout', 'track orders', 'manage profiles', and a hundred other application-specific tasks. The code that makes up 'checkout' and 'manage profile' can be grouped together neatly into a single microservice. Essentially, microservices architecture is a set of principles that, when applied to an application, determine how business logic is grouped and distributed across an 'application'. Microservices architecture aims to group that logic as tightly as it can, and in the smallest sustainable chunks possible. This aligns well with agile methodologies that demand rapid, frequent iterations over the code. By isolating chunks of business logic, each chunk can be iterated over in isolation to allow new features and functionality to be delivered in a rapid fashion. The size of the microservice is largely determined by the development organization's ability to maintain the code and speed of delivery. Too many microservices introduces complexity that slows down delivery. Thus, most organizations have settled on fewer microservices and moved from a purely functional approach to a more object-oriented strategy. Regardless, there are still more microservices that make up a modern application than there are in a traditional, three-tier web application. The Impact on NetOps So you might be thinking, so what? What does a NetOps care about how developers architect applications? The impact on NetOps is largely operational. That is, instead of deploying one application, you must now find time to deploy and operate all the application services needed to support many more applications. This imposes a burden on every operational group in IT that makes up the deployment pipeline as it is likely that no two applications share the same deployment schedule. This is true whether microservices are deployed in containers or not. Regardless of the deployment model - whether in individual VMs or in containers or even on bare-metal built servers - each microservice must be effectively treated as an individual application. There are also technical and architectural impacts on NetOps, largely due to reliance on API paths rather hosts to route requests to the appropriate service. This can mean insertion of new app routing services in the network architecture to support potentially multiple layers of HTTP-based routing. The simplicity or complexity of the routing environment is largely determined by how granular developers get during design and implementation. Highly granular, function-based microservices architectures can result in hundreds of tiny microservices. Less aggressive (and more reasonable in an enterprise setting) implementations will see smaller but still significant increases. Getting familiar with microservices will be a boon to NetOps and a necessary step toward supporting the next generation of applications built on this application architecture evolution.Microservices and HTTP/2
It's all about that architecture. There's a lot of things we do to improve the performance of web and mobile applications. We use caching. We use compression. We offload security (SSL and TLS) to a proxy with greater compute capacity. We apply image optimization and minification to content. We do all that because performance is king. Failure to perform can be, for many businesses, equivalent to an outage with increased abandonment rates and angry customers taking to the Internet to express their extreme displeasure. The recently official HTTP/2 specification takes performance very seriously, and introduced a variety of key components designed specifically to address the need for speed. One of these was to base the newest version of the Internet's lingua franca on SPDY. One of the impacts of this decision is that connections between the client (whether tethered or mobile) and the app (whether in the cloud or on big-iron) are limited to just one. One TCP connection per app. That's a huge divergence from HTTP/1 where it was typical to open 2, 4 or 6 TCP connections per site in order to take advantage of broadband. And it worked for the most part because, well, broadband. So it wouldn't be a surprise if someone interprets that ONE connection per app limitation to be a negative in terms of app performance. There are, of course, a number of changes in the way HTTP/2 communicates over that single connection that ultimately should counteract any potential negative impact on performance from the reduction in TCP connections. The elimination of the overhead of multiple DNS lookups (not insignificant, by the way) as well as TCP-related impacts from slow start and session setup as well as a more forgiving exchange of frames under the covers is certainly a boon in terms of application performance. The ability to just push multiple responses to the client without having to play the HTTP acknowledgement game is significant in that it eliminates one of the biggest performance inhibitors of the web: latency arising from too many round trips. We've (as in the corporate We) seen gains of 2-3 times the performance of HTTP/1 with HTTP/2 during testing. And we aren't alone; there's plenty of performance testing going on out there, on the Internets, that are showing similar improvements. Which is why it's important (very important) that we not undo all the gains of HTTP/2 with an architecture that mimics the behavior (and performance) of HTTP/1. Domain Sharding and Microservices Before we jump into microservices, we should review domain sharding because the concept is important when we look at how microservices are actually consumed and delivered from an HTTP point of view. Scalability patterns (i.e. architectures) include the notion of Y-axis scale which is a sharding-based pattern. That is, it creates individual scalability domains (or clusters, if you prefer) based on some identifiable characteristic in the request. User identification (often extricated from an HTTP cookie) and URL are commonly used information upon which to shard requests and distribute them to achieve greater scalability. An incarnation of the Y-axis scaling pattern is domain sharding. Domain sharding, for the uninitiated, is the practice of distributing content to a variety of different host names within a domain. This technique was (and probably still is) very common to overcome connection limitations imposed by HTTP/1 and its supporting browsers. You can see evidence of domain sharding when a web site uses images.example.com and scripts.example.com and static.example.com to optimize page or application load time. Connection limitations were by host (origin server), not domain, so this technique was invaluable in achieving greater parallelization of data transfers that made it appear, at least, that pages were loading more quickly. Which made everyone happy. Until mobile came along. Then we suddenly began to realize the detrimental impact of introducing all that extra latency (every connection requires a DNS lookup, a TCP handshake, and suffers the performance impacts of TCP slow start) on a device with much more limited processing (and network) capability. I'm not going to detail the impact; if you want to read about it in more detail I recommend reading some material from Steve Souder and Tom Daly or Mobify on the subject. Suffice to say, domain sharding has an impact on mobile performance, and it is rarely a positive one. You might think, well, HTTP/2 is coming and all that's behind us now. Except it isn't. Microservice architectures in theory, if not in practice, are ultimately a sharding-based application architecture that, if we're not careful, can translate into a domain sharding-based network architecture that ultimately negates any of the performance gains realized by adopting HTTP/2. That means the architectural approach you (that's you, ops) adopt to delivering microservices can have a profound impact on the performance of applications composed from those services. The danger is not that each service will be its on (isolated and localized) "domain", because that's the whole point of microservices in the first place. The danger is that those isolated domains will be presented to the outside world as individual, isolated domains, each requiring their own personal, private connection by clients. Even if we assume there are load balancing services in front of each service (a good assumption at this point) that still means direct connections between the client and each of the services used by the client application because the load balancing service acts as a virtual service, but does not eliminate the isolation. Each one is still its own "domain" in the sense that it requires a separate, dedicated TCP connection. This is essentially the same thing as domain sharding as each host requires its own IP address to which the client can connect, and its behavior is counterproductive to HTTP/2*. What we need to do to continue the benefits of a single, optimized TCP connection while being able to shard the back end is to architect a different solution in the "big black box" that is the network. To be precise, we need to take advantage of the advanced capabilities of a proxy-based load balancing service rather than a simple load balancer. An HTTP/2 Enabling Network Architecture for Microservices That means we need to enable a single connection between the client and the server and then utilize capabilities like Y-axis sharding (content switching, L7 load balancing, etc...) in "the network" to maintain the performance benefits of HTTP/2 to the client while enabling all the operational and development benefits of a microservices architecture. What we can do is insert a layer 7 load balancer between the client and the local microservice load balancers. The connection on the client side maintains a single connection in the manner specified (and preferred) by HTTP/2 and requires only a single DNS lookup, one TCP session start up, and incurs the penalties from TCP slow start only once. On the service side, the layer 7 load balancer also maintains persistent connections to the local, domain load balancing services which also reduces the impact of session management on performance. Each of the local, domain load balancing services can be optimized to best distribute requests for each service. Each maintains its own algorithm and monitoring configurations which are unique to the service to ensure optimal performance. This architecture is only minimally different from the default, but the insertion of a layer 7 load balancer capable of routing application requests based on a variety of HTTP variables (such as the cookies used for persistence or to extract user IDs or the unique verb or noun associated with a service from the URL of a RESTful API call) results in a network architecture that closely maintains the intention of HTTP/2 without requiring significant changes to a microservice based application architecture. Essentially, we're combining X- and Y-axis scalability patterns to architect a collaborative operational architecture capable of scaling and supporting microservices without compromising on the technical aspects of HTTP/2 that were introduced to improve performance, particularly for mobile applications. Technically speaking we're still doing sharding, but we're doing it inside the network and without breaking the one TCP connection per app specified by HTTP/2. Which means you get the best of both worlds - performance and efficiency. Why DevOps Matters The impact of new architectures - like microservices - on the network and the resources (infrastructure) that deliver those services is not always evident to developers or even ops. That's one of the reasons DevOps as a cultural force within IT is critical; because it engenders a breaking down of the isolated silos between ops groups that exist (all four of them) and enables greater collaboration that leads to more efficient deployment, yes, but also more efficient implementations. Implementations that don't necessarily cause performance problems that require disruptive modification to applications or services. Collaboration in the design and architectural phases will go along way towards improving not only the efficacy of the deployment pipeline but the performance and efficiency of applications across the entire operational spectrum. * It's not good for HTTP/1, either, as in this scenario there is essentially no difference** between HTTP/1 and HTTP/2. ** In terms of network impact. HTTP/2 still receives benefits from its native header compression and other performance benefits.Microservices – application fragments need application services
Microservices are a cool way to build applications – cool in terms of the attention they get in blogs, marketing materials and conference speaking slots, but also cool in terms of genuinely offering a way to build applications that are more agile and robust than the monolithic architectures we have become used to. But really what are microservices? They are just fragments of an application. All the things we need from our monolithic applications we need from our fragments. Security, availability, performance. Each microservice needs to be up and running, have capacity and not be compromised if the integrity of the whole is to be preserved. Sure, there may be some services that are less critical than others, but in general you want all your services secure, fast, and available. How can you achieve this? Well there are number of ways, but the easiest is to reuse a common design pattern from the monolithic stack – an application proxy. Application proxies allow you to insert a layer of logic and control between the microservice client and the server. Pretty basic stuff. But by de-coupling the client from the server, you essentially virtualize the service. That means you can scale, protect and manage the service without disruption to the clients. You can provide capacity without complexity and security without slowing the application. Application traffic flow in microservices can be complex – multiple services need to be able to communicate to create the application. These complex flows can make application monitoring and security more complex to deliver – plus you need to code in client side discovery patterns if you need to scale out the number of instances supporting the service. Adding an application proxy – like a BIG-IP allow you to shrink or grow the processing power for a particular microservice – and add security, availability and optimization services inline. This can help you with segmentation, reporting and performance challenges. With a programmable, API-driven application proxy you can design a responsive system that is able to respond to changes in the environment and add additional application layer logic easily and without changing application code. With a strategic point of control like an application proxy - you can provide application services to your microservices, adding security and scalability while keeping the agility and performance you need.The Impact of the Observer Effect on Microservices Architecture
Application availability is not just the measure of “being up”. Many apps can claim that status. Technically they are running and responding to requests, but at a rate which users would certainly interpret as being down. That’s because excessive load times can (and will be) interpreted as “not available.” That’s why it’s important to view ensuring application availability as requiring attention to all its composite parts: scalability, performance, and security. The paradox begins when we consider how we ensure scale, performance, and security: monitoring and measuring. That is, we observe certain characteristics about the network, compute, and application resources to gain an understanding of the status of the application. That necessarily means we have to interact with those components that need monitoring and measuring and thus we enter the world of physics. The Observer Effect simply states that observing something necessarily changes the thing being observed. When it’s a sentient being, this often takes the form of the Hawthorne Effect, which claims that sentient beings will change their behavior when they know they’re being observed. Go ahead, try it out on your kids. If they know they’re being watched they’re angels. But turn your back on them for a minute and wham! They’ve destroyed their play room and littered popcorn all over the floor. Within the realm of IT, the effect is no less active: In information technology, the observer effect is the potential impact of the act of observing a process output while the process is running. For example: if a process uses a log file to record its progress, the process could slow. Furthermore, the act of viewing the file while the process is running could cause an I/O error in the process, which could, in turn, cause it to stop. Another example would be observing the performance of a CPU by running both the observed and observing programs on the same CPU, which will lead to inaccurate results because the observer program itself affects the CPU performance (modern, heavily cached and pipelined CPUs are particularly affected by this kind of observation). -- https://en.wikipedia.org/wiki/Observer_effect_(information_technology) The act of measuring capacity and performance of a system* – say an app or an individual microservice - alters its state by consuming resources that in turn increase total load which, based on operational axiom #2 says, ultimately degrades both capacity and performance. This is one of the reasons agent-based monitoring has always been a less favorable choice for APM, because the presence of the agent on the system necessarily reduces capacity and performance. The Observer Effect is going to be particularly impactful on applications composed of microservices because of, well, math. If the act of measuring and monitoring one monolithic application degrades performance by X then the act of measuring and monitoring a microservices-based application is going to degrade performance by many more X. It could be argued that the impact on a microservices-based application is actually not X per service, but some fraction of X given that the point is to distribute services in such a way that not all services are being taxed at the same rate as in a monolithic application. That would be true if it the microservices were being used as part of a single application, but one of the benefits – and target uses – of microservices is reuse. That implies that multiple apps or APIs are going to make use of each service, thus increasing the need to measure and monitor the capacity and performance of each service. This is where architecture and technique matters. Where the design and implementation of the measuring and monitoring for performance and load of microservices becomes an important piece of ensuring availability. While each and every point of control – an API gateway or service discovery system or load balancer or proxy - can measure each microservice for which it performs its assigned tasks, it is likely to unnecessarily increase the impact of the Observer Effect on the microservice. That’s because most points of control take an active approach to monitoring and measuring load and performance. That is, they purposefully poll a system so as to enquire regarding the status and responsiveness of the system. They use ICMP pings, they use TCP half opens, and they use HTTP content requests to gather the data they need. Each of these methods interacts with the system in question and thus fulfills the Observers Effect prediction. The more systems gathering this data, the more interaction occurs, the greater the impact of the Observer Effect. That means there must be greater attention paid to the way in which microservices are monitoring and measured – including the techniques used to accomplish it. Passive approaches to measuring and monitoring provide one means of avoiding the Observer Effect. That’s because they – as the term implies – passively observe status and measure performance without actively probing systems for this data. This is typically achieved by leveraging intermediate systems like load balancers and proxies through which requests and responses necessarily flow to capture status information as it is passing through. The measurements are then used by the intermediary, of course, to manage distribution of load but are also exposed via APIs for collection by other systems. Those statistics gathered from an intermediary are likely* to have no impact on performance because they are managed by a system separate from the real-time execution of the intermediary. It is important to consider the availability of the statistics via APIs to external systems when architecting a solution based on passive monitoring and measurement techniques. If the system performing the monitoring and measuring makes available the data it has collected, it relieves other systems of needing to directly measure each services’ status and performance and further reduces the impact of the Observer Effect on the overall system. This is one of the ways in which the collaborative aspects of DevOps can provide significant value. As ops and net ops work together to establish a more efficient means of measuring and monitoring the availability of systems like microservices they can provide as valuable input to dev those statistics using APIs directly or through integration with other established systems. At an operational level this effort also establishes a more centralized location from which performance-related data can be retrieved (in real-time) and used to trigger other actions such as auto-scaling (up and down) – a critical capability when moving to microservices architectures in which the number and variability of usage and services requires a more automated approach to operations than their monolithic predecessors. * This is less applicable to virtual network appliances because they are purposefully designed to separate operational and actionable systems to ensure that management – measuring, monitoring, modifying – the system does not impact the performance of the actionable system. This is carried over from their roots in hardware, where “lights out” management is a requirement.Microservices, State, and the Network
Does moving to stateless microservices eliminate state in the network? One of the ways to increase scalability of services – and applications – is to go “stateless.” The reasons for this are many, but in general by eliminating the mapping between a single client and a single app or service instance you eliminate the need for resources to manage state in the app (overhead) and improve the distributability (I can make up words if I want) of requests across a pool of instances. The latter occurs because sessions don’t need to hang out and consume resources that could be used to serve other requests. Distribution should, in theory, be more even and enable better predictability. One request takes one second to respond. That’s it. This is important to “the network” because stateful services require special attention from certain types of proxies. Load balancing, for example. After an instance is selected to service the first request, all subsequent requests from that client must be routed to that same instance. That requires that “the network” maintain state, too. So one wonders as we begin to adopt microservices and its stateless approach whether or not that will extend upstream, into “the network”, too. The answer is yes and no. There are actually three places where state is maintained in the network: HTTP This is the application layer. State here is maintained as described above, by maintaining communication between a client and an application/service. TCP This is the transport layer. TCP is how connections are made. State here is maintained to ensure reliable delivery of data between client and an application/service. SSL This is a homeless layer between TCP and HTTP that provides confidentiality of data. State here is maintained because encryption and decryption relies on information unique to the connection between a client and an application/service. Now. Let’s assume that the application and/or services are stateless, as per best practices for microservices. This implies there is no need for maintaining HTTP “state” in the network. So it can go away. Poof! But that leaves TCP and SSL (or TLS, if you prefer). The answer for these depends on your architectural choices. If your load balancer (because you have one, I guarantee it) is terminating SSL/TLS, state is still required in the network. Architecturally you want to terminate SSL/TLS upstream of servers to eliminate the overhead and weight required not just to process SSL/TLS on web servers but to eliminate the cost and overhead associated with managing certificates across an elastic set of web servers. Similarly, if your load balancer is distributing requests based on HTTP-layer information, it’s likely terminating TCP. That means state has to be maintained in the network, in the load balancer at a minimum. If you’re using any kind of web application firewall to inspect data (inbound and outbound) then it’s terminating TCP connections, too, and thus maintaining state. And of course if your load balancer is doing any kind of application-layer DDoS protection (which it totes should be) state has to be maintained because it’s part of the detection process. So the answer to the question ends up being “maybe”. Ultimately, state in the network is related to architectural choices regarding the deployment of microservices, not the nature of microservices themselves.
