HashiCorp Consul Update Makes App-Level 'Service-Mesh' Architectures Easier
HashiCorp Consul 1.6 adds a 'mesh gateway' to its its rich set of network feature. The update makes it easier to deliver app-level end-to-end secure communication across regions, platforms and clouds.
HashiCorp Consul is looking to make a “service mesh” architecture easy to achieve by providing a distributed networking layer to connect, secure, and observe services across any runtime platform, as well as any public or private cloud.
The new HashiCorp Consul 1.6 features include: Layer 7 traffic management, multi-platform, multi-cloud service networking capabilities, and expanded integration with the HashiCorp Nomad scheduler.
Notably, Consul 1.6 also delivers a “mesh gateway” feature that transparently and securely routes service traffic across multiple regions, platforms, and clouds.
With these new features, Consul enhances its rich set of network features to add advanced traffic management and observability functions at the application layer. The result is a new version that enables end-to-end secure communication across multiple clusters, platforms, data centers, or clouds.
Consul is also able to network the widely diverse and increasingly dynamic environments in today’s enterprises – from various public cloud infrastructures, Kubernetes, as well as the more static, non-containerized legacy environments -- all using a common cloud operating model. Consul provides a central service registry to enable visibility into all running services and drives network middleware automation with its service discovery capabilities. For sophisticated environments using microservices, Consul provides a distributed service mesh to securely connect dynamic services across any runtime platform and any public or private cloud.
Details of HashiCorp Consul 1.6 include:
Layer 7 traffic management: Consul will provide dynamic routing at the application layer of the network to support different deployment strategies and improved resiliency.
Mesh gateway: Consul will provide simplified and secure cross-cluster communication without the need to configure a VPN nor create complex routing rules, enabling the scale needed for multi-platform and multi-cloud environments.
Service discovery: Consul provides a central registry to locate services across multiple runtime platforms and clouds dynamically.
Service segmentation: Consul provides a zero-trust network enabled by identity-based security policies.
Layer 7 observability: Consul provides insights into application behavior and performance through the networking metric collection, distributed tracking, and logging.
Armon Dadgar, CTO and co-founder of HashiCorp explained the service mesh focus of the latest HashiCorp Consul update this way:
“Networking becomes more dynamic and complex with the move to microservices and multi-cloud environments. . . Cloud applications often create independent network silos, and connectivity between these network environments has been challenging. We want to ‘flatten’ the network further and to easily bridge services across multiple network silos without the need for complex network configuration or VPN technologies. Consul can ensure end-to-end security is automatically enforced regardless of how service traffic traverses your network, “ Dadgar said in a statement.
He also described how Consul will specifically help developers in this context.
“It is very challenging for network operations and security teams to keep up with the updates to network middleware and manage the increasingly complex network topologies,” he said. “We designed HashiCorp Consul to enable service networking across these varied environments by pushing routing, authorization, and other core networking functionality from centralized middleware to the endpoints. This means that the network topology can be dramatically simplified by removing middleware from the network path.”
HashiCorp’s Consul 1.6 Architecture – Pushing the Service Mesh Limits
IDN takes a quick look at the HashiCorp Consul 1.6 update, and its client-service architecture approach to provide what the company calls a ‘control plane” for its service mesh.
Consul Service Mesh Architecture: Consul has a client-server architecture which is the “control plane” for the service mesh. Multiple servers are deployed for high availability, and a pool of clients runs on every host. Clients integrate with sidecar proxies, such as Envoy, that provide the “data plane” for the service mesh.
The centralized servers hold the service registry, access and traffic policies, configurations, and certificate authorities, which are efficiently transferred to the distributed clients in real-time. The clients configure local proxies, cache data, and policies, and provide health checking.
The Solution: As an automated and distributed approach to networking and Security, service mesh offers the following solutions:
- Improve Security with fine-grained authorization and mutual-TLS
- Increase productivity by automating changes and minimizing the management overhead of networks
- Reduce cost by avoiding a proliferation of network appliances
- Simplify Networks by pushing authorization and traffic management to the endpoints, avoiding complex topologies
- Regardless of where or on what platform a workload is running, Consul allows users to link together all of their applications in a single dedicated network layer
HashiCorp. Consul Team Outlines Vision, Implementation of ‘Service Mesh’
In a blog post tied to the latest Consul update, HashiCorp’s Consul Team described the ongoing “service mesh” journey:
A year ago, Consul 1.2 introduced our service mesh solution, Consul Connect. Our initial release focused on solving security challenges at Layer 4 and leveraging Consul’s service discovery feature to provide service-to-service identity and trust. Since that point, Connect has added support for additional proxies (Envoy), L7 observability, a simpler way to enable Consul ACLs and TLS, and platform integrations with Kubernetes.
Today we're proud to announce a major milestone in realizing our vision for service mesh. With the release of Consul 1.6 we are adding features for traffic management at Layer 7 and enabling transparent, cross-network connectivity with Mesh Gateways. Of course, these features work across platforms, with continued first-class support for Kubernetes and easy deployment across more traditional environments on any cloud or private network. This delivers on HashiCorp's goal for Consul to enable multi-cloud service networking.
The release also rolls out a “mesh gateway’ feature in beta.
This new feature “enable routing of Connect traffic between different Consul datacenters,” HashiCorp stated.
It went on to say: “Those datacenters can reside in different clouds or runtime environments where general interconnectivity between all services in all datacenters isn't feasible. These gateways operate by sniffing the SNI header out of the Connect session and then route the connection to the appropriate destination based on the server name requested. The data within the Connect session is not decrypted by the Gateway.”
The HashiCorp post described more specific benefits from the “mesh gateway” approach:
As organizations distribute their workloads across multiple platforms, data centers, and clouds, the underlying network becomes increasingly fragmented and complex. Services in their respective environments run on independent networks, leading to multiple network silos. Managing connections between multiple network environments is challenging. It requires careful network planning to avoid overlapping IP addresses and typically relies on point-to-point VPN, networking peering, or private links. These approaches add operational overhead to manage and troubleshoot.
Mesh gateways are Envoy proxies at the edge of a network, which enable services in separate networking environments to easily communicate with each other. They are configured by Consul using a similar mechanism as sidecar proxies. If a source service wants to connect with a destination service in a remote cluster/platform/cloud, the traffic is proxied through mesh gateways, which route the traffic to the destination service based on the Server Name Indication (SNI) that is sent as part of the TLS handshake. Because SNI is part of TLS, mesh gateways don't (and can't) decrypt the data of the payload and have no special access to it, which keeps data safe even if a mesh gateway is compromised. Mesh gateways are a routing tool for service-to-service connections and are not suitable for general purpose ingress from non-mesh traffic.
Core HashiCorp Consul Components
In summary, there are several core value props to HashiCorp Consul. Among the more notable, the HashiCorp website calls out
Service-to-Service Security
Consul Connect provides service-to-service connection authorization and encryption using mutual Transport Layer Security (TLS). Applications can use sidecar proxies in a service mesh configuration to automatically establish TLS connections for inbound and outbound connections without being aware of Connect at all. Applications may also natively integrate with Connect for optimal performance and security. Connect can help you secure your services and provide data about service-to-service communications.
Application Security
Consul Connect further enables secure deployment best-practices with “automatic” service-to-service encryption, and identity-based authorization. Connect uses the registered service identity (rather than IP addresses) to enforce access control with intentions.
This approach makes it easier to reason about access control and enables services to be rescheduled by orchestrators including Kubernetes and Nomad. Intention enforcement is network agnostic, which allows Consul Connect to work with physical networks, cloud networks, software-defined networks, cross-cloud, and more.
Observability
Provides a uniform and consistent view of all the services on a network, irrespective of their different programming languages and frameworks. This means when users configure Consul Connect to use sidecar proxies, those proxies "see" all service-to-service traffic and can collect data about it. Consul Connect can configure Envoy proxies to collect layer 7 metrics and export them to tools like Prometheus. Further, correctly instrumented application can also send open tracing data through Envoy.
HashiCorp Consul 1.6 is available now.
| Upcoming Events Click to Register |
|
|---|---|
| Feb 6, 2025 |
AI Deployment Summit |