Azure Service Bus – what to watch out for
I’ve been working with a few customers that use Azure Service Bus with (very) high volumes. Service Bus is a critical piece of infrastructure in their solution. When the service does not behave how they expect it to behave, it has direct consequences to their business operations, sometimes in critical ways. In this post, I’d like to showcase some of the points that aren’t sometimes obvious to see when using the service heavily.
Business critical workloads should have an active-active Service Bus across regions
Why am I making this recommendation?
Azure Service Bus is a managed platform service, which means Microsoft continuously updates its underlying components to keep the service secure, compliant, reliable, and performant. This is a normal part of the software development lifecycle for any cloud service.
To reduce risk, Microsoft deploys updates in rings rather than updating every region at once. This approach allows the team to closely monitor the service after deployment and quickly roll back changes if an issue is detected. As a result, multiple regions are not updated simultaneously.
During these updates, short service interruptions can occasionally occur. For most workloads, these interruptions are brief and have little to no noticeable impact. However, if your application has strict availability requirements and must always be able to access and process messages, it’s worth considering a multi-region design.
A common approach is to provision a second Service Bus namespace in a different Azure region and publish messages to both namespaces. Because the same message may be processed from either location, deduplication and idempotency needs to be handled by the application.
This design helps ensure that if one region experiences a brief interruption during a platform update, your application can continue receiving and processing messages from the secondary region. The product group targets update-related interruptions of 30 seconds or less, although longer interruptions can occasionally occur depending on the nature and scope of the update.
There is no such thing as zero risk in distributed systems. Even the best managed cloud services occasionally need maintenance, and infrastructure issues can still occur. That is why the recommendation is an active-active architecture for workloads where every minute of availability matters, i.e. when near-continuous availability and protection from both planned maintenance events and unplanned outages matter. Although the Service Bus team can continuously improve the service itself, they do not control every failure domain in the underlying infrastructure.
While it requires additional operational and application complexity, running equivalent namespaces across separate regions provides a powerful layer of protection against both maintenance activities and rare infrastructure failures. For organizations that view message delivery as mission-critical, this is often the architecture worth striving toward
How is this different than geo-replication?
As the documentation states, Geo-Replication provides active-passive replication of both metadata and message data between a primary region and a secondary region. Geo-Replication is used for most applications that must remain resilient to region outages and where business requirements have a low tolerance for message data loss.
Always use SDKs and shy away from HTTP whenever possible
While Service Bus provides HTTP REST APIs to communicate with the service, the Azure Service Bus SDKs provide built-in retry handling, connection resiliency, and recovery logic that have been battled tested by the product team. At first glance, that may sound like a small implementation detail. In reality, it can have a meaningful impact on resiliency, recovery times, and operational behavior.
HTTP works, but it leaves a lot of heavy lifting to the application
There is nothing inherently wrong with using the Service Bus REST APIs over HTTP. Microsoft supports them and they solve many integration scenarios.
However, when you use HTTP directly, your application becomes responsible for much of the resiliency logic. Things such as:
- Connection recovery
- Retry handling
- Transient fault detection
- Re-establishing sessions
- Managing network interruptions
- Optimizing client connections
Many of these concerns are already handled by the official SDKs. Those SDKs contain years of engineering work to deal with the realities of distributed systems and cloud networking.
One thing I found particularly interesting is the discussion around AMQP versus HTTP. Many teams continue to use HTTP because it is familiar and easy to work with, but Azure Service Bus is fundamentally designed around AMQP. AMQP gives the service and client libraries more opportunities to recognize and recover from short-lived interruptions before they become noticeable to the application. While it is not a silver bullet, it is another example of how using the native protocol often unlocks capabilities that are difficult to reproduce in custom implementations. And if opening additional ports is a concern, AMQP over WebSockets provides a practical middle ground that works well in many enterprise environments.
Can a custom HTTP implementation work? Absolutely.
But when availability, resiliency, and operational simplicity matter, the official SDKs and native protocols often provide capabilities that are difficult and expensive to reproduce yourself.
Distribute heavily-used workloads across additional namespaces
One of the more interesting discussions we often have is not about protocols or SDKs. It is about architecture.
Namespace Design Matters More Than Most Teams Realize
One customer I had the chance to deal with had several namespaces containing close to 1,000 queues each. On paper, that is entirely within the documented Service Bus limits. Yet they were experiencing maintenance-related impacts that appeared larger than expected.
A limit is not a target
There are customers running thousands of entities that are used very lightly and see no issues. Likewise, there are customers running very heavily utilized entities without problems because they only have a small number of them.
The challenges tend to appear when both dimensions are combined:
- Large numbers of entities
- Heavy activity
- Frequent state changes
- Additional Service Bus features such as deduplication windows
this combination can increase operational complexity during maintenance and recovery operations.
One concept that does not often appear in architecture diagrams is rehydration. During maintenance operations, Service Bus must preserve and reload state. The service is not simply restarting an empty process. It must reconstruct the operational state associated with queues, subscriptions, messages, sessions, and other internal metadata. Heavily loaded namespaces require more work during those transitions because additional state must be transferred and reloaded into memory.
The recommendation: spread busy workloads
The primary recommendation is to identify the busiest entities and evaluate whether they can be distributed across additional namespaces rather than concentrating them in a small number of very large namespaces. The goal is not necessarily to reduce the total number of queues. The goal is to reduce the operational concentration of workload and state.
Continue providing incident data to Microsoft
The product team takes these matters seriously (yes, really!). What you can do is continue opening support cases whenever you see odd behavior and provide:
- Namespace affected.
- Timestamps.
- Observed business impact.
This information gives the Service Bus product team valuable context to compare customer impact against their own telemetry and any active engineering investigations. It also feeds directly into their continuous improvement efforts. By studying patterns across namespaces experiencing sustained queue growth, the team can better understand real-world behaviors, identify opportunities for optimization, and validate that the service is operating as intended. This ongoing analysis helps improve the platform over time and ensures customers benefit from the learnings gathered across large-scale deployments.
