Skip to content

Commit 214f119

Browse files
richardpark-msftrichardpark-msft
authored
Add in a simple service bus troubleshooting guide. (#24741)
1 parent bb804de commit 214f119

File tree

2 files changed

+199
-0
lines changed

2 files changed

+199
-0
lines changed

‎sdk/messaging/azservicebus/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -269,6 +269,8 @@ To see some example code for receiving messages using the `Receiver`, see the [R
269269

270270
## Troubleshooting
271271

272+
For detailed troubleshooting information, refer to the [Service Bus Troubleshooting Guide][servicebus_troubleshooting].
273+
272274
### Logging
273275

274276
This module uses the classification-based logging implementation in `azcore`. To enable console logging for all SDK modules, set the environment variable `AZURE_SDK_GO_LOGGING` to `all`.
@@ -331,3 +333,4 @@ If you'd like to contribute to this library, please read the [contributing guide
331333
[godoc_newsender]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/messaging/azservicebus/#Client.NewSender
332334
[godoc_newreceiver_queue]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/messaging/azservicebus/#Client.NewReceiverForQueue
333335
[godoc_newreceiver_subscription]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/messaging/azservicebus/#Client.NewReceiverForSubscription
336+
[servicebus_troubleshooting]: https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/messaging/azservicebus/TROUBLESHOOTING.md

‎sdk/messaging/azservicebus/TROUBLESHOOTING.md‎

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
# Troubleshooting Azure Service Bus module issues
2+
3+
This troubleshooting guide contains instructions to diagnose frequently encountered issues while using the Azure Service Bus module for Go (`azservicebus`).
4+
5+
## Table of contents
6+
7+
- [General Troubleshooting](#general-troubleshooting)
8+
- [Error Handling](#error-handling)
9+
- [Logging](#logging)
10+
- [Common Error Scenarios](#common-error-scenarios)
11+
- [Unauthorized Access Errors](#unauthorized-access-errors)
12+
- [Connection Lost Errors](#connection-lost-errors)
13+
- [Lock Lost Errors](#lock-lost-errors)
14+
- [Performance Considerations](#performance-considerations)
15+
- [Receiver](#receiver)
16+
- [Sender](#sender)
17+
- [Connectivity Issues](#connectivity-issues)
18+
- [Enterprise Environments and Firewalls](#enterprise-environments-and-firewalls)
19+
- [Advanced Troubleshooting](#advanced-troubleshooting)
20+
- [Logs to collect](#logs-to-collect)
21+
- [Interpreting Logs](#interpreting-logs)
22+
- [Additional Resources](#additional-resources)
23+
- [Filing GitHub Issues](#filing-github-issues)
24+
25+
---
26+
27+
## General Troubleshooting
28+
29+
### Error Handling
30+
31+
`azservicebus` can return two types of errors: `azservicebus.Error`, which contains a code you can use programmatically, and generic `error`s which only contain an error message.
32+
33+
Here's an example of how to check the `Code` from an `azservicebus.Error`:
34+
35+
```go
36+
if err != nil {
37+
var sbErr *azservicebus.Error
38+
if errors.As(err, &sbErr) {
39+
switch sbErr.Code {
40+
case azservicebus.CodeUnauthorizedAccess:
41+
// Handle authentication errors
42+
case azservicebus.CodeConnectionLost:
43+
// Handle connection lost errors
44+
case azservicebus.CodeNotFound:
45+
// Handle queue/topic/subscription not existing
46+
default:
47+
// Handle other error codes.
48+
}
49+
}
50+
// Handle other error types
51+
}
52+
```
53+
54+
### Logging
55+
56+
Service Bus uses the classification-based logging implementation in `azcore`. You can enable logging for all Azure SDK modules by setting the environment variable `AZURE_SDK_GO_LOGGING` to `all`.
57+
58+
For more fine-grained control, use the `azcore/log` package to enable specific log events:
59+
60+
```go
61+
import (
62+
"fmt"
63+
64+
azlog "github.com/Azure/azure-sdk-for-go/sdk/azcore/log"
65+
"github.com/Azure/azure-sdk-for-go/sdk/messaging/azservicebus"
66+
)
67+
68+
// Print log output to stdout
69+
azlog.SetListener(func(event azlog.Event, s string) {
70+
fmt.Printf("[%s] %s\n", event, s)
71+
})
72+
73+
// Enable specific event types
74+
azlog.SetEvents(
75+
azservicebus.EventConn, // Connection-related events
76+
azservicebus.EventAuth, // Authentication events
77+
azservicebus.EventSender, // Sender operations
78+
azservicebus.EventReceiver, // Receiver operations
79+
azservicebus.EventAdmin, // operations in the azservicebus/admin.Client
80+
)
81+
```
82+
83+
---
84+
85+
## Common Error Scenarios
86+
87+
### Unauthorized Access Errors
88+
89+
If you receive a `CodeUnauthorizedAccess` error, it means the credentials provided are not valid for use with a particular entity, or they have expired.
90+
91+
**Common causes and solutions:**
92+
93+
- **Expired credentials**: If using SAS tokens, they expire after a certain duration. Generate a new token or use a credential that automatically refreshes, like one of the TokenCredential types from the [Azure Identity module][azidentity_tokencredentials].
94+
- **Missing permissions**: Ensure the identity you're using has the correct role assigned from the [built-in roles for Azure Service Bus](https://learn.microsoft.com/azure/service-bus-messaging/service-bus-authentication-and-authorization#microsoft-entra-id).
95+
- **Incorrect entity name**: Verify that the queue, topic, or namespace name is spelled correctly.
96+
97+
For more help with troubleshooting authentication errors when using Azure Identity, see the Azure Identity client library [troubleshooting guide][azidentity_troubleshooting].
98+
99+
### Connection Lost Errors
100+
101+
An `azservicebus.CodeConnectionLost` error indicates that the connection was lost and all retry attempts failed. This typically reflects an extended outage or connection disruption.
102+
103+
**Common causes and solutions:**
104+
105+
- **Network instability**: Check your network connection and try again after ensuring stability.
106+
- **Service outage**: Check the [Azure status page](https://status.azure.com) for any ongoing Service Bus outages.
107+
- **Firewall or proxy issues**: Ensure firewall rules aren't blocking the connection.
108+
109+
### Lock Lost Errors
110+
111+
A `CodeLockLost` error occurs when the lock on a session or message is lost due to exceeding the lock duration. If you find this happening, you can either:
112+
- Increase the [LockDuration](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/messaging/[email protected]/admin#QueueProperties) for your queue or subscription.
113+
- Call `Receiver.RenewMessageLock` or `SessionReceiver.RenewSessionLock`. Sample code for automatic lock renewal can be found here: [Example_autoRenewLocks](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/messaging/azservicebus#example-package-AutoRenewLocks).
114+
115+
### Performance Considerations
116+
117+
#### Receiver
118+
119+
- **Process messages in multiple goroutines**: Receiver settlement methods, like CompleteMessage, are goroutine-safe and can be called concurrently, allowing message processing to run in multiple goroutines.
120+
- **Increase client instances**: Add more receiver clients to distribute the load. The number of concurrent receivers for a session-enabled entity is limited by the number of sessions.
121+
122+
#### Sender
123+
124+
- **Use message batching**: [Sender.NewMessageBatch](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/messaging/azservicebus#Sender.NewMessageBatch) and [Sender.SendMessageBatch](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/messaging/azservicebus#Sender.SendMessageBatch) methods allow you to send batches of messages, reducing network traffic between the service and the client.
125+
126+
---
127+
128+
## Connectivity Issues
129+
130+
### Enterprise Environments and Firewalls
131+
132+
In corporate networks with strict firewall rules, you may encounter connectivity issues when connecting to Service Bus.
133+
134+
**Common solutions:**
135+
136+
1. **Allow the necessary endpoints**: See [Service Bus FAQ: "What ports do I need to open on the firewall?"](https://learn.microsoft.com/azure/service-bus-messaging/service-bus-faq#what-ports-do-i-need-to-open-on-the-firewall).
137+
2. **Use a proxy**: If you require a proxy to connect to Azure resources you can configure your client to use it. See [example using a proxy and/or Websockets][example_proxy_websockets].
138+
3. **Use Websockets**: If you can only connect to Azure resources using HTTPS (443) you can configure your client to use Websockets. See [example using a proxy and/or Websockets][example_proxy_websockets].
139+
4. **Configure network security rules**: If using Azure VNet integration, configure service endpoints or private endpoints as needed.
140+
141+
---
142+
143+
## Advanced Troubleshooting
144+
145+
### Logs to collect
146+
147+
When troubleshooting issues with Service Bus that you need to escalate to support or report in GitHub issues, collect the following logs:
148+
149+
1. **Enable debug logging**: To enable logs, see [logging](#logging).
150+
2. **Timeframe**: Capture logs from at least 5 minutes before until 5 minutes after the issue occurs.
151+
3. **Include timestamps**: Ensure your logging setup includes timestamps. By default, `AZURE_SDK_GO_LOGGING` logging includes timestamps.
152+
153+
### Interpreting Logs
154+
155+
When analyzing Service Bus logs:
156+
157+
1. **Connection errors**: Look for AMQP connection and link errors in `EventConn` logs.
158+
2. **Authentication failures**: Check `EventAuth` logs for credential or authorization failures.
159+
3. **Sender errors**: `EventSender` logs show message send operations and errors.
160+
4. **Receiver errors**: `EventReceiver` logs show message receive operations and session/message lock changes.
161+
162+
### Additional Resources
163+
164+
- [Service Bus Documentation](https://learn.microsoft.com/azure/service-bus-messaging/)
165+
- [Service Bus Pricing](https://azure.microsoft.com/pricing/details/service-bus/)
166+
- [Service Bus Quotas](https://learn.microsoft.com/azure/service-bus-messaging/service-bus-quotas)
167+
- [Service Bus FAQ](https://learn.microsoft.com/azure/service-bus-messaging/service-bus-faq)
168+
169+
### Filing GitHub Issues
170+
171+
To file an issue in GitHub, use this [link](https://github.com/Azure/azure-sdk-for-go/issues/new/choose) and include the following information:
172+
173+
1. **Service Bus details**:
174+
- Session-enabled?
175+
- SKU (Standard/Premium)
176+
2. **Client environment**:
177+
- Machine specifications
178+
- Number of client instances running
179+
- Package version
180+
- Go version
181+
3. **Message patterns**:
182+
- Average message size
183+
- Throughput (messages per second)
184+
- Whether traffic is consistent or bursty
185+
4. **Reproduction steps**:
186+
- A minimal code example that reproduces the issue
187+
- Steps to reproduce the problem
188+
5. **Logs**:
189+
- Include diagnostic logs from before, during, and after the failure. For instructions on enabling logging see the [Logs to collect](#logs-to-collect) section above.
190+
191+
**NOTE**: The information in Github issues and logs are publicly viewable. Please keep this in mind when posting any information.
192+
193+
<!-- LINKS -->
194+
[azidentity_troubleshooting]: https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/azidentity/TROUBLESHOOTING.md
195+
[azidentity_tokencredentials]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#readme-credential-chains
196+
[example_proxy_websockets]: https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/messaging/azservicebus/example_websockets_and_proxies_test.go

0 commit comments

Comments
 (0)