Go goroutine Leak Detection and Remediation Methods, pprof Tool Localization and Context Cancellation Specifications

by

Go goroutine Leak Detection and Remediation Methods, pprof Tool Localization and Context Cancellation Specifications

A Complete Guide to Goroutine Leak Detection and Remediation in Go

I. Analysis of Key Problem Points

The core characteristic of a goroutine leak is that a goroutine starts and then gets blocked on channel operations, lock waits or I/O operations, never being able to exit. Main causes:

– Channel blocking: Sending data to an unbuffered channel with no receiver, or reading from a channel that will never receive writes
– Missing context cancellation: Goroutines blocked on network I/O, `time.After`, or `select` without handling cancellation signals
– Deadlocks / lock contention: Unreleased `sync.Mutex`, mismatched `WaitGroup.Add` / `WaitGroup.Done` calls
– `for-range` iterating over a channel that is never closed

II. Detection Methods

Option 1: Troubleshooting with pprof (Production-ready, Recommended)

 

import _ "net/http/pprof"

func main() {
    go func() {
        http.ListenAndServe("localhost:6060", nil)
    }()
    // ...
}

Troubleshooting Commands：

# View all current goroutines and their call stacks
go tool pprof http://localhost:6060/debug/pprof/goroutine

# Execute within the interactive interface
(pprof) top          # Show the most concentrated stack traces
(pprof) traces       # Full call stacks
(pprof) list FuncName # Locate source code lines

# View full text directly (most intuitive; search for recurring stack traces)
curl http://localhost:6060/debug/pprof/goroutine?debug=1 > goroutines.txt

Judgment Tips: The number of goroutines in a healthy application remains relatively stable. If `http://localhost:6060/debug/pprof/goroutine?debug=1` shows a continuously rising goroutine count and a large number of goroutines share identical stack traces (e.g., all blocked on `chan send` / `chan receive`), you have found a leak point.

Option 2: Goroutine Leak Detection in Unit Tests with goleak (Open-sourced by Uber)

import "go.uber.org/goleak"

func TestMain(m *testing.M) {
    goleak.VerifyTestMain(m)
}

func TestWorker(t *testing.T) {
    defer goleak.VerifyNone(t) // Check for leaks after test completion
    // ...
}

Option 3: Rough Monitoring with runtime.NumGoroutine

// Expose metrics with expvar or Prometheus
gauge.Set(float64(runtime.NumGoroutine()))

If the count keeps increasing without dropping, a leak exists. Use pprof to locate the exact position.

Option 4: gops / runtime.Stack

buf := make([]byte, 1<<20)
n := runtime.Stack(buf, true)
fmt.Printf("%s", buf[:n]) // Print stack traces of all goroutines

III. Typical Leak Scenarios and Remediation

Scenario 1: Sending to an unbuffered channel with no receiver

// ❌ Leak: find times out and returns first. No one receives from resultChan, so find blocks
func find(ctx context.Context) (interface{}, error) {
    resultChan := make(chan interface{}) // Unbuffered channel
    go func() {
        resultChan <- expensiveCall()     // Blocks forever!
    }()
    select {
    case r := <-resultChan:
        return r, nil
    case <-ctx.Done():
        return nil, ctx.Err()
    }
}

Fix: Use a buffered channel with a capacity of 1. The sender can complete the send even if no receiver is available:

// ✅ resultChan := make(chan interface{}, 1)

Or make sends cancellable via select + ctx.

Scenario 2: select only listens for writes, ctx is forgotten.

// ✅ Standard practice: Associate ctx with all blocking operations
for {
    select {
    case <-ctx.Done():
        return ctx.Err() // Exit, goroutine can be reclaimed
    case v, ok := <-in:
        if !ok {
            return nil    // Channel closed, exit promptly
        }
        process(v)
    }
}

Scenario 3: HTTP request without a timeout set

// ❌ No timeout by default, risk of goroutine leak
http.Get(url)

// ✅
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
http.DefaultClient.Do(req)

Scenario 4: Mismatched WaitGroup / Recursive locking
Ensure `Add` and `Done` are strictly paired; use `defer mu.Unlock()`; aid detection with `go vet` and `-race`.

IV. Summary of Context Cancellation Specifications

VI. Summary：
Prevention first: Every goroutine must have a clear exit path. All channel operations shall be paired with `select + ctx.Done()`. Use buffered channels of size 1 for one-off sends. External I/O must use timeout-aware APIs such as `NewRequestWithContext`.

Test guardrails: Enforce `goleak` in CI to prevent new leaks from being introduced.

Production troubleshooting: Expose pprof. Check goroutine counts and duplicate stacks via `/debug/pprof/goroutine?debug=1` to quickly locate leak sources.

Code review: Enable `go vet`. Focus on goroutines spawned with `go func`, checking for unconsumed channel sends and unbounded blocking calls without timeouts.

Leave a Comment