notexOPENOverview
Notex/Languages/Test Concurrent Code with synctest
05.22LESSON 22 OF 29 · 7 MIN READ

Test Concurrent Code with synctest

Always free to read

You might be surprised to discover that a language with strong support for both testing and concurrency did not initially include strong support for testing concurrency. The release of Go 1.25 addressed this oversight with the synctest package.

There are three common situations that were difficult to test before the release of synctest. First, there’s code that depends on time passing. Say that you implement a simple cache with expiration. You can put values into it and read values from it. If a value isn’t read within the expiration period, it is no longer returned by the cache.

func (c *Cache[K, V]) Get(key K) (V, bool) {
    c.mu.Lock()
    defer c.mu.Unlock()
    e, ok := c.items[key]
    if !ok || time.Now().After(e.expiresAt) {
        delete(c.items, key)
        var zero V
        return zero, false
    }
    return e.value, true
}

This code gets a value from the cache. (The complete implementation is in the cachev1 directory) It compares the time that an entry was placed into the cache against the current time. Being a good developer, you want to write a test to make sure this code functions correctly. Since the cache has different behavior depending on how much time has passed, you need to include time.Sleep calls to put pauses into reads and writes to the cache.

Here’s a test that puts a value into the cache, sleeps until the value expires, and then confirms that the value isn’t present:

func TestCache_GetAfterExpiry_standard(t *testing.T) {
    c := cachev1.New[string, int](1 * time.Second)
    c.Set("answer", 42)

    // Jump past the TTL.
    time.Sleep(1100 * time.Millisecond)

    _, ok := c.Get("answer")
    if ok {
        t.Fatal("expected key to be expired after TTL")
    }
}

If you run this test, it passes, but that call to time.Sleep means it takes about a second to run. It’s only one second, but properly testing this cache requires additional tests that also require time.Sleep calls. Other parts of your program are liable to have their own tests that require the passage of time and this adds up. Unit tests should execute quickly to prevent developers from being distracted. That’s where synctest is useful.

The synctest package defines two functions: synctest.Test and synctest.Wait. I’ll talk about Test first. It has the signature:

func Test(t *testing.T, f func(*testing.T))

This looks a lot like the t.Run method used in “Running Table Tests”. Like t.Run, synctest.Test creates a sub-environment. In the case of synctest.Test, this environment has magic powers. Each call to synctest.Test creates what the Go team calls a “bubble”. In this bubble, there is a fake clock, which starts at January 1st, 2000 UTC. Within the bubble, every time.Sleep call or use of a time.Ticker or time.Timer is tracked across all goroutines launched in the bubble. This allows the fake clock to move forward without waiting for the specified time to elapse in the real world.

An example makes this easier to understand. Say you have a test that looks like this:

func TestTimeWaiting(t *testing.T) {
    start := time.Now() // real wall clock time/date
    go func() {
        time.Sleep(1 * time.Second)
        t.Log(time.Since(start)) // may not be exactly "1s"
        t.Log(time.Now())
    }()
    // the goroutine above will run before this Sleep returns
    time.Sleep(2 * time.Second)
    t.Log(time.Since(start))    // may not be exactly "2s"
    t.Log(time.Now())
}

Running this test takes 2 seconds to complete. But if you use synctest.Test, the test completes instantly:

func TestTime(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        start := time.Now() // always midnight UTC 2000-01-01
        go func() {
            time.Sleep(1 * time.Second)
            t.Log(time.Since(start)) // always logs "1s"
            t.Log(time.Now())
        }()
        // the goroutine above will run before this Sleep returns
        time.Sleep(2 * time.Second)
        t.Log(time.Since(start))    // always logs "2s"
        t.Log(time.Now())
    })
}

Note

synctest requires Go 1.25 or later. If you get an error like this, Check your go.mod file and update the specified version of Go:

panic: synctest.Run not supported with asynctimerchan!=0

If we apply this pattern to the TestCache_GetAfterExpiry_standard function from earlier, running the test is now instanteous.

func TestCache_GetAfterExpiry_synctest(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        c := cachev1.New[string, int](1 * time.Second)
        c.Set("answer", 42)

        // Jump past the TTL.
        time.Sleep(1100 * time.Millisecond)

        _, ok := c.Get("answer")
        if ok {
            t.Fatal("expected key to be expired after TTL")
        }
    })
}

Using synctest also helps ensure that all goroutines properly exit. As mentioned in the related discussion, you must make sure that every goroutine launched also exits. Let’s demonstrate this with an improved cache implementation. The first version doesn’t remove elements when they expire; it waits for them to be looked up again. This leaks memory if an entry is placed in the cache and never referenced again. Let’s add a background job to remove expired elements:

func (c *Cache[K, V]) clean(interval time.Duration) {
    ticker := time.NewTicker(interval)
    defer ticker.Stop()
    for range ticker.C {
        c.mu.Lock()
        now := time.Now()
        for k, e := range c.items {
            if now.After(e.expiresAt) {
                delete(c.items, k)
                c.Stats.RemovedBySweep++
            }
        }
        c.mu.Unlock()
    }
}

func New[K comparable, V any](ttl, sweepInterval time.Duration) *Cache[K, V] {
    c := &Cache[K, V]{
        items: make(map[K]entry[V]),
        ttl:   ttl,
        done:  make(chan struct{}),
    }
    go c.clean(sweepInterval)
    return c
}

Every time you create a new cache instance with the New function, it also launches a goroutine to remove any items that have already expired. (The rest of the code for this cache implementation is in the cachev2 directory.) Again, being a good developer, you write a test to validate that this code works. The implementation tracks how many entries are removed via clean, so you can check this value:

func TestCache_Clean(t *testing.T) {
    c := cachev2.New[string, string](2*time.Second, 1*time.Second)
    c.Set("key", "hello")
    time.Sleep(4 * time.Second)
    val, ok = c.Get("key")
    if ok {
        t.Fatal("expected key to not be present")
    }
    if c.Stats.RemovedByClean != 1 {
        t.Error("expected removed by Sweep to be 1")
    }
}

If you run this test, it will pass, but it takes 4 seconds for the test to complete. Wrapping it in synctest.Test to speed up the test produces a surprising result:

=== RUN   TestCache_Clean_synctest
--- FAIL: TestCache_Clean_synctest (0.00s)
panic: deadlock: main bubble goroutine has exited but blocked
    goroutines remain [recovered, repanicked]
...

The synctest bubble ensures that every goroutine created within the bubble exits before the bubble ends. (I wish the Go team had called this “popping”, but the software industry is not in a whimsical moment.) If any goroutines continue to run, the test panics. Looking at the implemmentation of clean, there’s no way to signal to the goroutine when to exit. Without synctest, problems like this are very hard to detect.

The following changes fix this. First we update our cache implementation:

func (c *Cache[K, V]) clean(interval time.Duration) {
    ticker := time.NewTicker(interval)
    defer ticker.Stop()
    for {
        select {
        case <-ticker.C:
            c.mu.Lock()
            now := time.Now()
            for k, e := range c.items {
                if !now.Before(e.expiresAt) {
                    delete(c.items, k)
                    c.Stats.RemovedBySweep++
                }
            }
            c.mu.Unlock()
        case <-c.done:
            return
        }
    }
}

func (c *Cache[K, V]) Done() {
    c.mu.Lock()
    defer c.mu.Unlock()
    select {
    case <-c.done:
        return
    default:
        close(c.done)
    }
}

Next, we fix the test to call c.Done:

func TestCache_Clean_synctest(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        c := cachev2.New[string, string](2*time.Second, 1*time.Second)
        t.Cleanup(c.Done)
        c.Set("key", "hello")
        time.Sleep(4 * time.Second)
        synctest.Wait()
        val, ok = c.Get("key")
        if ok {
            t.Error("expected key to not be present")
        }
        if c.Stats.RemovedBySweep != 1 {
            t.Error("expected removed by Sweep to be 1")
        }
    })
}

With the new clean implementation and the addition of the Done method, the test passes when it includes a call to t.Cleanup(c.Done).

This brings us to the third problem addressed by synctest. You might have noticed the call to synctest.Wait() in the fixed code and wondered what it does. Before I answer that question, think about this: how do you know that sleeping for 4 seconds is enough? You know that the background cleaner goroutine will receive a tick from its timer within 4 seconds, but how do you know that it will finish looking for expired cache entries to remove and get back to waiting for the next tick of its timer?

The fact is that you don’t know how long to wait; you are taking for granted that it should be enough time. If you write tests this way and run them on a machine with lots of load (like a shared CI/CD server), you might find enough isn’t enough. Unless you use an absurdly big value for time.Sleep, tests fail randomly. In a synctest bubble where time “passes” instanteously, it becomes more likely that the work won’t complete.

The synctest.Wait function provides a solution for this problem. When you call it from inside a closure that’s passed to synctest.Test, the test pauses until all other goroutines run to completion, are waiting to read from or write to a channel, sleeping with a time.Sleep, waiting on a WaitGroup.Wait, or waiting on a Cond.Wait. The synctest spec calls this being “durably blocked”.

To make our background cleaner test correct, we need to add a synctest.Wait call after the time.Sleep(4 * time.Second). This guarantees that the clean goroutine not only gets a chance to run, but that it runs until it is waiting to read the next tick from the ticker channel. This ensures that the entry was removed from the cache.

The synctest package solves many of the problems in testing concurrent code, but it does have some limitations. For more details, look at the package documentation and the blog post from the Go team that introduced synctest.

Progress is saved on this device.
KEEP EXPLORING

One note leads to another.

Back to the library →
⌕
Search across the entire library ⌘ K