Nate Woods

Nate Woods

Go 1.23 Iterators

2024-07-06

Reading Why People are Angry over Go 1.23 Iterators (or having thePrimeagen read it) stirred the following idea.

I think this little adapter converts what thePrimeagen was thinking about with “iterable interfaces”, to what Russ Cox (rsc) and the go team are actually planning on using in go 1.23: functional iterators.

type Iterable[V any] interface {
    Init()
    Next() (V, bool)
    Done()
}

func New[V any](core Iterable[V]) iter.Seq[V] {
    return nil // TODO
}

Background

The go team is looking to add iterable types to the language. Folks were expecting an interface to enable for range-based iteration over custom types. Instead, the go team is using a new type iter.Seq or iter.Seq2; And it’s not quite what folks were expecting.

The iter.Seq (or it’s sibling iter.Seq2) are defined by the about-to-be-added iter package as:

type (
	Seq[V any]     func(yield func(V) bool)
	Seq2[K, V any] func(yield func(K, V) bool)
)

Here the go team is looking to support for v := range myIterable and for k, v := range myIterable syntax. While the usage these iterable types is fairly simple, the implementation is a bit more complex.

The crux of the problem is that there is a bit of a control flow issue, the for range loop needs to be able to control the iteration, but the iterator needs to be able to control the loop. To do this, the Seq* types are functions that take a function that is called for each value in the sequence. The function passed to the Seq* type is called a “yield” function. Yield returns a bool indicating if the loop should continue or not.

Whew, that’s kind of a lot to take in. But, generally speaking, there are a lot of good things packed into this design.

By having the iterator itself be a function, it can use defer to clean up resources when the loop is done. This is a subtle, but powerful side-effect of the design; one of those strategic decisions that comes from minds that have been doing this for a long time.

The Adapter

I’ve roughed out a bit of “glue code” that can be written to convert what folks were expecting to what the go team is actually doing.

type Iterable[V any] interface {
    Init()
    Next() (V, bool)
    Done()
}

func Walk[V any](core Iterable[V]) iter.Seq[V] {
	return func(yield func(V) bool) bool {
		core.Init()
		defer core.Done()
		for {
			v, done := core.Next()
			if !done {
				return true
			}
			if !yield(v) {
				return false
			}
		}
	}
}

The Iterable is the interface that thePrimeagen was thinking about. It’s a simple interface that has three methods: Init, Next, and Done. The Init method is called once before any calls to Next to initialize the iterator. The Next method returns the next value in the sequence and a boolean indicating if there are more values. The Done method is called after the last call to Next.

Oh, and what about Seq2, well… we will need another interface and constructor for that.

type Iterable2[V any] interface {
    Init()
    Next() (V, bool)
    Done()
}

func Walk2[K, V any](core Iterable2[K, V]) iter.Seq2[K, V] {
	return func(yield func(K, V) bool) bool {
		core.Init()
		defer core.Done()
		for {
			k, v, done := core.Next()
			if !done {
				return true
			}
			if !yield(k, v) {
				return false
			}
		}
	}
}

Conclusion

Now, I haven’t put a ton of thought into this, but I think it’s a simple enough bit of code. I’m sure there are some optimizations that could be made, or some deep/technical concept that I’m missing. But, I think it gets the point across.

I’m fairly certain that this was already considered by the go-team, and they decided to go with the Seq* types for a good reason. But, I didn’t see any harm in sharing this with you all.

Best of luck with your Go 1.23 adventures!

×