Go 1.23 Iterators: Learn to Build and Use Golang Iterators

Go 1.23 introduces a new way to work with sequences of data: iterators. This feature fundamentally changes how developers can handle collections of items, such as lists, map entries, or data streams. 

Think of an iterator as a smart helper that provides items one by one, precisely when you need them. This "lazy" approach is especially beneficial for very large datasets or continuous streams of information, as it can significantly save memory and make programs more efficient.

What Are Iterators in Go?

Go 1.23 iterators are functions that "yield" values sequentially. This mechanism allows a function to pause its execution, return a value, and then resume later to produce the next value, all without the overhead of creating a complete collection upfront.

Core Iterator Concepts: Seq and Seq2

Go 1.23 formally defines iterator types in the new iter package. These types serve as the foundation for creating and using iterators.

iter.Seq[V any] (Sequence of single values):

What it is: Represents an iterator that produces a sequence of individual items (e.g., a list of numbers: 1, then 2, then 3).

How it works (simplified): It's a function type: func(yield func(V) bool). The outer function is your iterator. It receives a yield callback function as an argument. 

Your iterator calls this yield function for each item it wants to produce. The yield function returns a boolean: true to continue iteration, false to stop early. V is a generic type parameter representing the type of item in the sequence.

iter.Seq2[K, V any] (Sequence of key-value pairs):

What it is: Designed for sequences where each item consists of two parts, like a key and its corresponding value (e.g., from a map: "name" with "Alice", then "age" with 30).

How it works (simplified): Similar to Seq, but its type is func(yield func(K, V) bool). The yield callback receives two arguments (a key K and a value V) for each step.

Example: Seq and Seq2 basic structure

// exampleSeqAndSeq2 demonstrates the basic structure of Seq and Seq2 iterators.*
func exampleSeqAndSeq2() {
	printSection("Core Iterator Types: Seq and Seq2")

	// 1. Seq[V any] Example: An iterator for numbers 0, 1, 2*
	numbersIterator := func(yield func(int) bool) {
		fmt.Println("  (Seq Iterator: Starting)")
		for i := 0; i < 3; i++ {
			fmt.Printf("  (Seq Iterator: About to yield %d)\n", i)
			if !yield(i) { // Call yield for each number*
				fmt.Printf("  (Seq Iterator: Yield for %d returned false, stopping)\n", i)
				return // Stop if yield returns false*
			}
			fmt.Printf("  (Seq Iterator: Successfully yielded %d)\n", i)
		}
		fmt.Println("  (Seq Iterator: Finished)")
	}

	fmt.Println("Consuming Seq iterator directly:")
	// Manually "consume" the iterator by providing a yield function*
	numbersIterator(func(n int) bool {
		fmt.Printf("  Consumer of Seq: Got %d\n", n)
		// Let's stop early after getting the number 1*
		if n == 1 {
			fmt.Println("  Consumer of Seq: Deciding to stop early.")
			return false // Signal to stop*
		}
		return true // Signal to continue*
	})

	// 2. Seq2[K, V any] Example: An iterator for simple key-value pairs*
	pairsIterator := func(yield func(string, int) bool) {
		fmt.Println("\n  (Seq2 Iterator: Starting)")
		if !yield("first", 100) { fmt.Println("  (Seq2 Iterator: Yield for 'first' returned false, stopping)"); return }
		fmt.Println("  (Seq2 Iterator: Successfully yielded 'first', 100)")
		if !yield("second", 200) { fmt.Println("  (Seq2 Iterator: Yield for 'second' returned false, stopping)"); return }
		fmt.Println("  (Seq2 Iterator: Successfully yielded 'second', 200)")
		fmt.Println("  (Seq2 Iterator: Finished)")
	}

	fmt.Println("\nConsuming Seq2 iterator directly:")
	pairsIterator(func(k string, v int) bool {
		fmt.Printf("  Consumer of Seq2: Got Key='%s', Value=%d\n", k, v)
		return true *// Continue for all items*
	})
}

Using Iterators with for...range Loops

A key enhancement in Go 1.23 is the extension of the for...range loop to work directly with functions that match the iterator signatures. This makes using iterators feel natural and idiomatic in Go.

• How it works: When you use an iterator function in a for...range loop, Go's compiler synthesizes the yield callback function. The iterator provides items to the loop body one at a time.
• Stopping Early: If you use a break statement inside your for...range loop (or a return from the enclosing function), the synthesized yield function will return false, signaling the iterator to stop producing more items. This makes loops efficient, as they don't perform unnecessary work.

Example: for...range with an iterator

// countdownIterator yields numbers from v down to 0.// It matches the signature for a single-value iterator for range.*
func countdownIterator(startValue int) iter.Seq[int] {
	return func(yield func(int) bool) {
		fmt.Printf("  (Countdown Iterator(%d): Starting)\n", startValue)
		for i := startValue; i >= 0; i-- {
			fmt.Printf("  (Countdown Iterator: About to yield %d)\n", i)
			if !yield(i) {
				fmt.Printf("  (Countdown Iterator: Yield for %d returned false, stopping early)\n", i)
				return
			}
			fmt.Printf("  (Countdown Iterator: Successfully yielded %d)\n", i)
		}
		fmt.Printf("  (Countdown Iterator(%d): Finished all iterations)\n", startValue)
	}
}

func exampleForRangeWithIterators() {
	printSection("for...range with Iterators")

	fmt.Println("Looping with for...range over countdownIterator(3):")
	for val := range countdownIterator(3) {
		fmt.Printf("  Loop Body: Received %d\n", val)
		if val == 1 {
			fmt.Println("  Loop Body: Value is 1, breaking loop.")
			break *// This will cause the iterator's yield to return false*
		}
	}
	fmt.Println("Looping finished.")
}

Iterators in Go's Standard Library

Go's standard library, particularly the slices and maps packages, has been significantly enhanced to provide and consume iterators.

Working with Slices (slices package)

The slices package now includes functions that return iterators:

• slices.All(mySlice): Returns an iterator (iter.Seq2[int, T]) to go through mySlice from start to end, yielding index-value pairs.
• slices.Values(mySlice): Returns an iterator (iter.Seq[T]) for just the values in mySlice.
• slices.Backwards(mySlice): Returns an iterator (iter.Seq2[int, T]) to go through mySlice from end to start, yielding index-value pairs.
• slices.Collect(iterator): A utility function that gathers all items from any iter.Seq[T] iterator and puts them into a new slice []T.

Example: slices package iterators

func exampleStdLibSlices() {
	printSection("Standard Library: slices Package Iterators")
	myFruits := []string{"apple", "banana", "cherry", "date"}

	fmt.Println("\nIterating with slices.All() (index-value pairs):")
	for i, fruit := range slices.All(myFruits) {
		fmt.Printf("  Index: %d, Fruit: %s\n", i, fruit)
	}

	fmt.Println("\nIterating with slices.Values():")
	for fruit := range slices.Values(myFruits) {
		fmt.Printf("  Fruit: %s\n", fruit)
	}

	fmt.Println("\nIterating with slices.Backwards() (index-value pairs):")
	for i, fruit := range slices.Backwards(myFruits) {
		fmt.Printf("  Index: %d, Fruit: %s\n", i, fruit)
	}

	// Example of slices.Collect() with a custom iterator// (using countdownIterator from previous example for simplicity)*
	collectedCountdown := slices.Collect(countdownIterator(2)) *// Collects 2, 1, 0*
	fmt.Printf("\nCollected from countdownIterator(2) using slices.Collect(): %v\n", collectedCountdown)
}

Working with Maps (maps package)

Similarly, the maps package provides iterator functions:

• maps.All(myMap): Returns an iterator (iter.Seq2[K, V]) for key-value pairs in myMap.
• maps.Keys(myMap): Returns an iterator (iter.Seq[K]) for just the keys in myMap.
• maps.Values(myMap): Returns an iterator (iter.Seq[V]) for just the values in myMap.
• maps.Collect(iterator): Gathers all key-value pairs from an iter.Seq2[K,V] and puts them into a new map map[K]V.

Example: maps package iterators

func exampleStdLibMaps() {
	printSection("Standard Library: maps Package Iterators")
	myAges := map[string]int{"Alice": 30, "Bob": 25, "Charlie": 35}

	fmt.Println("\nIterating with maps.All() (key-value pairs):")
	*// Note: Map iteration order is not guaranteed*
	for name, age := range maps.All(myAges) {
		fmt.Printf("  Name: %s, Age: %d\n", name, age)
	}

	fmt.Println("\nIterating with maps.Keys():")
	var keys []string
	for name := range maps.Keys(myAges) {
		keys = append(keys, name) *// Collect keys to sort for predictable output*
	}
	sort.Strings(keys) *// Sort for consistent output*
	for _, name := range keys {
		fmt.Printf("  Key: %s\n", name)
	}

	fmt.Println("\nIterating with maps.Values():")
	var values []int
	for age := range maps.Values(myAges) {
		values = append(values, age)
	}
	sort.Ints(values) *// Sort for consistent output*
	for _, age := range values {
		fmt.Printf("  Value: %d\n", age)
	}

	*// Example for maps.Collect()// Create a simple key-value iterator function*
	simplePairIter := func(yield func(string, int) bool) {
		yield("x", 1)
		yield("y", 2)
	}
	collectedMap := maps.Collect(iter.Seq2[string,int](simplePairIter))
	fmt.Printf("\nCollected from custom pair iterator using maps.Collect(): %v\n", collectedMap)
}

>> Read more:

• Understanding Golang Ordered Map with Code Examples
• An In-Depth Guide for Using Go sync.Map with Code Sample

Combining Operations with Iterators

Iterators shine when you chain operations. Functions that consume one iterator and produce another allow for powerful and readable data transformation pipelines, often without allocating intermediate collections. slices.Sorted is an example; it can take an iterator and produce a new iterator that yields sorted values.

Example: Combining maps.Keys, slices.Sorted, and slices.Collect

func exampleCombiningOperations() {
	printSection("Combining Iterator Operations")
	fruitPrices := map[string]float64{"banana": 0.5, "apple": 1.2, "cherry": 2.5, "date": 1.8}
	fmt.Printf("Original fruit prices: %v\n", fruitPrices)

	// Goal: Get a sorted list of fruit names (keys)// 1. maps.Keys(fruitPrices) -> returns iter.Seq[string] (unsorted keys)
	keyIterator := maps.Keys(fruitPrices)

	// 2. slices.Sorted(iterator) -> takes an iter.Seq[E] and returns a new iter.Seq[E] that yields sorted items//    This is a new iterator that will yield the keys in sorted order.
	sortedKeyIterator := slices.Sorted(keyIterator)

	// 3. slices.Collect(iterator) -> collects items from the sortedKeyIterator into a slice
	sortedFruitNames := slices.Collect(sortedKeyIterator)

	// The above can be done in one line:// sortedFruitNames := slices.Collect(slices.Sorted(maps.Keys(fruitPrices)))

	fmt.Printf("Sorted fruit names: %v\n", sortedFruitNames)
}

Creating Your Own Custom Iterators

The real power comes from creating your own functions that act as iterators. This allows you to define custom iteration logic for any data structure or sequence generation rule.

How: Define a function that returns an iter.Seq[V] or iter.Seq2[K, V]. Inside this function, you return another function (a closure) that implements the yield pattern.

Example: A custom iterator for Fibonacci numbers

// fibonacciIterator returns an iterator (iter.Seq[int])// that yields Fibonacci numbers up to a given limit.*
func fibonacciIterator(limit int) iter.Seq[int] {
	return func(yield func(int) bool) {
		fmt.Printf("  (Fibonacci Iterator up to %d: Starting)\n", limit)
		a, b := 0, 1
		for a <= limit {
			fmt.Printf("  (Fibonacci Iterator: About to yield %d)\n", a)
			if !yield(a) {
				fmt.Printf("  (Fibonacci Iterator: Yield for %d returned false, stopping early)\n", a)
				return
			}
			fmt.Printf("  (Fibonacci Iterator: Successfully yielded %d)\n", a)
			a, b = b, a+b
		}
		fmt.Printf("  (Fibonacci Iterator up to %d: Finished all iterations)\n", limit)
	}
}

func exampleCustomIterator() {
	printSection("Custom Iterator: Fibonacci Example")

	fmt.Println("Fibonacci numbers up to 20 (consuming all):")
	for n := range fibonacciIterator(20) {
		fmt.Printf("  Loop Body: Received Fibonacci number %d\n", n)
	}

	fmt.Println("\nFibonacci numbers up to 50 (stopping early if > 30):")
	for n := range fibonacciIterator(50) {
		fmt.Printf("  Loop Body: Received Fibonacci number %d\n", n)
		if n > 30 {
			fmt.Println("  Loop Body: Number > 30, breaking.")
			break
		}
	}
}

This iterator generates Fibonacci numbers up to a given limit.

"Lazy Evaluation": Only Get What You Need

The primary advantage of iterators is lazy evaluation.

What it means: Iterators produce items only when they are requested by the consumer (e.g., the for...range loop). If you have a potentially vast sequence but only need the first few items, an iterator will only generate those few.

Benefits:

• Memory Savings: Your program doesn't need to allocate memory for an entire collection if it's not all needed at once. This is crucial for large datasets or infinite/streaming data.
• Speed for Partial Processing: If a loop terminates early (e.g., a search finds an item), the iterator doesn't waste time and CPU cycles preparing subsequent, unneeded items.

Example: Demonstrating Lazy Evaluation

// potentiallyLargeSequenceIterator simulates an iterator that could produce many items.// We add a small delay to make the lazy evaluation more observable.*
func potentiallyLargeSequenceIterator(maxItems int) iter.Seq[int] {
	return func(yield func(int) bool) {
		fmt.Printf("  (Large Sequence Iterator (max %d): Starting)\n", maxItems)
		for i := 0; i < maxItems; i++ {
			fmt.Printf("  (Large Sequence Iterator: About to yield item #%d)\n", i)
			time.Sleep(100 * time.Millisecond) *// Simulate work to produce item*
			if !yield(i) {
				fmt.Printf("  (Large Sequence Iterator: Yield for item #%d returned false, stopping early)\n", i)
				return
			}
			fmt.Printf("  (Large Sequence Iterator: Successfully yielded item #%d)\n", i)
		}
		fmt.Printf("  (Large Sequence Iterator (max %d): Finished all iterations)\n", maxItems)
	}
}

func exampleLazyEvaluation() {
	printSection("Lazy Evaluation Example")

	fmt.Println("Taking only the first 3 items from a potentially large sequence (max 10):")
	itemsProcessed := 0
	for val := range potentiallyLargeSequenceIterator(10) {
		fmt.Printf("  Loop Body: Received %d\n", val)
		itemsProcessed++
		if itemsProcessed >= 3 {
			fmt.Println("  Loop Body: Processed 3 items, breaking loop.")
			break
		}
	}
	fmt.Println("Looping finished after lazy evaluation.")
	// Observe that the iterator doesn't print "About to yield" for items beyond the break.*
}

Resource Management with Iterators

Iterators provide a natural and robust way to manage resources that are tied to a sequence, such as open files, database connections, or network streams.

How it helps: An iterator can acquire a resource when it begins and ensure its release when the iteration completes (either fully or due to early termination signaled by yield returning false).

Reliability: This pattern helps prevent resource leaks. If an error occurs or the consumer stops early, the iterator can perform cleanup.

Example: Simulating Resource Management

// fileLinesIterator simulates reading lines from a file.// It demonstrates opening/closing a "resource".*
func fileLinesIterator(filePath string) iter.Seq[string] {
	return func(yield func(string) bool) {
		fmt.Printf("  (File Iterator for '%s': Opening simulated file...)\n", filePath)
		// In a real scenario: file, err := os.Open(filePath); if err != nil { /* handle */ return }// It's crucial to handle cleanup if the yield loop exits.// The `defer` keyword is excellent for ensuring cleanup if the *iterator function itself*// returns normally or panics. However, if the `yield` loop exits early because `yield`// returns `false`, the iterator function doesn't necessarily "return" immediately from// the outer scope. Explicit cleanup after the loop or when `!yield` is true is more direct.*
		closed := false
		defer func() {
			if !closed { *// Ensure it's closed if panic or unexpected exit happens before explicit close*
				fmt.Printf("  (File Iterator for '%s': Closing simulated file via DEFER - should be already closed if exited cleanly)\n", filePath)
			}
		}()

		simulatedLines := []string{"Line 1 from " + filePath, "Line 2 from " + filePath, "Line 3 from " + filePath}

		for i, line := range simulatedLines {
			fmt.Printf("  (File Iterator for '%s': About to yield line %d: '%s')\n", filePath, i+1, line)
			if !yield(line) {
				fmt.Printf("  (File Iterator for '%s': Yield returned false. Closing simulated file NOW.)\n", filePath)
				// In a real scenario: file.Close()*
				closed = true
				return
			}
			fmt.Printf("  (File Iterator for '%s': Successfully yielded line %d)\n", filePath, i+1)
		}

		fmt.Printf("  (File Iterator for '%s': Finished all lines. Closing simulated file NOW.)\n", filePath)
		// In a real scenario: file.Close()*
		closed = true
	}
}

func exampleResourceManagement() {
	printSection("Resource Management Example (Simulated File Reading)")

	fmt.Println("\nConsuming all lines from 'report.txt':")
	for line := range fileLinesIterator("report.txt") {
		fmt.Printf("  Loop Body: Read: %s\n", line)
	}

	fmt.Println("\nConsuming only the first 2 lines from 'data.log':")
	linesRead := 0
	for line := range fileLinesIterator("data.log") {
		fmt.Printf("  Loop Body: Read: %s\n", line)
		linesRead++
		if linesRead >= 2 {
			fmt.Println("  Loop Body: Read 2 lines, breaking.")
			break
		}
	}
	fmt.Println("Looping for 'data.log' finished.")
}

Moving to the New Iterator Style

Before Go 1.23, Go developers often implemented iterator-like patterns using:

• Interfaces with a Next() bool method and a Value() method.
• Go channels to send items one by one.

While these patterns are functional, the new language-integrated iterators generally offer:

• Simpler Syntax: The for...range integration is more concise.
• Better Performance: Function-call-based iterators can be more efficient than channel-based ones, avoiding go-routine scheduling overhead.
• Clearer Control Flow: The yield mechanism is explicit about continuation.

Migrating existing code might involve reframing the iteration logic to fit the func(yield func(...) bool) pattern. The result is often cleaner, more idiomatic, and potentially more performant Go code.

>> Read more: A Comprehensive Guide to Concurrency in Golang

From Experiment to Standard Feature

The iterator functionality (often called "range over func") was first introduced as an experimental feature in Go 1.22, enabled by the GOEXPERIMENT=rangefunc flag. This trial period allowed the Go team and the community to test, provide feedback, and refine the design. Its promotion to a standard, non-experimental language feature in Go 1.23 signifies its maturity, stability, and readiness for widespread adoption.

Conclusion

Go 1.23’s iterators are a major step forward, offering a cleaner, more efficient way to work with data sequences. With lazy evaluation and smooth integration into for...range loops and the standard library, they make complex operations simpler, use less memory, and run faster, especially when you don’t need every item. This feature gives developers a strong, flexible tool for building efficient, data-driven Go applications.

>>> Follow and Contact Relia Software for more information!