GoCQ: High-Performance Concurrent Queue for Gophers

Package gocq offers a concurrent queue system using channels and goroutines, supporting both FIFO and priority operations, with options for result-returning and void (non-returning) queues. Zero dependency just install, import and use any where in your go program.

🌟 Features

• Generic type support for both data and results
• Result returning and void (non-returning) queue variants
• Configurable concurrency limits with auto-scaling based on CPU cores
• FIFO queue with O(1) operations
• Priority queue support with O(log n) operations
• Non-blocking job submission
• Control job after submission
• Thread-safe operations
• Pause/Resume functionality
• Clean and graceful shutdown mechanisms

🔧 Installation

go get github.com/fahimfaisaal/gocq/v2

🚀 Quick Start

package main

import (
    "fmt"
    "time"
    "github.com/fahimfaisaal/gocq/v2"
)

func main() {
    // Create a queue with 2 concurrent workers
    queue := gocq.NewQueue(2, func(data int) (int, error) {
        time.Sleep(500 * time.Millisecond)
        return data * 2, nil
    })
    defer queue.Close()

    // Add a single job
    result, err := queue.Add(5).Result()
    fmt.Println(result, err) // Output: 10 nil

    // Add multiple jobs
    for result := range queue.AddAll([]int{1, 2, 3, 4, 5}).Results() {
        if result.Err != nil {
            fmt.Printf("Error: %v\n", result.Err)
            continue
        }
        fmt.Println(result.Data) // Output: 2, 4, 6, 8, 10 (unordered)
    }
}

💡 Examples

Priority Queue

queue := gocq.NewPriorityQueue(1, func(data int) (int, error) {
    time.Sleep(500 * time.Millisecond)
    return data * 2, nil
})
defer queue.WaitAndClose()

// import "github.com/fahimfaisaal/gocq/v2/shared/types"
items := []types.PQItem[int]{
    {Value: 1, Priority: 2}, // Lowest priority
    {Value: 2, Priority: 1}, // Medium priority
    {Value: 3}, // Highest priority
}

for result := range queue.AddAll(items).Results() {
    fmt.Println(result.Data) // Output: 6, 4, 2 (processed by priority)
}

Void Queue

queue := gocq.NewVoidQueue(2, func(data int) error {
    fmt.Printf("Processing: %d\n", data)
    time.Sleep(500 * time.Millisecond)
    fmt.Printf("Processed: %d\n", data)
    return nil
})
defer queue.WaitAndClose()

queue.Add(1).Drain()
queue.AddAll([]int{2, 3, 4}).Drain()

Documentation

For detailed API documentation, please refer to the API Reference.

🚀 Performance

Benchmark Results

# Normal Queue

goos: linux
goarch: amd64
pkg: github.com/fahimfaisaal/gocq/v2/internal/concurrent_queue
cpu: 13th Gen Intel(R) Core(TM) i7-13700
BenchmarkQueue_Operations/Add-24                         1113538              1467 ns/op             376 B/op          8 allocs/op
BenchmarkQueue_Operations/AddAll-24                        15778            131428 ns/op           17353 B/op        510 allocs/op
BenchmarkPriorityQueue_Operations/Add-24                 1382113             878.6 ns/op             352 B/op          8 allocs/op
BenchmarkPriorityQueue_Operations/AddAll-24                10000            121044 ns/op           14951 B/op        510 allocs/op

# Void Queue

goos: linux
goarch: amd64
pkg: github.com/fahimfaisaal/gocq/v2/internal/concurrent_queue/void_queue
cpu: 13th Gen Intel(R) Core(TM) i7-13700
BenchmarkVoidQueue_Operations/Add-24                     1987044              1001 ns/op             240 B/op          7 allocs/op
BenchmarkVoidQueue_Operations/AddAll-24                    10000            134711 ns/op           16989 B/op        512 allocs/op
BenchmarkVoidPriorityQueue_Operations/Add-24             1326402              1323 ns/op             240 B/op          7 allocs/op
BenchmarkVoidPriorityQueue_Operations/AddAll-24            11812            108261 ns/op           16965 B/op        512 allocs/op

Queue Type	Operation	Variant	ns/op	B/op	allocs/op
Non-Priority Queue	Add	Normal	1467	376	8
Non-Priority Queue	Add	Void	1001	240	7

Note: Void queue is ~25% faster than the standard queue according to benchmarks. even the memory usage is also lower.

Run Benchmarks

go test -bench=. -benchmem ./internal/concurrent_queue/
go test -bench=. -benchmem ./internal/concurrent_queue/void_queue/

👤 Author

• GitHub: @fahimfaisaal
• LinkedIn: in/fahimfaisaal
• Twitter: @FahimFaisaal

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.