巧用Golang泛型,简化代码编写
GEEK TALK
01
前言
泛型功能是Go语言在1.18版本引入的功能,可以说是Go语言开源以来最大的语法特性变化,其改动和影响都很大, 所以整个版本的开发周期,测试周期都比以往要长很多。接下来为了大家更好的理解文章中的代码示例,先再简单介绍一下 Go语言在1.18版本加入的泛型的基本使用方法。
从官方的资料来看,泛型增加了三个新的重要内容:
函数和类型新增对类型形参(type parameters)的支持。
将接口类型定义为类型集合,包括没有方法的接口类型。
支持类型推导,大多数情况下,调用泛型函数时可省略类型实参(type arguments)。
1.1 Type Parameter
参数泛型类型(Type Parameter)可以说是泛型使用过程应用最多的场景了, 一般应用于方法或函数的形参或返回参数上。
参数泛型类型基本的使用格式可参见如下:func FuncName[P, Q constraint1, R constraint2, ...](parameter1 P, parameter2 Q, ...) (R, Q, ...)
说明: 参数泛型类定义后,可以用于函数的形参或返回参数上。
下面是一个应用参数泛型类型的代码示例:
// Min return the min one
func Min[E int | int8 | int16 | int32 | int64 | uint | uint8 | uint16 | uint32 | uint64 | float32 | float64 | uintptr | ~string](x, y E) E {
if x < y {
return x
}
return y
}
1.2 类型集合 Type Set
类型集合是为了简化泛型约束的使用,提升阅读性,同时增加了复用能力,方式他通过接口定义的方式使用。
编写格式参见如下:
type Constraint1 interface {
Type1 | ~Type2 | ...
}
以下示例定义了有符号整型与无符号整型的泛型约束:
// Signed is a constraint that permits any signed integer type.
// If future releases of Go add new predeclared signed integer types,
// this constraint will be modified to include them.
type Signed interface {
~int | ~int8 | ~int16 | ~int32 | ~int64
}
// Unsigned is a constraint that permits any unsigned integer type.
// If future releases of Go add new predeclared unsigned integer types,
// this constraint will be modified to include them.
type Unsigned interface {
~uint | ~uint8 | ~uint16 | ~uint32 | ~uint64 | ~uintptr
}
类型集合也支持继承的方式,简化复用。使用方式也接口的继承是完全一致。
以下示例定义把Signed和Unsigned进行了组合,用于表达对整型泛型约束的定义。
// Integer is a constraint that permits any integer type.
// If future releases of Go add new predeclared integer types,
// this constraint will be modified to include them.
type Integer interface {
Signed | Unsigned
}
1.3 类型推导
引入类型推导,可以简化我们代码工作,目前支持2种类型推导功能。
通过函数的实参推导出来具体的类型以前提到的Min函数为例,可能通过传入的传数类型来判断推导。
var a, b uint
minUint := Min(a, b) // 不再需要这样写 Min[uint](a, b)
fmt.Println(minUint)
minInt := Min(10, 100) // 常量数字,go语言会默认为 int类型,不再需要这样写 Min[int](a, b)
fmt.Println(minInt)
GEEK TALK
02
巧用泛型,实现通用排序函数
// An implementation of Interface can be sorted by the routines in this package.
// The methods refer to elements of the underlying collection by integer index.
type Interface interface {
// Len is the number of elements in the collection.
Len() int
// Less reports whether the element with index i
// must sort before the element with index j.
//
// If both Less(i, j) and Less(j, i) are false,
// then the elements at index i and j are considered equal.
// Sort may place equal elements in any order in the final result,
// while Stable preserves the original input order of equal elements.
//
// Less must describe a transitive ordering:
// - if both Less(i, j) and Less(j, k) are true, then Less(i, k) must be true as well.
// - if both Less(i, j) and Less(j, k) are false, then Less(i, k) must be false as well.
//
// Note that floating-point comparison (the < operator on float32 or float64 values)
// is not a transitive ordering when not-a-number (NaN) values are involved.
// See Float64Slice.Less for a correct implementation for floating-point values.
Less(i, j int) bool
// Swap swaps the elements with indexes i and j.
Swap(i, j int)
}
这样导致我们对不同元素类型的数组,都需要重复实现这个接口,编写了很多类似的代码。下面是官方给的一个排序的示例,可以看到实现这样的排序功能,要写这么多代码。
type Person struct {
Name string
Age int
}
// ByAge implements sort.Interface for []Person based on
// the Age field.
type ByAge []Person
func (a ByAge) Len() int { return len(a) }
func (a ByAge) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
func (a ByAge) Less(i, j int) bool { return a[i].Age < a[j].Age }
func main() {
people := []Person{
{"Bob", 31},
{"John", 42},
{"Michael", 17},
{"Jenny", 26},
}
fmt.Println(people)
// There are two ways to sort a slice. First, one can define
// a set of methods for the slice type, as with ByAge, and
// call sort.Sort. In this first example we use that technique.
sort.Sort(ByAge(people))
fmt.Println(people)
// Output:
// [Bob: 31 John: 42 Michael: 17 Jenny: 26]
// [Michael: 17 Jenny: 26 Bob: 31 John: 42]
}
下面我们应用泛型,编写一个通用的排序功能,可以支持任何类型的数组。大致思路是封装一个接受任何类型(any)的结构体sortable,来实现sort.Interface接口,把需要扩展的部分(比较处理)进行可设置化。主要的代码如下:
// 通用化的排序实现
type sortable[E any] struct {
data []E
cmp base.CMP[E]
}
func (s sortable[E]) Len() int { return len(s.data) }
func (s sortable[E]) Swap(i, j int) { s.data[i], s.data[j] = s.data[j], s.data[i] }
func (s sortable[E]) Less(i, j int) bool {
return s.cmp(s.data[i], s.data[j]) >= 0
}
接入下来,实现一个支持泛型的排序函数,对任何类型的数组进行排序。
func Sort[E any](data []E, cmp base.CMP[E]) {
sortobject := sortable[E]{data: data, cmp: cmp}
sort.Sort(sortobject)
}
至此,我们就已经实现一个通用的排序函数了, 应用这个函数,上面官方给出的排序实现就可以简化如下:
type Person struct {
Name string
Age int
}
people := []Person{
{"Bob", 31},
{"John", 42},
{"Michael", 17},
{"Jenny", 26},
}
people = Sort(people, func(e1, e2 Person) int {
return e1.Age - e2.Age
})
// Output:
// [Michael: 17 Jenny: 26 Bob: 31 John: 42]
可以看到, 应用泛型后,只需要简单的一个函数调用就可以了。
完整的代码实现可参见:https://github.com/jhunters/goassist/blob/main/arrayutil/array.go
GEEK TALK
03
巧用泛型,简化strconv.Append系列函数
Go语言内置的strconv包的api也是日常开发经常使用的, 它提供的Append系列函数可以实现高效的字符串拼接功能,但因为Go语言不支持重载,所以会看到因为接受参数类型的不同,需要选择不同的函数。
func AppendBool(dst []byte, b bool) []byte
func AppendFloat(dst []byte, f float64, fmt byte, prec, bitSize int) []byte
func AppendInt(dst []byte, i int64, base int) []byte
func AppendQuote(dst []byte, s string) []byte
func AppendQuoteRune(dst []byte, r rune) []byte
func AppendQuoteRuneToASCII(dst []byte, r rune) []byte
func AppendQuoteRuneToGraphic(dst []byte, r rune) []byte
func AppendQuoteToASCII(dst []byte, s string) []byte
func AppendQuoteToGraphic(dst []byte, s string) []byte
func AppendUint(dst []byte, i uint64, base int) []byte
所以我们不得不面临以下使用的窘境。
// append bool
b := []byte("bool:")
b = strconv.AppendBool(b, true)
fmt.Println(string(b))
// append int
b10 := []byte("int (base 10):")
b10 = strconv.AppendInt(b10, -42, 10)
fmt.Println(string(b10))
// append quote
b := []byte("quote:")
b = strconv.AppendQuote(b, `"Fran & Freddie's Diner"`)
fmt.Println(string(b))
接下来,我们用泛型来简化一下代码,让其只需要一个函数就能搞定, 直接上代码如下:
// Append convert e to string and appends to dst
func Append[E any](dst []byte, e E) []byte {
toAppend := fmt.Sprintf("%v", e)
return append(dst, []byte(toAppend)...)
}
再来看看应用后的效果,修改之前的示例:
// append bool
b := []byte("bool:")
b = conv.Append(b, true)
fmt.Println(string(b))
// append int
b10 := []byte("int:")
b10 = conv.Append(b10, -42)
fmt.Println(string(b10))
// append quote
b = []byte("quote:")
b = conv.Append(b, `"Fran & Freddie's Diner"`)
fmt.Println(string(b))
GEEK TALK
04
巧用泛型,实现通用heap容器,简化使用
Go语言container/heap包提供了一个优先级队列功能, 以实现在Pop数里时,总是优先获得优先级最高的节点。
同样的问题,如果要应用heap包的功能,针对不同的对象,必须要 实现 heap.Interface接口, 包括5个方法。
// The Interface type describes the requirements
// for a type using the routines in this package.
// Any type that implements it may be used as a
// min-heap with the following invariants (established after
// Init has been called or if the data is empty or sorted):
//
// !h.Less(j, i) for 0 <= i < h.Len() and 2*i+1 <= j <= 2*i+2 and j < h.Len()
//
// Note that Push and Pop in this interface are for package heap's
// implementation to call. To add and remove things from the heap,
// use heap.Push and heap.Pop.
type Interface interface {
sort.Interface
Push(x any) // add x as element Len()
Pop() any // remove and return element Len() - 1.
}
下面的代码示例是来自Go语言官方,实现了对Int类型元素的优先级队列实现:
import (
"container/heap"
"fmt"
)
// An IntHeap is a min-heap of ints.
type IntHeap []int
func (h IntHeap) Len() int { return len(h) }
func (h IntHeap) Less(i, j int) bool { return h[i] < h[j] }
func (h IntHeap) Swap(i, j int) { h[i], h[j] = h[j], h[i] }
func (h *IntHeap) Push(x any) {
// Push and Pop use pointer receivers because they modify the slice's length,
// not just its contents.
*h = append(*h, x.(int))
}
func (h *IntHeap) Pop() any {
old := *h
n := len(old)
x := old[n-1]
*h = old[0 : n-1]
return x
}
// This example inserts several ints into an IntHeap, checks the minimum,
// and removes them in order of priority.
func Example_intHeap() {
h := &IntHeap{2, 1, 5}
heap.Init(h)
heap.Push(h, 3)
fmt.Printf("minimum: %d\n", (*h)[0])
for h.Len() > 0 {
fmt.Printf("%d ", heap.Pop(h))
}
// Output:
// minimum: 1
// 1 2 3 5
}
看到上面写了这么多的代码才把功能实现, 想必大家都觉得太繁琐了吧? 那我们用泛型来改造一下,大致思路如下:
实现一个支持泛型参数的结构体heapST,实现heap.Interface接口。
开放比较函数的功能,用于使用方来更灵活的设置排序要求。
封装一个全新的带泛型参数传入Heap结构体, 来封装Pop与Push方法的实现。
主要的代码实现如下:
type heapST[E any] struct {
data []E
cmp base.CMP[E]
}
// implments the methods for "heap.Interface"
func (h *heapST[E]) Len() int { return len(h.data) }
func (h *heapST[E]) Less(i, j int) bool {
v := h.cmp(h.data[i], h.data[j])
return v < 0
}
func (h *heapST[E]) Swap(i, j int) { h.data[i], h.data[j] = h.data[j], h.data[i] }
func (h *heapST[E]) Push(x any) {
// Push and Pop use pointer receivers because they modify the slice's length,
// not just its contents.
v := append(h.data, x.(E))
h.data = v
}
func (h *heapST[E]) Pop() any {
old := h.data
n := len(old)
x := old[n-1]
h.data = old[0 : n-1]
return x
}
// Heap base on generics to build a heap tree for any type
type Heap[E any] struct {
data *heapST[E]
}
// Push pushes the element x onto the heap.
// The complexity is O(log n) where n = h.Len().
func (h *Heap[E]) Push(v E) {
heap.Push(h.data, v)
}
// Pop removes and returns the minimum element (according to Less) from the heap.
// The complexity is O(log n) where n = h.Len().
// Pop is equivalent to Remove(h, 0).
func (h *Heap[E]) Pop() E {
return heap.Pop(h.data).(E)
}
func (h *Heap[E]) Element(index int) (e E, err error) {
if index < 0 || index >= h.data.Len() {
return e, fmt.Errorf("out of index")
}
return h.data.data[index], nil
}
// Remove removes and returns the element at index i from the heap.
// The complexity is O(log n) where n = h.Len().
func (h *Heap[E]) Remove(index int) E {
return heap.Remove(h.data, index).(E)
}
func (h *Heap[E]) Len() int {
return len(h.data.data)
}
// Copy to copy heap
func (h *Heap[E]) Copy() *Heap[E] {
ret := heapST[E]{cmp: h.data.cmp}
ret.data = make([]E, len(h.data.data))
copy(ret.data, h.data.data)
heap.Init(&ret)
return &Heap[E]{&ret}
}
// NewHeap return Heap pointer and init the heap tree
func NewHeap[E any](t []E, cmp base.CMP[E]) *Heap[E] {
ret := heapST[E]{data: t, cmp: cmp}
heap.Init(&ret)
return &Heap[E]{&ret}
}
完整的代码获取:https://github.com/jhunters/goassist/blob/main/container/heapx/heap.go
接入来可以改写之前的代码, 代码如下:
// An IntHeap is a min-heap of ints.
type IntHeap []int
// This example inserts several ints into an IntHeap, checks the minimum,
// and removes them in order of priority.
func Example_intHeap() {
h := heapx.NewHeap(IntHeap{2, 1, 5}, func(p1, p2 int) int {
return p1 - p2
})
h.Push(3)
for h.Len() > 0 {
fmt.Printf("%d ", h.Pop())
}
// Output:
// 1 2 3 5
}
可以看到改写后,代码量大量减少,而且代码的可读性也大大提升. 完整的使用示例可参见:https://github.com/jhunters/goassist/blob/main/container/heapx/heap_test.go
GEEK TALK
05
巧用泛型,提升Pool容器可读性与安全性
Go语言内存的sync包下Pool对象, 提供了可伸缩、并发安全的临时对象池的功能,用来存放已经分配但暂时不用的临时对象,通过对象重用机制,缓解 GC 压力,提高程序性能。需要注意的是Pool 是一个临时对象池,适用于储存一些会在 goroutine 间共享的临时对象,其中保存的任何项都可能随时不做通知地释放掉,所以不适合当于缓存或对象池的功能。
Pool的框架代码如下:
type Pool struct {
// New optionally specifies a function to generate
// a value when Get would otherwise return nil.
// It may not be changed concurrently with calls to Get.
New func() interface{}
// contains filtered or unexported fields
}
// Get 从 Pool 中获取元素。当 Pool 中没有元素时,会调用 New 生成元素,新元素不会放入 Pool 中。若 New 未定义,则返回 nil。
func (p *Pool) Get() interface{}
// Put 往 Pool 中添加元素 x。
func (p *Pool) Put(x interface{})
官方Pool的API使用起来已经是非常方便,下面是摘取官方文档中的示例代码:
package sync_test
import (
"bytes"
"io"
"os"
"sync"
"time"
)
var bufPool = sync.Pool{
New: func() any {
// The Pool's New function should generally only return pointer
// types, since a pointer can be put into the return interface
// value without an allocation:
return new(bytes.Buffer)
},
}
// timeNow is a fake version of time.Now for tests.
func timeNow() time.Time {
return time.Unix(1136214245, 0)
}
func Log(w io.Writer, key, val string) {
b := bufPool.Get().(*bytes.Buffer)
b.Reset()
// Replace this with time.Now() in a real logger.
b.WriteString(timeNow().UTC().Format(time.RFC3339))
b.WriteByte(' ')
b.WriteString(key)
b.WriteByte('=')
b.WriteString(val)
w.Write(b.Bytes())
bufPool.Put(b)
}
func ExamplePool() {
Log(os.Stdout, "path", "/search?q=flowers")
// Output: 2006-01-02T15:04:05Z path=/search?q=flowers
}
从上面的代码,可以看到一个问题就是从池中获取对象时,要强制进行转换,如果转换类型不匹配,就会出现Panic异常,这种场景正是泛型可以很好解决的场景,我们改造代码如下, 封装一个全新的带泛型参数传入 Pool 结构体:
package syncx
import (
"sync"
"github.com/jhunters/goassist/base"
)
type Pool[E any] struct {
New base.Supplier[E]
internal sync.Pool
}
// NewPoolX create a new PoolX
func NewPool[E any](f base.Supplier[E]) *Pool[E] {
p := Pool[E]{New: f}
p.internal = sync.Pool{
New: func() any {
return p.New()
},
}
return &p
}
// Get selects an E generic type item from the Pool
func (p *Pool[E]) Get() E {
v := p.internal.Get()
return v.(E)
}
// Put adds x to the pool.
func (p *Pool[E]) Put(v E) {
p.internal.Put(v)
}
接下来,使用新封装的Pool对象改写上面的官方示例代码:
var bufPool = syncx.NewPool(func() *bytes.Buffer {
return new(bytes.Buffer)
})
// timeNow is a fake version of time.Now for tests.
func timeNow() time.Time {
return time.Unix(1136214245, 0)
}
func Log(w io.Writer, key, val string) {
b := bufPool.Get() // 不再需要强制类型转换
b.Reset()
// Replace this with time.Now() in a real logger.
b.WriteString(timeNow().UTC().Format(time.RFC3339))
b.WriteByte(' ')
b.WriteString(key)
b.WriteByte('=')
b.WriteString(val)
w.Write(b.Bytes())
bufPool.Put(b)
}
func ExamplePool() {
Log(os.Stdout, "path", "/search?q=flowers")
// Output: 2006-01-02T15:04:05Z path=/search?q=flowers
}
完整的代码实现与使用示例可参见:https://github.com/jhunters/goassist/tree/main/concurrent/syncx
GEEK TALK
06
巧用泛型,增强sync.Map容器功能
sync.Map是Go语言官方提供的一个map映射的封装实现,提供了一些更实用的方法以更方便的操作map映射,同时它本身也是线程安全的,包括原子化的更新支持。
type Map
func (m *Map) Delete(key any)
func (m *Map) Load(key any) (value any, ok bool)
func (m *Map) LoadAndDelete(key any) (value any, loaded bool)
func (m *Map) LoadOrStore(key, value any) (actual any, loaded bool)
func (m *Map) Range(f func(key, value any) bool)
func (m *Map) Store(key, value any)
接入来我们要用泛型功能,给sync.Map增加如下功能:
所有的操作支持泛型,以省去对象强制转换功能
引入泛型后,保障了key与value类型的一致性,可以扩展支持 Key或Value是否存在, 查询最小最大Key或Value的功能
另外还增加了StoreAll 从另一个map导入, ToMap转成原生map结构, Clear清空map, 以数组结构导出key或value等实用功能
增加后Map的API列表如下:
type Map
func NewMap[K comparable, V any]() *Map[K, V]
func (m *Map[K, V]) Clear()
func (m *Map[K, V]) Copy() *Map[K, V]
func (m *Map[K, V]) Exist(key K) bool
func (m *Map[K, V]) ExistValue(value V) (k K, exist bool)
func (m *Map[K, V]) ExistValueWithComparator(value V, equal base.EQL[V]) (k K, exist bool)
func (m *Map[K, V]) Get(key K) (V, bool)
func (m *Map[K, V]) IsEmpty() (empty bool)
func (m *Map[K, V]) Keys() []K
func (m *Map[K, V]) MaxKey(compare base.CMP[K]) (key K, v V)
func (m *Map[K, V]) MaxValue(compare base.CMP[V]) (key K, v V)
func (m *Map[K, V]) MinKey(compare base.CMP[K]) (key K, v V)
func (m *Map[K, V]) MinValue(compare base.CMP[V]) (key K, v V)
func (m *Map[K, V]) Put(key K, value V) V
func (m *Map[K, V]) Range(f base.BiFunc[bool, K, V])
func (m *Map[K, V]) Remove(key K) bool
func (m *Map[K, V]) Size() int
func (m *Map[K, V]) ToMap() map[K]V
func (m *Map[K, V]) Values() []V
部分泛型代码后的代码如下:
// Map is like a Go map[interface{}]interface{} but is safe for concurrent use
// by multiple goroutines without additional locking or coordination.
// Loads, stores, and deletes run in amortized constant time.
// By generics feature supports, all api will be more readable and safty.
//
// The Map type is specialized. Most code should use a plain Go map instead,
// with separate locking or coordination, for better type safety and to make it
// easier to maintain other invariants along with the map content.
//
// The Map type is optimized for two common use cases: (1) when the entry for a given
// key is only ever written once but read many times, as in caches that only grow,
// or (2) when multiple goroutines read, write, and overwrite entries for disjoint
// sets of keys. In these two cases, use of a Map may significantly reduce lock
// contention compared to a Go map paired with a separate Mutex or RWMutex.
//
// The zero Map is empty and ready for use. A Map must not be copied after first use.
type Map[K comparable, V any] struct {
mp sync.Map
empty V
mu sync.Mutex
}
// NewMap create a new map
func NewMap[K comparable, V any]() *Map[K, V] {
return &Map[K, V]{mp: sync.Map{}}
}
// NewMapByInitial create a new map and store key and value from origin map
func NewMapByInitial[K comparable, V any](mmp map[K]V) *Map[K, V] {
mp := NewMap[K, V]()
if mmp == nil {
return mp
}
for k, v := range mmp {
mp.Store(k, v)
}
return mp
}
// Exist return true if key exist
func (m *Map[K, V]) Exist(key K) bool {
_, ok := m.mp.Load(key)
return ok
}
// ExistValue return true if value exist
func (m *Map[K, V]) ExistValue(value V) (k K, exist bool) {
de := reflectutil.NewDeepEquals(value)
m.Range(func(key K, val V) bool {
if de.Matches(val) {
exist = true
k = key
return false
}
return true
})
return
}
// ExistValue return true if value exist
func (m *Map[K, V]) ExistValueWithComparator(value V, equal base.EQL[V]) (k K, exist bool) {
m.Range(func(key K, val V) bool {
if equal(value, val) {
exist = true
k = key
return false
}
return true
})
return
}
// ExistValue return true if value exist
func (m *Map[K, V]) ExistValueComparable(v base.Comparable[V]) (k K, exist bool) {
m.Range(func(key K, val V) bool {
if v.CompareTo(val) == 0 {
exist = true
k = key
return false
}
return true
})
return
}
// MinValue to return min value in the map
func (m *Map[K, V]) MinValue(compare base.CMP[V]) (key K, v V) {
return selectByCompareValue(m, func(o1, o2 V) int {
return compare(o1, o2)
})
}
// MaxValue to return max value in the map
func (m *Map[K, V]) MaxValue(compare base.CMP[V]) (key K, v V) {
return selectByCompareValue(m, func(o1, o2 V) int {
return compare(o2, o1)
})
}
// MinKey to return min key in the map
func (m *Map[K, V]) MinKey(compare base.CMP[K]) (key K, v V) {
return selectByCompareKey(m, func(o1, o2 K) int {
return compare(o1, o2)
})
}
// MaxKey to return max key in the map
func (m *Map[K, V]) MaxKey(compare base.CMP[K]) (key K, v V) {
return selectByCompareKey(m, func(o1, o2 K) int {
return compare(o2, o1)
})
}
GEEK TALK
07
总结
END
[1] Go语言官方泛型使用介绍:https://golang.google.cn/doc/tutorial/generics
[2]《using generics in go by Ian》Go语言泛型专题分享:
https://www.bilibili.com/video/BV1KP4y157rn/?p=1&share_medium=iphone&share_plat=ios&share_session_id=B63420EF-45D0-4464-A195-F27C13188C75&share_source=WEIXIN&share_tag=s_i×tamp=1636179512&unique_k=ORZiX1
[3]专为Go语言开发者提供一套基础api库,包括了非常多的泛型应用:https://github.com/jhunters/goassist