grpc

• grpc
• grpc-go

grpc seamlessly integrates protobuf

protobuf

• For those of you accustomed to using Json and XML data storage formats, I believe most have never heard of Protocol Buffer.
• Protocol Buffer is actually a lightweight & efficient structured data storage format developed by Google, and its performance is truly much stronger than Json and XML!
• protobuf has gone through protobuf2 and protobuf3, with pb3 being much simpler than pb2, and pb3 is currently the mainstream version.

Downloading protoc

Download protoc andgo get github.com/golang/protobuf/protoc-gen-go

syntax = "proto3";

option go_package = "."; //这名不写现在编译不了

message HelloRequest {
  string name = 1; //1 是编号不是值
}

protoc --proto_path=. \
  --go_out=. \
  --go-grpc_out=. \
  ./helloworld.proto

A helloworld.pb.go file will be generated in the directory

Comparison of compression ratio

package main

import (
 "RpcLearn/protobuf"
 "encoding/json"
 "fmt"
 "github.com/golang/protobuf/proto"
)

// TIP <p>To run your code, right-click the code and select <b>Run</b>.</p> <p>Alternatively, click
// the <icon src="AllIcons.Actions.Execute"/> icon in the gutter and select the <b>Run</b> menu item from here.</p>
type Hello struct {
 Name string `json:"name"`
}

func main() {
 //TIP <p>Press <shortcut actionId="ShowIntentionActions"/> when your caret is at the underlined text
 // to see how GoLand suggests fixing the warning.</p><p>Alternatively, if available, click the lightbulb to view possible fixes.</p>
 //s := "gopher"
 //fmt.Printf("Hello and welcome, %s!\n", s)
 //
 //for i := 1; i <= 5; i++ {
 // //TIP <p>To start your debugging session, right-click your code in the editor and select the Debug option.</p> <p>We have set one <icon src="AllIcons.Debugger.Db_set_breakpoint"/> breakpoint
 // // for you, but you can always add more by pressing <shortcut actionId="ToggleLineBreakpoint"/>.</p>
 // fmt.Println("i =", 100/i)
 //}
 jsonStruct := Hello{
  Name: "gopher",
 }
 jsonReq, _ := json.Marshal(jsonStruct)
 fmt.Println(string(jsonReq))
 fmt.Println(len(jsonReq))
 req := __.HelloRequest{
  Name: "gopher",
 }
 rsp, _ := proto.Marshal(&req)
 fmt.Println(rsp)
 fmt.Println(len(rsp))

}

Roughly a twofold difference

Explanation of Changes in Protobuf Usage in Go Language

📌 Current Behavior Explanation

Starting from Go's Protobuf plugin v1.20+, gRPC support has been split into a separate plugin, protoc-gen-go-grpc, used to generate service interface related code. This approach is more modular and has clearer responsibilities.

Therefore, execute the following command:

protoc --proto_path=. \
  --go_out=. \
  --go-grpc_out=. \
  ./your_file.proto

This will generate:

• your_file.pb.go: containing message structures (e.g., Request, Response)
• your_file_grpc.pb.go: containing service interface definitions (e.g., YourServiceServer)

It is recommended to keep and synchronize these two files.

A brief overview of the gRPC setup process

1. Create stub files and method definitions, then compile

// proto/helloworld.proto
syntax = "proto3";
option go_package = ".;proto";

service Greeter{
    rpc SayHello (HelloRequest) returns (HelloReply);
}
message HelloRequest {
    string name = 1;
}
message HelloReply {
    string message = 1;
}

1. Server-side implementation of method calls, note server implementation (duck typing)

package main

import (
 "RpcLearn/grpc_test/proto"
 "context"
 "google.golang.org/grpc"
 "log"
 "net"
)

type Server struct {
 proto.UnimplementedGreeterServer // 👈 重点：嵌入这个类型
}

func (s *Server) SayHello(ctx context.Context, req *proto.HelloRequest) (*proto.HelloReply, error) {
 reply := &proto.HelloReply{
  Message: "Hello " + req.Name,
 }
 return reply, nil
}

func main() {
 g := grpc.NewServer()                     // 创建一个新的gRPC服务器
 proto.RegisterGreeterServer(g, &Server{}) // 注册服务
 listener, err := net.Listen("tcp", ":9090")
 if err != nil {
  log.Fatalf("failed to listen: %v", err)
 }
 err = g.Serve(listener)
 if err != nil {
  log.Fatalf("failed to serve: %v", err)
 }
}

1. Client-side call implementation

package main

import (
 "RpcLearn/grpc_test/proto"
 "context"
 "fmt"
 "google.golang.org/grpc"
 "google.golang.org/grpc/credentials/insecure"
)

func main() {
 // Create a new gRPC server
 //conn, err := grpc.Dial("localhost:9090", grpc.WithInsecure())
 // 使用安全连接
 //creds := credentials.NewTLS(&tls.Config{InsecureSkipVerify: true})
 conn, err := grpc.NewClient("localhost:9090",
  //grpc.WithInsecure()
  grpc.WithTransportCredentials(insecure.NewCredentials()), // ✅ 新方式
 )
 if err != nil {
  panic(err)
 }
 defer conn.Close()
 c := proto.NewGreeterClient(conn)
 r, err := c.SayHello(context.Background(), &proto.HelloRequest{Name: "world"})
 if err != nil {
  panic(err)
 }
 fmt.Println(r.Message)

}

gRPC Stream Mode

1. Simple mode (simple RPC)
2. Server-side data stream
3. Client-side data stream
4. Bidirectional

Focused Learning on gRPC

Default values

Default Value Explanation

In Protobuf, different field types have the following default values:

• String type (string): The default value is an empty string.
• Byte type (bytes): The default value is an empty byte sequence.
• Boolean type (bool): The default value is false.
• Numeric types: The default value is 0.
• Enum type: The default value is the first defined enum value, which must be 0.
• Message type (message): The default value is unset (i.e., null).

Notes

1. For scalar message fields: Once a message is parsed, it's impossible to tell whether a field was set to its default value (e.g., whether a boolean is false) or if the field was not set at all.
2. Boolean usage recommendation: Avoid defining boolean fields with a default value of false, as this might lead to unintended behavior.
3. Serialization of scalar messages: If a scalar message field is set to its default value, that value will not be serialized for transmission.

You can refer to the Generated Code Guide for a more detailed understanding of how Protobuf default values are handled.

Enum Type Definition

Enum types can be used when you need to specify a set of "predefined value sequences" for a field in a message type. Each enum value corresponds to an integer value.

Example

Below is an example of an enum definition:

message SearchRequest {
  enum Corpus {
    UNIVERSAL = 0;
    WEB = 1;
    IMAGES = 2;
    LOCAL = 3;
    NEWS = 4;
    PRODUCTS = 5;
    VIDEO = 6;
  }
  Corpus corpus = 1;
}

• Server-side data stream (for real-time data acquisition)
• Client-side data stream (for data reporting)
• Bidirectional data stream

 syntax = "proto3";

option go_package = ".;proto";

service Greeter {
    rpc GetStream (StreamReqData) returns (stream StreamResData); // 服务端流模式
    rpc PutStream (stream StreamReqData) returns ( StreamResData); // 客户端流模式
    rpc AllStream (stream StreamReqData) returns (stream StreamResData); // 双向流模式
}

message StreamReqData {
    string data = 1;
}

message StreamResData {
    string message = 1;
}

Compile and generate Go protobuf

protoc --proto_path=. \
  --go_out=. \
  --go-grpc_out=. \
  ./helloworld.proto

Server-side code

package main

import (
 "RpcLearn/stream_grpc_test/proto"
 "fmt"
 "google.golang.org/grpc"
 "log"
 "net"
 "sync"
 "time"
)

const PORT = ":50052"

type server struct {
 proto.UnimplementedGreeterServer // 👈 重点：嵌入这个类型
}

func (s server) GetStream(data *proto.StreamReqData, res proto.Greeter_GetStreamServer) error {
 //TODO implement me
 //panic("implement me")
 i := 0
 for {
  i++
  _ = res.Send(&proto.StreamResData{
   Message: fmt.Sprintf("%v", time.Now().Unix()),
  })
  time.Sleep(time.Second)
  if i >= 10 {
   break
  }
 }
 return nil
}

func (s server) PutStream(cliStr proto.Greeter_PutStreamServer) error {
 //TODO implement me
 //panic("implement me")//
 for {
  if a, err := cliStr.Recv(); err != nil {
   fmt.Println(err)
   break
  } else {
   fmt.Println(a)
  }

 }
 return nil
}

func (s server) AllStream(allStr proto.Greeter_AllStreamServer) error {
 //TODO implement me
 //panic("implement me")
 wg := sync.WaitGroup{}
 wg.Add(2)
 go func() {
  defer wg.Done()
  for {
   if a, err := allStr.Recv(); err != nil {
    fmt.Println(err)
    break
   } else {
    fmt.Println("客户端发送的消息：", a)
   }
  }

 }()
 go func() {
  defer wg.Done()
  for {
   allStr.Send(&proto.StreamResData{
    Message: fmt.Sprintf("我是服务端数据:%v\n", time.Now().Unix()),
   })
   time.Sleep(time.Second)
  }

 }()
 wg.Wait()
 return nil
}

func (s *server) SomeRPC(ctx context.Context, in *pb.SomeRequest) (*pb.SomeResponse, error) {
    md, ok := metadata.FromIncomingContext(ctx)
    if (!ok) {
        // handle missing metadata
    }
    // do something with metadata
    return &pb.SomeResponse{}, nil
}

func main() {
 // 创建一个新的gRPC服务器
 g := grpc.NewServer()
 proto.RegisterGreeterServer(g, &server{}) // 注册服务
 listener, err := net.Listen("tcp", PORT)
 if err != nil {
  log.Fatalf("failed to listen: %v", err)
 }
 err = g.Serve(listener)
 if err != nil {
  log.Fatalf("failed to serve: %v", err)
 }
}

Client-side code

package main

import (
 "RpcLearn/stream_grpc_test/proto"
 "context"
 "fmt"
 "google.golang.org/grpc"
 "google.golang.org/grpc/credentials/insecure"
 "sync"
 "time"
)

func main() {
 // Create a new gRPC server
 conn, err := grpc.("localhost:50052",
  //grpc.WithInsecure()
  grpc.WithTransportCredentials(insecure.NewCredentials()), // ✅ 新方式
 )
 if err != nil {
  panic(err)
 }
 defer conn.Close()
 c := proto.NewGreeterClient(conn)
 //res, _ := c.GetStream(context.Background(), &proto.StreamReqData{Data: "慕课网"})
 //for {
 // r, err := res.Recv()
 // if err != nil {
 //  break
 // }
 // fmt.Println(r)
 //}
 //// 客户端流模式
 //putS, _ := c.PutStream(context.Background())
 //i := 0
 //for {
 // i++
 // putS.Send(&proto.StreamReqData{Data: fmt.Sprintf("慕课网%v", i)})
 // time.Sleep(time.Second)
 // if i >= 10 {
 //  break
 // }
 //}

 //双向流模式
 allStr, _ := c.AllStream(context.Background())
 wg := sync.WaitGroup{}
 wg.Add(2)
 go func() {
  defer wg.Done()
  for {
   if a, err := allStr.Recv(); err != nil {
    fmt.Println(err)
    break
   } else {
    fmt.Println("我是客户端:%v", a)
   }
  }
 }()
 go func() {
  defer wg.Done()
  i := 0
  for {
   i++
   allStr.Send(&proto.StreamReqData{Data: fmt.Sprintf("双向的慕课网:%v", i)})
   time.Sleep(time.Second)
   if i >= 10 {
    break
   }
  }
 }()
 wg.Wait()
}

Protobuf Type to Go Type Mapping (with Encoding Recommendations)

option go_package

Specifying the compilation output directory

How to import other protos in a proto file

import "base.proto"

import "google/protobuf/empty.proto"

Nested messages

Enum types

enum Gender{
  MALE = 0;
  FEMALE = 1;
}

Map type map<string,string> mp = 4

How to use timestamps google/protobuf/timestamp.proto

message HelloRequest {
  string name = 1; // 姓名 相当于文档
  string url = 2;
  Gender g = 3;
  map<string, string> mp = 4;
  google.protobuf.Timestamp addTime = 5;
}

More gRPC Features

gRPC allows us to implement remote calls just like local calls. In every RPC call, there might be some useful data, and this data can be transmitted via metadata. Metadata stores data in key-value pairs, where the key is a string type and the value is `[]string`, i.e., a string array type. Metadata enables the client and server to provide information about the current call to each other, similar to the RequestHeader and ResponseHeader of an HTTP request. Just as the lifecycle of an HTTP header is one HTTP request, the lifecycle of metadata is one RPC call.

metadata

Not directly intrusive to business code

// 新建
type MD mp[string][]string
// 创建
// 第一种方式
md := metadata.New(map[string]string{"key1": "val1", "key2": "val2"})

// 第二种方式 key 不区分大小写，会被统一转成小写。
md := metadata.Pairs(
    "key1", "val1",
    "key1", "val1-2", // "key1" will have map value []string{"val1", "val1-2"}
    "key2", "val2",
)

Sending metadata

md := metadata.Pairs("key", "val")

// 新建一个有 metadata 的 context
ctx := metadata.NewOutgoingContext(context.Background(), md)

// 单向 RPC
response, err := client.SomeRPC(ctx, someRequest)

Receiving metadata

func (s *server) SomeRPC(ctx context.Context, in *pb.SomeRequest) (*pb.SomeResponse, error) {
    md, ok := metadata.FromIncomingContext(ctx)
    if !ok {
        // handle missing metadata
    }
    // do something with metadata
    return &pb.SomeResponse{}, nil
}

gRPC Interceptors

Server-side and client-side interceptors

package main

import (
 "RpcLearn/grpc_test/proto"
 "context"
 "google.golang.org/grpc"
 "log"
 "net"
)

type Server struct {
 proto.UnimplementedGreeterServer // 👈 重点：嵌入这个类型
}

func (s *Server) SayHello(ctx context.Context, req *proto.HelloRequest) (*proto.HelloReply, error) {
 reply := &proto.HelloReply{
  Message: "Hello " + req.Name,
 }
 return reply, nil
}

func main() {
 interceptor := func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (resp interface{}, err error) {
  log.Println("Request:接收到一个新的请求", req)
  resp, err = handler(ctx, req)
  if err != nil {
   log.Println("Error:", err)
  }
  log.Println("Response:处理这个新请求完成", resp)
  return resp, err
 }
 opt := grpc.UnaryInterceptor(interceptor)
 g := grpc.NewServer(opt)                  // 创建一个新的gRPC服务器
 proto.RegisterGreeterServer(g, &Server{}) // 注册服务
 listener, err := net.Listen("tcp", ":9090")
 if err != nil {
  log.Fatalf("failed to listen: %v", err)
 }
 err = g.Serve(listener)
 if err != nil {
  log.Fatalf("failed to serve: %v", err)
 }
}

package main

import (
 "RpcLearn/grpc_test/proto"
 "context"
 "fmt"
 "google.golang.org/grpc"
 "google.golang.org/grpc/credentials/insecure"
 "time"
)

func main() {
 // Create a new gRPC server
 //conn, err := grpc.Dial("localhost:9090", grpc.WithInsecure())
 // 使用安全连接
 //creds := credentials.NewTLS(&tls.Config{InsecureSkipVerify: true})
 // 添加拦截器
 clientInterceptor := func(ctx context.Context, method string, req, reply interface{}, cc *grpc.ClientConn, invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error {
  start := time.Now()
  err := invoker(ctx, method, req, reply, cc, opts...)
  fmt.Println("耗时", time.Since(start), err)
  return err
 }
 opt := grpc.WithUnaryInterceptor(clientInterceptor)
 conn, err := grpc.NewClient("localhost:9090",
  //grpc.WithInsecure()
  grpc.WithTransportCredentials(insecure.NewCredentials()), // ✅ 新方式
  opt,
 )
 if err != nil {
  panic(err)
 }
 defer conn.Close()
 c := proto.NewGreeterClient(conn)
 r, err := c.SayHello(context.Background(), &proto.HelloRequest{Name: "world"})
 if err != nil {
  panic(err)
 }
 fmt.Println(r.Message)

}

gRPC Login Verification via Metadata

// server
package main

import (
 "RpcLearn/grpc_test/proto"
 "context"
 "google.golang.org/grpc"
 "google.golang.org/grpc/codes"
 "google.golang.org/grpc/metadata"
 "google.golang.org/grpc/status"
 "log"
 "net"
)

type Server struct {
 proto.UnimplementedGreeterServer // 👈 重点：嵌入这个类型
}

func (s *Server) SayHello(ctx context.Context, req *proto.HelloRequest) (*proto.HelloReply, error) {
 reply := &proto.HelloReply{
  Message: "Hello " + req.Name,
 }
 return reply, nil
}

func main() {
 interceptor := func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (resp interface{}, err error) {
  log.Println("Request:接收到一个新的请求", req)
  md, ok := metadata.FromIncomingContext(ctx)
  if !ok {
   log.Println("没有metadata")
   return resp, status.Error(codes.Unauthenticated, "没有metadata")
  }

  var (
   appid  string
   appkey string
  )
  if val1, ok := md["appid"]; ok {
   appid = val1[0]
  }
  if val2, ok := md["appkey"]; ok {
   appkey = val2[0]
  }
  if appid != "1010101" || appkey != "123456" {
   return resp, status.Error(codes.Unauthenticated, "无token认证信息")
  }

  resp, err = handler(ctx, req)
  if err != nil {
   log.Println("Error:", err)
  }
  log.Println("Response:处理这个新请求完成", resp)
  return resp, err
 }
 opt := grpc.UnaryInterceptor(interceptor)
 g := grpc.NewServer(opt)                  // 创建一个新的gRPC服务器
 proto.RegisterGreeterServer(g, &Server{}) // 注册服务
 listener, err := net.Listen("tcp", ":9090")
 if err != nil {
  log.Fatalf("failed to listen: %v", err)
 }
 err = g.Serve(listener)
 if err != nil {
  log.Fatalf("failed to serve: %v", err)
 }
}

package main

import (
 "RpcLearn/grpc_test/proto"
 "context"
 "fmt"
 "google.golang.org/grpc"
 "google.golang.org/grpc/credentials/insecure"
)

type customCredentials struct {
}

func (c *customCredentials) GetRequestMetadata(ctx context.Context, uri ...string) (map[string]string, error) {
 return map[string]string{
  "appid":  "1010101",
  "appkey": "123456",
 }, nil
}
func (c *customCredentials) RequireTransportSecurity() bool {
 return false
}

func main() {

 // 添加拦截器
 //clientInterceptor := func(ctx context.Context, method string, req, reply interface{}, cc *grpc.ClientConn, invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error {
 // start := time.Now()
 // //生成metadata
 // md := metadata.New(map[string]string{
 //  "appid":  "1010101",
 //  "appkey": "123456",
 // })
 // ctx = metadata.NewOutgoingContext(ctx, md)
 // err := invoker(ctx, method, req, reply, cc, opts...)
 // fmt.Println("耗时", time.Since(start), err)
 // return err
 //}
 //opt := grpc.WithUnaryInterceptor(clientInterceptor
 // 另一种方式
 opt := grpc.WithPerRPCCredentials(&customCredentials{}) // 有一个专门的metadata的拦截器
 conn, err := grpc.NewClient("localhost:9090",
  //grpc.WithInsecure()
  grpc.WithTransportCredentials(insecure.NewCredentials()), // ✅ 新方式
  opt,
 )
 if err != nil {
  panic(err)
 }
 defer conn.Close()
 c := proto.NewGreeterClient(conn)
 r, err := c.SayHello(context.Background(), &proto.HelloRequest{Name: "world"})
 if err != nil {
  panic(err)
 }
 fmt.Println(r.Message)

}

grpc(protoc-gen-validate)

protoc-gen-validate (PGV for short) is a Protocol Buffers plugin used to add validation logic for struct fields in generated Go code.

It automatically generates validation code for each field by adding validate rules to .proto files, saving you from manually writing validation logic.

syntax = "proto3";

import "validate/validate.proto";

message User {
  string name = 1 [(validate.rules).string.min_len = 3];
  int32 age = 2 [(validate.rules).int32.gte = 18];
}

View the introduction to protoc-gen-validate

grpc_proto_validate

syntax = "proto3";

import "validate.proto";

option go_package = ".;proto";


service Greeter {
  rpc SayHello (Person) returns (Person);
}

message Person {
  uint64 id = 1 [(validate.rules).uint64.gt = 999];

  string email = 2 [(validate.rules).string.email = true];

  string mobile = 3 [
    (validate.rules).string = {
      pattern: "^1[3-9]\\d{9}$"
    }
  ];
}

Compile this proto by adding --validate_out="lang=go", first install go install github.com/bufbuild/protoc-gen-validate@latest

protoc \
  --proto_path=. \
  --go_out=. \
  --go-grpc_out=. \
  --validate_out=lang=go:. \
  ./helloworld.proto

package main

import (
 "RpcLearn/grpc_validate_test/proto"
 "context"
 "google.golang.org/grpc"
 "google.golang.org/grpc/codes"
 "google.golang.org/grpc/status"
 "log"
 "net"
)

type Server struct {
 proto.UnimplementedGreeterServer // 👈 重点：嵌入这个类型
}

func (s *Server) SayHello(ctx context.Context, request *proto.Person) (*proto.Person, error) {
 // Simulate some processing
 return &proto.Person{
  Id:     request.Id,
  Mobile: request.Mobile,
  Email:  request.Email,
 }, nil
}

type Validator interface {
 Validate() error
}

func main() {
 //p := new(proto.Person)
 //err := p.Validate()
 //if err != nil {
 // panic(err)
 //}
 var interceptor grpc.UnaryServerInterceptor
 interceptor = func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (resp interface{}, err error) {
  if v, ok := req.(Validator); ok {
   if err := v.Validate(); err != nil {
    return nil, status.Error(codes.InvalidArgument, err.Error())
   }
  }
  return handler(ctx, req)
 }
 opt := grpc.UnaryInterceptor(interceptor)
 g := grpc.NewServer(opt)                  // 创建一个新的gRPC服务器
 proto.RegisterGreeterServer(g, &Server{}) // 注册服务
 listener, err := net.Listen("tcp", ":50051")
 if err != nil {
  log.Fatalf("failed to listen: %v", err)
 }
 err = g.Serve(listener)
 if err != nil {
  log.Fatalf("failed to serve: %v", err)
 }
}

package main

import (
 "RpcLearn/grpc_validate_test/proto"
 "context"
 "fmt"
 "google.golang.org/grpc"
 "google.golang.org/grpc/credentials/insecure"
 "time"
)

type customCredential struct{}

func main() {
 // 拦截器（可选）
 clientInterceptor := func(ctx context.Context, method string, req, reply interface{}, cc *grpc.ClientConn, invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error {
  start := time.Now()
  err := invoker(ctx, method, req, reply, cc, opts...)
  fmt.Println("调用耗时:", time.Since(start), "错误:", err)
  return err
 }

 // 使用新方式连接（推荐）
 conn, err := grpc.NewClient("localhost:50051",
  grpc.WithTransportCredentials(insecure.NewCredentials()), // 替代 grpc.WithInsecure()
  grpc.WithUnaryInterceptor(clientInterceptor),
 )
 if err != nil {
  panic(err)
 }
 defer conn.Close()

 // 构建 gRPC 客户端
 c := proto.NewGreeterClient(conn)

 // 调用 RPC
 rsp, err := c.SayHello(context.Background(), &proto.Person{
  Id:     1000,
  Email:  "bobby@test.com",
  Mobile: "13222223333",
 })
 if err != nil {
  panic(err)
 }

 fmt.Println("返回 ID:", rsp.Id)
}

Exception Handling in gRPC

1. gRPC Status Codes

gRPC Status Codes Documentation

Go Exception Handling

1. Server-side

st := status.New(codes.InvalidArgument, "invalid username")

2. Client-side

st, ok := status.FromError(err)
if !ok {
    // Error was not a status error
}
st.Message()
st.Code()

Timeout Mechanism

ctx,_ = content.WithTimeout(context.Background(),time.Second*3)