Differences between GOPATH and Go Modules

1. Concepts

• GOPATH

• Is Go's early dependency management mechanism.

• All Go projects and dependency packages must be placed in the GOPATH directory (default is ~/go).
• GO111MODULE=off must be set.
• Project paths must be organized according to the src/package_name structure.
• Does not support version control; dependency management needs to be handled manually (e.g., go get).

• The order to find dependency packages is to look in gopath/src, and if not found, then look in the goroot/src directory

• Go Modules

• Is a modular dependency management mechanism introduced in Go 1.11, enabled by default after Go 1.13.
• Does not depend on GOPATH; projects can be placed in any directory.
• Each project has an independent go.mod file for managing dependencies and versions.
• GO111MODULE=on must be enabled.

// vendor

2. Dependency Management

• GOPATH

• Dependency packages are uniformly stored in the pkg directory of GOPATH.

• Dependency versions are not fixed; for example, running go get will pull the latest version, making it impossible to lock a specific version.

• Lacks a modular management mechanism, leading to potential dependency conflicts across multiple projects.

• Go Modules

• Uses go.mod and go.sum files to record dependency and version information.
• Supports version control, explicitly specifying dependency versions.
• Supports isolation between modules, avoiding dependency conflicts.

3. Project Organization

• GOPATH

• Projects must be located under the GOPATH/src directory.

• Fixed directory structure: GOPATH/src/github.com/username/project.

• GO11MODULE=off must be set.

• Go Modules

• Projects can be placed in any directory, independent of GOPATH.
• More flexible, developers can freely choose the directory structure.

4. Applicable Scenarios

• GOPATH

• Suitable for older Go projects or toolchains.

• Suitable for simple projects that do not require complex dependency management.

• Go Modules

• Is Go's recommended standard, suitable for modern projects.
• More suitable for projects requiring dependency version management.

5. Command Differences

• GOPATH

• go get: Used to fetch dependencies.

• go build: Finds dependencies in GOPATH and builds.

• Go Modules

• go mod init: Initializes a module, generating a go.mod file.
• go mod tidy: Cleans up and synchronizes dependencies.
• go mod vendor: Downloads dependencies to the vendor directory.

Summary

• For new projects, Go Modules should be used as much as possible, as it provides more powerful features and flexibility.
• If old projects need to be maintained, GOPATH might continue to be used.

Go Language Coding Standards

1. Why code standards are needed

2. Code standards are not mandatory, meaning that code written without following them will still run perfectly fine.

3. The purpose of code standards is to facilitate a unified code style within a team, improving code readability, standardization, and consistency. This standard will explain naming conventions, comment conventions, code style, and commonly used tools provided by the Go language.
4. Standards are not unique; theoretically, each company can formulate its own standards, but generally, the overall differences in standards will not be significant.

2. Code Standards

1. Naming Conventions

Naming is a very important part of code standards; unified naming rules help improve code readability. Good naming allows sufficient information to be obtained just from the name.

• a. When a name (including constants, variables, types, function names, struct fields, etc.) starts with an uppercase letter:

• E.g.: Group1, then objects using this form of identifier can be used by code in external packages (client programs need to import this package first).

• This is called exporting (similar to public in object-oriented languages).

• b. If a name starts with a lowercase letter:

• Then it is not visible outside the package, but it is visible and available within the entire package (similar to private in object-oriented languages).

1.1 Package Name: package

• Keep the package name consistent with the directory name, and try to use meaningful package names.
• Short, meaningful, and try not to conflict with the standard library.
• Package names should be lowercase words, without underscores or mixed case.

package model
package main

What is RPC

1. RPC (Remote Procedure Call), simply understood as one node requesting a service provided by another node.
2. The counterpart to RPC is local procedure call, and function call is the most common procedure call.
3. Turning a local procedure call into a remote procedure call faces various issues.
4. The original local function runs on another server. But many new problems are introduced.
5. Call ID mapping
6. Serialization and deserialization (important)
7. Network transmission (important)

New Problems Introduced by Remote Procedure Calls

1. Call ID Mapping

2. In local calls, the function body is directly pointed to via a function table.

3. In remote calls, all functions must have their own unique ID.
4. The client and server each maintain a mapping table of (function <--> Call ID).

5. When the client calls, it needs to find the corresponding Call ID, and the server finds the corresponding function via the Call ID.

6. Serialization and Deserialization

7. The client needs to serialize parameters into a byte stream and pass them to the server.

8. The server receives the byte stream and deserializes it into parameters.

9. Calls between different languages require a unified serialization protocol.

10. Network Transmission

11. All data needs to be transmitted over the network.
12. The network transport layer needs to pass the Call ID and serialized parameters to the server.
13. After processing, the server returns the result to the client.

Implementing a Simple RPC via HTTP

// server
package main

import (
 "encoding/json"
 "fmt"
 "net/http"
 "strconv"
)

func main() {
 // path 相当于callID
 // 返回格式：json {data:3}
 // http://127.0.0.1:8000/add?a=1&b=2
 // 网络传输协议
 http.HandleFunc("/add", func(w http.ResponseWriter, r *http.Request) {
  _ = r.ParseForm() // 解析参数
  fmt.Println("path:", r.URL.Path)
  aList, aOK := r.Form["a"]
  bList, bOK := r.Form["b"]
  if !aOK || !bOK || len(aList) == 0 || len(bList) == 0 {
   http.Error(w, `{"error":"missing parameter"}`, http.StatusBadRequest)
   return
  }
  a, _ := strconv.Atoi(aList[0])
  b, _ := strconv.Atoi(bList[0])
  w.Header().Set("Content-Type", "application/json; charset=utf-8")
  jData, _ := json.Marshal(map[string]int{
   "data": a + b,
  })
  _, _ = w.Write(jData)
 })
 _ = http.ListenAndServe(":8000", nil)
}

client

// client
package main

import (
 "encoding/json"
 "fmt"
 "github.com/kirinlabs/HttpRequest"
)

type ResponseData struct {
 Data int `json:"data"`
}

// rpc远程过程调用，如何做到像本地调用
func Add(a, b int) int {
 req := HttpRequest.NewRequest()

 res, _ := req.Get(fmt.Sprintf("http://127.0.0.1:8000/%s?a=%d&b=%d", "add", a, b))
 body, _ := res.Body()
 //fmt.Println(string(body))
 rspData := ResponseData{}
 _ = json.Unmarshal(body, &rspData)
 return rspData.Data

}
func main() {
 //conn, err := net.Dial("tcp", "127.0.0.1:8000")
 fmt.Println(Add(1, 2))
}

Four Key Elements of RPC Development

RPC technology consists of four parts in its architectural design: Client, Client Stub, Server, and Server Stub.

• Client: The initiator of a service call, also known as the service consumer.

• Client Stub: This program runs on the client's computer and is mainly used to store the address of the server to be called. In addition, this program is responsible for packaging the client's request data for the remote server program into data packets and sending them to the Server Stub program via the network; secondly, it also receives the call result data packets sent by the Server Stub program and parses them back to the client.

• Server: A program running on a remote computer, containing the methods that the client wants to call.

• Server Stub: Receives the request message data packet sent by the Client Stub program over the network, calls the actual program function method in the server to complete the function call; secondly, processes and packages the result of the server's execution of the call and sends it to the Client Stub program.

RPC based on Go language package (helloworld)

package main

import (
 "net"
 "net/rpc"
 "net/rpc/jsonrpc"
)

type HelloService struct{}

func (s *HelloService) Hello(request string, reply *string) error {
 *reply = "hello:" + request
 return nil

}

func main() {
 // 启动rpc服务
 // 1. 实例化一个Server
 // 2. 调用Server的Register方法注册rpc服务
 // 3. 调用Server的Serve方法监听端口(启动服务)
 listener, err1 := net.Listen("tcp", ":1234")
 if err1 != nil {
  return
 }

 // go 语言是有一个内置的rpc包的，可以用来实现rpc服务
 err := rpc.RegisterName("HelloService", new(HelloService))
 if err != nil {
  return
 }

 for {
  conn, er := listener.Accept()
  if er != nil {
   return
  }
  go rpc.ServeCodec(jsonrpc.NewServerCodec(conn)) // 当一个新的连接进来的时候，rpc会处理这个连接
 }

}

// python调用rpc服务
//import json
//import socket
//
//request = {
//"id":0,
//"params":["imooc"],
//"method":"HelloService.Hello"
//}
//client = socket.create_connection(("localhost", 1234))
//client.send(json.dumps(request).encode('utf-8'))
//
//# 获取服务器返回的数据
//rsp = client.recv(1024)
//rsp = json.loads(rsp.decode('utf-8'))
//print(rsp)

package main

import (
 "fmt"
 "net"
 "net/rpc"
 "net/rpc/jsonrpc"
)

func main() {
 // 连接rpc服务
 connn, err := net.Dial("tcp", "localhost:1234") // 连接rpc服务
 if err != nil {
  panic("连接失败")
 }
 var reply string
 client := rpc.NewClientWithCodec(jsonrpc.NewClientCodec(connn))
 err = client.Call("HelloService.Hello", "json grpc", &reply)
 if err != nil {
  panic("调用失败")
 }
 fmt.Println(reply)
}

Can it listen for HTTP requests?

package main

import (
 "io"
 "net/http"
 "net/rpc"
 "net/rpc/jsonrpc"
)

type HelloService struct{}

func (s *HelloService) Hello(request string, reply *string) error {
 // 返回值是通过修改replay的值
 *reply = "Hello, " + request
 return nil
}

func main() {

 _ = rpc.RegisterName("HelloService", new(HelloService))

 http.HandleFunc("/jsonrpc", func(w http.ResponseWriter, r *http.Request) {
  var conn io.ReadWriteCloser = struct {
   io.Writer
   io.ReadCloser
  }{
   Writer:     w,
   ReadCloser: r.Body,
  }
  rpc.ServeRequest(jsonrpc.NewServerCodec(conn))
 })
 http.ListenAndServe(":1234", nil)
}

request = {
    "id": 0,
    "params": ["bobby"],
    "method": "HelloService.Hello"
}

import requests
rsp = requests.post("http://localhost:1234/jsonrpc", json=request)
print(rsp.text)

Focus on new_helloworld in my code

• handler handles business logic

package handler

const HelloServiceName = "handler/HelloService"

// 服务端的业务逻辑
type NewHelloService struct{}

// 业务逻辑
func (s *NewHelloService) Hello(request string, reply *string) error {
 // 返回值是通过修改replay的值
 *reply = "Hello, " + request
 return nil
}

• client initiates via client_proxy

// client
package main

import (
 "fmt"

 "RpcLearn/new_helloworld/client_proxy"
)

func main() {
 // 建立连接
 client := client_proxy.NewHelloServiceClient("tcp", "127.0.0.1:1234")
 var reply string
 err := client.Hello("bobby", &reply)
 if err != nil {
  fmt.Println(err)
  panic(err)
 }
 fmt.Println(reply)
}

// client_proxy
package client_proxy

import (
 "net/rpc"

 "RpcLearn/new_helloworld/handler"
)

type HelloServiceStub struct {
 *rpc.Client
}

// 在go中没有类、对象，就意味着没有初始化方法
func NewHelloServiceClient(protocol, address string) *HelloServiceStub {
 conn, err := rpc.Dial(protocol, address)
 if err != nil {
  panic("dial error")
 }
 return &HelloServiceStub{conn}
}
func (c *HelloServiceStub) Hello(request string, reply *string) error {
 err := c.Client.Call(handler.HelloServiceName+".Hello", request, reply)
 if err != nil {
  return err
 }
 return err
}

• server responds via server_proxy

// server
package main

import (
 "RpcLearn/new_helloworld/handler"
 "net"
 "net/rpc"

 "RpcLearn/new_helloworld/server_proxy"
)

func main() {

 // 1. 实例化server
 listener, err := net.Listen("tcp", ":1234")
 //2. 注册处理逻辑
 //_ = rpc.RegisterName(handler.HelloServiceName, new(handler.NewHelloService))
 err = server_proxy.RegisterHelloService(new(handler.NewHelloService))
 if err != nil {
  return
 }
 //3. 启动服务
 for {
  conn, _ := listener.Accept() // 当一个新的连接进来的时候，
  go rpc.ServeConn(conn)
 }
}

// server_proxy
package server_proxy

import (
 "RpcLearn/new_helloworld/handler"
 "net/rpc"
)

type HelloServicer interface {
 Hello(request string, reply *string) error
}

// 如何做到解耦呢，我们关心的是函数 鸭子类型
func RegisterHelloService(srv HelloServicer) error {
 return rpc.RegisterName(handler.HelloServiceName, srv)
}

• These concepts have corresponding counterparts in gRPC.
• A soul-searching question: Can server_proxy and client_proxy be automatically generated for multiple languages?
• If both can be satisfied, it's gRPC+protobuf.

Summary of Main Differences between go install and go get

Usage of protoc

# 先安装protoc
# go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
# go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
# 安装protobuf support

syntax="proto3";

package helloworld;

option go_package = ".";

message HelloRequest {
  string name = 1; // 1 是编号不是值
  int32 age = 2; // 2 是编号不是值
}

# 生成普通的 .pb.go 文件（用于消息结构定义）：
protoc --go_out=. --go_opt=paths=source_relative helloworld.proto
# 生成 gRPC 的 .grpc.pb.go 文件：
protoc --go-grpc_out=. --go-grpc_opt=paths=source_relative helloworld.proto
# protoc --go_out=. helloworld.proto 我在proto这个目录下直接执行

Comparison with JSON

package main

import (
 "encoding/json"
 "fmt"
 "github.com/golang/protobuf/proto"
 "learngo/proto"
)

// 结构休的对比
type Hello struct {
 Name string `json:"name"`
 Age  int    `json:"age"`
}

func main() {
 req := helloworld.HelloRequest{
  Name: "gRPC",
  Age:  18,
 }
 jsonStruct := Hello{
  Name: req.Name,
  Age:  int(req.Age),
 }
 jsonRep, _ := json.Marshal(jsonStruct)
 fmt.Println(len(jsonRep))
 rsp, _ := proto.Marshal(&req) // 具体的编码是如何做到的 那大可以自行学习
 newReq := helloworld.HelloRequest{}
 proto.Unmarshal(rsp, &newReq)
 fmt.Println(newReq.Name, newReq.Age)
 fmt.Println(len(rsp))
}

Stub not generated (stub)

Add some method declarations to the proto, and add gRPC appearance parameters during compilation.

protoc -I . --go-grpc_out=. --go-grpc_opt=paths=source_relative helloworld.proto # 这里没有用到 --go-grpc_opt=paths=source_relative
# 这个命令会额外生成grpc调用的一些内容
# 这里要注意生成  protoc --go_out=. helloworld.proto 不要删除它们两两个不冲突

Use the latest protoc as much as possible

syntax="proto3";

option go_package = ".:proto";

service Greeter {
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

message HelloRequest {
  string name = 1;
}

message HelloReply {
  string message = 1;
}

protoc -I helloworld.proto --go_out=. --go-grpc_out=.
protoc -I . --go_out=. --go-grpc_out=. --go_opt=paths=source_relative --go-grpc_opt=paths=source_relative helloworld.proto
# 是的，新版本的 protoc 工具将生成的文件分为两个部分：

# Protobuf 本身的定义文件（.pb.go）：

# 生成普通的 Protobuf 消息定义代码。
# 包含消息的结构体、枚举等内容。
# 使用 --go_out 选项生成。
# 支持 gRPC 的服务定义文件（_grpc.pb.go）：

# 生成与 gRPC 服务相关的代码。
# 包括服务接口、客户端代码和服务端代码。
# 使用 --go-grpc_out 选项生成。
# 原因：

# 这种拆分更清晰，允许你在不使用 gRPC 的情况下单独使用 Protobuf 消息定义。
# 提高了灵活性和扩展性。例如，你可以仅使用 Protobuf 的定义，而不依赖 gRPC 的功能。

server.go

package main

import (
 "context"
 "google.golang.org/grpc"
 "net"

 "learngo/grpc_test/proto"
)

type Server struct {
 proto.UnimplementedGreeterServer // 嵌入 UnimplementedGreeterServer
}

func (s *Server) SayHello(ctx context.Context, request *proto.HelloRequest) (*proto.HelloReply, error) {
 //注意定义的时候name是小写的，编译完成后是大小的，所以这里可以直接调用
 return &proto.HelloReply{Message: "Hello " + request.Name}, nil

}

func main() {
 // 实例化grpc server
 g := grpc.NewServer()
 // 注册HelloService
 proto.RegisterGreeterServer(g, &Server{})
 // 监听端口
 lis, err := net.Listen("tcp", "0.0.0.0:1234")
 if err != nil {
  panic(err)
 }
 // 启动服务
 err = g.Serve(lis)
 if err != nil {
  panic("failed to serve: " + err.Error())
 }
}

client.go

package main

import (
 "context"
 "fmt"
 "time"

 "google.golang.org/grpc"
 "google.golang.org/grpc/credentials/insecure"
 "learngo/grpc_test/proto"
)

func main() {
 // 创建一个带超时的上下文
 _, cancel := context.WithTimeout(context.Background(), time.Second*5)
 defer cancel()

 // 使用 grpc.DialContext 连接服务端
 conn, err := grpc.NewClient(
  "localhost:1234", // 服务端地址
  grpc.WithTransportCredentials(insecure.NewCredentials()), // 非安全连接（开发环境使用）
 )
 if err != nil {
  panic(fmt.Sprintf("Failed to connect to server: %v", err))
 }
 defer conn.Close()

 // 创建 gRPC 客户端
 client := proto.NewGreeterClient(conn)

 // 调用服务方法
 resp, err := client.SayHello(context.Background(), &proto.HelloRequest{Name: "World"})
 if err != nil {
  panic(fmt.Sprintf("Failed to call SayHello: %v", err))
 }
 fmt.Println(resp.Message) // 输出服务器返回的消息
}