API Gateway and Continuous Deployment Introduction (Kong & Jenkins)

Corresponds to the resource directory "Chapter 2: Getting Started with Jenkins" and "Chapter 3: Deploying Services with Jenkins", this article organizes the practical path of Kong and Jenkins in enterprise-level continuous delivery. Even with zero foundation, you can follow the steps to build your own gateway + continuous deployment pipeline.

Pre-class Guide: What is an API Gateway

An API Gateway sits between clients and backend microservices, acting as the system's "unified entry point". It is responsible for receiving external requests, routing them to different backend services based on configuration, and performing a series of common capabilities at the entry layer (authentication, authorization, rate limiting, monitoring, protocol conversion, etc.), thereby simplifying client call processes and reducing coupling between services.

graph LR
    ClientA[Web 客户端] --> APIGateway[API Gateway]
    ClientB[移动端] --> APIGateway
    ClientC[第三方合作方] --> APIGateway
    APIGateway -->|路由/聚合| ServiceA[用户服务]
    APIGateway -->|协议转换| ServiceB[订单服务]
    APIGateway -->|安全控制| ServiceC[支付服务]

Core Responsibilities

• Unified Entry and Routing: All requests first go to the gateway, then are forwarded according to rules such as path, domain, and Header.
• Security Protection: Centralized handling of security policies such as authentication, authorization, rate limiting, IP whitelisting, and WAF.
• Protocol and Format Conversion: Supports REST, gRPC, GraphQL, etc., unifying various protocols externally.
• Observability and Operations: Collects logs, metrics, and tracing, provides resilience measures such as circuit breaking, retries, and caching.
• Aggregation and Orchestration: Combines multiple backend interfaces, returns customized responses, and reduces client call complexity.

Why an API Gateway is Needed

• Growth in Microservice Count: Clients do not need to care about the address and authentication method of each service; the gateway exposes them uniformly.
• Centralized Governance Policies: Strategies such as rate limiting, circuit breaking, and monitoring are uniformly implemented at the gateway layer, avoiding redundant implementations across services.
• Facilitates Evolution and Canary Releases: Achieves canary releases, A/B testing, and version switching through routing rules.
• Enhances Security and Compliance: A unified entry point facilitates auditing and risk control.

sequenceDiagram
    participant Client as 客户端
    participant Gateway as API 网关
    participant Auth as 身份验证服务
    participant Order as 订单服务
    participant Billing as 计费服务

    Client->>Gateway: 请求 /v1/order
    Gateway->>Auth: 校验 Token
    Auth-->>Gateway: 返回用户信息
    Gateway->>Order: 调用订单详情
    Gateway->>Billing: 查询支付状态
    Gateway-->>Client: 聚合后的订单响应

Common products: Nginx + Lua, Kong, Tyk, APISIX, AWS API Gateway, Spring Cloud Gateway, etc.

Chapter 2: Getting Started with Kong in Practice

Combining videos 2-1 to 2-6, master common operations from installation to security hardening.

2-1 Kong's 8001, 8000, and 1337 Port Numbers

Kong opens multiple ports by default, each carrying different responsibilities. Understanding their roles first makes troubleshooting easier.

graph TD
    subgraph 业务面
        Client[客户端流量] --> Proxy[8000/8443 Proxy]
        Proxy --> UpstreamPool[上游服务]
    end
    subgraph 运维面
        AdminAPI[8001/8444 Admin API] --> KongCore[Kong 核心]
        Portal[1337 Manager] --> KongCore
    end
    KongCore --> Metrics[日志/指标输出]

Quick self-check commands:

# 确认端口监听状态
docker exec kong netstat -tnlp | grep 800
# 用 Admin API 获取节点信息
curl http://localhost:8001/

2-2 Basic Routing Configuration

The first thing to do when getting started is to make a URL correctly forward to a backend service.

1. Register Service: Define upstream address and protocol.
2. Create Route: Determine which requests hit the above Service.
3. Verify Forwarding: Access the proxy entry via curl or browser.

# 1. 注册服务
curl -X POST http://localhost:8001/services \
  --data "name=user-service" \
  --data "url=http://mockbin.org"

# 2. 绑定路由
curl -X POST http://localhost:8001/services/user-service/routes \
  --data "paths[]=/users" \
  --data "strip_path=true"

# 3. 访问 Kong 代理
curl -i http://localhost:8000/users

Common pitfalls: hosts, paths, and methods are "AND" relationships; the more matching conditions, the lower the hit probability; ensure that the client's Host Header is consistent with the route configuration.

2-3 Kong's Service, Route, Upstream

Service, Route, and Upstream form Kong's core abstractions; understanding their relationships helps you flexibly combine traffic strategies.

• Service: Describes backend applications (protocol, domain, port, timeout).
• Route: Defines how requests are matched to a specific Service.
• Upstream: Provides an "instance pool" for Services, supporting load balancing and health checks.
• Target: A specific instance (IP:Port) within an Upstream.

graph TD
    Client --> Route[Route: hosts/paths]
    Route --> Service[Service: http://orders.internal]
    Service --> Upstream[Upstream: order-upstream]
    Upstream --> TargetA[10.0.1.10:9000]
    Upstream --> TargetB[10.0.1.11:9000]
    Route -- Plugin 链 --> Plugins[认证/限流/日志]

Best practices:

• Create a separate Upstream for each upstream service to facilitate subsequent rolling upgrades.
• Plugins can be attached to Routes, Services, or globally; global policies have the lowest priority.
• Use tags to group resources for easier operational search.

2-4 Kong Integration with Consul for Service Discovery and Health Checks

When backend instances change frequently, entrusting health checks to Consul can reduce manual maintenance pressure.

1. Prepare Consul: Start the Consul Agent and register your application instances (you can use a service.json file).
2. Configure Service Discovery: Add a new host in the form of service.consul to Kong's Service, and enable service_discovery in KONG_DATABASE or kong.conf.
3. Declare Upstream to use Consul:

curl -X POST http://localhost:8001/upstreams \
  --data "name=order.consul" \
  --data "service=orders" \
  --data "healthchecks.active.type=http"

1. Consul Maintains Instances: When instances go online or offline, Consul automatically synchronizes, and Kong reads the latest health status.

graph LR
    Consul -->|服务健康| KongUpstream
    Jenkins -->|部署后注册| Consul
    KongProxy --> KongUpstream --> BackendPods[订单服务实例]

Tip: Ensure that Kong and Consul are network reachable, and enable TTL or HTTP health checks in Consul.

2-5 Kong Configuration of JWT for Login Verification

The JWT plugin makes Kong a unified authentication entry point. The typical process is as follows.

1. Enable plugin and create consumer:

# 添加一个消费者
curl -X POST http://localhost:8001/consumers \
  --data "username=mobile-app"

# 为消费者生成密钥对
curl -X POST http://localhost:8001/consumers/mobile-app/jwt

1. Enable JWT plugin on Route or Service:

curl -X POST http://localhost:8001/services/user-service/plugins \
  --data "name=jwt" \
  --data "config.claims_to_verify=exp"

1. Client constructs and carries the Token:

curl http://localhost:8000/users \
  -H "Host: api.example.com" \
  -H "Authorization: Bearer <签名后的JWT>"

sequenceDiagram
    participant Client as 客户端
    participant Kong as Kong 网关
    participant JWT as JWT 插件
    participant Upstream as 上游服务

    Client->>Kong: 带 Authorization 的请求
    Kong->>JWT: 校验签名、过期时间
    JWT-->>Kong: 校验通过/失败
    opt 校验通过
        Kong->>Upstream: 转发请求
        Upstream-->>Kong: 返回业务响应
        Kong-->>Client: 200 OK
    end
    opt 校验失败
        Kong-->>Client: 401 未授权
    end

Tip: Combining with key-auth or rate-limiting plugins can further differentiate access permissions and frequencies for different clients.

2-6 Kong Configuration of Anti-Crawler IP Blacklist

Common methods to combat malicious requests are "whitelisting + blacklisting + rate limiting".

# 开启 IP 限制插件
curl -X POST http://localhost:8001/services/user-service/plugins \
  --data "name=ip-restriction" \
  --data "config.deny[]=192.168.1.100" \
  --data "config.deny[]=203.0.113.0/24"

Combined approach:

• Blacklist: Immediately add malicious IPs to the deny list upon discovery.
• Whitelist: For admin backends and BFF interfaces, an allow list can be set, with other sources denied by default.
• Rate Limiting: Use with rate-limiting or bot-detection plugins to smooth out peaks.
• Auditing: Deliver Kong logs to ELK/Loki to identify abnormal UAs and Referers.

graph TD
    ClientReq[请求] --> IPCheck[IP 限制插件]
    IPCheck -->|黑名单| Block[返回 403]
    IPCheck -->|通过| RateLimit[限流插件]
    RateLimit --> Backend[上游服务]

Deploying Services with Jenkins

Corresponding to videos 3-1 to 3-11, this guides you from Jenkins installation and plugin selection to Pipeline orchestration and release automation.

3-1 Continuous Integration Pain Points in Agile Development

• Frequent code merges, low efficiency of manual packaging and release.
• Significant dependency differences across environments, easily leading to "it runs on my machine" issues.
• Lack of unified quality gates, test omissions leading to production rollbacks.
• Lack of traceability for deployment results, inability to quickly locate failed nodes.

Agile Development: We don't really know what to develop, let's just take it one step at a time.
User Story: The boss said this feature needs to go live tomorrow, I don't care how you implement it.
Rapid Iteration: The last feature had too low a click-through rate, change the ad to full screen.
User Pain Point: A user complained yesterday, adjust the ad.
Embrace Change: The boss has new ideas every day, everyone needs to adapt (don't blame me).
Continuous Delivery: Every version has problems, always continuously delivered to testing, delivery interval 10 minutes.
Pair Programming: Too many bugs, just go to the tester's desk and fix them while testing.
Code Review: You reviewed this code, you'll be responsible if problems arise later.
Flexible Work: No fixed quitting time, you can only leave after fixing the bugs.
Four Meetings: (Scrum Meeting) Starts at 9:00 AM every day, no way to be late for work.

The "joke-style" passage above satirizes common agile misconceptions in the industry: equating agile with no planning, treating user stories as ad-hoc requirements, packaging rough launches as "rapid iteration", and even neglecting quality gates in the name of continuous delivery. True agile emphasizes:

• Rhythmic Planning: Before each iteration, requirements still need to be clarified, workloads estimated, and deliverable goals synchronized externally.
• User Stories Focus on Value: They describe user roles, scenarios, and benefits, helping the team understand "why" something is being done, not just arbitrary feature commands.
• Continuous Delivery is Premised on Quality: Automated testing, code reviews, and monitoring alerts are fundamental safeguards; frequent delivery is for quickly validating value, not for continuously "throwing defects to testing".
• Embracing Change Must Have Boundaries: Manage changes through product backlog and iteration rhythm, adjusting priorities when necessary instead of ad-hoc insertions.
• Scrum Four Meetings (Planning Meeting, Daily Stand-up, Review Meeting, Retrospective Meeting) are for synchronizing information, identifying risks, and continuous improvement, not for formalistic clocking in.

Understanding these principles is essential to combine agile and continuous integration, allowing Jenkins pipelines to serve "continuous delivery of value" rather than "continuous delivery of problems".

3-2 Installing Jenkins and Disabling Firewall

1. Docker installation is the quickest:

docker run -d --name jenkins \
  -p 8080:8080 -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  jenkins/jenkins:lts

1. Initialization Wizard: Follow the prompts to enter the unlock password, install recommended plugins, and create an administrator.
2. Server Firewall: If using CentOS, you can first disable or allow port 8080.

sudo systemctl stop firewalld
sudo firewall-cmd --permanent --add-port=8080/tcp
sudo firewall-cmd --reload

3-3 Jenkins Build Service Deployment Process

sequenceDiagram
    participant Dev as 开发者
    participant Git as 代码仓库
    participant Jenkins
    participant Registry as 镜像仓库
    participant K8s as Kubernetes
    participant Kong

    Dev->>Git: 提交代码
    Git-->>Jenkins: Webhook/轮询触发
    Jenkins->>Jenkins: 编译 + 测试
    Jenkins->>Registry: 推送镜像/制品
    Jenkins->>K8s: 滚动更新
    Jenkins->>Kong: 调用 Admin API 刷新路由
    Jenkins-->>Dev: 通知结果（钉钉/邮件）

Key points of the process:

• Use Credentials to store access keys for Git, Registry, and Kubernetes.
• Timely emailext or robot notifications during failure stages facilitate immediate handling.
• Specify different notification actions for success and failure using the post block.

3-4 Installing Common Jenkins Plugins

• Git/Git Parameter: Supports multi-repository, multi-branch builds.
• Pipeline + Blue Ocean: Declarative pipelines and visual display.
• Credentials Binding: Injects sensitive credentials into environment variables.
• Kubernetes / Docker: Containerized build capabilities.
• Email Extension, DingTalk/WeChat Work Plugins: Build notifications.
• Config File Provider: Centralized management of files like Maven settings, Kubeconfig, etc.

3-5 Building Projects with Freestyle Style

1. Create a new Freestyle project, configure a description to help the team understand its purpose.
2. Fill in the Git address and credentials in Source Code Management.
3. In Build Triggers, check GitHub hook trigger or Poll SCM.
4. Add Shell to the Build step, for example:

go test ./...
go build -o bin/order-service ./cmd/order

1. Post-build Actions can archive binaries and publish notifications.

Suitable for simple projects or quick PoCs, can be migrated to Pipeline later.

3-6 Uploading Code from Build Server to Runtime Environment

Common practices:

• SCP/RSYNC: Push compiled artifacts to the target host.

scp bin/order-service deploy@10.0.0.12:/srv/apps/order/
ssh deploy@10.0.0.12 "systemctl restart order.service"

• Artifactory/Nexus: Upload build artifacts, then the deployment machine pulls them.
• Docker Registry: Deploy using kubectl or docker stack after building the image.

Remember to configure SSH credentials in Jenkins and restrict the permissions of execution nodes.

3-7 Achieving Continuous Integration with Pipeline

Declarative Pipeline allows processes to be written as code, facilitating reuse and review.

pipeline {
  agent any
  environment {
    REGISTRY = "registry.example.com"
    APP = "order-service"
  }
  stages {
    stage('Checkout') {
      steps { git url: 'https://github.com/demo/order-service.git', branch: 'main' }
    }
    stage('Test') {
      steps { sh 'go test ./...' }
    }
    stage('Build Image') {
      steps { sh 'docker build -t $REGISTRY/$APP:$BUILD_NUMBER .' }
    }
    stage('Push Image') {
      steps { sh 'docker push $REGISTRY/$APP:$BUILD_NUMBER' }
    }
    stage('Deploy') {
      steps {
        sh '''
        kubectl set image deploy/order-service order=$REGISTRY/$APP:$BUILD_NUMBER
        kubectl rollout status deploy/order-service
        '''
      }
    }
    stage('Notify Kong') {
      steps { sh 'curl -X POST http://kong-admin:8001/services/order-service/reload' }
    }
  }
  post {
    success { emailext subject: "Order Service #${BUILD_NUMBER} 成功", to: "team@example.com" }
    failure { emailext subject: "Order Service #${BUILD_NUMBER} 失败", to: "devops@example.com" }
  }
}

3-8 Managing Build Pipelines with Jenkinsfile

• Store Jenkinsfile in the root directory of the code repository, versioning the build process.
• Code reviews should not only examine business code but also pipeline changes.
• You can create multi-branch Pipelines, allowing Jenkins to automatically discover Jenkinsfiles in new branches.

Directory example:

.
├── Jenkinsfile
├── cmd/
├── deploy/
└── infra/helm/

3-9 Triggering Builds Remotely and from Other Projects

• Remote Trigger: In Job settings, check Trigger builds remotely, generate a Token, and use curl to invoke.

curl "http://jenkins.example.com/job/order-service/build?token=deploy"

• Upstream Project Trigger: In Build Triggers, check Build after other projects are built to chain pipelines.
• Trigger via Script: Integrate with ChatOps robots using Jenkins CLI or REST API.

3-10 Scheduled Builds and Poll SCM Builds

• Poll SCM: Enter a Cron expression, for example, H/5 * * * * means checking Git for new commits every 5 minutes.
• Build periodically: Executes on a schedule even without code changes, for example, running security scans at night.
• Cron tip: Use H to let Jenkins automatically distribute execution times for different jobs, avoiding task congestion.

3-11 Parameterized Pipeline Builds

Parameterization makes pipelines more flexible, commonly used for selecting branches, versions, or deployment environments.

pipeline {
  agent any
  parameters {
    choice(name: 'ENV', choices: ['dev', 'staging', 'prod'], description: '部署环境')
    string(name: 'IMAGE_TAG', defaultValue: 'latest', description: '镜像版本')
    booleanParam(name: 'SKIP_TEST', defaultValue: false, description: '是否跳过测试')
  }
  stages {
    stage('Checkout') {
      steps { git branch: params.ENV == 'prod' ? 'main' : params.ENV, url: 'https://github.com/demo/order-service.git' }
    }
    stage('Test') {
      when { expression { return !params.SKIP_TEST } }
      steps { sh 'go test ./...' }
    }
    stage('Deploy') {
      steps { sh "kubectl set image deploy/order-service order=${params.IMAGE_TAG}" }
    }
  }
}

Common usage: A single Pipeline covering multiple environments, or providing a quick entry point for manual rollbacks.

Review and Practice Checklist

• Draw your company's API Gateway topology, marking the relationships between Service, Route, Plugin, and traffic sources.
• Start Kong with Docker and reproduce the port check, routing, JWT, and blacklist configurations from 2-1 to 2-6.
• Deploy Jenkins in a test environment, complete one Freestyle and one Pipeline, and integrate with Kong's hot update.
• Incorporate Jenkinsfile into the code review process, and try adding parameterized releases for main branches.
• Use deck or decK to export Kong configurations, combine with Git management, to achieve "configuration as code".