Getting Started with API Gateway and Continuous Deployment (Kong & Jenkins)

Corresponds to the resource directory "Chapter 2 Getting Started with Jenkins" and "Chapter 3 Deploying Services via Jenkins", organizing the practical path of Kong and Jenkins in enterprise-level continuous delivery. Even with zero foundation, you can follow the steps to build your own gateway + continuous deployment pipeline.

Course Preview: What is an API Gateway

An API Gateway sits between clients and backend microservices, acting as the system's "unified entry point". It is responsible for receiving external requests, routing them to different backend services based on configuration, and performing a series of common capabilities at the entry layer (authentication, authorization, rate limiting, monitoring, protocol conversion, etc.), thereby simplifying client invocation processes and reducing coupling between services.

graph LR
    ClientA[Web Client] --> APIGateway[API Gateway]
    ClientB[Mobile Client] --> APIGateway
    ClientC[Third-Party Partner] --> APIGateway
    APIGateway -->|Routing/Aggregation| ServiceA[User Service]
    APIGateway -->|Protocol Conversion| ServiceB[Order Service]
    APIGateway -->|Security Control| ServiceC[Payment Service]

Core Responsibilities

• Unified Entry Point and Routing: All requests first go to the gateway, then are forwarded according to rules such as path, domain, and Header.
• Security Protection: Centralized handling of security policies such as authentication, authorization, rate limiting, IP whitelisting, and WAF.
• Protocol and Format Conversion: Supports REST, gRPC, GraphQL, etc., unifying various protocols externally.
• Observability and Operations: Collects logs, metrics, tracing, and provides resilience measures such as circuit breaking, retries, and caching.
• Aggregation and Orchestration: Combines multiple backend interfaces, returns customized responses, and reduces client invocation complexity.

Why an API Gateway is Needed

• Growth in Microservice Count: Clients do not need to care about the address and authentication method of each service; the gateway exposes them uniformly.
• Centralization of Governance Policies: Policies such as rate limiting, circuit breaking, and monitoring are uniformly implemented at the gateway layer, avoiding redundant implementations by individual services.
• Facilitates Evolution and Canary Releases: Achieves canary releases, A/B testing, and version switching through routing rules.
• Enhances Security and Compliance: A unified entry point facilitates auditing and risk control.

sequenceDiagram
    participant Client as Client
    participant Gateway as API Gateway
    participant Auth as Authentication Service
    participant Order as Order Service
    participant Billing as Billing Service

    Client->>Gateway: Request /v1/order
    Gateway->>Auth: Validate Token
    Auth-->>Gateway: Return User Info
    Gateway->>Order: Call Order Details
    Gateway->>Billing: Query Payment Status
    Gateway-->>Client: Aggregated Order Response

Common products: Nginx + Lua, Kong, Tyk, APISIX, AWS API Gateway, Spring Cloud Gateway, etc.

Chapter 2: Getting Started with Kong in Practice

Combine with videos 2-1 to 2-6 to master common operations from installation to security hardening.

2-1 Kong's 8001, 8000, and 1337 Port Numbers

Kong opens multiple ports by default, each carrying different responsibilities. Understanding their roles first makes it easier to troubleshoot issues.

graph TD
    subgraph Business Plane
        Client[Client Traffic] --> Proxy[8000/8443 Proxy]
        Proxy --> UpstreamPool[Upstream Services]
    end
    subgraph Operations Plane
        AdminAPI[8001/8444 Admin API] --> KongCore[Kong Core]
        Portal[1337 Manager] --> KongCore
    end
    KongCore --> Metrics[Logs/Metrics Output]

Quick self-check commands:

# Confirm port listening status
docker exec kong netstat -tnlp | grep 800
# Get node info using Admin API
curl http://localhost:8001/

2-2 Basic Routing Configuration

The first thing to do when getting started is to ensure a URL can be correctly forwarded to a backend service.

1. Register Service: Define upstream address and protocol.
2. Create Route: Determine which requests hit the aforementioned Service.
3. Verify Forwarding: Access the proxy entry point via curl or a browser.

# 1. Register service
curl -X POST http://localhost:8001/services \
  --data "name=user-service" \
  --data "url=http://mockbin.org"

# 2. Bind route
curl -X POST http://localhost:8001/services/user-service/routes \
  --data "paths[]=/users" \
  --data "strip_path=true"

# 3. Access Kong proxy
curl -i http://localhost:8000/users

Common pitfall: hosts, paths, methods are "AND" relationships; the more matching conditions, the lower the hit probability; ensure the client's Host Header is consistent with the route configuration.

2-3 Kong's Service, Route, Upstream

Service, Route, and Upstream constitute Kong's core abstractions; understanding their relationships helps you flexibly combine traffic strategies.

• Service: Describes backend applications (protocol, domain, port, timeout).
• Route: Defines how requests are matched to a specific Service.
• Upstream: Provides an "instance pool" for a Service, supporting load balancing and health checks.
• Target: A specific instance (IP:Port) within an Upstream.

graph TD
    Client --> Route[Route: hosts/paths]
    Route --> Service[Service: http://orders.internal]
    Service --> Upstream[Upstream: order-upstream]
    Upstream --> TargetA[10.0.1.10:9000]
    Upstream --> TargetB[10.0.1.11:9000]
    Route -- Plugin Chain --> Plugins[Auth/Rate Limit/Logs]

Best practices:

• Create a separate Upstream for each upstream service to facilitate subsequent rolling upgrades.
• Plugins can be attached to a Route, a Service, or globally; global policies have the lowest priority.
• Use tags to group resources for easier operational search.

2-4 Kong Integration with Consul for Service Discovery and Health Checks

When backend instances change frequently, entrusting health checks to Consul can reduce manual maintenance pressure.

1. Prepare Consul: Start the Consul Agent and register your application instances (you can use a service.json file).
2. Configure Service Discovery: In Kong's Service, add a host in the service.consul format, and enable service_discovery in KONG_DATABASE or kong.conf.
3. Declare Upstream to Use Consul:

curl -X POST http://localhost:8001/upstreams \
  --data "name=order.consul" \
  --data "service=orders" \
  --data "healthchecks.active.type=http"

1. Consul Maintains Instances: When instances go online or offline, Consul automatically synchronizes, and Kong reads the latest health status.

graph LR
    Consul -->|Service Health| KongUpstream
    Jenkins -->|Register After Deployment| Consul
    KongProxy --> KongUpstream --> BackendPods[Order Service Instances]

Tip: Ensure Kong and Consul are network reachable, and enable TTL or HTTP health checks in Consul.

2-5 Kong Configuration for JWT to Implement Login Verification

The JWT plugin makes Kong a unified authentication entry point. The typical process is as follows.

1. Enable Plugin and Create Consumer:

# Add a consumer
curl -X POST http://localhost:8001/consumers \
  --data "username=mobile-app"

# Generate key pair for consumer
curl -X POST http://localhost:8001/consumers/mobile-app/jwt

1. Enable JWT Plugin on Route or Service:

curl -X POST http://localhost:8001/services/user-service/plugins \
  --data "name=jwt" \
  --data "config.claims_to_verify=exp"

1. Client Constructs and Carries Token:

curl http://localhost:8000/users \
  -H "Host: api.example.com" \
  -H "Authorization: Bearer <签名后的JWT>"

sequenceDiagram
    participant Client as Client
    participant Kong as Kong Gateway
    participant JWT as JWT Plugin
    participant Upstream as Upstream Service

    Client->>Kong: Request with Authorization
    Kong->>JWT: Verify signature, expiration time
    JWT-->>Kong: Verification passed/failed
    opt Verification passed
        Kong->>Upstream: Forward request
        Upstream-->>Kong: Return business response
        Kong-->>Client: 200 OK
    end
    opt Verification failed
        Kong-->>Client: 401 Unauthorized
    end

Tip: Combining with key-auth or rate-limiting plugins can further differentiate access permissions and frequencies for different clients.

2-6 Kong Configuration for Anti-Crawler IP Blacklist

Common methods to combat malicious requests are "whitelisting + blacklisting + rate limiting".

# Enable IP restriction plugin
curl -X POST http://localhost:8001/services/user-service/plugins \
  --data "name=ip-restriction" \
  --data "config.deny[]=192.168.1.100" \
  --data "config.deny[]=203.0.113.0/24"

Combined approach:

• Blacklist: Immediately add malicious IPs to the deny list upon discovery.
• Whitelist: For admin backends and BFF interfaces, an allow list can be set, denying other sources by default.
• Rate Limiting: Use with rate-limiting or bot-detection plugins to mitigate traffic spikes.
• Auditing: Deliver Kong logs to ELK/Loki to identify abnormal UAs and Referers.

graph TD
    ClientReq[Request] --> IPCheck[IP Restriction Plugin]
    IPCheck -->|Blacklist| Block[Return 403]
    IPCheck -->|Pass| RateLimit[Rate Limiting Plugin]
    RateLimit --> Backend[Upstream Service]

Deploying Services via Jenkins

Corresponds to videos 3-1 to 3-11, guiding you from Jenkins installation and plugin selection to Pipeline orchestration and release automation.

3-1 Continuous Integration Pain Points in Agile Development

• Frequent code merges, low efficiency of manual packaging and release.
• Significant dependency differences across environments, easily leading to "it works on my machine" issues.
• Lack of unified quality gates, testing omissions leading to online rollbacks.
• Lack of traceability for deployment results, unable to quickly pinpoint failed nodes.

Agile Development: We don't really know what to develop, let's just take it one step at a time.
User Story: The boss said this feature needs to go live tomorrow, I don't care how you implement it.
Rapid Iteration: The feature we did last time had too low a click-through rate, change the ad to full screen.
User Pain Point: A user complained yesterday, adjust the ad.
Embrace Change: The boss has new ideas every day, everyone needs to adapt (don't blame me).
Continuous Delivery: Every version has problems, always continuously delivered to testing, delivery interval 10 minutes.
Pair Programming: Too many bugs, just go to the tester's desk and fix them while testing.
Code Review: You reviewed this code, you'll be responsible if there's a problem later.
Flexible Work: No fixed quitting time, can only leave after fixing bugs.
Four Meetings: (Scrum Meeting) Starts at 9:00 AM every day, no way to be late for work.

The "joke" above pokes fun at common agile misconceptions in the industry: equating agile with no planning, treating user stories as temporary demands, packaging rough launches as "rapid iterations," and even neglecting quality gates in the name of continuous delivery. True agile emphasizes:

• Rhythmic Planning: Before iteration, requirements still need to be sorted out, workloads estimated, and deliverable goals synchronized externally.
• User Stories Focus on Value: Describe user roles, scenarios, and benefits, helping the team understand "why" something is done, not just arbitrary feature commands.
• Continuous Delivery is Premised on Quality: Automated testing, code reviews, and monitoring alerts are fundamental safeguards. Frequent delivery is for quickly validating value, not for continuously "throwing defects to testing".
• Embracing Change Also Has Boundaries: Manage changes through product backlog and iteration rhythm, adjusting priorities when necessary instead of ad-hoc insertions.
• Scrum Four Meetings (Planning Meeting, Daily Stand-up, Review Meeting, Retrospective Meeting) are for synchronizing information, identifying risks, and continuous improvement, not for formalistic check-ins.

Understanding these principles is essential to combine agile with continuous integration, allowing Jenkins pipelines to serve "continuous delivery of value" rather than "continuous delivery of problems".

3-2 Installing Jenkins and Disabling Firewall

1. Docker installation is the fastest:

docker run -d --name jenkins \
  -p 8080:8080 -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  jenkins/jenkins:lts

1. Initialization Wizard: Follow the prompts to enter the unlock password, install recommended plugins, and create an administrator.
2. Server Firewall: If using CentOS, you can first disable or open port 8080.

sudo systemctl stop firewalld
sudo firewall-cmd --permanent --add-port=8080/tcp
sudo firewall-cmd --reload

3-3 Jenkins Build Service Deployment Process

sequenceDiagram
    participant Dev as Developer
    participant Git as Code Repository
    participant Jenkins
    participant Registry as Image Repository
    participant K8s as Kubernetes
    participant Kong

    Dev->>Git: Commit Code
    Git-->>Jenkins: Webhook/Polling Trigger
    Jenkins->>Jenkins: Compile + Test
    Jenkins->>Registry: Push Image/Artifact
    Jenkins->>K8s: Rolling Update
    Jenkins->>Kong: Call Admin API to Refresh Routes
    Jenkins-->>Dev: Notify Result (DingTalk/Email)

Key points of the process:

• Use Credentials to store access keys for Git, Registry, Kubernetes.
• Timely emailext or robot notifications at failure stages for prompt handling.
• Specify different notification actions for success and failure using the post block.

3-4 Installing Common Jenkins Plugins

• Git/Git Parameter: Supports multi-repository, multi-branch builds.
• Pipeline + Blue Ocean: Declarative pipelines and visual display.
• Credentials Binding: Inject sensitive credentials into environment variables.
• Kubernetes / Docker: Containerized build capabilities.
• Email Extension, DingTalk/WeChat Work Plugins: Build notifications.
• Config File Provider: Centralized management of Maven settings, Kubeconfig, and other files.

3-5 Building Projects with Freestyle Style

1. Create a new Freestyle project, configure a description to help the team understand its purpose.
2. Fill in the Git address and credentials in Source Code Management.
3. In Build Triggers, check GitHub hook trigger or Poll SCM.
4. Add Shell to the Build step, for example:

go test ./...
go build -o bin/order-service ./cmd/order

1. Post-build Actions can archive binaries and publish notifications.

Suitable for simple projects or quick PoCs, can be migrated to Pipeline later.

3-6 Uploading Code from Build Server to Runtime Environment

Common practices:

• SCP/RSYNC: Push compiled artifacts to the target host.

scp bin/order-service deploy@10.0.0.12:/srv/apps/order/
ssh deploy@10.0.0.12 "systemctl restart order.service"

• Artifactory/Nexus: Upload build artifacts, then the deployment machine pulls them.
• Docker Registry: After building the image, deploy using kubectl or docker stack.

Remember to configure SSH credentials in Jenkins and restrict permissions for execution nodes.

3-7 Implementing Continuous Integration via Pipeline

Declarative Pipelines allow processes to be written as code, facilitating reuse and review.

pipeline {
  agent any
  environment {
    REGISTRY = "registry.example.com"
    APP = "order-service"
  }
  stages {
    stage('Checkout') {
      steps { git url: 'https://github.com/demo/order-service.git', branch: 'main' }
    }
    stage('Test') {
      steps { sh 'go test ./...' }
    }
    stage('Build Image') {
      steps { sh 'docker build -t $REGISTRY/$APP:$BUILD_NUMBER .' }
    }
    stage('Push Image') {
      steps { sh 'docker push $REGISTRY/$APP:$BUILD_NUMBER' }
    }
    stage('Deploy') {
      steps {
        sh '''
        kubectl set image deploy/order-service order=$REGISTRY/$APP:$BUILD_NUMBER
        kubectl rollout status deploy/order-service
        '''
      }
    }
    stage('Notify Kong') {
      steps { sh 'curl -X POST http://kong-admin:8001/services/order-service/reload' }
    }
  }
  post {
    success { emailext subject: "Order Service #${BUILD_NUMBER} Success", to: "team@example.com" }
    failure { emailext subject: "Order Service #${BUILD_NUMBER} Failure", to: "devops@example.com" }
  }
}

3-8 Managing Build Pipelines with Jenkinsfile

• Store Jenkinsfile in the root directory of the code repository to version control the build process.
• Code reviews should not only examine business code but also pipeline changes.
• Multi-branch Pipelines can be created, allowing Jenkins to automatically discover Jenkinsfiles in new branches.

Directory example:

.
├── Jenkinsfile
├── cmd/
├── deploy/
└── infra/helm/

3-9 Triggering Builds Remotely and from Other Projects

• Remote Trigger: In Job settings, check Trigger builds remotely, generate a Token, and use curl to invoke.

curl "http://jenkins.example.com/job/order-service/build?token=deploy"

• Upstream Project Trigger: In Build Triggers, check Build after other projects are built to chain pipelines.
• Trigger via Script: Utilize Jenkins CLI or REST API to integrate with ChatOps robots.

3-10 Scheduled Builds and Poll SCM Builds

• Poll SCM: Enter a Cron expression, for example, H/5 * * * * means checking Git for new commits every 5 minutes.
• Build periodically: Executes on a schedule even without code changes, for example, running security scans at night.
• Cron tip: Use H to let Jenkins automatically distribute execution times for different jobs, avoiding task pile-ups.

3-11 Parameterized Pipeline Builds

Parameterization makes pipelines more flexible, commonly used for selecting branches, versions, or deployment environments.

pipeline {
  agent any
  parameters {
    choice(name: 'ENV', choices: ['dev', 'staging', 'prod'], description: 'Deployment environment')
    string(name: 'IMAGE_TAG', defaultValue: 'latest', description: 'Image version')
    booleanParam(name: 'SKIP_TEST', defaultValue: false, description: 'Skip tests')
  }
  stages {
    stage('Checkout') {
      steps { git branch: params.ENV == 'prod' ? 'main' : params.ENV, url: 'https://github.com/demo/order-service.git' }
    }
    stage('Test') {
      when { expression { return !params.SKIP_TEST } }
      steps { sh 'go test ./...' }
    }
    stage('Deploy') {
      steps { sh "kubectl set image deploy/order-service order=${params.IMAGE_TAG}" }
    }
  }
}

Common usage: A single Pipeline covering multiple environments, or providing a quick entry point for manual rollbacks.

Review and Practice Checklist

• Draw your company's API Gateway topology, marking the relationships between Service, Route, Plugin, and traffic sources.
• Start Kong with Docker and reproduce the port check, routing, JWT, and blacklist configurations from 2-1 to 2-6.
• Deploy Jenkins in a test environment, complete one Freestyle and one Pipeline, and integrate with Kong's hot update.
• Incorporate Jenkinsfile into the code review process, and try adding parameterized releases for main branches.
• Use deck or decK to export Kong configurations, combine with Git management, to achieve "configuration as code".