# CI Security Validation

Your CI pipeline is a security boundary. Code running during builds and tests can make network requests—to package registries, external APIs, analytics endpoints, or worse. Without visibility into this traffic, you can't detect supply chain attacks, accidental credential exfiltration, or compliance violations before they reach production.

This guide shows how to use Qtap in GitHub Actions to observe ALL network traffic during CI, enforce network allow-lists, and fail builds when unexpected hosts are contacted.

## Why This Matters

### The Problem

**Your CI environment makes network calls you don't know about:**

* **Supply chain attacks** - A compromised npm/pip package phones home during `npm install` or `pip install`
* **Accidental production calls** - Integration tests hit production APIs instead of staging
* **Secrets exfiltration** - Malicious code in dependencies POSTs environment variables to attacker-controlled servers
* **Compliance violations** - HIPAA/SOC2/PCI require proving no data leaves approved networks
* **Dependency drift** - New transitive dependencies fetch from unexpected registries

Traditional approaches don't help:

* **Network policies** - Not available in most CI runners (GitHub Actions, CircleCI, etc.)
* **egress proxies** - Require app code changes and don't catch kernel-level traffic
* **Log scraping** - Only shows what apps choose to log, misses silent exfiltration

### The Solution

**Qtap captures ALL network traffic at the Linux kernel level** using eBPF:

* **Zero code changes** - Observes traffic passively, no proxy configuration needed
* **Complete visibility** - Sees inside TLS/HTTPS before encryption happens
* **Process attribution** - Know exactly which command made each request
* **Enforcement** - Fail builds automatically when disallowed hosts are contacted

### Use Cases

This guide demonstrates the pattern with a simple curl example, but the same approach works for:

1. **Package manager validation** - Ensure `npm install`, `pip install`, `go get` only fetch from approved registries
2. **Integration test guardrails** - Verify your app only calls staging/test APIs, never production
3. **Docker build security** - Confirm `apt-get`, `curl` in Dockerfiles only reach approved mirrors
4. **Compliance auditing** - Generate tamper-proof logs of all CI network activity for security reviews
5. **Secrets leak prevention** - Detect when dependencies try to exfiltrate AWS keys, tokens, etc.

## Prerequisites

* Repository uses GitHub Actions with Docker available on runners.
* Basic familiarity with `docker compose`.
* Python 3 (for the helper script).

## Quick Start: Basic Pattern

This section demonstrates the core pattern using a simple curl command. Once you understand the mechanics, see the production examples below for real-world use cases.

### Directory Layout

The snippets below assume this directory layout:

```
.
├── docker-compose.ci.yaml
├── qtap-ci.yaml
├── scripts/ci/verify_allowed_hosts.py
└── .github/workflows/qtap-security.yml
```

### Step 1 – Qtap Configuration (`qtap-ci.yaml`)

```yaml
version: 2

services:
  # Stream metadata to stdout so the pipeline can collect it
  event_stores:
    - type: stdout

  # Stream HTTP request/response summaries to stdout as well
  object_stores:
    - type: stdout

stacks:
  ci_monitoring:
    plugins:
      - type: http_capture
        config:
          level: summary
          format: json
          rules:
            # Flag suspicious traffic loudly in the logs
            - name: "ALERT: Disallowed host"
              expr: |
                http.req.host != "localhost" &&
                http.req.host != "localhost:8000" &&
                http.req.host != "approved-api" &&
                http.req.host != "approved-api:8000"
              level: full

tap:
  direction: all
  ignore_loopback: false
  http:
    stack: ci_monitoring
```

We're using `format: json` to make downstream parsing easy.

### Step 2 – Compose File (`docker-compose.ci.yaml`)

```yaml
version: '3.9'

services:
  approved-api:
    image: kennethreitz/httpbin
    container_name: approved-api
    ports:
      - "8000:80"

  qtap:
    image: us-docker.pkg.dev/qpoint-edge/public/qtap:v0
    container_name: qtap-ci
    privileged: true
    user: "0:0"
    cap_add:
      - CAP_BPF
      - CAP_SYS_ADMIN
    pid: host
    network_mode: host
    volumes:
      - /sys:/sys
      - /var/run/docker.sock:/var/run/docker.sock
      - ./qtap-ci.yaml:/app/config/qtap.yaml
    environment:
      - TINI_SUBREAPER=1
    ulimits:
      memlock: -1
    command:
      - --log-level=info
      - --log-encoding=console
      - --config=/app/config/qtap.yaml
```

The GitHub runner host can reach `approved-api` on `localhost:8000`, and Qtap observes both the host traffic and container cross-talk.

### Step 3 – Verification Script (`scripts/ci/verify_allowed_hosts.py`)

```python
#!/usr/bin/env python3
"""
Fail the pipeline if Qtap saw traffic to hosts outside the allow list.
"""
from __future__ import annotations

import re
import subprocess
import sys
from pathlib import Path

QTAP_CONTAINER = "qtap-ci"
ALLOWED = {"localhost:8000", "localhost", "approved-api", "approved-api:8000"}
LOG_PATH = Path("artifacts/qtap.log")

# Match the "authority" field from Qtap's JSON output (e.g., "authority": "example.com")
AUTHORITY_REGEX = re.compile(r'"authority"\s*:\s*"([^"]+)"')


def collect_logs() -> str:
    result = subprocess.run(
        ["docker", "logs", QTAP_CONTAINER],
        capture_output=True,
        text=True,
        check=True,
    )
    LOG_PATH.parent.mkdir(parents=True, exist_ok=True)
    LOG_PATH.write_text(result.stdout, encoding="utf-8")
    return result.stdout


def extract_hosts(log_text: str) -> list[str]:
    return [match.group(1) for match in AUTHORITY_REGEX.finditer(log_text)]


def main() -> int:
    log_text = collect_logs()
    hosts = extract_hosts(log_text)
    disallowed = sorted(set(h for h in hosts if h not in ALLOWED))

    if disallowed:
        print("Disallowed hosts detected by Qtap:", file=sys.stderr)
        for host in disallowed:
            print(f"  - {host}", file=sys.stderr)
        print(f"\nQtap logs saved to {LOG_PATH}", file=sys.stderr)
        return 1

    print("Qtap host allow-list check passed.")
    return 0


if __name__ == "__main__":
    sys.exit(main())
```

The script downloads the entire Qtap log, saves it as an artifact, and fails if any host falls outside the allowed set.

### Step 4 – GitHub Actions Workflow (`.github/workflows/qtap-security.yml`)

```yaml
name: Validate Outbound Traffic

on:
  pull_request:
  push:
    branches: [ main ]

jobs:
  qtap-network-guard:
    runs-on: ubuntu-latest
    timeout-minutes: 15

    steps:
      - uses: actions/checkout@v4

      - name: Start Qtap and approved API
        run: docker compose -f docker-compose.ci.yaml up -d qtap approved-api

      - name: Warm up Qtap
        run: sleep 6

      - name: Generate expected traffic
        run: |
          docker run --rm --network host curlimages/curl:8.10.1 \
            --silent --show-error http://localhost:8000/get > /dev/null

      - name: (Optional) Generate forbidden traffic example
        if: env.QTAP_NEGATIVE_TEST == 'true'
        run: |
          docker run --rm curlimages/curl:8.10.1 \
            --silent --show-error https://example.com > /dev/null

      - name: Verify allowed hosts
        run: python scripts/ci/verify_allowed_hosts.py

      - name: Upload Qtap logs
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: qtap-logs
          path: artifacts/qtap.log

      - name: Tear down
        if: always()
        run: docker compose -f docker-compose.ci.yaml down -v
```

Set `QTAP_NEGATIVE_TEST=true` in workflow dispatch to watch the job fail on purpose. In normal runs you’d omit that step, keeping the allow-list tight.

## Understanding the Qtap Output

Qtap emits one JSON object per line for each HTTP transaction. Here's what the output looks like:

### Allowed Host Example

Traffic to `localhost:8000` (on the allow-list):

```json
{
  "metadata": {
    "process_id": "57081",
    "process_exe": "/usr/bin/curl",
    "bytes_sent": 81,
    "bytes_received": 419,
    "connection_id": "d3rvps07p3qhrpga199g",
    "endpoint_id": "localhost"
  },
  "request": {
    "method": "GET",
    "url": "http://localhost:8000/get",
    "scheme": "http",
    "path": "/get",
    "authority": "localhost:8000",
    "protocol": "http1",
    "request_id": "d3rvps07p3qhrpga19ag",
    "user_agent": "curl/8.10.1"
  },
  "response": {
    "status": 200,
    "content_type": "application/json"
  },
  "transaction_time": "2025-10-21T21:36:48.691937519Z",
  "duration_ms": 2,
  "direction": "egress-external"
}
```

Notice the `"authority": "localhost:8000"` field—the verification script extracts this to confirm it's on the allow-list. This request shows **summary level** capture (no `headers` or `body` fields) because it doesn't match the "Disallowed host" rule.

### Disallowed Host Example

Traffic to `example.com` (NOT on the allow-list) - captured at **full level** because it triggers the "Disallowed host" rule:

```json
{
  "metadata": {
    "process_id": "57156",
    "process_exe": "/usr/bin/curl",
    "bytes_sent": 37,
    "bytes_received": 679,
    "connection_id": "d3rvpso7p3qhrpga1a80",
    "endpoint_id": "example.com"
  },
  "request": {
    "method": "GET",
    "url": "https://example.com/",
    "scheme": "https",
    "path": "/",
    "authority": "example.com",
    "protocol": "http2",
    "request_id": "d3rvpso7p3qhrpga1a8g",
    "user_agent": "curl/8.10.1",
    "headers": {
      ":authority": "example.com",
      ":method": "GET",
      ":path": "/",
      ":scheme": "https",
      "Accept": "*/*",
      "User-Agent": "curl/8.10.1"
    }
  },
  "response": {
    "status": 200,
    "content_type": "text/html",
    "headers": {
      ":status": "200",
      "Alt-Svc": "h3=\":443\"; ma=93600",
      "Cache-Control": "max-age=86000",
      "Content-Length": "513",
      "Content-Type": "text/html",
      "Date": "Tue, 21 Oct 2025 21:36:51 GMT",
      "Etag": "\"bc2473a18e003bdb249eba5ce893033f:1760028122.592274\"",
      "Last-Modified": "Thu, 09 Oct 2025 16:42:02 GMT"
    },
    "body": "PCFkb2N0eXBlIGh0bWw+PGh0bWwgbGFuZz0iZW4iPjxoZWFkPjx0aXRsZT5FeGFtcGxlIERvbWFpbjwvdGl0bGU+PG1ldGEgbmFtZT0idmlld3BvcnQiIGNvbnRlbnQ9IndpZHRoPWRldmljZS13aWR0aCwgaW5pdGlhbC1zY2FsZT0xIj48c3R5bGU+Ym9keXtiYWNrZ3JvdW5kOiNlZWU7d2lkdGg6NjB2dzttYXJnaW46MTV2aCBhdXRvO2ZvbnQtZmFtaWx5OnN5c3RlbS11aSxzYW5zLXNlcmlmfWgxe2ZvbnQtc2l6ZToxLjVlbX1kaXZ7b3BhY2l0eTowLjh9YTpsaW5rLGE6dmlzaXRlZHtjb2xvcjojMzQ4fTwvc3R5bGU+PGJvZHk+PGRpdj48aDE+RXhhbXBsZSBEb21haW48L2gxPjxwPlRoaXMgZG9tYWluIGlzIGZvciB1c2UgaW4gZG9jdW1lbnRhdGlvbiBleGFtcGxlcyB3aXRob3V0IG5lZWRpbmcgcGVybWlzc2lvbi4gQXZvaWQgdXNlIGluIG9wZXJhdGlvbnMuPHA+PGEgaHJlZj0iaHR0cHM6Ly9pYW5hLm9yZy9kb21haW5zL2V4YW1wbGUiPkxlYXJuIG1vcmU8L2E+PC9kaXY+PC9ib2R5PjwvaHRtbD4K"
  },
  "transaction_time": "2025-10-21T21:36:51.525118272Z",
  "duration_ms": 47,
  "direction": "egress-external"
}
```

Here `"authority": "example.com"` triggers the verification script to fail because `example.com` is not in the `ALLOWED` set.

### Key Fields to Note

* `metadata.process_exe` – Identifies which process made the request (`/usr/bin/curl` in these examples)
* `request.authority` – The host:port being accessed (what the verification script checks)
* `direction` – `egress-external` means outbound to external hosts
* `request.headers` – Only present at `level: full` (disallowed hosts in this example)
* `response.headers` – Only present at `level: full`
* `response.body` – Only present at `level: full` (base64-encoded)

**Capture Levels:**

* **Summary level** (allowed hosts): Basic metadata only - no headers or bodies
* **Full level** (disallowed hosts): Complete request/response including headers and bodies

If the job fails, download `qtap-logs` from the Actions run to inspect exactly which host triggered the alert.

## Production Examples

Now that you understand the basic pattern, here's how to apply it to real-world scenarios.

### Example 1: Validate npm install

**Scenario:** Ensure `npm install` only fetches packages from your approved registries (npmjs.com, your internal registry).

**Complete Qtap configuration (`qtap-npm.yaml`):**

```yaml
version: 2

services:
  event_stores:
    - type: stdout
  object_stores:
    - type: stdout

stacks:
  npm_monitoring:
    plugins:
      - type: http_capture
        config:
          level: summary
          format: json
          rules:
            - name: "ALERT: Unexpected npm registry"
              expr: |
                http.req.host != "registry.npmjs.org" &&
                http.req.host != "registry.yarnpkg.com" &&
                http.req.host != "npm.company.internal"
              level: full

tap:
  direction: all
  ignore_loopback: false
  http:
    stack: npm_monitoring
```

**Workflow changes:**

```yaml
- name: Start Qtap
  run: docker compose -f docker-compose.ci.yaml up -d qtap

- name: Warm up Qtap
  run: sleep 6

- name: Install dependencies
  run: npm ci

- name: Verify npm only used approved registries
  run: python scripts/ci/verify_allowed_hosts.py
  env:
    QTAP_ALLOWED_HOSTS: "registry.npmjs.org,registry.yarnpkg.com,npm.company.internal"
```

**What this catches:**

* Malicious packages that phone home during install scripts
* Typosquatting packages from unexpected registries
* Dependency confusion attacks using public registries when internal ones expected

### Example 2: Validate Integration Tests

**Scenario:** Your app has integration tests that should only call staging APIs, never production.

**Complete Qtap configuration (`qtap-integration-tests.yaml`):**

```yaml
version: 2

services:
  event_stores:
    - type: stdout
  object_stores:
    - type: stdout

stacks:
  integration_test_monitoring:
    plugins:
      - type: http_capture
        config:
          level: summary
          format: json
          rules:
            - name: "ALERT: Production API called during tests"
              expr: |
                http.req.host matches /^.*\.company\.com$/ &&
                http.req.host != "api.staging.company.com" &&
                http.req.host != "auth.staging.company.com" &&
                http.req.host != "api.test.company.com" &&
                http.req.host != "auth.test.company.com"
              level: full

tap:
  direction: all
  ignore_loopback: false
  http:
    stack: integration_test_monitoring
```

**Workflow changes:**

```yaml
- name: Start Qtap and test dependencies
  run: docker compose -f docker-compose.ci.yaml up -d

- name: Warm up Qtap
  run: sleep 6

- name: Run integration tests
  run: npm test

- name: Verify tests only called staging
  run: python scripts/ci/verify_allowed_hosts.py
  env:
    QTAP_ALLOWED_HOSTS: "api.staging.company.com,auth.staging.company.com,localhost,localhost:3000"
```

**What this catches:**

* Hardcoded production URLs accidentally committed
* Environment variable misconfigurations
* Tests that modify production data

### Example 3: Validate Docker Build

**Scenario:** Your Dockerfile runs `apt-get`, `curl`, `wget` - ensure they only fetch from approved mirrors.

**Complete Qtap configuration (`qtap-docker-build.yaml`):**

```yaml
version: 2

services:
  event_stores:
    - type: stdout
  object_stores:
    - type: stdout

stacks:
  docker_build_monitoring:
    plugins:
      - type: http_capture
        config:
          level: summary
          format: json
          rules:
            - name: "ALERT: Docker build fetched from unexpected host"
              expr: |
                http.req.host != "deb.debian.org" &&
                http.req.host != "security.debian.org" &&
                http.req.host != "archive.ubuntu.com" &&
                http.req.host != "security.ubuntu.com" &&
                http.req.host != "github.com" &&
                http.req.host != "mirror.company.internal"
              level: full

tap:
  direction: all
  ignore_loopback: false
  http:
    stack: docker_build_monitoring
```

**Workflow changes:**

```yaml
- name: Start Qtap
  run: docker compose -f docker-compose.ci.yaml up -d qtap

- name: Warm up Qtap
  run: sleep 6

- name: Build Docker image
  run: docker build -t myapp:ci .

- name: Verify build only used approved mirrors
  run: python scripts/ci/verify_allowed_hosts.py
  env:
    QTAP_ALLOWED_HOSTS: "deb.debian.org,security.debian.org,archive.ubuntu.com,github.com,mirror.company.internal"
```

**What this catches:**

* Malicious RUN commands in Dockerfile
* Compromised base images with modified package sources
* wget/curl commands fetching from unexpected locations

## Adapting to Your Environment

### Update the Allow-List

The verification script accepts environment variables for flexibility:

```bash
export QTAP_ALLOWED_HOSTS="host1.com,host2.com,localhost:8080"
python scripts/ci/verify_allowed_hosts.py
```

### Add More Rulekit Rules

Beyond host validation, you can flag other suspicious patterns:

```yaml
rules:
  # Flag requests with Authorization headers (potential secret leak)
  - name: "ALERT: Authorization header detected"
    expr: http.req.headers.authorization != ""
    level: full

  # Flag POST requests (writes/uploads)
  - name: "ALERT: POST request to external host"
    expr: |
      http.req.method == "POST" &&
      http.req.host != "localhost" &&
      http.req.host != "approved-api"
    level: full

  # Flag non-TLS traffic to external hosts
  - name: "ALERT: Unencrypted external traffic"
    expr: |
      http.req.scheme == "http" &&
      http.req.host != "localhost" &&
      http.req.host != "approved-api"
    level: full
```

{% hint style="warning" %}
**Rulekit Limitation:** The `not` operator does not currently work in rule expressions or macros. Use explicit negation with `!=` and `&&` operators instead.

For example, instead of:

```yaml
expr: not (http.req.host == "localhost" || http.req.host == "approved-api")
```

Use:

```yaml
expr: http.req.host != "localhost" && http.req.host != "approved-api"
```

{% endhint %}

### Archive for Compliance

For HIPAA/SOC2/PCI compliance, archive Qtap logs permanently:

```yaml
- name: Upload Qtap logs to S3 (compliance archive)
  if: always()
  run: |
    aws s3 cp artifacts/qtap.log \
      s3://compliance-artifacts/ci-logs/${{ github.run_id }}/qtap.log \
      --metadata "repo=${{ github.repository }},commit=${{ github.sha }}"
```

## Summary

With this pattern, network policy drift becomes visible during every pull request—long before changes reach production. You can:

* **Prevent supply chain attacks** by validating package manager traffic
* **Enforce environment boundaries** by blocking accidental production calls
* **Detect secrets exfiltration** by flagging unexpected outbound requests
* **Generate compliance evidence** with complete network audit trails

Continuous Integration now enforces both functional AND security expectations.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.qpoint.io/guides/qtap-guides/advanced-use-cases/ci-security-validation-with-qtap.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.