Production Debugging with HTTPS Visibility

Get HTTPS traffic visibility in 30 seconds for urgent production debugging - no permanent installation needed.

Who This Is For

Use this guide when you:

Have a production issue RIGHT NOW that needs immediate investigation
Need to see inside HTTPS traffic without certificates or proxies
Want to debug authentication failures (401/403), API errors, or rate limiting
Need temporary visibility without permanent installation

This IS for:

Emergency troubleshooting and incident response
Short-term traffic capture (hours to days)
One-off debugging sessions
Proof that Qtap works in your environment

This is NOT for:

Long-term production monitoring (use Complete Guide for that)
Selective capture with complex rules (use Complete Guide for rulekit)
Multi-environment visibility (consider Qplane)

Time to results: 30 seconds (demo mode) or 5 minutes (full install)

🔒 Your Data Stays Local
This guide uses standalone Qtap with stdout output—all captured traffic stays on your server. No data is sent to Qpoint or any external service. Everything you capture remains in your terminal or log files under your control.
Want centralized visibility across a fleet with alerting? That's Qplane (our cloud control plane), but this guide keeps everything local and private.

Temporary Installation

The fastest way to see HTTPS traffic. No installation, no configuration file—just one command that downloads qtap to /tmp and starts capturing immediately.

One-Line Demo

SSH into your production server and run:

curl -s https://get.qpoint.io/demo | sudo LOG_LEVEL=warn sh

Prefer not to pipe curl to sudo? We get it. See manual installation instructions to download and verify the binary yourself before running.

Qtap will:

Download the latest qtap binary to /tmp (no permanent installation)
Start capturing all HTTP/HTTPS traffic with full visibility
Display readable console output in real-time (access_logs plugin, full mode)

What You'll See

Captured traffic appears immediately:

=============================================================================================================
■ /usr/bin/curl → GET https://httpbin.org/get 200 OK
=============================================================================================================

------------------ META ------------------
PID: 12345
Exe: /usr/bin/curl
Direction: egress-external
Bytes Sent: 41
Bytes Received: 395

------------------ REQUEST ------------------
GET httpbin.org http2
:method: GET
:path: /get
:scheme: https
User-Agent: curl/8.10.1
Accept: */*
:authority: httpbin.org

------------------ REQUEST BODY ------------------
(empty)

------------------ RESPONSE ------------------
200 OK
Content-Type: application/json
Content-Length: 255
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
:status: 200
Date: Wed, 29 Oct 2025 19:51:23 GMT

------------------ RESPONSE BODY ------------------
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Host": "httpbin.org",
    "User-Agent": "curl/8.10.1"
  },
  "origin": "73.71.138.108",
  "url": "https://httpbin.org/get"
}

Stop the Demo

Press Ctrl+C to stop qtap. The binary remains in /tmp and will be cleaned up on system reboot.

Pro tip: The demo script uses a default configuration (access_logs, full mode, all traffic). For production-safe filtering and custom rules, continue to the sections below.

Full Installation

This option gives you full control with custom configuration. Perfect for production-safe debugging with filters and rules.

Prerequisites

SSH access to production server (Linux with kernel 5.10+)
Root/sudo privileges
Running application to debug

Step 1: Create Minimal Configuration

SSH into your production server and create /tmp/qtap-debug.yaml:

# Minimal production debugging configuration
# Captures all HTTP traffic with readable console output
version: 2

services:
  event_stores:
    - type: stdout  # Connection metadata to console
  object_stores:
    - type: stdout  # Request/response bodies to console

stacks:
  debug:
    plugins:
      - type: access_logs
        config:
          mode: full      # (summary|details|full) - Capture everything
          format: console # (console|json) - Human-readable output

tap:
  direction: all          # (egress|egress-external|egress-internal|ingress|all)
  ignore_loopback: true   # Skip localhost traffic
  audit_include_dns: false  # Skip DNS queries
  http:
    stack: debug

Step 2: Install and Start Qtap

Install qtap with a single command:

# Install qtap
curl -s https://get.qpoint.io/install | sudo sh

# Verify installation
qtap --version

Prefer not to pipe curl to sudo? See manual installation instructions to download, extract, and verify the binary yourself.

Start qtap (runs in foreground, shows captured traffic in real-time):

sudo qtap --config=/tmp/qtap-debug.yaml --log-level=warn --log-encoding=console | tee /tmp/qtap-output.log

Qtap displays captured HTTPS traffic in real-time. Output is also saved to /tmp/qtap-output.log for later analysis.

Step 3: Generate Test Traffic

Open a second SSH session to your server (keep qtap running in the first session), then generate test traffic:

# Test authentication failure (returns 401)
curl https://httpbin.org/status/401

# Test successful request (returns 200)
curl https://httpbin.org/get

# Test error response (returns 500)
curl https://httpbin.org/status/500

Or simply wait for your production application to generate traffic—you'll see it appear in real-time in your first SSH session.

Step 4: What You'll See

Readable console output (example 401 error from httpbin):

=========================================================================
■ /usr/bin/curl → GET https://httpbin.org/status/401 401 Unauthorized
=========================================================================

------------------ META ------------------
PID: 12345
Exe: /usr/bin/curl
Direction: egress-external
Bytes Sent: 54
Bytes Received: 0

------------------ REQUEST ------------------
GET httpbin.org http2
User-Agent: curl/8.10.1
Accept: */*
:authority: httpbin.org
:method: GET
:path: /status/401
:scheme: https

------------------ RESPONSE ------------------
401 Unauthorized
:status: 401
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Content-Length: 0
Date: Wed, 29 Oct 2025 19:30:00 GMT
Server: gunicorn/19.9.0

------------------ RESPONSE BODY ------------------
(empty)

Key indicators of successful capture:

✅ Process identified (/usr/bin/curl)
✅ Request headers visible (including Authorization token!)
✅ Response body readable (no base64 encoding)
✅ Complete HTTPS traffic despite encryption

Step 5: Cleanup

When debugging is complete, press Ctrl+C in the qtap session to stop it, then:

# Remove configuration and logs
rm /tmp/qtap-debug.yaml /tmp/qtap-output.log

Production Debugging Configuration

The quick start captures everything at full detail. For production use, this single configuration works for all common debugging scenarios—just adjust what you look for in the output.

Need advanced capture options? This guide uses the access_logs plugin for simplicity. For selective capture with rules, custom filtering, or other plugins (like http_capture), see Traffic Processing with Plugins.

Base Configuration

Create /tmp/qtap-debug.yaml:

# Production debugging configuration
# Captures all HTTP/HTTPS traffic with full details
version: 2

services:
  event_stores:
    - type: stdout
  object_stores:
    - type: stdout

stacks:
  debug:
    plugins:
      - type: access_logs
        config:
          mode: full        # (summary|details|full) - Full capture for debugging
          format: console   # (console|json) - Readable output

tap:
  direction: all            # (egress|egress-external|egress-internal|ingress|all)
  ignore_loopback: true     # Skip localhost traffic
  audit_include_dns: false  # Skip DNS queries
  http:
    stack: debug

Start capturing:

sudo qtap --config=/tmp/qtap-debug.yaml --log-level=warn --log-encoding=console | tee /tmp/qtap-output.log

Now use a second SSH session to generate traffic (or just wait for your production app to make or receive requests).

Common Debugging Scenarios

Same config, different things to look for in the captured output.

Scenario 1: Authentication Failures

Problem: Third-party API returning 401/403 errors

Generate test traffic:

curl https://api.stripe.com/v1/customers -H "Authorization: Bearer sk_test_invalid"
curl https://httpbin.org/status/403

What to look for in qtap output:

Status codes: 401 Unauthorized or 403 Forbidden
Authorization header: Check if token is present and formatted correctly
Response body: Error message often explains the issue (expired token, wrong scope, etc.)
Process: Which service is making the failing request (exe: /usr/bin/python3, container name)

Scenario 2: Slow API Responses

Problem: External APIs taking too long to respond

What to look for in qtap output:

Duration: Response time shown in milliseconds at top of each capture
Bytes Sent/Received: Large payloads can cause slowness
Process: Which service is making slow requests
Host: Which external API is slow

Pro tip: Use mode: summary instead of full if you don't need bodies—just timing data.

Scenario 3: Rate Limiting

Problem: Hitting API rate limits (429 responses)

Generate test traffic:

curl https://httpbin.org/status/429

What to look for in qtap output:

Status code: 429 Too Many Requests
Rate-limit headers: X-RateLimit-Remaining, X-RateLimit-Reset, Retry-After
Request frequency: Count how many requests to the same host in a short time window
Process: Which service is making too many requests

Scenario 4: External API Errors

Problem: Third-party API integration suddenly failing

What to look for in qtap output:

Status codes: 500, 502, 503, 504 (server errors)
Error response bodies: Often contain specific error messages or codes
Request bodies: Verify payload format is correct
Headers: Check required headers are present (Content-Type, API version headers)
Process: Which service is affected

Reducing Noise with Filters

In busy production environments, you'll capture a lot of traffic. Use filters to focus on what matters.

Filter by Process

Ignore noisy health checks, monitoring agents, or specific services:

version: 2

services:
  event_stores:
    - type: stdout
  object_stores:
    - type: stdout

stacks:
  debug:
    plugins:
      - type: access_logs
        config:
          mode: full
          format: console

tap:
  direction: all
  ignore_loopback: true
  audit_include_dns: false
  http:
    stack: debug

  filters:
    groups:
      - qpoint              # Don't capture qtap's own traffic
    custom:
      - exe: /usr/bin/health-check
        strategy: exact     # Exact path match
      - exe: /usr/bin/
        strategy: prefix    # All executables in /usr/bin/
      - exe: .*node.*
        strategy: regex     # Any path matching pattern

Common filters:

Health checks: /usr/bin/health-check, /usr/bin/curl (if used for monitoring)
Monitoring agents: .*datadog.*, .*newrelic.*
Internal tools: /usr/local/bin/metrics-collector

Filter by Domain

Capture only specific domains by setting the default stack to none and overriding for specific endpoints:

version: 2

services:
  event_stores:
    - type: stdout
  object_stores:
    - type: stdout

stacks:
  debug:
    plugins:
      - type: access_logs
        config:
          mode: full
          format: console

tap:
  direction: all
  ignore_loopback: true
  audit_include_dns: false

  # Default: capture nothing
  http:
    stack: none

  # Override: capture only these specific domains
  endpoints:
    - domain: 'api.stripe.com'
      http:
        stack: debug
    - domain: 'api.github.com'
      http:
        stack: debug

How it works: The default http.stack applies to all traffic. Specific domains under endpoints override the default. With stack: none as the default, only the listed domains are captured.

When to use domain filtering:

Debugging a specific third-party integration
High-volume environments where you need to focus
Investigating issues with a particular external service

Adjust Capture Level

Start with less detail, increase as needed by changing the mode value in your config:

version: 2

services:
  event_stores:
    - type: stdout
  object_stores:
    - type: stdout

stacks:
  debug:
    plugins:
      - type: access_logs
        config:
          mode: summary     # Change this: summary | details | full
          format: console

tap:
  direction: all
  ignore_loopback: true
  audit_include_dns: false
  http:
    stack: debug

Available modes:

summary: Method, URL, status, timing - no headers or bodies (minimal overhead)
details: Summary + all headers - no bodies (auth/rate-limit debugging)
full: Everything including headers and bodies (complete debugging)

Tip: Start with summary, look at the list of requests, then restart qtap with full mode to dig into specific issues.

Filter by Response Code

Focus on errors, authentication failures, or rate limiting by capturing only specific HTTP status codes:

version: 2

services:
  event_stores:
    - type: stdout
  object_stores:
    - type: stdout

rulekit:
  macros:
    - name: is_error
      expr: http.res.status >= 400

stacks:
  errors_only:
    plugins:
      - type: access_logs
        config:
          mode: summary     # Start with summary for all traffic
          format: console
          rules:
            - name: "Capture errors at full detail"
              expr: is_error()
              mode: full    # Upgrade to full mode for errors

tap:
  direction: all
  ignore_loopback: true
  audit_include_dns: false
  http:
    stack: errors_only

How it works: All traffic is captured at summary level (minimal overhead), but requests with status codes ≥ 400 are automatically upgraded to full mode with complete headers and bodies.

Common response code filters:

Authentication issues: http.res.status == 401 || http.res.status == 403
Rate limiting: http.res.status == 429
Client errors: http.res.status >= 400 && http.res.status < 500
Server errors: http.res.status >= 500

When to use response code filtering:

High-volume production debugging where you only care about failures
Authentication troubleshooting (capture 401/403 fully, ignore successful 200s)
Rate limit investigation (capture 429 responses with all headers)
Error diagnosis without capturing thousands of successful requests

Understanding Captured HTTPS Data

Qtap's eBPF hooks capture traffic at the kernel level before TLS encryption, giving you complete visibility into HTTPS traffic without proxies or certificate management.

What You Can See

Request details:

HTTP method (GET, POST, PUT, DELETE, etc.)
Full URL (scheme, host, path, query parameters)
All request headers (Authorization, Content-Type, custom headers)
Request body (when mode is full)

Response details:

Status code (200, 401, 500, etc.)
All response headers (Content-Type, rate-limit headers, cookies)
Response body (when mode is full)
Response timing (duration in milliseconds)

Process attribution (unique to eBPF-based capture):

Executable path (/usr/bin/curl, /usr/bin/python3, etc.)
Process ID (PID)
Container name (Docker/Kubernetes)
Container/pod labels

TLS information:

TLS detected (is_tls: true)
TLS library used (tlsProbeTypesDetected: ["openssl"])
Protocol version (http1, http2)

Capture Levels Explained

Level

What's Captured

Use Case

summary

Method, URL, status, timing, process info

High-level monitoring, minimal overhead

details

Summary + all headers (no bodies)

Debug headers, auth tokens, rate limits

full

Everything (headers + request/response bodies)

Complete debugging, error diagnosis

Example Captured Request (Full Mode)

This is what you'll see with mode: full and format: console:

=================================================================================
■ /usr/local/bin/python3 → POST https://api.stripe.com/v1/charges 402 Payment Required
=================================================================================

------------------ META ------------------
PID: 54321
Exe: /usr/local/bin/python3
Container: payment-service
Direction: egress-external
Bytes Sent: 156
Bytes Received: 287
Duration: 187ms

------------------ REQUEST ------------------
POST api.stripe.com http2
:method: POST
:path: /v1/charges
:scheme: https
:authority: api.stripe.com
Authorization: Bearer sk_test_...
Content-Type: application/x-www-form-urlencoded
Stripe-Version: 2023-10-16

------------------ REQUEST BODY ------------------
amount=2000&currency=usd&source=tok_visa

------------------ RESPONSE ------------------
402 Payment Required
:status: 402
Content-Type: application/json
Request-Id: req_abc123
Stripe-Version: 2023-10-16

------------------ RESPONSE BODY ------------------
{
  "error": {
    "type": "card_error",
    "code": "card_declined",
    "message": "Your card was declined."
  }
}

Production Safety

Performance

Start with mode: summary for minimal overhead
Use process or domain filters to reduce volume
Typical overhead: 1-3% CPU

Sensitive Data

Captured data may contain API keys and PII. For quick debugging:

Data stays on your server in /tmp
Remove logs when done: rm /tmp/qtap-*.log

Cleanup

# Stop qtap (Ctrl+C in qtap session)
rm /tmp/qtap-*.yaml /tmp/qtap-*.log

Need help? For additional resources:

Traffic Processing with Plugins - Complete plugin and rules syntax
Linux Kernel Lockdown for eBPF Applications - Permission issues
Contact [email protected]

Previous5-Minute Quickstart NextComplete Guide: Hello World to Production

Last updated 3 days ago