Stacks & Plugins

This guide explains how to configure traffic processing in Qplane using Stacks and Plugins. When you use Qplane (cloud-connected mode), you manage stacks visually through the UI instead of editing YAML files directly.

Understanding Stacks & Plugins

Stacks are named collections of plugins that work together to process your traffic. Think of a stack as a processing pipeline: traffic flows in, plugins process it according to their rules, and results flow out to your configured destinations.

Plugins are individual processing components that perform specific functions:

  • Report Usage: Send anonymized metrics to Qplane for analytics and dashboards

  • Detect Errors: Capture detailed information when errors occur and upload to your object store

  • Scan Payloads & Data: Classify sensitive data (PII, credentials, secrets) using Qscan

  • Access Logs: Write formatted logs to stdout (console output)

  • HTTP Metrics: Expose Prometheus metrics for HTTP traffic

You can create multiple stacks for different traffic types. For example, use one stack for general traffic and another for payment APIs that need extra scrutiny.

The Default Configuration

When you first set up Qplane, a default stack called "Basic - Reporting and Error Detection" is automatically created. This stack includes these plugins:

Report Usage

Sends anonymized usage metrics to Qplane (Pulse) so you can see traffic patterns, vendor analytics, and endpoint usage in your dashboards.

What it does:

  • Tracks connection counts, request volumes, and response times

  • Groups metrics by vendor, endpoint, and client

  • Powers the Traffic, Inventory, and Dashboard pages

  • Required for alerting to work

Configuration: None required. This plugin runs automatically when added to a stack.

Detect Errors

Captures full request/response details when HTTP errors occur and uploads them to your object store for debugging.

What it does:

  • Monitors HTTP status codes

  • When a matching error occurs, captures headers and bodies

  • Uploads captured data to your configured object store

  • Optionally reports the error as an "issue" in Qplane for visibility

Default Rules:

The default stack includes these pre-configured error detection rules:

  1. App Error (500)

    • Triggers on: 500 Internal Server Error

    • Captures: Request headers, body, response headers, body

    • Reports as issue: Yes

  2. Infrastructure Outage (502, 503, 520-523, 525-526, 530)

    • Triggers on: Bad Gateway, Service Unavailable, Cloudflare errors

    • Captures: Full request/response details

    • Reports as issue: Yes

  3. Client Error (400)

    • Triggers on: 400 Bad Request

    • Captures: Full request/response details

    • Reports as issue: Yes

  4. Authentication Error (401, 403, 407)

    • Triggers on: Unauthorized, Forbidden, Proxy Authentication Required

    • Captures: Full request/response details

    • Reports as issue: Yes

  5. Rate Limited (429)

    • Triggers on: 429 Too Many Requests

    • Captures: Full request/response details

    • Reports as issue: Yes

  6. Not Found (404)

    • Triggers on: 404 Not Found

    • Captures: Full request/response details

    • Reports as issue: Yes

Important: The Detect Errors plugin uploads payloads to your object store. Make sure you have configured an object store in Settings → Deploy → Services (by default, this will send to QPlane at app.qpoint.io).

Navigating the Plugins Interface

Viewing All Stacks

Location: Plugins → Stacks

Here you'll see all your configured stacks. By default, you'll only see:

  • Basic - Reporting and Error Detection: The default stack with metrics, reports, and error detection capabilities

You can create additional stacks for different purposes such as:

  • Sensitive Data Stack: For traffic requiring PII/credential scanning

  • Debug Stack: For detailed troubleshooting with full logging

  • Or any custom stack for your specific needs

Each stack shows:

  • Name

  • Description

  • "Manage" button to configure plugins

Creating a New Stack

  1. Click "+ New Stack" button (top right)

  2. Enter a name and description

  3. Click into the new stack to add plugins

  4. Click "+ Add Plugin" to see available plugins

  5. Select plugins and configure them

  6. Enable "Auto generate release on every change" to push updates to agents immediately

Managing Plugins in a Stack

When you click "Manage" on a stack, you see the stack's configuration interface:

Stack Settings Tab

Name: The stack identifier used in routing rules

Description: Human-readable description of the stack's purpose

Auto generate release on every change:

  • When enabled (recommended), changes propagate to connected agents upon saving

  • When disabled, changes are saved but not deployed until you manually create a release

Plugins Tab

This shows all plugins currently in the stack. You can:

  • Reorder plugins by dragging the handle icons

  • Click the plugin to configure individual plugin settings

    • Click "Delete Plugin" to remove

Reliability Tab (Rules)

This tab shows specialized rules that can trigger based on traffic patterns when the Detect Errors plugin is enabled. Default rules include:

  • Debug: Enable debug mode on all traffic (disabled by default)

  • App Error: Errors within the application

  • Infrastructure Outage: Networks down or infrastructure offline

  • Client Error: Malformed or bad requests

  • Authentication Error: Invalid authentication or unauthorized requests

  • Rate Limited: Rate limiting threshold exceeded

  • Not Found: Requested resource doesn't exist

  • Slow Response: Application taking too long to respond

Each rule can be enabled/disabled with a toggle and configured by clicking "Manage".

Available Plugins in Detail

1. Report Usage

Purpose: Powers Qplane analytics and dashboards

When to use:

  • Always include in production stacks

  • Required for Qplane's Traffic, Inventory, and Dashboard pages

  • Necessary for alerting to function

Configuration: None. Just add it to your stack.

Output: Sends anonymized event metadata to Pulse (ClickHouse)

YAML equivalent:

plugins:
  - type: report_usage

2. Detect Errors

Purpose: Capture and upload full request/response details when errors occur

When to use:

  • Debugging integration problems

  • Understanding why API calls fail

  • Compliance requirements to log errors

Important Note: While this plugin is marked as deprecated in the local YAML documentation (which recommends using http_capture or access_logs instead), Qplane still uses it extensively. Qplane uses detect_errors for:

  • Automatic error categorization (App Error, Infrastructure Outage, etc.)

  • Selective capture based on status codes

  • Issue reporting in the Qplane UI

  • Integration with the Reliability dashboard

Configuration Options:

Trigger Status Codes: Which HTTP status codes trigger capture

  • Individual codes: 404, 500

  • Ranges: 4xx, 5xx

Categories: Only capture errors from specific sources

  • app: Application errors only

  • Leave empty to capture all

Capture Settings:

  • Record request headers: Yes/No

  • Record request body: Yes/No

  • Record response headers: Yes/No

  • Record response body: Yes/No

Report as Issue: Flag the error for visibility in Qplane's issues list

How it Works:

  1. Traffic flows through the stack

  2. Detect Errors monitors HTTP status codes

  3. When a configured status code occurs (e.g., 500), the plugin captures the specified headers/bodies

  4. Data uploads to your configured object store

  5. If "Report as Issue" is enabled, the error appears in Qplane for investigation

YAML equivalent:

plugins:
  - type: detect_errors
    config:
      rules:
        - name: "Server Errors"
          trigger_status_codes:
            - '5xx'
          only_categories:
            - app
          report_as_issue: true
          record_req_headers: true
          record_req_body: true
          record_res_headers: true
          record_res_body: true

3. Scan Payloads & Data (Beta)

Purpose: Classify sensitive data in your traffic using Microsoft Presidio

When to use:

  • Compliance requirements (GDPR, CCPA, PCI-DSS, HIPAA)

  • Security audits of third-party data sharing

  • Understanding what sensitive data your apps send

What it detects (20+ types):

  • Personal: Names, emails, phone numbers, locations, dates of birth

  • Financial: Credit cards, crypto addresses, IBAN codes, bank accounts

  • Government IDs: SSNs, driver's licenses, passport numbers

  • Technical: IP addresses, URLs, API keys, credentials

  • Medical: License numbers

Configuration Options:

Cache Settings:

  • Cache TTL: How long to keep URLs in cache (default: 24h)

  • Cache Size: Maximum URLs to track (default: 4096)

Sampling:

  • Sample Baseline: Always sample first N requests per URL (default: 10)

  • Sample Rate: Percentage to sample after baseline (default: 0.9 = 90%)

Storage:

  • Record matching metadata: Store detection results (path, score) in object store

Data Types: Enable/disable detection for each type individually. For each enabled type:

  • Store value with metadata: Save the actual detected value (e.g., the email address) or just store that something was detected

How it Works:

  1. Traffic flows through the stack

  2. Qscan plugin sends payloads to your Qscan service

  3. Qscan uses Microsoft Presidio to analyze content

  4. Detected sensitive data is classified and optionally stored

  5. Results appear in Qplane for analysis

Important: Requires a Qscan service running in your environment. Configure in Settings → Deploy → Services → Qscan.

YAML equivalent:

plugins:
  - type: qscan
services:
  qscan:
    type: client
    url: https://qscan.internal.example.com
    token:
      type: env
      value: QSCAN_TOKEN

4. Access Logs

Purpose: Generate formatted HTTP access logs to stdout (console)

When to use:

  • Viewing traffic in real-time during development

  • Integration with log aggregation systems (Fluentd, Logstash)

  • Lightweight logging without object store uploads

Configuration Options:

Mode: Default logging level

  • summary: Method, path, status code

  • details: Includes headers

  • full: Complete headers and bodies

Format: Output style

  • console: Human-readable format

  • json: Structured JSON logs

Rules: Override the default mode for specific traffic (uses Rulekit expressions)

How it Works:

  1. Traffic flows through the stack

  2. Access Logs plugin formats each request

  3. Logs write to stdout (visible in agent logs)

  4. NO uploads to object store occur

YAML equivalent:

plugins:
  - type: access_logs
    config:
      mode: details
      format: console
      rules:
        - name: "Full logging for errors"
          expr: res.status >= 400
          mode: full

5. HTTP Metrics

Purpose: Expose Prometheus-compatible metrics for HTTP traffic

When to use:

  • Integration with existing Prometheus/Grafana monitoring

  • Custom metrics dashboards

  • Advanced observability pipelines

Configuration: Typically none. The plugin exposes metrics on the agent's metrics endpoint.

Metrics Exposed:

  • Request counts by method, status, endpoint

  • Response time histograms

  • Error rates

  • Bytes sent/received

YAML equivalent:

plugins:
  - type: http_metrics

Routing: Assigning Domains to Stacks

Location: Plugins → Routing

Routing lets you send specific domains to specific stacks. This enables fine-grained control over how different traffic is processed. By default, all traffic uses your default stack (Basic - Reporting and Error Detection), but you can create routing rules to send specific domains to custom stacks you've created.

Default Stack

All traffic not matching a specific endpoint rule uses this stack. By default, this is set to "Basic - Reporting and Error Detection", but you can select any stack you've created.

Endpoints

Map domains to stacks:

Example Setup:

Domain
Stack
Purpose

github.com

Debug Stack

Full logging for debugging

postman-echo.com

Sensitive Data Stack

Scan test payloads

echo.free.beeceptor.com

Sensitive Data Stack

Scan test traffic

httpbin.org

Sensitive Data Stack

Scan public test API

To Add an Endpoint:

  1. Click "+ Add Endpoint"

  2. Enter domain (e.g., api.stripe.com)

  3. Select stack from dropdown

  4. Click Save

Domain Matching:

  • Exact: api.stripe.com matches only that domain

Use Cases:

Payment APIs:

Domain: api.stripe.com
Stack: Sensitive Data Stack (with Qscan enabled)
Reason: Compliance requires PII detection on payment flows

Internal Services:

Domain: api.internal.company.com
Stack: Default Stack (basic reporting only)
Reason: Internal traffic doesn't need deep inspection

Debugging Specific Integrations:

Domain: problematic-vendor.com
Stack: Debug Stack (with Access Logs at full level)
Reason: Troubleshooting integration issues

Common Stack Configurations

Here are examples of stacks you might create for different use cases. Remember, only "Basic - Reporting and Error Detection" exists by default - you'll need to create these others yourself.

Basic - Reporting and Error Detection (Default)

Purpose: Standard visibility with error capture

Plugins:

  • Report Usage

  • Detect Errors (with all default rules)

Use Case: Most production deployments. This is created automatically.

Minimal Production Stack

Purpose: Basic visibility without payload capture

Plugins:

  • Report Usage

Use Case: Simple monitoring, minimal data storage. Good for high-volume internal traffic.

How to Create:

  1. Click "+ New Stack"

  2. Name it "Minimal Production" or similar

  3. Add only the "Report Usage" plugin

  4. Save

Compliance Stack

Purpose: Detect and classify sensitive data

Plugins:

  • Report Usage

  • Scan Payloads & Data (configured for required data types)

  • Detect Errors (for error investigation)

Use Case: GDPR, CCPA, PCI-DSS, HIPAA compliance

Routing: Apply to domains handling sensitive data (payment APIs, user data APIs)

How to Create:

  1. Click "+ New Stack"

  2. Name it "Compliance" or "Sensitive Data"

  3. Add "Report Usage" plugin

  4. Add "Scan Payloads & Data" plugin and configure data types

  5. Add "Detect Errors" plugin

  6. Save

  7. In Plugins → Routing, route sensitive domains to this stack

Debug Stack

Purpose: Maximum visibility for troubleshooting

Plugins:

  • Report Usage

  • Access Logs (mode: full, format: console)

  • Detect Errors (capture everything)

Use Case: Temporary debugging of specific integrations

Routing: Apply to problematic domains during investigation, then revert

How to Create:

  1. Click "+ New Stack"

  2. Name it "Debug"

  3. Add "Report Usage" plugin

  4. Add "Access Logs" plugin, set mode to "full"

  5. Add "Detect Errors" plugin

  6. Save

  7. Route specific problematic domains to this stack when needed

Ignore Stack

Purpose: Disable processing for specific traffic

Plugins: None

Use Case:

  • High-volume internal traffic that doesn't need monitoring

  • Reducing data storage costs

  • Excluding specific domains from analysis (health checks, monitoring endpoints)

Routing: Apply to domains you want to ignore

How to Create:

  1. Click "+ New Stack"

  2. Name it "Ignore"

  3. Don't add any plugins

  4. Save

  5. Route noisy domains to this stack

Configuration Workflow

1. Plan Your Stacks

The default "Basic - Reporting and Error Detection" stack works well for most traffic, but consider creating additional stacks for special cases:

Questions to ask:

  • Do we need PII detection for compliance? → Create a Compliance/Sensitive Data stack

  • Which vendors handle our most sensitive data? → Route those domains to the Compliance stack

  • Are we troubleshooting specific integrations? → Create a Debug stack temporarily

  • Do we have high-volume health checks or internal endpoints? → Create an Ignore stack

  • What's our data retention strategy? → Minimal stacks reduce storage costs

2. Create Stacks in Qplane

Start with the default "Basic - Reporting and Error Detection" stack. Create additional stacks only when you need different processing:

  1. Navigate to Plugins → Stacks

  2. Click "+ New Stack" for each additional stack you need

  3. Name descriptively (e.g., "Payment APIs - PII Scanning", "Debug Mode", "Ignore Health Checks")

  4. Add relevant plugins to each stack

  5. Configure plugin settings

3. Configure Plugins

For each plugin:

  1. Click "Manage" to open configuration

  2. Set appropriate options (capture levels, data types, etc.)

  3. Save changes

4. Set Up Routing

  1. Navigate to Plugins → Routing

  2. Verify your default stack (usually "Basic - Reporting and Error Detection")

  3. Add endpoint rules only for domains that need different processing than the default

  4. Test with a few domains first before expanding

Example: If you created a "Sensitive Data" stack with Qscan enabled, route your payment API domain to it:

  • Domain: api.stripe.com

  • Stack: Sensitive Data

  • Result: Payment API traffic gets PII scanning; everything else uses default stack

5. Deploy and Monitor

  1. Enable "Auto generate release on every change" for production stacks

  2. Watch the Snapshot YAML to see configuration updates

  3. Monitor Traffic and Inventory pages to verify agents are processing correctly

  4. Adjust rules and thresholds based on observed patterns

6. Iterate

As you learn more about your traffic:

  • Refine routing rules

  • Adjust capture levels to balance visibility with storage costs

  • Add or remove plugins based on changing needs

  • Create new stacks for new use cases

Services Configuration

Qplane automatically configures these services for cloud-connected agents:

services:
  event_stores:
    - id: qpoint_cloud
      type: pulse
      url: https://api-pulse.qpoint.io
      token: <auto-generated>
  
  object_stores:
    - id: qpoint_cloud
      type: qpoint
      url: https://warehouse.qpoint.io
      token: <auto-generated>
  
  qscan:
    type: client
    url: https://api-s.qpoint.io

You don't configure these manually; they're managed by Qplane. However, you can add your own object stores and Qscan instances in Settings → Deploy → Services.

PreviousHow It Fits TogetherNextSettings

Last updated