Self-Hosted Setup: PII Scanning with Local Storage

Set up a complete QScan pipeline on your local machine -- from object storage to PII detections in the Pulse dashboard. This is the fastest way to see QScan in action.

Who This Is For

Use this guide if you want to:

Try QScan for the first time
See the full scanning pipeline working end-to-end
Evaluate PII detection capabilities before a production rollout

Choose something else if you:

Already have Qtap and object storage running -- see the QScan Installation docs
Want production Kubernetes deployment -- see Kubernetes Installation

Time to complete: 30 minutes

Prerequisites

Before you begin, make sure you have:

Docker and Docker Compose installed and running
A registration token from app.qpoint.io (Settings -> API Tokens)
Basic familiarity with YAML configuration

The registration token connects Qtap and QScan to the Pulse service, which coordinates scan jobs and displays results. You can create one from the Settings page in the Qpoint dashboard.

Step 1: Set Up Object Storage

QScan reads captured HTTP artifacts from S3-compatible object storage. For this guide, you will use MinIO as a lightweight local store.

Create a project directory and add a docker-compose.yml:

mkdir qscan-quickstart && cd qscan-quickstart

Add the MinIO service to docker-compose.yml:

docker-compose.yml

services:
  minio:
    image: minio/minio:latest
    container_name: minio
    command: server /data --console-address ":9001"
    ports:
      - "9000:9000"
      - "9001:9001"
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    volumes:
      - minio-data:/data

volumes:
  minio-data:

Start MinIO:

docker compose up -d minio

Create the bucket that Qtap will write artifacts to:

docker run --rm --network host \
  --entrypoint sh minio/mc -c "
    mc alias set local http://localhost:9000 minioadmin minioadmin &&
    mc mb local/qpoint --ignore-existing
  "

You now have an S3-compatible store running at http://localhost:9000 with a qpoint bucket.

Step 2: Configure Qtap for Artifact Capture

Create a Qtap configuration that captures HTTP traffic and stores artifacts in MinIO with the QScan plugin enabled.

qtap.yaml

version: 2

services:
  object_stores:
    - id: minio
      type: s3
      endpoint: localhost:9000
      bucket: qpoint
      region: us-east-1
      insecure: true
      access_key:
        type: env
        value: AWS_ACCESS_KEY_ID
      secret_key:
        type: env
        value: AWS_SECRET_ACCESS_KEY

stacks:
  scan_stack:
    plugins:
      - type: http_capture
        config:
          level: full
          format: json
      - type: qscan
        config:
          record_document: true
          sample_baseline: 10
          sample_rate: 1.0
          monitors:
            - type: PERSON
              record_value: false
            - type: EMAIL_ADDRESS
              record_value: false
            - type: PHONE_NUMBER
              record_value: false
            - type: CREDIT_CARD
              record_value: false
            - type: US_SSN
              record_value: false

tap:
  direction: egress
  http:
    stack: scan_stack

Key configuration choices:

sample_rate: 1.0 scans every captured request. In production, lower this to reduce cost and load.
sample_baseline: 10 ensures the first 10 requests to each endpoint are always scanned, regardless of sample rate.
record_value: false on each monitor means QScan reports that PII was found, but does not store the actual sensitive values.

To use Qpoint's hosted QScan cloud service instead of self-hosted, set qscan_cloud: true in the plugin config. When enabled, Qtap sends artifacts to Qpoint's cloud for scanning -- no QScan deployment needed on your side.

Step 3: Deploy Qtap

Add the Qtap service to your docker-compose.yml:

docker-compose.yml (add to services)

  qtap:
    image: us-docker.pkg.dev/qpoint-edge/public/qtap:v0
    container_name: qtap
    restart: unless-stopped
    privileged: true
    pid: host
    network_mode: host
    command: >-
      --log-level=info
      --log-encoding=console
      --config=/app/config/qpoint.yaml
    volumes:
      - /sys:/sys
      - /var/run/docker.sock:/var/run/docker.sock
      - ./qtap.yaml:/app/config/qpoint.yaml:ro
    environment:
      - REGISTRATION_TOKEN=${REGISTRATION_TOKEN}
      - AWS_ACCESS_KEY_ID=minioadmin
      - AWS_SECRET_ACCESS_KEY=minioadmin
      - TINI_SUBREAPER=1
    ulimits:
      memlock:
        soft: -1
        hard: -1

Start Qtap:

docker compose up -d qtap

Check that it starts cleanly:

docker logs qtap --tail 20

You should see log lines indicating that Qtap has loaded the configuration and is tapping traffic.

Step 4: Deploy QScan

Add the QScan service to your docker-compose.yml:

docker-compose.yml (add to services)

  qscan:
    image: us-docker.pkg.dev/qpoint-edge/public/qscan:latest
    container_name: qscan
    restart: unless-stopped
    depends_on:
      - minio
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 12G
        reservations:
          cpus: '0.5'
          memory: 256M
    environment:
      - REGISTRATION_TOKEN=${REGISTRATION_TOKEN}
      - S3_ENDPOINT_URL=http://minio:9000
      - S3_BUCKET_NAME=qpoint
      - S3_REGION_NAME=us-east-1
      - AWS_ACCESS_KEY_ID=minioadmin
      - AWS_SECRET_ACCESS_KEY=minioadmin
      - METRICS_PORT=8080
      - NUM_POLLERS=1
      - NUM_SCANNERS=1
      - LOG_LEVEL=info

QScan loads PII detection models into memory. The 12 GB memory limit is recommended for reliable operation. Machines with less available memory may experience out-of-memory errors during scanning.

Start QScan:

docker compose up -d qscan

Verify it connects and begins polling for scan jobs:

docker logs qscan --tail 20

Look for log messages indicating that QScan has connected to Pulse and is polling for work.

Step 5: Generate Test Traffic

With the pipeline running, generate some HTTP requests containing fake PII. Qtap will capture these, store the artifacts in MinIO, and Pulse will schedule scan jobs for QScan.

Run a few curl commands from a container on the host (so Qtap can observe the traffic):

# Request with fake PII in the body
curl -X POST https://httpbin.org/post \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Jane Smith",
    "email": "[email protected]",
    "phone": "555-867-5309",
    "ssn": "123-45-6789",
    "card": "4111-1111-1111-1111"
  }'

# A few more to build up data
curl -X POST https://httpbin.org/post \
  -H "Content-Type: application/json" \
  -d '{"customer": "John Doe", "contact": "[email protected]"}'

curl -X POST https://httpbin.org/post \
  -H "Content-Type: application/json" \
  -d '{"patient": "Alice Johnson", "phone": "415-555-0142", "ssn": "987-65-4321"}'

httpbin.org echoes your request body back in the response, so both the request and response will contain PII for QScan to detect.

Wait a minute or two for the pipeline to process:

Qtap captures the requests and writes artifacts to MinIO
Pulse receives metadata and schedules scan jobs
QScan picks up the jobs, downloads artifacts, and runs PII detection

Step 6: Verify PII Detections

Check QScan Logs

docker logs qscan --tail 50

Look for log entries showing scan activity -- messages about downloading artifacts, running detection, and reporting results.

Check the Pulse Dashboard

Open app.qpoint.io and navigate to your environment. You should see:

PII findings associated with the httpbin.org endpoint
Detected entity types such as PERSON, EMAIL_ADDRESS, PHONE_NUMBER, US_SSN, and CREDIT_CARD
Confidence scores for each detection

The dashboard shows which endpoints are transmitting sensitive data, what types of PII were found, and how frequently it appears.

Check MinIO (Optional)

You can browse the stored artifacts through the MinIO console at http://localhost:9001 (login with minioadmin / minioadmin). Look in the qpoint bucket for captured request and response data.

Step 7: Next Steps

You now have a working QScan pipeline. Here are paths forward:

Tune sampling rates -- Lower sample_rate for production traffic and rely on sample_baseline to ensure coverage of new endpoints. See QScan Configuration.
Deploy to production -- Run QScan on Kubernetes or Cloud Run for production workloads.
Add more monitors -- QScan supports additional entity types like LOCATION, STREET_ADDRESS, US_BANK_NUMBER, and US_DRIVER_LICENSE.
Enable GPU acceleration -- For higher throughput scanning, QScan can use GPU resources to accelerate PII detection models.
Target specific endpoints -- Use Qtap's endpoint configuration to apply the qscan plugin only to specific domains (e.g., AI providers, external APIs).

Complete Docker Compose File

For convenience, here is the full docker-compose.yml with all three services ready to run:

docker-compose.yml

services:
  minio:
    image: minio/minio:latest
    container_name: minio
    command: server /data --console-address ":9001"
    ports:
      - "9000:9000"
      - "9001:9001"
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    volumes:
      - minio-data:/data

  qtap:
    image: us-docker.pkg.dev/qpoint-edge/public/qtap:v0
    container_name: qtap
    restart: unless-stopped
    privileged: true
    pid: host
    network_mode: host
    depends_on:
      - minio
    command: >-
      --log-level=info
      --log-encoding=console
      --config=/app/config/qpoint.yaml
    volumes:
      - /sys:/sys
      - /var/run/docker.sock:/var/run/docker.sock
      - ./qtap.yaml:/app/config/qpoint.yaml:ro
    environment:
      - REGISTRATION_TOKEN=${REGISTRATION_TOKEN}
      - AWS_ACCESS_KEY_ID=minioadmin
      - AWS_SECRET_ACCESS_KEY=minioadmin
      - TINI_SUBREAPER=1
    ulimits:
      memlock:
        soft: -1
        hard: -1

  qscan:
    image: us-docker.pkg.dev/qpoint-edge/public/qscan:latest
    container_name: qscan
    restart: unless-stopped
    depends_on:
      - minio
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 12G
        reservations:
          cpus: '0.5'
          memory: 256M
    environment:
      - REGISTRATION_TOKEN=${REGISTRATION_TOKEN}
      - S3_ENDPOINT_URL=http://minio:9000
      - S3_BUCKET_NAME=qpoint
      - S3_REGION_NAME=us-east-1
      - AWS_ACCESS_KEY_ID=minioadmin
      - AWS_SECRET_ACCESS_KEY=minioadmin
      - METRICS_PORT=8080
      - NUM_POLLERS=1
      - NUM_SCANNERS=1
      - LOG_LEVEL=info

volumes:
  minio-data:

To run the full stack:

# Set your registration token
export REGISTRATION_TOKEN=your-token-from-app-qpoint-io

# Create the bucket and start everything
docker compose up -d minio
docker run --rm --network host \
  --entrypoint sh minio/mc -c "
    mc alias set local http://localhost:9000 minioadmin minioadmin &&
    mc mb local/qpoint --ignore-existing
  "
docker compose up -d

Then generate test traffic with the curl commands from Step 5 and check results in the Pulse dashboard.

PreviousQScan Guides NextAWS Setup: PII Scanning on EKS

Last updated 1 day ago

hashtagWho This Is For

hashtagPrerequisites

hashtagStep 1: Set Up Object Storage

hashtagStep 2: Configure Qtap for Artifact Capture

hashtagStep 3: Deploy Qtap

hashtagStep 4: Deploy QScan

hashtagStep 5: Generate Test Traffic

hashtagStep 6: Verify PII Detections

hashtagCheck QScan Logs

hashtagCheck the Pulse Dashboard

hashtagCheck MinIO (Optional)

hashtagStep 7: Next Steps

hashtagComplete Docker Compose File

Who This Is For

Prerequisites

Step 1: Set Up Object Storage

Step 2: Configure Qtap for Artifact Capture

Step 3: Deploy Qtap

Step 4: Deploy QScan

Step 5: Generate Test Traffic

Step 6: Verify PII Detections

Check QScan Logs

Check the Pulse Dashboard

Check MinIO (Optional)

Step 7: Next Steps

Complete Docker Compose File