checkpoint/erm-generic-ioc-feeder repository overview

⁠ERM-Panorama IOC feeder

The ERM-Panorama IOC Feeder provides automated integration between ERM and Palo Alto Panorama.

Daily IOC feeds are retrieved from ERM using configurable filters for severity level, confidence level, and detected activities. IOCs are automatically categorized by type (IP, URL, Domain, File) and saved to their respective directories in predefined formats.

⁠Usage

The fetch job will start automatically after running the app. Once the fetch job is finished, you can access the IOC data by going to the following paths:

• {docker_ip:docker_port}/ioc/feed/ip/ip_list.txt
• {docker_ip:docker_port}/ioc/feed/url/url_list.txt
• {docker_ip:docker_port}/ioc/feed/domain/domain_list.txt
• {docker_ip:docker_port}/ioc/feed/file/file_list.txt

Make sure to add http:// or https:// prefix to the path (based on the ENABLE_HTTPS definition in your .env file).

⁠Configuration

The following environment variables are required to run the app:

Please create the .env file in the root directory and set the variables.

⁠Enable HTTPS requests

By default the application supports HTTP requests. To enable HTTPS requests, please supply the SSL_KEY_PATH, SSL_CERT_PATH,SSL_CA_CERT_PATH environment variables and set the ENABLE_HTTPS to 'True'.

⁠Monitoring

The application exposes endpoints for health monitoring and Prometheus metrics scraping, suitable for OpenShift/Kubernetes environments.

⁠Health Check Endpoint

A stale feed must not fail liveness -- restarting the pod does not refresh it -- so /health stays 200 and only /ready fails. Point the readiness probe at /ready.

Example /health body when the fetch job is failing:

{
  "status": "degraded",
  "version": "6.13",
  "feed": {
    "ready": false,
    "stale": false,
    "last_successful_fetch": null,
    "consecutive_failures": 3,
    "last_error": "RestError: Failed to execute request: ...",
    "entries": {}
  }
}

⁠Prometheus Metrics Endpoint

The /metrics endpoint exposes the following metrics:

• http_requests_total - Total HTTP requests by method, path, and status code
• http_request_duration_seconds - Request latency histogram
• http_request_size_bytes - Request size histogram
• http_response_size_bytes - Response size histogram
• Python process metrics (memory, CPU, garbage collection stats)

Connector and feed health:

• connector_build_info{version} - Running connector version
• ioc_feed_ready - 1 once a fetch has succeeded and the feed is being served
• ioc_feed_last_success_timestamp_seconds - Unix timestamp of the last successful fetch
• ioc_feed_age_seconds - Age of the currently served feed
• ioc_feed_entries{ioc_type} - Entries written to each feed file by the last fetch
• ioc_feed_fetch_failures_total - Failed fetch attempts
• connector_update_available - 1 when a newer connector version is published

Alert on ioc_feed_ready == 0 and on ioc_feed_age_seconds exceeding your refresh window: both indicate the firewall is enforcing a list that is no longer being updated.

⁠OpenShift/Kubernetes Configuration

Configure your deployment with the following probes:

livenessProbe:
  httpGet:
    path: /health
    port: 8000
  initialDelaySeconds: 10
  periodSeconds: 30

readinessProbe:
  httpGet:
    path: /ready
    port: 8000
  initialDelaySeconds: 5
  periodSeconds: 10

For Prometheus metrics scraping, create a ServiceMonitor or add pod annotations:

annotations:
  prometheus.io/scrape: "true"
  prometheus.io/path: "/metrics"
  prometheus.io/port: "8000"

⁠URL Processing

URL IOCs are processed to be compatible with Palo Alto URL EDL format:

• Protocol is always stripped (https://example.com/path → example.com/path).
• Query parameters are stripped for regular URLs, since they typically contain ephemeral tokens (SAS, pre-signed URLs) rather than meaningful IOC data.
• Redirect-service URLs (e.g. google.com/url) receive special handling. Stripping query parameters from these would produce overly broad entries that block all legitimate traffic through the redirect service. Instead, the feeder:
  1. Keeps the full URL with query parameters so the block targets only the specific malicious redirect.
  2. Extracts the actual malicious target URL from the redirect query parameter (url, q, or u) and adds it as a separate entry.

Currently recognized redirect services: google.com/url, www.google.com/url. The list can be extended in REDIRECT_SERVICES.