pfelg

OPNsense/pfSense firewall log analysis with GeoIP visualization for Grafana.

Inspired by pfelk but simplified: no custom scripts, just the official Elasticsearch and Logstash Docker images with configuration files.

Features

Firewall log parsing (block/pass events)
GeoIP enrichment with map visualization
Minimal RAM footprint
Automatic data retention via ILM
ECS-compliant field structure

Architecture

OPNsense ──── Syslog (RFC5424, UDP 5140) ────► Logstash ────► Elasticsearch ────► Grafana
                                               (GeoIP)        (Storage)          (Dashboards)

Prerequisites

Docker
MaxMind GeoLite2 databases (free account at maxmind.com)

Dashboard Preview

Just import the JSON-file from dashboard/

Installation

1. Directory Structure

/your/path/pfelg/
├── elasticsearch/
├── logstash/
│   ├── config/
│   │   └── logstash.yml
│   ├── pipeline/
│   │   ├── 01-inputs.conf
│   │   ├── 02-firewall.conf
│   │   ├── 05-apps.conf
│   │   ├── 30-geoip.conf
│   │   ├── 49-cleanup.conf
│   │   └── 50-outputs.conf
│   └── patterns/
│       ├── pfelk.grok
│       └── openvpn.grok
└── maxmind/
    ├── GeoLite2-City.mmdb
    └── GeoLite2-ASN.mmdb

Copy all files from this repository to the corresponding directories. Download MaxMind databases and place them in the maxmind/ folder.

⚠️ IMPORTANT: Ensure all directories have read/write permissions. Both Elasticsearch and Logstash will fail to start without proper permissions.

2. Elasticsearch

docker run -d \
  --name pfelg-elasticsearch \
  -p 9200:9200 \
  -v /your/path/pfelg/elasticsearch:/usr/share/elasticsearch/data \
  -e "discovery.type=single-node" \
  -e "xpack.security.enabled=false" \
  -e "ES_JAVA_OPTS=-Xms256m -Xmx256m" \
  -e "cluster.name=pfelg" \
  --ulimit memlock=-1:-1 \
  docker.elastic.co/elasticsearch/elasticsearch:8.17.0

3. Logstash

docker run -d \
  --name pfelg-logstash \
  --link pfelg-elasticsearch:elasticsearch \
  -p 5140:5140/udp \
  -p 5140:5140/tcp \
  -v /your/path/pfelg/logstash/pipeline:/usr/share/logstash/pipeline \
  -v /your/path/pfelg/logstash/patterns:/usr/share/logstash/patterns \
  -v /your/path/pfelg/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml \
  -v /your/path/pfelg/maxmind:/usr/share/logstash/maxmind \
  -e "LS_JAVA_OPTS=-Xms256m -Xmx256m" \
  -e "XPACK_MONITORING_ENABLED=false" \
  docker.elastic.co/logstash/logstash:8.17.0

4. ILM Policy

Concept

┌─────────┐    ┌─────────┐    ┌─────────┐
│   HOT   │ →  │  WARM   │ →  │  COLD   │ →  (optional: DELETE)
│  0-24h  │    │  1-7d   │    │   7d+   │
│  fast   │    │ Compress│    │  Archiv │
└─────────┘    └─────────┘    └─────────┘

Choose a retention policy or change it:

Keep forever (cold storage after 7 days):

curl -X PUT "http://localhost:9200/_ilm/policy/pfelk-forever?pretty" \
  -H 'Content-Type: application/json' -d '
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "set_priority": { "priority": 100 }
        }
      },
      "warm": {
        "min_age": "1d",
        "actions": {
          "set_priority": { "priority": 50 },
          "forcemerge": { "max_num_segments": 1 }
        }
      },
      "cold": {
        "min_age": "7d",
        "actions": {
          "set_priority": { "priority": 0 },
          "readonly": {}
        }
      }
    }
  }
}'

Delete after 7 days:

curl -X PUT "http://localhost:9200/_ilm/policy/pfelk-7days?pretty" \
    -H 'Content-Type: application/json' -d '
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "set_priority": { "priority": 100 }
        }
      },
      "warm": {
        "min_age": "1d",
        "actions": {
          "set_priority": { "priority": 50 },
          "forcemerge": { "max_num_segments": 1 },
        }
      },
      "delete": {
        "min_age": "7d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}'

5. Index Template

Replace POLICY_NAME with your chosen policy (pfelk-forever or pfelk-7days):

curl -X PUT "http://localhost:9200/_index_template/pfelg" -H 'Content-Type: application/json' -d '
{
  "index_patterns": ["pfelk-*"],
  "priority": 100,
  "template": {
    "settings": {
      "index": {
        "number_of_shards": 1,
        "number_of_replicas": 0,
        "refresh_interval": "30s",
        "codec": "best_compression",
        "lifecycle": { "name": "POLICY_NAME" },
        "mapping": { "total_fields": { "limit": "10000" } }
      }
    },
    "mappings": {
      "dynamic_templates": [
        { "strings_as_keyword": { "match_mapping_type": "string", "mapping": { "type": "keyword", "ignore_above": 1024 } } }
      ],
      "date_detection": false,
      "properties": {
        "@timestamp": { "type": "date" },
        "source": {
          "properties": {
            "ip": { "type": "ip" },
            "port": { "type": "long" },
            "geo": {
              "properties": {
                "city_name": { "type": "keyword" },
                "continent_code": { "type": "keyword" },
                "country_iso_code": { "type": "keyword" },
                "country_name": { "type": "keyword" },
                "location": { "type": "geo_point" },
                "region_iso_code": { "type": "keyword" },
                "region_name": { "type": "keyword" },
                "timezone": { "type": "keyword" },
                "postal_code": { "type": "keyword" }
              }
            },
            "as": {
              "properties": {
                "number": { "type": "long" },
                "organization": { "properties": { "name": { "type": "keyword" } } }
              }
            }
          }
        },
        "destination": {
          "properties": {
            "ip": { "type": "ip" },
            "port": { "type": "long" },
            "geo": {
              "properties": {
                "city_name": { "type": "keyword" },
                "continent_code": { "type": "keyword" },
                "country_iso_code": { "type": "keyword" },
                "country_name": { "type": "keyword" },
                "location": { "type": "geo_point" },
                "region_iso_code": { "type": "keyword" },
                "region_name": { "type": "keyword" },
                "timezone": { "type": "keyword" },
                "postal_code": { "type": "keyword" }
              }
            },
            "as": {
              "properties": {
                "number": { "type": "long" },
                "organization": { "properties": { "name": { "type": "keyword" } } }
              }
            }
          }
        },
        "client": {
          "properties": {
            "ip": { "type": "ip" },
            "port": { "type": "long" },
            "mac": { "type": "keyword" },
            "user": { "properties": { "name": { "type": "keyword" } } },
            "geo": {
              "properties": {
                "city_name": { "type": "keyword" },
                "country_iso_code": { "type": "keyword" },
                "country_name": { "type": "keyword" },
                "location": { "type": "geo_point" }
              }
            }
          }
        },
        "network": {
          "properties": {
            "direction": { "type": "keyword" },
            "type": { "type": "keyword" },
            "transport": { "type": "keyword" },
            "iana_number": { "type": "long" }
          }
        },
        "event": {
          "properties": {
            "action": { "type": "keyword" },
            "dataset": { "type": "keyword" },
            "reason": { "type": "keyword" },
            "category": { "type": "keyword" },
            "severity": { "type": "long" },
            "sequence": { "type": "keyword" },
            "original": { "type": "keyword", "index": false }
          }
        },
        "rule": {
          "properties": {
            "id": { "type": "keyword" },
            "uuid": { "type": "keyword" },
            "reference": { "type": "keyword" },
            "version": { "type": "keyword" }
          }
        },
        "interface": {
          "properties": {
            "name": { "type": "keyword" },
            "alias": { "type": "keyword" }
          }
        },
        "log": {
          "properties": {
            "syslog": {
              "properties": {
                "appname": { "type": "keyword" },
                "hostname": { "type": "keyword" },
                "priority": { "type": "long" },
                "procid": { "type": "keyword" },
                "version": { "type": "keyword" },
                "facility": {
                  "properties": {
                    "code": { "type": "long" },
                    "name": { "type": "keyword" }
                  }
                },
                "severity": {
                  "properties": {
                    "code": { "type": "long" },
                    "name": { "type": "keyword" }
                  }
                }
              }
            },
            "level": { "type": "keyword" }
          }
        },
        "host": {
          "properties": {
            "ip": { "type": "ip" }
          }
        },
        "kea": {
          "properties": {
            "dhcp": {
              "properties": {
                "operation": { "type": "keyword" },
                "client": {
                  "properties": {
                    "ip": { "type": "ip" },
                    "mac": { "type": "keyword" }
                  }
                },
                "lease": {
                  "properties": {
                    "id": { "type": "keyword" },
                    "duration": { "type": "long" }
                  }
                },
                "id": { "type": "keyword" },
                "hardware_type": { "type": "keyword" }
              }
            },
            "message": { "type": "keyword" }
          }
        },
        "dhcp": {
          "properties": {
            "operation": { "type": "keyword" },
            "message": { "type": "keyword" },
            "error": { "properties": { "code": { "type": "keyword" } } }
          }
        },
        "dhcpv4": {
          "properties": {
            "client": {
              "properties": {
                "ip": { "type": "ip" },
                "mac": { "type": "keyword" },
                "address": { "type": "keyword" }
              }
            },
            "server": { "properties": { "ip": { "type": "ip" } } },
            "relay": { "properties": { "ip": { "type": "ip" } } },
            "option": { "properties": { "hostname": { "type": "keyword" } } },
            "lease": { "properties": { "duration": { "type": "long" } } },
            "query": {
              "properties": {
                "ip": { "type": "ip" },
                "id": { "type": "keyword" },
                "mac": { "type": "keyword" },
                "associated": { "type": "long" }
              }
            }
          }
        },
        "dns": {
          "properties": {
            "question": {
              "properties": {
                "name": { "type": "keyword" },
                "type": { "type": "keyword" },
                "class": { "type": "keyword" },
                "registered_domain": { "type": "keyword" },
                "top_level_domain": { "type": "keyword" }
              }
            }
          }
        },
        "suricata": {
          "properties": {
            "eve": { "type": "object", "enabled": true },
            "rule": {
              "properties": {
                "uuid": { "type": "keyword" },
                "id": { "type": "keyword" },
                "version": { "type": "keyword" },
                "description": { "type": "keyword" },
                "category": { "type": "keyword" }
              }
            },
            "priority": { "type": "long" }
          }
        },
        "vulnerability": {
          "properties": {
            "description": { "type": "keyword" },
            "classification": { "type": "keyword" }
          }
        },
        "haproxy": {
          "properties": {
            "frontend_name": { "type": "keyword" },
            "backend_name": { "type": "keyword" },
            "server_name": { "type": "keyword" },
            "termination_state": { "type": "keyword" },
            "client": {
              "properties": {
                "ip": { "type": "ip" },
                "port": { "type": "long" }
              }
            },
            "time_request": { "type": "long" },
            "time_queue": { "type": "long" },
            "time_backend_connect": { "type": "long" },
            "time_backend_response": { "type": "long" },
            "bytes_read": { "type": "long" },
            "connections": {
              "properties": {
                "active": { "type": "long" },
                "frontend": { "type": "long" },
                "backend": { "type": "long" },
                "server": { "type": "long" },
                "retries": { "type": "long" }
              }
            }
          }
        },
        "nginx": {
          "properties": {
            "access": {
              "properties": {
                "method": { "type": "keyword" },
                "url": { "type": "keyword" },
                "http_version": { "type": "keyword" },
                "response_code": { "type": "long" },
                "body_sent": { "properties": { "bytes": { "type": "long" } } },
                "referrer": { "type": "keyword" },
                "agent": { "type": "keyword" },
                "user_name": { "type": "keyword" },
                "forwarder": { "type": "keyword" }
              }
            }
          }
        },
        "http": {
          "properties": {
            "request": {
              "properties": {
                "method": { "type": "keyword" },
                "referrer": { "type": "keyword" }
              }
            },
            "response": {
              "properties": {
                "status_code": { "type": "long" },
                "bytes": { "type": "long" },
                "mime_type": { "type": "keyword" },
                "body": { "properties": { "status_code": { "type": "long" } } }
              }
            },
            "version": { "type": "keyword" }
          }
        },
        "url": {
          "properties": {
            "scheme": { "type": "keyword" },
            "domain": { "type": "keyword" },
            "port": { "type": "long" },
            "path": { "type": "keyword" }
          }
        },
        "process": {
          "properties": {
            "pgid": { "type": "long" },
            "thread": { "properties": { "id": { "type": "long" } } }
          }
        },
        "observer": {
          "properties": {
            "ingress": {
              "properties": {
                "interface": { "properties": { "alias": { "type": "keyword" } } },
                "zone": { "type": "keyword" }
              }
            }
          }
        },
        "labels": {
          "properties": {
            "request_status": { "type": "keyword" },
            "hierarchy_status": { "type": "keyword" }
          }
        },
        "service": { "properties": { "type": { "type": "keyword" } } },
        "ecs": { "properties": { "version": { "type": "keyword" } } },
        "tags": { "type": "keyword" }
      }
    }
  }
}'

6. OPNsense Configuration

System → Settings → Logging / targets → Add:

Setting	Value
Transport	UDP(4)
Applications	filter (filterlog)
Hostname	Your Docker host IP
Port	5140
RFC5424	✅ MUST be enabled

⚠️ Without RFC5424, timestamps will be missing timezone info and data will appear with incorrect times in Grafana.

7. Grafana Data Source

Setting	Value
Type	Elasticsearch
URL	`http://YOUR_HOST:9200`
Index name	`pfelk-firewall*`
Time field	`@timestamp`
Version	8.0+

Setting	Value
Type	Elasticsearch
URL	`http://YOUR_HOST:9200`
Index name	`pfelk-dhcp*`
Time field	`@timestamp`
Version	8.0+

In Grafana: Dashboards → Import → Upload JSON file or go to grafana.com and copy the id

Available Fields

Field	Description
`event.action`	pass / block
`source.ip` / `destination.ip`	IP addresses
`source.port` / `destination.port`	Ports
`source.geo.location` / `destination.geo.location`	Coordinates (for maps)
`source.geo.country_name` / `destination.geo.country_name`	Country names
`network.transport`	tcp / udp / icmp
`interface.name`	wan / lan / etc.
`rule.id`	Firewall rule ID

Query Examples

event.action:block
event.action:block AND destination.port:443
event.action:pass AND destination.port:22
source.geo.country_name:Germany

Useful Commands

# List indices
curl -s "http://localhost:9200/_cat/indices/pfelk-*?v&h=index,docs.count,store.size"

# Check ILM status
curl -s "http://localhost:9200/pfelk-*/_ilm/explain?pretty" | grep -E "(index|phase)"

# Delete all indices
for i in $(curl -s "http://localhost:9200/_cat/indices/pfelk-*?h=index"); do curl -X DELETE "http://localhost:9200/$i"; done

# Logstash logs
docker logs pfelg-logstash --tail 50

Troubleshooting

No data in Grafana: Check if indices exist (curl localhost:9200/_cat/indices/pfelk-*), verify Logstash logs, expand time range.

Fields have .keyword suffix: Reinstall template, delete existing indices, restart Logstash.

GeoIP not working: Verify MaxMind files exist in mounted directory.

Wrong timestamps: Enable RFC5424 in OPNsense logging settings.

RAM Usage

Heap	Total RAM
ES: 256 MB / LS: 256 MB	~1.5 GB
ES: 512 MB / LS: 512 MB	~2.5 GB

Unraid User Scripts

Manual cleanup scripts for Unraid. Add via Settings → User Scripts → Add New Script.

Delete indices older than X days

#!/bin/bash
# CONFIGURATION - adjust as needed:
DAYS_TO_KEEP=30

ES_HOST="http://localhost:9200"

echo "Deleting indices older than ${DAYS_TO_KEEP} days..."

for INDEX in $(curl -s "${ES_HOST}/_cat/indices/pfelk-*?h=index" 2>/dev/null); do
    INDEX_DATE=$(echo "$INDEX" | grep -oE '[0-9]{4}\.[0-9]{2}\.[0-9]{2}$')
    
    if [ -z "$INDEX_DATE" ]; then
        continue
    fi
    
    INDEX_EPOCH=$(date -d "${INDEX_DATE//./-}" +%s 2>/dev/null)
    CUTOFF_EPOCH=$(date -d "${DAYS_TO_KEEP} days ago" +%s)
    
    if [ "$INDEX_EPOCH" -lt "$CUTOFF_EPOCH" ]; then
        echo "Deleting: $INDEX"
        curl -s -X DELETE "${ES_HOST}/${INDEX}"
    fi
done

echo ""
echo "Remaining indices:"
curl -s "${ES_HOST}/_cat/indices/pfelk-*?v&h=index,docs.count,store.size"

Delete all indices

#!/bin/bash
ES_HOST="http://localhost:9200"

echo "Deleting ALL pfelk indices..."

for INDEX in $(curl -s "${ES_HOST}/_cat/indices/pfelk-*?h=index" 2>/dev/null); do
    echo "Deleting: $INDEX"
    curl -s -X DELETE "${ES_HOST}/${INDEX}"
done

echo "Done!"

🙏 Credits

Based on pfELK Project by pfelk.

Name		Name	Last commit message	Last commit date
dashboard		dashboard
img		img
logstash		logstash
README.md		README.md

Navigation Menu

Repository files navigation

pfelg

Features

Architecture

Prerequisites

Dashboard Preview

Installation

1. Directory Structure

2. Elasticsearch

3. Logstash

4. ILM Policy

Concept

5. Index Template

6. OPNsense Configuration

7. Grafana Data Source

Available Fields

Query Examples

Useful Commands

Troubleshooting

RAM Usage

Unraid User Scripts

Delete indices older than X days

Delete all indices

🙏 Credits

About

Releases

Packages

Contributors

Latest commit

History

Folders and files

Repository files navigation

pfelg

Features

Architecture

Prerequisites

Dashboard Preview

Installation

1. Directory Structure

2. Elasticsearch

3. Logstash

4. ILM Policy

Concept

5. Index Template

6. OPNsense Configuration

7. Grafana Data Source

Available Fields

Query Examples

Useful Commands

Troubleshooting

RAM Usage

Unraid User Scripts

Delete indices older than X days

Delete all indices

🙏 Credits

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors