eBPF and Linux observability: Falco, Cilium and kernel monitoring in 2026

Linux monitoring has long relied on the same tools: userspace agents that read /proc, parse logs and expose metrics through HTTP endpoints. This approach works, but it has structural limits. The agent only sees what the kernel is willing to expose through its standard interfaces. It cannot observe system calls in real time, trace network packets inside the kernel, or detect suspicious behavior before it leaves a trace in a log file.

eBPF changes the game. This technology makes it possible to run code directly inside the Linux kernel, without modifying the kernel or loading custom modules. In 2026, eBPF has moved from a niche tool for kernel developers to the foundation of an entire generation of security and observability tools. Falco uses it for runtime intrusion detection. Cilium uses it for Kubernetes networking. bpftrace uses it for system profiling. And these tools are only getting started.

This article explains how eBPF works, breaks down the three major tools in the ecosystem, and shows how to put them into production to transform observability across your Linux servers.

eBPF in 30 seconds: the essentials

eBPF (extended Berkeley Packet Filter) is a virtual machine built into the Linux kernel. It lets you attach programs to hooks inside the kernel: system calls, network functions, tracepoints, kprobes. These programs run in a sandbox that the kernel verifies before loading, which guarantees they cannot crash the system or access arbitrary memory.

Fundamental architecture

An eBPF program follows a precise cycle. The developer writes the program (in restricted C or with frameworks like libbpf). The compiler turns it into eBPF bytecode. The kernel verifier analyzes the bytecode to guarantee its safety: no infinite loops, no out-of-bounds memory access, no dangerous instructions. If verification passes, the program is attached to the requested hook and runs on every matching event.

    eBPF program (restricted C)
            |
            v
    Compilation (clang/LLVM)
            |
            v
    eBPF bytecode (.o)
            |
            v
    Kernel verifier ----> Rejected if unsafe
            |
            v
    Load + Attach to hook
            |
            v
    Execution in the kernel
            |
            v
    eBPF maps (share data kernel <-> userspace)

Maps: the kernel/userspace bridge

eBPF maps are data structures shared between the kernel program and userspace. Hash maps, arrays, ring buffers, LRU caches: the choice of structure depends on the use case. An eBPF program attached to system calls can increment a counter in a map on every open(), and a userspace daemon reads that map periodically to expose the metric. Zero data copies, zero context switches: performance is close to native.

Available hooks

eBPF's power comes from the diversity of its hooks.

• kprobes / kretprobes: attach to any kernel function. Useful for tracing internal calls.
• tracepoints: stable instrumentation points defined by the kernel. More reliable than kprobes across versions.
• XDP (eXpress Data Path): execution at the network driver level, even before the TCP/IP stack. Ideal for high-performance filtering.
• TC (Traffic Control): network hook at the packet scheduler level. Used by Cilium for routing.
• LSM (Linux Security Modules): security hooks for mandatory access control. Used by Tetragon.
• uprobes: attach to userspace functions. Let you trace applications without modifying them.

Falco: runtime intrusion detection

Falco is a runtime threat detection engine created by Sysdig and now a graduated CNCF project. Its principle is simple: observe system calls in real time via eBPF and compare them against a set of rules to detect suspicious behavior. A container spawning an interactive shell, a process reading /etc/shadow, an unexpected outbound connection: Falco sees everything and alerts immediately.

Unlike auditd, which writes to log files you analyze after the fact, Falco operates in real time with a latency of a few microseconds. And unlike network IDS such as Suricata, Falco observes system behavior, not just network traffic.

Installation and configuration

Falco deploys either directly on the host or as a Kubernetes DaemonSet. The eBPF version is recommended over the kernel module (more portable, safer).

# Installation on Debian/Ubuntu via the official repo
curl -fsSL https://falco.org/repo/falcosecurity-packages.asc | \
  sudo gpg --dearmor -o /usr/share/keyrings/falco-archive-keyring.gpg

echo "deb [signed-by=/usr/share/keyrings/falco-archive-keyring.gpg] \
  https://download.falco.org/packages/deb stable main" | \
  sudo tee /etc/apt/sources.list.d/falcosecurity.list

sudo apt update && sudo apt install -y falco

# Check that the eBPF driver is loaded
sudo falco --version
# Falco version: 0.40.x (driver: ebpf)

# /etc/falco/falco.yaml - Essential configuration
engine:
  kind: ebpf
  ebpf:
    buf_size_preset: 4  # Ring buffer size (4 = 8 MB)

rules_files:
  - /etc/falco/falco_rules.yaml        # Default rules
  - /etc/falco/falco_rules.local.yaml  # Custom rules (take priority)

outputs:
  rate: 0           # No rate limiting
  max_burst: 1000   # Maximum burst

stdout_output:
  enabled: true

http_output:
  enabled: true
  url: "http://localhost:2801/"  # Falcosidekick for forwarding

Custom detection rules

Falco rules use a declarative DSL that filters syscall events. Each rule defines a condition (when to alert), a formatted output (what to log) and a priority.

# /etc/falco/falco_rules.local.yaml

# Detect a shell being spawned in a container
- rule: Interactive shell in a container
  desc: An interactive shell was spawned in a container
  condition: >
    spawned_process and container and
    proc.name in (bash, sh, zsh, dash) and
    proc.tty != 0
  output: >
    Interactive shell detected (container=%container.name
    image=%container.image.repository command=%proc.cmdline
    user=%user.name pid=%proc.pid)
  priority: WARNING
  tags: [container, shell, mitre_execution]

# Detect reads of sensitive files
- rule: Reading credential files
  desc: A process reads files containing credentials
  condition: >
    open_read and fd.name in (/etc/shadow, /etc/gshadow,
    /root/.ssh/authorized_keys, /root/.bash_history) and
    not proc.name in (sshd, passwd, su, sudo, login)
  output: >
    Sensitive file read (file=%fd.name command=%proc.cmdline
    user=%user.name container=%container.name)
  priority: CRITICAL
  tags: [filesystem, credentials, mitre_credential_access]

# Detect suspicious outbound connections
- rule: Outbound connection from a web container
  desc: A web container initiates an unexpected outbound connection
  condition: >
    outbound and container and
    container.image.repository contains "nginx" and
    not fd.sport in (80, 443, 8080)
  output: >
    Suspicious outbound connection (container=%container.name
    image=%container.image.repository connection=%fd.name
    command=%proc.cmdline)
  priority: ERROR
  tags: [network, container, mitre_command_and_control]

Kubernetes integration

In a Kubernetes environment, Falco deploys as a DaemonSet to cover every node. Integration with Falcosidekick lets you route alerts to Slack, PagerDuty, Elasticsearch or any webhook.

# Deployment via Helm
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update

helm install falco falcosecurity/falco \
  --namespace falco --create-namespace \
  --set driver.kind=ebpf \
  --set falcosidekick.enabled=true \
  --set falcosidekick.config.slack.webhookurl="https://hooks.slack.com/..." \
  --set falcosidekick.config.elasticsearch.hostPort="http://elastic:9200"

Cilium: eBPF-native networking and security

If Falco observes syscalls, Cilium controls the network. Cilium is a Kubernetes CNI (Container Network Interface) that replaces iptables with eBPF programs for all networking: routing, load balancing, network policies, observability. Performance is significantly better than iptables-based solutions because eBPF avoids traversing linear rule chains.

L3/L4/L7 network policies

Standard Kubernetes NetworkPolicies operate at layers 3 and 4 (IP and port). Cilium goes further with CiliumNetworkPolicies that filter at L7: HTTP method, path, headers, gRPC content. This lets you define policies such as "the orders service may GET /api/products but not DELETE".

# CiliumNetworkPolicy - L7 HTTP filtering
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: api-access-control
  namespace: production
spec:
  endpointSelector:
    matchLabels:
      app: api-backend
  ingress:
    - fromEndpoints:
        - matchLabels:
            app: frontend
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
          rules:
            http:
              - method: "GET"
                path: "/api/v2/products.*"
              - method: "POST"
                path: "/api/v2/orders"
    - fromEndpoints:
        - matchLabels:
            app: monitoring
      toPorts:
        - ports:
            - port: "9090"
              protocol: TCP
          rules:
            http:
              - method: "GET"
                path: "/metrics"

This granularity is essential to implement a Zero Trust architecture inside a Kubernetes cluster. Each service only reaches the specific endpoints it needs.

Hubble: network observability

Hubble is Cilium's observability component. It captures network flows in real time with L3/L4/L7 visibility, with no significant overhead thanks to eBPF. The CLI and web UI let you visualize inter-service dependencies, debug connectivity issues and detect unauthorized flows.

# Install the Hubble CLI
export HUBBLE_VERSION=$(curl -s \
  https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
curl -L --remote-name-all \
  https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-amd64.tar.gz
tar xzf hubble-linux-amd64.tar.gz
sudo mv hubble /usr/local/bin/

# Observe flows in real time
hubble observe --namespace production --protocol http

# Filter dropped flows
hubble observe --verdict DROPPED --namespace production

# Export flows to Prometheus
# Hubble exposes metrics on :9965/metrics by default

Sidecar-less service mesh

Cilium offers an eBPF-native service mesh that eliminates the need for sidecars (unlike Istio/Envoy). mTLS, L7 load balancing and retries are handled directly in the kernel by eBPF programs. The benefit is twofold: lower memory consumption (no proxy per pod) and reduced latency (no extra hop). For production Docker and Kubernetes clusters, that is a considerable operational gain.

Complementary tools: bpftrace, Tetragon, Pixie

Beyond Falco and Cilium, the eBPF ecosystem includes several specialized tools that cover specific needs.

bpftrace: the Swiss army knife of tracing

bpftrace is a high-level tracing language for eBPF, inspired by DTrace and SystemTap. It lets you write one-liners to observe any aspect of the system in real time. It is the ideal tool for ad-hoc debugging and performance investigation.

# Count system calls per process
sudo bpftrace -e 'tracepoint:raw_syscalls:sys_enter {
  @syscalls[comm] = count();
}'

# Latency histogram of disk reads
sudo bpftrace -e 'tracepoint:block:block_rq_issue {
  @start[args->dev, args->sector] = nsecs;
}
tracepoint:block:block_rq_complete /@start[args->dev, args->sector]/ {
  @usecs = hist((nsecs - @start[args->dev, args->sector]) / 1000);
  delete(@start[args->dev, args->sector]);
}'

# Trace TCP connections with PID and destination
sudo bpftrace -e 'tracepoint:sock:inet_sock_set_state
/args->newstate == 1/ {
  printf("%-8d %-16s %s:%d\n", pid, comm,
    ntop(args->daddr), args->dport);
}'

# Top 10 most-opened files
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_openat {
  @files[str(args->filename)] = count();
}
END { print(@files, 10); }'

Tetragon: enforcement and runtime security

Tetragon, also developed by Isovalent (the creators of Cilium), goes further than Falco. Where Falco detects and alerts, Tetragon can block malicious actions in real time thanks to eBPF LSM hooks. It can stop a process from executing a binary, block a file access or kill a process that violates a policy, all directly in the kernel without going through userspace.

# Tetragon TracingPolicy - Block miner execution
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: block-crypto-miners
spec:
  kprobes:
    - call: "security_bprm_check"
      syscall: false
      args:
        - index: 0
          type: "linux_binprm"
      selectors:
        - matchBinaries:
            - operator: "In"
              values:
                - "/usr/bin/xmrig"
                - "/tmp/minerd"
          matchActions:
            - action: Sigkill

Pixie: auto-instrumented observability

Pixie (a CNCF project) uses eBPF to automatically capture HTTP requests, SQL queries, gRPC calls and system metrics without any instrumentation of the application code. No SDK, no sidecar, no changes to deployments. Pixie installs on the cluster and immediately starts collecting observability data. This is particularly useful for legacy applications or environments where adding OpenTelemetry instrumentation is not possible.

Kernel monitoring with eBPF: custom metrics

Beyond packaged tools, eBPF lets you create bespoke kernel metrics that are impossible to obtain with classic exporters. Here are the most relevant production use cases.

I/O and scheduling latency

Classic tools like iostat give averages. eBPF gives distributions. The difference is crucial: an average latency of 5 ms can hide a P99 of 200 ms. The BCC tools (BPF Compiler Collection) provide ready-to-use scripts to measure these distributions.

# I/O latency distribution per disk (BCC tools)
sudo biolatency -D 10 1
# Displays a latency histogram per device over 10 seconds

# Scheduling latency (time between wakeup and execution)
sudo runqlat 10 1
# Shows how long processes wait in the run queue

# TCP retransmissions (symptom of congestion or loss)
sudo tcpretrans
# Displays each retransmission with source, destination, state

# DNS request latency
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_sendmsg
/comm == "systemd-resolve"/ {
  @start[tid] = nsecs;
}
tracepoint:syscalls:sys_exit_recvmsg
/comm == "systemd-resolve" && @start[tid]/ {
  @dns_latency_us = hist((nsecs - @start[tid]) / 1000);
  delete(@start[tid]);
}'

Production-ready bpftrace one-liners

These commands are usable directly during an investigation. Each answers a precise question that a classic Prometheus/Grafana stack does not cover natively.

# Who is consuming network bandwidth? (per process)
sudo bpftrace -e 'tracepoint:net:net_dev_xmit {
  @bytes[comm] = sum(args->len);
}'

# Which files are written most often? (I/O storm detection)
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_write {
  @writes[comm, pid] = count();
}'

# Time spent in page faults (sign of memory pressure)
sudo bpftrace -e 'software:page-faults:1 {
  @faults[comm] = count();
}'

# Delay between a packet arriving and the app consuming it
sudo bpftrace -e 'tracepoint:net:netif_receive_skb {
  @qlen[comm] = hist(args->len);
}'

eBPF vs traditional agents: a comparison

The point is not to replace Prometheus or auditd, but to understand what eBPF adds on top and where it outperforms classic approaches.

In practice, the best approach is hybrid. Keep Prometheus and node_exporter for standard system metrics and long-term history. Add Falco for runtime detection. Use Cilium for networking if you are on Kubernetes. And pull out bpftrace when you need to answer a precise question about kernel behavior.

Production deployment: best practices and pitfalls

Deploying eBPF tools in production requires rigor. Here are the lessons learned from real deployments.

1. Check kernel compatibility

Not all kernels support all eBPF features. The minimum viable for Falco and Cilium is kernel 5.8. For Tetragon and LSM hooks, aim for 5.15+. Kernel 6.x brings CO-RE (Compile Once, Run Everywhere), which eliminates the dependency on kernel headers.

# Check the kernel version
uname -r

# Check available eBPF features
sudo bpftool feature probe kernel

# Check that BTF (BPF Type Format) is enabled (required for CO-RE)
ls /sys/kernel/btf/vmlinux && echo "BTF OK" || echo "BTF missing"

2. Deploy in audit mode first

As with server security rules, never block anything on the first deployment. Falco in alert-only mode. Cilium in log-only mode for network policies. Tetragon in observe mode without Sigkill. Collect data for at least two weeks to understand normal flows before enabling enforcement.

3. Manage false positives

Default Falco rules generate noise. A cron job running a bash script inside a container triggers the "shell in container" alert. The solution is to maintain contextual exception lists and tune rules progressively. Never disable an entire rule: add precise exceptions.

# Exception for a legitimate cron job
- rule: Interactive shell in a container
  append: true
  condition: and not (container.image.repository = "registry.internal/cron-jobs")

4. Monitor the eBPF programs themselves

eBPF programs consume resources: memory for maps, CPU for execution. Monitor the number of loaded programs, the size of maps and lost events (dropped events in the ring buffer).

# List loaded eBPF programs
sudo bpftool prog list

# View maps and their memory consumption
sudo bpftool map list

# Program execution statistics
sudo bpftool prog show id <ID> --json | jq '.run_cnt, .run_time_ns'

5. Plan ring buffer capacity

If the system generates more events than the ring buffer can hold, events are lost. On a high-traffic server, increase the buffer size and set up a metric to track lost events. For Falco, the buf_size_preset parameter controls this size.

6. Secure the eBPF tools

eBPF programs run with elevated privileges. Restrict access to the bpftool and bpftrace commands to administrators only. In a Kubernetes environment, Falco and Cilium pods run in privileged mode: isolate them in dedicated namespaces with strict RBAC.

Conclusion

eBPF has fundamentally changed what it is possible to observe and control on a Linux system. The tools built on this technology (Falco, Cilium, Tetragon, bpftrace) offer visibility and responsiveness that traditional userspace agents cannot match. Real-time detection of suspicious behavior, kernel-level network filtering and instrumentation-free application tracing are now within reach of any system administrator willing to invest in learning.

The learning curve is real. eBPF requires understanding kernel concepts, hooks, maps and how the verifier works. But high-level tools like Falco and Cilium abstract away most of that complexity. You do not need to write eBPF programs to benefit from it.

Start with Falco for runtime security. Add bpftrace for performance debugging. Migrate to Cilium when your Kubernetes infrastructure justifies it. And above all, deploy in audit mode before enabling any blocking. Linux observability has never been this powerful: it would be a shame not to take advantage of it.