Dissecting Data Streams: My Journey Using Python to Monitor and Analyze IoT Network Traffic

From passive packet sniffing to anomaly detection pipelines — a deep dive into building a full-fledged Python-based network traffic analysis system for smart environments.

🔹 1. Introduction: Why IoT Traffic Needs Special Attention

Smart devices don’t communicate like traditional computers. Their protocols are often lightweight, custom, and noisy. Worse? They chat non-stop — sometimes even when you think they’re idle.

That’s why I decided to use Python to build my own IoT network traffic analysis toolchain. I wanted visibility — and more importantly — control.

Here’s how I did it.

🔹 2. Setting Up the Environment for Packet Capture

I started with scapy, one of the most powerful packet crafting and sniffing tools available in Python. It gives raw access to layers of network data.

🧪 Code: Passive Packet Sniffer

from scapy.all import sniff, IP, TCP

def packet_callback(packet):
    if IP in packet and TCP in packet:
        print(f"{packet[IP].src} -> {packet[IP].dst} | {packet[TCP].sport} → {packet[TCP].dport}")

sniff(prn=packet_callback, filter="tcp", store=0)

This simple script listens to all TCP packets and logs the source/destination addresses and ports. You’d be surprised how much chatter a smart plug produces.

🔹 3. Filtering Device-Specific Traffic

For meaningful insights, you have to isolate device-specific traffic. I mapped MAC/IP addresses for each IoT device on my network.

🔍 Code: Filtering Based on Device IP

DEVICE_IP = "192.168.1.42"

def device_filter(packet):
    return IP in packet and (packet[IP].src == DEVICE_IP or packet[IP].dst == DEVICE_IP)

sniff(prn=packet_callback, lfilter=device_filter, store=0)

This filter zeroes in on a single device, which is especially useful for profiling communication patterns.

🔹 4. Protocol Dissection: Understanding MQTT and CoAP

Most IoT devices use lightweight protocols like MQTT or CoAP. Here’s a simple MQTT dissection using scapy.

📦 Code: Decoding MQTT Packets

from scapy.layers.inet import TCP
from scapy.packet import Raw

def mqtt_sniffer(packet):
    if Raw in packet and TCP in packet:
        payload = packet[Raw].load
        if payload.startswith(b'\x10'):  # MQTT CONNECT control packet
            print("[+] MQTT CONNECT detected")
            print(payload)

sniff(prn=mqtt_sniffer, filter="tcp port 1883", store=0)

By targeting control packets, we can analyze session start behavior and payloads.

🔹 5. Storing Packets for Offline Analysis

Real-time inspection is useful, but deeper insights require logging.

💾 Code: Writing Packets to a PCAP File

from scapy.utils import wrpcap

packets = sniff(count=1000)
wrpcap('iot_traffic.pcap', packets)

Later, you can parse and visualize these with Wireshark or Pandas.

🔹 6. Parsing PCAP Data Using Python for Analysis

Once captured, PCAP files become gold mines. I used pyshark (a Python wrapper for tshark) to parse them.

📊 Code: Extracting Sessions from PCAP

import pyshark

cap = pyshark.FileCapture('iot_traffic.pcap')

for pkt in cap:
    try:
        print(f"{pkt.ip.src} -> {pkt.ip.dst} | Protocol: {pkt.highest_layer}")
    except AttributeError:
        continue

This helps in identifying session flows and unexpected protocol behavior.

🔹 7. Building Anomaly Detection with Scikit-learn

Once you extract features like packet rate, size, protocol type — you can run basic anomaly detection.

🧠 Code: Isolation Forest for Detecting Abnormal Traffic

from sklearn.ensemble import IsolationForest
import pandas as pd

data = pd.read_csv("traffic_features.csv")  # features: pkt_size, pkt_rate, etc.

clf = IsolationForest(contamination=0.05)
data['anomaly'] = clf.fit_predict(data)

anomalies = data[data['anomaly'] == -1]
print(anomalies)

This detects outliers that could indicate suspicious device behavior.

🔹 8. Live Dashboard Using Streamlit

Finally, I wrapped the whole system in a live dashboard using Streamlit to monitor devices and highlight anomalies in real-time.

📺 Code: Real-time Monitoring UI

import streamlit as st
import pandas as pd

st.title("IoT Traffic Monitor")

df = pd.read_csv("traffic_features.csv")
st.line_chart(df["pkt_rate"])

if "anomaly" in df.columns:
    st.write("Detected anomalies:")
    st.dataframe(df[df["anomaly"] == -1])

You can deploy this locally and keep tabs on all devices in real time.

🔹 9. Final Thoughts: What I Learned

Using Python to analyze network traffic gave me an incredible lens into how IoT devices behave — and misbehave. Some things I learned:

• Many devices ping cloud servers continuously.
• Some send unencrypted data, including usernames and passwords.
• Firmware updates? Often done over plain HTTP.

Python gave me full control — from packet capture to machine learning.

📌 Want the full toolchain as a project repo? Let me know. I’m happy to drop the full build with all feature extractors, models, and dashboards.