Anomaly Detection · Gil Eskayo

The story

Developed an anomaly detection pipeline on the

01

CTU-IoT Malware dataset

to identify malicious vs. benign network connections. This project demonstrates applying

02

unsupervised learning

techniques to cybersecurity data, including feature engineering, preprocessing, and model evaluation.

Stack: Python, pandas, scikit-learn, NumPy, Matplotlib, Seaborn.

View on GitHub

03

Key Contributions

Exploratory Data Analysis: Inspected 23 features across 23k+ network traffic entries. Explored data types, distributions, and correlations.

04

Data Preprocessing

Handled missing values and categorical encodings.
Converted IPs and ports into categorical features.
Engineered new features (e.g., rolling connection counts over time windows).

Pipeline Construction: Built preprocessing pipelines with ColumnTransformer and Pipeline to standardize numeric features and encode categorical ones.

Anomaly Detection Models: Experimented with clustering and unsupervised methods to detect unusual patterns that may indicate malware activity.

Cybersecurity Application: Interpreted anomalies in the context of malicious traffic detection.

05

Skills Demonstrated

Machine Learning (unsupervised learning, clustering, anomaly detection)
Feature engineering for network traffic data
Python ML stack: pandas, scikit-learn, numpy, matplotlib, seaborn
Cybersecurity analytics (malware/attack traffic detection)
Model pipeline design & evaluation

06

Links

Rendered Notebook PDF: ML unsupervised learning - anomaly detection pipeline
CTU-IoT Malware Kaggle Dataset: Malware Detection in Network Traffic Data