AFA Studio
← Back to Gallery

Data Science & Unsupervised Machine Learning

3D Semantic Cluster Universe

Dimensionality Reduction (3D PCA) and Unsupervised Topological Clustering applied to aggregated multi-source metadata streams.

Dimensionality

High-Dim → 3D PCA

Algorithm

Unsupervised K-Means

Visualization Engine

Plotly WebGL 3D
🛡️

Data Privacy Disclaimer (GDPR Compliance): For data protection and privacy reasons, all personal identifiable information (PII) exposed in this interactive 3D WebGL visualization—including email addresses, names, and subjects—has been masked, scrambled, and anonymized to serve strictly as an illustrative demonstration dataset.

⚡ Quick End-to-End System Flow Architecture Summary 📐

1. INGESTION & EDA mailbox_map.json Pandas & NLTK Cleaned Metadata 2. FEATURES & VECTOR Regex Domain Isolation OneHot + Scaler Tabular Feature Matrix 3. LLM EMBEDDING all-MiniLM-L6-v2 K-Means (384-Dim) Semantic Vectors 4. 3D PCA & WEBGL 3-Component PCA Projection Plotly WebGL 3D Scatter 3D Interactive Cluster Universe

1. Data Ingestion & EDA

Extracted JSON records into Pandas DataFrames. Performed Datetime Normalization, regex domain extraction, and NLTK Stop-Word Removal.

2. Feature Engineering

Engineered interaction metrics and encoded sparse matrices via OneHotEncoder and StandardScaler, validated with the Elbow Method.

3. Transformer Embeddings

Generated 384-dimensional dense vectors using Sentence-Transformers (all-MiniLM-L6-v2) and executed K-Means Clustering.

4. 3D PCA & WebGL

Reduced 384D vectors to 3 orthogonal axes using Principal Component Analysis (PCA). Cross-validated clusters using Seaborn Heatmaps.

⚙️ Detailed Data Science Pipeline & Techniques

01

Data Ingestion & Exploratory Data Analysis (EDA)

  • Raw Data Parsing: Parsed JSON payload arrays into tabular Pandas DataFrames.
  • Temporal Standardization: Converted UNIX millisecond timestamps and string timestamps to ISO-8601 Datetime Objects (date_dt) for time-series aggregation.
  • Text Mining & Natural Language Processing (NLP): Used NLTK (Natural Language Toolkit) for stop-word removal, corpus cleaning, and Frequency Distribution Analysis via WordCloud generation.
02

Tabular Feature Engineering & Supervised Evaluation

  • Pattern Mining & Regular Expressions: Isolated domain entities using Regex Pattern Matching ([\w\.-]+@([\w\.-]+)) across multi-recipient header fields.
  • Feature Construction: Extracted continuous numerical features including recipient counts (num_to, num_cc, total_recipients) and domain frequency distributions.
  • Scikit-Learn Pipeline Vectorization: Built a ColumnTransformer pipeline combining One-Hot Encoding for categorical domains and Standard Scaling (Z-Score Normalization) for continuous interaction features.
  • Hyperparameter Tuning & Elbow Method: Calculated Sum of Squared Errors (SSE / Inertia) across candidate cluster counts ($k \in [2, 15]$) to identify optimal cluster density breakpoints.
03

Transformer-Based Semantic Vector Embeddings

  • Text Concatenation & Context Aggregation: Constructed unified semantic strings by merging header subjects with domain meta-tags.
  • Transformer Embedding Extraction: Leveraged Hugging Face’s Sentence-Transformers with the pre-trained all-MiniLM-L6-v2 model to generate dense 384-dimensional latent representations.
  • Unsupervised High-Dimensional Clustering: Executed K-Means Vector Clustering directly over the 384D embedding space to categorize latent textual intent.
04

3D Dimensionality Reduction & WebGL Rendering

  • Principal Component Analysis (PCA): Applied Dimensionality Reduction to project 384-dimensional embeddings into 3 orthogonal latent axes (pca_3d_1, pca_3d_2, pca_3d_3) while maximizing variance retention.
  • Ground-Truth Matrix Benchmarking: Cross-tabulated unsupervised semantic topics against rule-based ground-truth categories using Seaborn Alignment Heatmaps.
  • Interactive WebGL Export: Exported 3D spatial coordinate matrices into an interactive, GPU-accelerated Plotly WebGL Scatter Graph (cinematic_3d_semantic_universe.html).