MotorMind - Sentiment Analysis & Customer Feedback Management System

Undergraduate Thesis Project (Tugas Akhir)
An end-to-end sentiment analysis web application and feedback intelligence platform for motorcycle steam wash businesses (Cuci Steam Motor), featuring a hybrid Multinomial Naive Bayes + Lexicon-Based classification engine, Explainable AI (XAI), and dual role-based dashboards for staff and business owners.

---

Important Notice: Version 2 (API Migration)

Important

This repository represents Version 1 (v1) — a full-stack monolithic application powered by Python Flask, Jinja2 server-side rendering, and an integrated machine learning pipeline.

Version 2 (v2) has transitioned the sentiment analysis engine into a decoupled, high-performance RESTful API:
sentimen-api (GitHub Repository)

If you are building a modern frontend (e.g., React, Vue, Next.js, mobile app) or integrating sentiment analysis as a microservice, please refer to the sentimen-api repository.

Architecture Evolution (v1 vs. v2)

Feature / Aspect	v1 (Tugas-Akhir — Current Repo)	v2 (sentimen-api — Next Gen)
Architecture	Monolithic Web Application (Flask + Jinja2)	Decoupled RESTful API Microservice
Interface	Server-side rendered HTML UI + Captive Portal	Stateless JSON Endpoints
Integration	Tight coupling with local database & templates	Language-agnostic HTTP/REST endpoints
Use Case	Complete standalone turnkey business portal	Microservice backend for web/mobile apps
Repository	dmsfzn/Tugas-Akhir	dmsfzn/sentimen-api

---

Table of Contents

• Overview & Motivation
• System Architecture
• Core Features
  • 1. Customer Captive Portal (WiFi Simulation)
  • 2. Hybrid Sentiment Analysis Engine
  • 3. Pegawai (Staff) Dashboard & Lexicon Management
  • 4. Owner (Executive) Dashboard & Business Insights
  • 5. Reporting & Data Export
• Machine Learning Pipeline
  • Preprocessing & Tokenization
  • Feature Extraction & Class Balancing
  • Model Training & Evaluation
  • Explainable AI (XAI)
• Tech Stack
• Directory Structure
• Getting Started
  • Prerequisites
  • Installation
  • Database Setup
  • Model Training (Optional / Reproducibility)
  • Running the Application
• Default User Roles
• License

---

Overview & Motivation

Customer satisfaction is vital in service industries, including automotive care and motorcycle steam washes. However, traditional paper comment cards or manual reviews are rarely completed and hard to analyze.

MotorMind solves this with a two-fold approach:

1. Seamless Feedback Gathering: Disguised as a WiFi access captive portal, customers waiting for their motorcycle wash submit feedback effortlessly before accessing internet amenities.
2. Intelligent Sentiment Pipeline: Customer inputs are automatically cleaned, tokenized, and classified into Positive or Negative sentiments via a hybrid Machine Learning (Multinomial Naive Bayes) + Rule-based Lexicon scoring system.
3. Data-Driven Operations: Staff monitor daily satisfaction levels, while business owners view trend charts, identify frequently mentioned pain points, and export printable operational audits.

---

System Architecture

flowchart TD
    subgraph Customer ["Customer Touchpoint"]
        A["Customer Connects to WiFi"] --> B["WiFi Captive Portal / Form"]
        B -->|Submits Review & Motor Type| C["POST /"]
    end

    subgraph Backend ["Flask Application (web/app.py)"]
        C --> D["Database Lexicon Lookup"]
        C --> E["ML Pipeline (src/predict.py)"]
        
        subgraph NLP ["NLP & Hybrid Engine"]
            E --> E1["Text Cleaning & Slang Normalization"]
            E1 --> E2["Sastrawi Stopword Filter & StemmID Stemming"]
            E2 --> E3["TF-IDF Vectorization"]
            E3 --> E4["Multinomial Naive Bayes Prediction"]
            E4 --> E5["Dynamic Lexicon Scoring Adjustment"]
            E5 --> E6["XAI Feature Importance"]
        end

        E6 --> F[("MySQL Database (motormind_db)")]
        D -.-> E5
    end

    subgraph StaffPortal ["Pegawai (Staff) Portal"]
        F --> G1["Staff Dashboard (Daily Metrics)"]
        F --> G2["Feedback History & Multi-filter"]
        F --> G3["Lexicon Dictionary Management (CRUD)"]
    end

    subgraph OwnerPortal ["Owner (Executive) Portal"]
        F --> H1["Owner Dashboard (7-Day Trends)"]
        F --> H2["Negative Review Word Frequency Insights"]
        F --> H3["Employee Account Management"]
        F --> H4["Printable PDF/HTML Reports & CSV Export"]
    end

Loading

---

Core Features

1. Customer Captive Portal (WiFi Simulation)

• Responsive mobile-friendly portal simulating public WiFi login redirect.
• Collects customer name (optional/anonymous), motorcycle model/type, review text, and improvement suggestions.
• Real-time sentiment prediction on submit with instantaneous success redirect.

2. Hybrid Sentiment Analysis Engine

• Combines the statistical strength of Multinomial Naive Bayes with the precision of a customizable Lexicon Dictionary.
• Prevents misclassification of domain-specific customer jargon or slang.
• Computes confidence scores and polarity intensities.

3. Pegawai (Staff) Dashboard & Lexicon Management

• Dashboard: Quick view of total feedback received, positive vs. negative volume, and latest 5 submissions.
• Review History: Comprehensive tabular list with full-text search, sentiment filtering (all, positif, negatif), date range picker, and pagination.
• Lexicon Manager: Directly add, modify, or remove polarity words in the database to iteratively improve sentiment scoring without retraining the model.

4. Owner (Executive) Dashboard & Business Insights

• Executive Analytics: 7-day daily submission trend graphs, positive/negative percentage distribution cards.
• Pain Point Analysis (Insight): Automatic extraction of top 15 most frequent keywords appearing specifically in negative reviews across an 8-week rolling window.
• Employee Management: Create, edit, and deactivate staff operator accounts (users table).

5. Reporting & Data Export

• CSV Data Export: Download filtered feedback data directly into Excel/CSV format with full metadata (customer, motorcycle, text, confidence, lexicon score, timestamp).
• Printable Reports: Dedicated print-optimized templates for:
  • Full feedback history audit
  • Registered employee list
  • Lexicon dictionary snapshot
  • 30-day statistical breakdown & vehicle popularity rankings
  • Negative complaint keyword insights

---

Machine Learning Pipeline

Raw Text 
  -> Regex Cleaning (URLs, special characters) 
  -> Slang Replacement ("nyuci" -> "cuci") 
  -> Stopword Removal (Custom Sastrawi) 
  -> Indonesian Stemming (StemmID) 
  -> TF-IDF Vectorizer (n-gram 1-2, sublinear TF) 
  -> Multinomial Naive Bayes (alpha = 0.5) 
  -> Lexicon Threshold Adjustment 
  -> Final Label & Confidence

Preprocessing & Tokenization

• URL & Noise Stripping: Removes links, special characters, and excessive whitespace.
• Domain Normalization: Preserves sentiment-critical negations (tidak, kurang, belum, ga, tak) while filtering non-informative words.
• Stemming: Powered by stemmid for fast and accurate Indonesian morphological root extraction.

Feature Extraction & Class Balancing

• TF-IDF Vectorizer: Configured with ngram_range=(1, 2), sublinear_tf=True, min_df=2, and max_df=0.95.
• RandomOverSampler (imblearn): Balances positive and negative classes during training to mitigate skewed datasets.

Model Training & Evaluation

• Classifier: MultinomialNB(alpha=0.5).
• Validation: 5-Fold Stratified Cross-Validation ensuring model generalization.
• Outputs artifacts: model/model.pkl and model/vectorizer.pkl.

Explainable AI (XAI)

• Local feature contribution extraction using log-probability variance: $$\text{Strength} = |\log P(w \mid \text{Positif}) - \log P(w \mid \text{Negatif})|$$
• Identifies the exact words that tipped the classification into positive or negative territory.

---

Tech Stack

• Language: Python 3.9+
• Web Framework: Flask 3.0.2, Jinja2, Werkzeug
• Database: MySQL 8.0+ / MariaDB via mysql-connector-python
• Machine Learning & NLP:
  • scikit-learn 1.8.0 (MultinomialNB, TfidfVectorizer, metrics)
  • imbalanced-learn 0.14.1 (RandomOverSampler)
  • stemmid 0.3 (Indonesian Stemmer)
  • Sastrawi 1.0.1 (Indonesian Stopword Removal Factory)
  • joblib 1.5.3 (Model persistence)
  • pandas 2.3.3 & openpyxl 3.1.5 (Data processing)
• Frontend: Vanilla CSS, HTML5, Chart.js, Feather/FontAwesome Icons

---

Directory Structure

Tugas Akhir/
├── data/
│   ├── dataset_TA.xlsx              # Raw dataset (training/validation)
│   ├── dataset_preprocessed.csv     # Cleaned and stemmed dataset
│   └── data_ujian.csv               # Test split data
├── model/
│   ├── model.pkl                    # Trained MultinomialNB artifact
│   └── vectorizer.pkl               # Fitted TfidfVectorizer artifact
├── notebook/
│   └── TA_notebook.ipynb            # Jupyter Notebook for experimentation & research
├── src/
│   ├── config.py                    # Global paths and seed configuration
│   ├── preprocess.py                # Text normalization and preprocessing script
│   ├── train.py                     # ML training, cross-validation & evaluation
│   ├── predict.py                   # Prediction logic, lexicon scoring & XAI
│   └── run_all.py                   # Master pipeline executor
├── web/
│   ├── app.py                       # Flask server, route controllers & auth guards
│   ├── static/                      # CSS styling, client-side JS, images
│   └── templates/                   # Jinja2 templates (dashboard, portal, reports)
├── requirements.txt                 # Python package dependencies
├── schema.sql                       # MySQL database DDL & initial seed records
└── README.md                        # Documentation

---

Getting Started

Prerequisites

• Python 3.9 or higher installed on your system.
• MySQL / MariaDB database server running (e.g., via Laragon, XAMPP, or native MySQL Service).

Installation

1. Clone the repository:

   git clone https://github.com/dmsfzn/Tugas-Akhir.git
   cd Tugas-Akhir

2. Create and activate a virtual environment:

   # Windows (PowerShell)
   python -m venv venv
   .\venv\Scripts\Activate.ps1
   
   # Linux / macOS
   python3 -m venv venv
   source venv/bin/activate

3. Install dependencies:

   pip install -r requirements.txt

Database Setup

1. Make sure your MySQL service is running on localhost:3306.
2. Import the schema file using MySQL CLI, phpMyAdmin, or your preferred GUI tool:

   mysql -u root -p < schema.sql

   (By default, this creates a database named motormind_db with default seed data and tables).
3. Verify or adjust your MySQL connection credentials in web/app.py:

   DB_CONFIG = {
       'host': 'localhost',
       'user': 'root',
       'password': '',
       'database': 'motormind_db'
   }

Model Training (Optional / Reproducibility)

Pre-trained artifacts (model.pkl and vectorizer.pkl) are already included in the model/ folder. If you wish to retrain the model with fresh data:

# Run the entire pipeline (Preprocessing -> Training -> Interactive Predictor)
python src/run_all.py

# Or execute individually:
python src/preprocess.py
python src/train.py

Running the Application

Start the Flask web server:

python web/app.py

Once started, open your web browser and navigate to:

• Customer WiFi Portal: http://localhost:5000/
• Internal Staff / Owner Login: http://localhost:5000/login

---

Default User Roles

The database seed script (schema.sql) provides initial accounts for testing:

Role	Default Identifier	Access Permissions
Owner (Pemilik)	OWN001	Full administrative control: View 7-day & 30-day analytics, negative keyword insights, manage employee accounts, manage lexicon, export CSV, and print reports.
Pegawai (Staff)	EMP001 / EMP002	Operational control: View personal/daily metrics, browse review history with filters, and manage lexicon entries.

(Authentication uses MD5 hashing for passwords as defined in schema.sql).

---

Next Steps: Exploring the REST API (v2)

Looking for an API-first approach? Check out Version 2:

Version 2 isolates the preprocessing and inference steps into lightweight HTTP JSON endpoints, making it simple to plug sentiment classification into third-party apps, chatbots, or modern frontend frameworks.

---

License

This project was developed for academic and practical research under the Tugas Akhir curriculum. Distributed under the MIT License.