Neural Network Project (NNN)

A complete neural network ecosystem with a custom deep learning framework, genetic hyperparameter optimization, and chess position analysis.

Modules

1. my_torch - Neural Network Framework

A lightweight, NumPy-based deep learning library.

Key Features:

• Multiple activation functions (Sigmoid, ReLU, Leaky ReLU, Softmax)
• Loss functions for regression (MSE) and classification (Cross-Entropy)
• Mini-batch gradient descent with learning rate scheduling
• Early stopping and gradient clipping
• Model persistence

See my_torch/README.md for detailed documentation.

2. generator - Genetic Hyperparameter Optimization

Trains neural networks using genetic algorithms to find optimal hyperparameters.

Usage:

# Train with genetic algorithm
python -m generator.main --genetics dataset.chess 10

# Train from config file
python -m generator.main example/config_file_xor.json 1000

What it optimizes:

• Learning rate and decay strategies
• Batch size
• Network architecture (number and size of layers)

3. analyzer - Chess Position Analysis

Evaluate and train models for chess position analysis.

Usage:

# Make predictions
python -m analyzer.main --predict model.pkl positions.chess

# Train existing model
python -m analyzer.main --train --save new_model.pkl model.pkl training_data.chess

Installation

# Install dependencies
pip install -r requirements.txt

# Requirements:
# - numpy==2.3.3
# - python-chess
# - pyinstaller

Quick Start

Using my_torch Library

import numpy as np
from my_torch.core.neural_network import NeuralNetwork
from my_torch.utils.activation_functions import sigmoid, sigmoid_derivation
from my_torch.utils.cost_functions import mse, mse_derivation
from my_torch.utils.init_functions import he_initialization, bias_zeros

# Create and train a simple network
nn = NeuralNetwork(
    learning_rate=0.5,
    lr_decay_type="step",
    lr_decay_rate=0.9,
    lr_step_size=100,
    batch_size=4,
    gradient_clipping=10.0,
    loss_func=mse,
    loss_deriv=mse_derivation
)

nn.add_input(2, 4, sigmoid, sigmoid_derivation, he_initialization, bias_zeros)
nn.add_layer(1, sigmoid, sigmoid_derivation, he_initialization, bias_zeros)

# XOR problem
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([[0], [1], [1], [0]])

nn.train_batch(X, y, epochs=1000)

Configuration Files

Example configuration for chess classification (see example/config_file_chess.json):

{
  "neural_network": {
    "learning_rate": 0.08,
    "lr_decay_type": "inverse_time",
    "lr_decay_rate": 0.005,
    "batch_size": 64,
    "gradient_clipping": 10,
    "loss_function": "cross_entropy",
    "input_size": 832,
    "layers": [
      {"size": 256, "activation_function": "leaky_relu"},
      {"size": 128, "activation_function": "leaky_relu"},
      {"size": 64, "activation_function": "leaky_relu"},
      {"size": 3, "activation_function": "softmax"}
    ]
  }
}

Architecture Overview

my_torch Framework

Input Layer → Hidden Layers → Output Layer
     ↓             ↓              ↓
 [Features]    [ReLU/Sigmoid]  [Softmax/Sigmoid]
                    ↓
            Backpropagation
                    ↓
         Gradient Descent Update

Training Process:

1. Forward pass through all layers
2. Compute loss
3. Backward pass (compute gradients)
4. Update weights with learning rate
5. Optional: learning rate decay, early stopping

Genetic Algorithm

Generation 0: Random population
     ↓
Evaluate fitness (train & measure loss)
     ↓
Select best performers
     ↓
Crossover + Mutation
     ↓
Generation 1: Improved population
     ↓
Repeat...

Development

Testing

# Test XOR problem
python -m generator.main example/config_file_xor.json 1000

# Test genetic training
python -m generator.main --genetics dataset.chess 5

API Documentation

All classes and functions in my_torch include comprehensive docstrings:

# View documentation
help(NeuralNetwork)
help(Layer)
help(sigmoid)

Limitations

• my_torch: CPU-only, no GPU acceleration
• Architecture: Feedforward networks only (no CNN/RNN/LSTM)
• Optimization: Basic SGD only (no Adam, RMSprop, etc.)
• Use Case: Educational/research purposes

For production applications, use PyTorch or TensorFlow.

License

Educational project - Epitech Tek3 Mathematics Module