This project implements a simple feedforward neural network in C++ to approximate the sine function over one period. The network consists of an input layer, a hidden layer with 32 neurons, and an output layer. Training is performed using gradient descent with backpropagation, and predictions are visualized using a Python script.
- NeuralNetwork.cpp: Core C++ implementation of the neural network, including utility functions for matrix operations, activation functions (tanh and sigmoid), and the main training loop.
- predictions.csv: Output file containing the predicted and actual values of the sine function.
- visualizer.py: Script to plot the predicted values against the ground truth.
- Understanding Neural Network Fundamentals: Learned how neural networks model non-linear functions by combining linear transformations and activation functions. Explored the structure and flow of data through input, hidden, and output layers.
- Matrix Operations in C++: Gained hands-on experience with matrix and vector operations using C++ standard library containers. Implemented custom functions for matrix multiplication, addition, subtraction, and transposition.
- Activation Functions: Explored the role of activation functions like tanh and sigmoid in introducing non-linearity and enabling the network to approximate complex functions.
- Loss Functions and Training: Implemented mean squared error (MSE) as the loss function and trained the network using gradient descent with backpropagation. Learned how to update weights and biases based on the computed gradients.
- Backpropagation: Developed an understanding of how error is propagated backward through the network to adjust weights and biases, improving the model’s predictions over time.
- Data Preprocessing and Normalization: Practiced normalizing input and output data to improve training stability and convergence.
- Visualization: Used Python and matplotlib to visualize the network’s predictions compared to the actual sine function, reinforcing the importance of model evaluation and interpretation.
- Challenges and Insights: Encountered and addressed challenges related to weight initialization, learning rate selection, and the impact of network architecture on model performance. Learned that even simple networks can approximate non-linear functions, but tuning hyperparameters and architecture is crucial for good results.
This project provided a solid foundation in implementing neural networks from scratch, deepening my understanding of both the theory and practical aspects of machine learning in C++. It also highlighted the importance of experimentation and visualization in developing and evaluating machine learning models.