Georgia Institute of Technology

School of Electrical and Computer Engineering

ECE 4803: Fundamentals of Machine Learning (FunML) (3-0-3)

Spring 2021, Spring 2022

Instructor:    Prof. Ghassan AlRegib | alregib@gatech.edu | www.ghassanalregib.info  

TAs: Mr. Ahmad Mustafa and Mr. Chen Zhou

Textbook: No required textbook but the following books are excellent references for this class:

1. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, The MIT Press (November 18, 2016), ISBN-10: 0262035618, ISBN-13: 978-0262035613 Available online: https://www.deeplearningbook.org/
2. Aggelos Konstantinos Katsaggelos, Jeremy Watt, and Reza Borhani, Machine Learning Refined: Foundations, Algorithms, and Applications, Cambridge University Press; 2 edition (March 12, 2020), ISBN-10: 1108480721, ISBN-13: 978-1108480727

Prerequisite: ECE 2026 [min C] (or equivalent)

Prerequisites with concurrency: ECE 3077 [min C] or ISYE/MATH/CEE 3770 [min C] or MATH 3670 [min C]

Course Objective: An introduction to the fundamentals and applications of Machine Learning  

Assignments: Assignments will have both analytical part and a hands-on part. The hands-on parts are coding assignments and they will look like mini projects. Students are expected to have some background in Python. Students are encouraged to utilize Google’s Colab for their codes in this course. These hands-on assignments vary between adding to existing codes, writing codes from scratch, searching the literature for codes for a specific application, or summarizing papers in the literature. 

Programming Language: We will utilize Python throughout the course. We will also utilize a library of Jupyter notebooks in Colab.  

Topical Outline:

• Overview
  • History of Pattern recognition, Development of an ANN
  • Types of Learning i.e., Supervised, Semi-supervised, Weakly supervised, Un-supervised
  • General features of a supervised learning system i.e. features, training/validation set, labels, model complexity and overfitting etc.
  • Simple overview of Optimization
• Classification     
  • Algorithms: Nearest Neighbors, Logistic Regression, Decision Trees, Random Forest, SVM, ANN
  • Classification Performance Evaluation  
    • Cross-Validation using k-fold, Confusion Matrix, Precision, Recall, and F1 score, ROC,
  • A set of hand-on exercises on Colab
• Regression:
  • Linear Regression        
    • Linear Regression, Performance Measures
    • Cost Function
  • Polynomial Regression 
    • High Degree Polynomial Regression
  • Regularized Linear Models       
    • Lasso vs. Ridge Regularization (L1/L2 regularization)
    • Dealing with high dimensional feature space – PCA
  • A set of hand-on exercises on Colab
• Clustering
  • Introduction     
  • Proximity Measures     
  • Similarity vs. Dissimilarity
  • Distance Measures
  • Common Clustering Methods   
    • k-Means, GMM, Mean-shift, Spectral Clustering
  • Evaluating Clustering Performance
  • Image Segmentation as a clustering problem
  • A set of hand-on exercises on Colab      
• Neural Networks
  • Introduction to Artificial Neural Network:
    • Artificial Neuron
    • PyTorch
  • Non-linearity, Activations, Losses         
  • ConvNets        
    • convolutional layer, pooling, FC, training
  • A set of hand-on exercises on Colab
• Autoencoders         
  • Fully Connected autoencoders, Conv AE, VAE
  • A set of hand-on exercises on Colab
• Advanced Topics
  • Generative models and GANs
  • Sequence models         
    • Plain RNN
    • GRUs and LSTMs
  • Boosting, Bagging, Stacking
  • Transfer learning
  • Data augmentation