The amateur’s guide to explore machine learning

Welcome.

I created this documentation as my exploration of machine learning & data science.

I usually forget logics, methods, algorithms and code. Hence to keep everything at one place so that I can go through the topic quickly as referenced.

Source of the content mostly from a free course, video, research paper and book (most of the references are mentioned with the topic).

If I haven’t mentioned your reference correctly then let me know, I’ll add it asap.

email	nishantbaheti.it19@gmail.com
github	https://www.github.com/nishantbaheti
linkedin	https://www.linkedin.com/in/nishantbaheti/

Basics

• Basics of Statistics
  • Type of data
  • Location Estimates
  • Variability Estimates
  • Multi Variate Analysis
• Covariance and Correlations
  • Covariance
  • Correlation Coefficient
• Sampling
  • Sample
  • Random Sampling
  • Sampling Bias
  • Random Seeding
  • Random Variable
  • Standard Error of Mean (SEM)
  • Bootstrap
  • Confidence Intervals
• Probability
  • References
  • Definition
  • Infinite Series
  • Spaces
  • Conditional Probability & Independence
  • Compound Probability
• Distributions
  • References
  • Data Distributions
  • Normal(Gaussian) Distribution (Continuous)
  • Chi-squared Distribution (Continuous)
  • Student’s t distribition (Continuous)
  • F Distribution (Continuous)
  • Uniform Distribution (Continuous)
  • Beta Distribution (Continuous)
  • Binomial Distribution (Discrete)
  • Hypergeometric Distribution (Discrete)
  • Poisson Distribution (Discrete)
  • Exponential Distribution (Discrete)
  • Weibull Distribution (Discrete)
  • Points to note
• Probability Estimation
  • References
  • Intro
  • Maximum Likelihood Estimation(MLE)
  • Naive Bayes
  • Bayes Classifier
  • The Bayesian Way
  • Maximum A Posteriori Probability Estimation(MAP)
  • Bayesian Statisics vs Frequentist Statisitcs
  • Connecting this to ML
• Bias-Variance Tradeoff
  • Reference
  • Concept
  • Bias Error
  • Variance Error
  • Data Prep
  • Proof

Experimentation

• Experiments & Testing
  • A/B test
  • Metrics
  • Hypothesis Test/ Significance Test
  • Types of tests
  • Resampling
  • Statisitcal Significance
  • Hypothesis Errors
  • Statistical Tests
  • t-Tests
  • Multiple Tests
  • Chi-squared(\(\chi^2\)) test (pearson’s)
  • ANOVA or Fisher analysis of variance
• Additional Statistics Examples
  • 1 Sample Quantile-Quantile(Q-Q) plot
  • 2 Samples Quantile-Quantile(Q-Q) plot

Linear Models

• Perceptron
  • Algorithm Assumptions
  • Classifier
  • Algorithm
  • Implementation
  • Hyperplane
  • Intuition
  • Margin
• Linear Regression
  • References
  • Intro
  • Generate Data
  • Check for bias column(column 0)
  • Regression Models Error Evaluation Functions
  • Algorithm
  • Generate Prediction
  • Batch Gradient Descent
  • Stochastic gradient Descent
  • Normal Equation (Closed Form)
• Ridge(L2 Regularization) Regression
  • References
  • Theory
  • Generate Data
  • Ridge with Normal Equation
  • Ridge with BGD
  • Ridge with SGD
• Lasso(L1 Regularization) Regression
  • References
  • Theory
  • Generate Data
  • Lasso with BGD
  • Lasso with SGD
• Logistic Regression
  • Cost Function
  • Sigmoid Function
  • Gradient Descent Algorithm
  • Logitsic Regression without Regularization
  • Logistic Regression with Regularization
• Trend, Slope & Regression
  • Trend
  • polynomial regression
• Support Vector Machine
  • References
  • Assumptions
  • Concept
  • Margin
  • Algorithm

Proximity Models

• Vector Distances
  • Minkowski distance
  • Manhattan distance
  • Euclidean distance
  • Cosine distance & Cosine Similarity
• K-Nearest Neighbors
  • References
  • Dataset Setup
  • Assumption
  • Algorithm
  • Training
  • Prediction & Decision Boundary
• Curse Of Dimensionality
  • References
  • Concept
  • Proofs

Tree Based and Ensemble Models

• Decision Tree Algorithm
  • CART(Classification and Regression Trees)
  • Question(Partition concept of the dataset)
  • Partition rows
  • Tree Node - Leaf node, Decision Node
  • Build Tree
  • Classification Tree
  • Regression Tree
• Random Forest Algorithm
  • Bootstrap (Resampling)
  • Algorithm of sampling for Random Forest
  • Random Forest Algorithm
  • Random Forest Classifier Model
  • Random Forest Regressor Model
• Boosting Methods
  • Adaboost Classfication
  • Gradient Boosting Regression
  • Gradient Boosting Classification

Image & Signal Procressing

• Convolution
  • Signal Convolution
  • matrix convolution
• Image Convolution
  • Compress image
  • converting to float
  • Edge Detection
  • Sharpen
  • Box Blur
  • Gaussian Blur 3X3
  • Gaussian Blue 5X5
  • Brighten
  • Costum Function
  • Vertical Edge
  • Horizontal Edge
  • Sobel Operator
  • Scharr Operator
• Image Processing
  • BGR to Grayscale
  • from scratch
  • using pillow
  • using cv2
• Image Segmentation
  • Loading Image
  • Flattening
  • Convert to gray scale
  • Gaussian smoothing
  • OpenCV Canny Edge Detection
• Fourier Transform
  • Euler’s Number(e)
  • Sine wave
  • Sine Wave Animation
  • Discrete Fourier Transform
  • Symmetries in DFT
  • Fast Fourier Transform
  • Inverse Frequency Fourier Transform
  • Using numpy fft methods
• Denoising data using Fast Fourier
  • Genereting a noisy signal
  • Calculating FFT
  • Power Spectral Density
  • Pickup index based on PSD threshold
  • Cleanup using PSD
  • Cleanup frequency transforms
  • Regenerate signal using Inverse Fourier Transform
• Audio Processing
  • Setup pyaudio
  • Plot data
  • Live audio setup
  • FFT processing & Power Spectral Density

Unsupervised

• Singular Value Decomposition
  • Understanding SVD
  • U & V orthogonality (Unitary matrices)
  • Eckart-Young Theorem (Low rank approximation)
  • SVD and Correlations
  • Method of Snapshots (SVD using eigen values and vectors)
  • Vector outer product
  • SVD and Image Compression
  • Log and Cumulative plots
  • Rotation matrix
  • Unitary transformation
• Principal Component Analysis
  • Statistical interpretation of SVD
  • PCA on Gaussian Distribution
  • Breast Cancer Data Anlysis with PCA
• SVD,Linear Systems and Least Square
  • Types of Non-Square Matrix solutions
  • Minimum Norm Solution
  • Least Squares Solution (Calculating \(\tilde{\theta}\))
  • Moore-Penrose Inverse
  • Understanding spaces
  • Calculate with Data
  • Another method to calcualte \(\tilde{\theta}\)
  • Boston housing data regression analysis

Feature Engineering

• Weight of Evidence & Information Value
• Moving Window Frame
  • Iteration By Row
  • Iteration By Column
• Sparse Matrix
  • Compressed Sparse Row Matrix
  • Block Sparse Row Matrix
  • COOrdinate Matrix
  • Diagonal Matrix
• Feature Scaling Techniques
  • Standard Scaler/ z-scale
  • Robust Scaler
  • Min Max Scaler
  • Max Absolute Scaler
  • Normalizer
  • Normalizer vs Scaler
  • Data based comparison

Intuition & Evaluation

• Vectors
  • dot product
  • cross product
  • magnitude \(||\vec{A}||\)
  • project \(\vec{A}\) to \(\vec{B}\)
  • Projection
• Algorithm’s Classification Boundaries
  • 2 classes, simple spiral data
  • 3 classes, simple spiral data
  • 3 classes, Complex spiral data
  • 5 classes, simple spiral data
• Classification Evaluation Metrics
  • References
  • Conditions
  • Confusion Matrix
  • Errors
  • Accuracy
  • Precision
  • Recall / Sensitivity / True Positive Rate (TPR) / Hit Rate
  • F1-score
  • Classification Report
  • Specificity / Selectvity / True Negative Rate (TNR)
  • Fall Out / False Positive Rate (FPR)/ False Alarm
  • Miss rate / False Negative Rate (FNR)
  • PRC (Precision-Recall Curve)
  • ROC (Receiver Operating Characteristic Curve)
• Matrix Multiplication
  • References
  • How it happens
  • Why it happens

Visualizations

• Matplotlib Visualization
• Live Visual
  • 2D Live Visual
  • 3D Live Visual
• Plot 3D
  • Surface
  • wireframe
  • Line in 3D plane
  • Scatter in 3D plane
  • Sphere
  • Paraboloid
• Plotly Exploration
  • Scatter Plot
  • Line Plot
  • CandleStick Plot
  • Histogram Plot
  • Box Plot

Miscellaneous

• Statsmodels
  • Linear Models
  • Linear Regression
  • Non-Linear Models
  • Design Matrices
  • Creating a Model
  • Fitting a Model
  • View Model Summary
  • Construct Design Matrices
  • Design Matrix with Numpy
  • Design Matrix with patsy
  • Linear Model Creation using statsmodels
  • Logistic Regression
  • Poisson Regression Model
• Graph Network
  • References
  • Undirected Network
  • Directed Network

Additional Docs

• Recommendation Systems Doc
• Neural Network Doc
• NLP Doc
• mightypy
• graphpkg

Work In Progress

• Bayes Estimation
  • References
  • Naive Bayes
  • Bayes Classifier
• Anomaly Detection using Scikit-Learn
  • Isolation Forest
  • Elliptic Envelope
• Time Series Forecasting
  • References
  • Introduction to Time Series
  • Load Data
  • AutoCorrelation Function
  • Partial AutoCorrelation Function
  • Seasonal Decompostion
• Markov Chains
  • Reference