01040_Neurons

Create a Neural Network in Java

Introduction and Perceptron

01010_Intro (6:24)
01020_Why_Write_a_Neural_Network (4:57)
01025_Getting_the_Most_Out_of_This_Course (5:16)
01030_Java_vs_Python (9:32)
01040_Neurons (3:40)
01050_Perceptron (5:20)
01060_Project_With_JUnit_Support (6:18)
01070_Coding_Perceptron (6:34)
01080_Where_to_Find_the_Source_Code (3:01)
01090_Eclipse_Formatters (1:19)
01100_Logic_Gates (5:56)
01110_Perceptron_AND (9:37)
01120_OR_NOR_NAND (6:40)
01130_XOR_and_XNOR (5:04)
01140_Linear_Separability (6:36)
01150_Some_Layer_Terminology (5:10)
01160_Labelling_Weights (5:49)
01170_Matrices (5:07)
01180_Some_Mathematical_Terminology (4:08)

Matrix Mathematics

02010_A_Matrix_Class (4:30)
02020_Initialising_the_Matrix (8:36)
02030_Matrix_toString_Method (6:40)
02040_Testing_the_toString_Method (9:58)
02050_The_Apply_Method (3:58)
02060_Multiplying_Matrices_By_a_Value (5:21)
02070_Comparing_Matrices (5:07)
02080_Using_the_Equals_Method (6:08)
02090_Adding_Matrices (5:28)
02110_Motivation_for_Matrix_Multiplication (4:19)
02120_Multiplying_the_Tables
02130_Matrix_Multiplication (6:00)
02140_Matrix_Multiplication_Rule (4:47)
02150_Matrix_Multiplication_Summary (4:49)
02160_Matrix_Multiplication_Examples (8:09)
02170_Assertions (5:06)
02180_2D_to_1D (5:52)
02190_Iterating_Over_Multiplicand_Rows (4:00)
02200_Completing_the_Multiplication_Implementation (5:29)
02210_Timing_Matrix_Multiplication (6:07)
02220_Optimising_Matrix_Multiplication (6:36)

Activation Functions

03010_Neural_Net_Test_Class (4:35)
03020_Modifying_Matrices (3:12)
03030_Adding_Bias (1:45)
03040_Multiple_Columns_of_Input (4:59)
03050_ReLu (4:36)
03060_A_ReLu_Test (5:30)
03070_Matrix_forEach (4:22)
03080_Implementing_ReLu (4:48)
03090_Introducing_Softmax (4:48)
03100_Softmax_Worked_Example (2:37)
03110_Summing_Columns (4:55)
03120_Implementing_Softmax (3:15)
03130_Testing_Softmax (6:36)
03140_The_Engine (6:22)
03150_Deciding_Weight_Matrix_Sizes (5:54)
03160_An_Untrained_Network (7:02)
03170_Configuring_Dense_Layers (5:21)
03180_Adding_Multiple_Layers (5:54)
03190_Running_the_Engine (8:00)

Information Theory and Cross Entropy

04010_Mean_Squares_Loss (6:51)
04020_What_is_Information (4:22)
04030_Symbol_Spaces (6:30)
04040_Entropy (6:07)
04050_An_Optimal_Encoding_Strategy (6:26)
04060_Unequally_Probable_Symbols (4:30)
04070_Calculating_the_Information_Assoicated_With_a_Symbol (4:49)
04080_Entropy_for_Unequally_Probable_Symbols (5:45)
04090_Introducing_Cross_Entropy (5:28)
04100_Cross_Entropy_Example (7:36)
04110_Cross_Entropy_as_a_Loss_Function (6:38)
04120_Implementing_Cross_Entropy (7:08)
04130_A_Cross_Entropy_Test (6:51)
04140_Implementing_the_Cross_Entropy_Test (5:45)

Calculus and Backpropagation

05010_Training_the_Network (3:36)
05020_Gradient_Descent (4:50)
05030_Gradients_and_Neural_Networks (4:15)
05040_A_Calculus_Class (5:54)
05050_Implementing_Differentiation (6:50)
05060_Basic_Mathematical_Notation (5:45)
05070_Partial_Derivatives (5:06)
05080_Overview_of_a_3_Layer_Network (6:29)
05090_The_Network_as_a_Transform (5:18)
05100_Approximator (5:38)
05110_Mock_Expected_Data (5:40)
05120_Implementing_the_Transform (5:37)
05130_Examining_Loss (5:04)
05140_An_AddIncrement_Method (7:01)
05150_Completing_the_Approximator (5:02)
05160_Recap_and_Natural_Logarithms (6:18)
05170_Finishing_the_Approximator_Test (5:30)
05180_Back_Propagation (5:31)
05190_Obtaining_Softmax_Cross_Entropy_Gradient (7:22)
05200_Backpropagating_Errors_Through_Weighted_Sums (3:11)
05210_Introducing_the_Chain_Rule (8:04)
05220_Programming_the_Chain_Rule (7:21)
05230_Chain_Rule_for_Functions_of_Multiple_Variables (7:00)
05240_Programming_Multi_Variable_Chain_Rule (8:29)
05250_More_Mathematical_Terminology (7:45)
05260_Matrix_Transpose (5:30)
05270_Implementing_Transpose (5:18)
05280_Backpropagation_Through_Weights (4:42)
05290_Calculating_the_Backpropagation_Transform_for_Weights (8:40)
05300_Creating_a_Test_for_Weights_Backprop (5:22)
05310_Creating_Weights_and_Biases (3:53)
05320_Completing_the_Weights_Backprop_Test (6:23)
05330_Improving_the_Weight_Backprop_Test (3:05)
05340_Backpropagating_Through_ReLu (4:44)
05350_Implementing_Backpropagation_Through_ReLu (7:41)

The Neural Network Engine

06010_Adding_a_RunBackwards_Method (5:58)
06020_A_Test_Data_Class (3:04)
06030_A_Batch_Result_Class (2:42)
06040_Configuring_the_Engine (4:09)
06050_Checking_the_Loss_and_Final_Transform (5:41)
06060_Storing_Errors (2:44)
06070_Initiating_Backpropagation (3:33)
06080_Checking_Initial_Backpropagation (5:05)
06090_Addiing_Backpropagation_Through_Weights (6:16)
06100_Adding_Backpropagation_Through_ReLu (2:49)
06110_Turning_Off_Backpropagation_to_Input (2:24)

Training the Neural Network

07010_Calculating_the_Weight_Gradients_from_the_Error (9:42)
07020_Practical_Weight_Gradient_Calculation_via_Coding (6:37)
07030_Completing_the_Weight_Gradients_Test (6:24)
07040_A_Useful_Fact_Involving_Matrix_Transpose (5:31)
07050_Online_Training_versus_Batch_Training (4:37)
07060_Training_the_Network_in_Full (5:23)
07070_Testing_Average_Column (4:58)
07080_Averaging_Columns (3:21)
07090_Evaluating_Average_Loss (5:38)
07100_Getting_Weight_Inputs_and_Errors (6:10)
07110_Generating_Trainable_Data (7:22)
07120_Configuring_the_Engine_Test (3:57)
07125_Finishing_the_Adjust_Method (6:21)
07130_Getting_Greatest_Row_Numbers (7:08)
07140_Adding_Percent_Correct (6:30)
07150_Repeated_Training (7:55)
07160_Generating_Spherically_Symmetric_Distributions (6:36)
07170_Better_Training_Data (9:20)
07180_Trying_the_New_Data (6:03)
07190_A_Running_Averages_Class (5:01)
07200_Using_Running_Averages (6:17)
07210_Variable_Learning_Rates (6:53)
07220_Testing_Engine_Performance (2:47)

Epochs, Batches and Multithreading

08010_The_MetaData_Interface (5:34)
08020_The_BatchData_Interface (2:00)
08030_The_Loader_Interface (1:52)
08040_The_Neural_Network_Class (4:14)
08050_Initializing_Matrixes_with_Double_Arrays (3:25)
08060_Generating_Training_Arrays (5:24)
08070_Some_Abstract_Convenience_Classes (4:40)
08080_Creating_a_Test_Loader (7:16)
08090_Completing_the_Test_Loader (5:35)
08100_Testing_the_Test_Loader (6:30)
08105_Summing_Matrix_Elements (1:20)
08110_Finishing_the_Loader_Test (6:07)
08120_The_Fit_Method (6:59)
08130_Running_with_the_Test_Loader (3:18)
08140_Running_the_Epochs (2:54)
08150_Running_the_Batches (2:30)
08160_Implementing_RunBatch (6:11)
08170_Adding_Multithreading (5:10)
08180_Consuming_the_Batches (7:12)
08190_Fixing_the_Execution_Exception (6:20)
08200_Outputting_Metrics (6:00)
08210_Preventing_NaN (5:07)
08220_Improving_toString (6:14)
08230_Saving_the_Network (6:14)
08240_Adding_a_Load_Method (5:42)
08250_Dealing_with_the_Lock (4:19)
08260_Adding_a_Predict_Method (5:31)

Loading MNIST Images

09010_The_MNIST_Dataset (6:02)
09020_Command_Line_Arguments (6:09)
09030_Creating_the_Image_Loader (5:14)
09040_Opening_the_MNIST_files (6:49)
09050_The_MNIST_File_Format (5:17)
09060_Reading_Label_Metadata (5:07)
09070_Reading_the_Image_MetaData (3:56)
09080_Storing_the_MetaData (8:49)
09090_Structuring_Batch_Reading (7:56)
09100_Reading_Image_Data (7:10)
09110_Converting_Bytes_to_Doubles (5:05)
09120_Reading_Batches (2:48)
09130_Reading_the_Labels (6:04)
09135_An_ImageWriter_Class (5:14)
09140_Writing_Images (5:16)
09150_Determining_Canvas_Width (4:17)
09160_Calculating_Pixel_Location (6:10)
09170_Writing_the_Montage_Images (7:06)
09180_Converting_One_Hot_to_Int (5:44)
09190_Writing_the_Labels (6:31)

Putting It All Together

10010_Putting_it_All_Together (6:32)
10020_Fixing_a_Nasty_Bug (2:57)
10030_Scaling_Initial_Weights (4:17)
10040_Improving_One_Hot_Conversion (4:03)
10050_Getting_Predictions (6:04)
10060_Colorising_the_Images (5:51)

Conclusion