Deep Learning - Machine Learning - Matrix Theory Notes

These are the collections of my notes from my projects in Deep Learning, Machine Learning, Metric Learning, and Matrix Theory.

• Parallel Recursive Skeletonization Solver for Dense Linear Systems on GPU-Accelerated Computers
  
  
  View summary of Parallel Skeletonization Solver / Hide summary View my notes in Parallel Skeletonization Solver
  The dense linear systems in large-scale kernel approximation in machine learning, low-rank Schur complements of a large sparse matrix factorization, and discretization of boundary integral equations in physics and engineering often employ a multilevel structure of lowrank off-diagonal blocks. To solve such large systems efficiently, we present a GPU-based parallel recursive skeletonization solver utilizing batched dense linear algebra to allow for adjustable precision and achieve a linear rate with the matrix size given fixed ranks.
• Discrete Empirical Interpolation Method (DEIM)
  
  
  View summary of DEIM / Hide summary View my notes in DEIM
  Discrete Empirical Interpolation Method (DEIM) is a technique used in the field of nonlinear model reduction to efficiently approximate high-dimensional nonlinear systems. DEIM reduces the computational complexity of such systems by creating a lower-dimensional approximation that retains the essential dynamics of the original model. It works by constructing an empirical basis for the nonlinear terms and then projecting these terms onto a reduced space. This approach is particularly useful in scenarios where direct simulation of the full model is computationally prohibitive, allowing for faster and more efficient analyses while preserving the accuracy of the system's behavior. DEIM is commonly applied in engineering and scientific computing, where it facilitates real-time simulations and optimizations of complex systems.
• Deep Neural Networks
  
  
  View summary of DNNs / Hide summary View my notes in DNNs
  Deep neural networks are a type of artificial intelligence that mimic the way the human brain processes information to recognize patterns and make decisions. These networks consist of multiple layers of interconnected nodes, or neurons, each of which performs a simple mathematical operation on the data. The architecture of a deep learning network typically includes an input layer, several hidden layers, and an output layer. Each layer is made up of many neurons that receive input from the previous layer, process it, and pass it on to the next layer.
  
  When data enters the network, it begins at the input layer, which represents the raw information, such as images, text, or audio. The data is then passed through the hidden layers, where each neuron performs a weighted sum of the inputs and applies a non-linear function to this sum. This series of transformations helps the network learn complex features and patterns in the data.
  
  To enable the network to make accurate predictions or classifications, it undergoes a training process involving large datasets. During training, the network adjusts the weights of the connections between neurons based on the errors it makes. This adjustment is guided by a technique called backpropagation, which helps the network refine its predictions by minimizing the difference between its outputs and the actual results.
  
  As the network processes more data and adjusts its weights through many iterations, it becomes better at recognizing patterns and making accurate predictions. Once training is complete, the network can be used to make predictions on new, unseen data by passing it through the trained layers, resulting in an output that reflects the learned patterns. In summary, deep learning networks enhance their performance by learning from data through multiple layers of processing, adjusting connections based on errors, and continually improving their ability to make accurate decisions.