Data Sparse Algorithms And Mathematical Theory For Large Scale Machine Learning Problems
DOWNLOAD
Download Data Sparse Algorithms And Mathematical Theory For Large Scale Machine Learning Problems PDF/ePub or read online books in Mobi eBooks. Click Download or Read Online button to get Data Sparse Algorithms And Mathematical Theory For Large Scale Machine Learning Problems book now. This website allows unlimited access to, at the time of writing, more than 1.5 million titles, including hundreds of thousands of titles in various foreign languages. If the content not found or just blank you must refresh this page
Data Sparse Algorithms And Mathematical Theory For Large Scale Machine Learning Problems
DOWNLOAD
Author : Ruoxi Wang
language : en
Publisher:
Release Date : 2018
Data Sparse Algorithms And Mathematical Theory For Large Scale Machine Learning Problems written by Ruoxi Wang and has been published by this book supported file pdf, txt, epub, kindle and other format this book has been release on 2018 with categories.
This dissertation presents scalable algorithms for high-dimensional large-scale datasets in machine learning applications. The ability to generate data at the scale of millions and even billions has increased rapidly, posing computational challenges to most machine learning algorithms. I propose fast kernel-matrix-based algorithms that avoid intensive kernel matrix operations and neural-network-based algorithms that efficiently learn feature interactions. My contributions include: 1) A structured low-rank approximation method--the Block Basis Factorization (BBF)--that reduces the training time and memory for kernel methods from quadratic to linear and enjoys better accuracy than state-of-art kernel approximation algorithms. 2) Mathematical theories for the ranks of RBF kernel matrices generated from high-dimensional datasets. 3) A parallel black-box fast multipole method (FMM) software library--PBBFMM3D--that evaluates particle interactions in 3D. 4) A neural network--the Deep & Cross Network (DCN)--for web-scale data predictions that requires no exhaustive feature searching nor manual feature engineering and efficiently learns bounded-degree feature interactions combined with complex deep representations. Chapter 2 presents BBF, which accelerates kernel methods by factorizing an n by n kernel matrix into a sparse representation with O(n) nonzero entries as compared to O(n^2). By identifying the low-rank properties of certain blocks, BBF extends the domain of applicability of low-rank approximation methods to the cases where traditional low-rank approximations are inefficient. By leveraging the knowledge from numerical linear algebra and randomized algorithms, the factorization can be constructed in O(n) time complexity while being accurate and stable. Our empirical results demonstrate the stability and superiority over the state-of-art kernel approximation algorithms. Chapter 3 presents a theoretical analysis of the RBF kernel matrix rank. Our three main results are as follows. First, we study the kernel rank, which for a fixed precision grows algebraically with the data dimension (in the worst case), and where the power is related to the accuracy. Second, we derive precise error bounds for the low-rank approximation in the L_infty norm in terms of the function smoothness and the domain diameters. And third, we analyze a group pattern in the magnitude of the singular values of the RBF kernel matrix. We explain this pattern by a grouping of the expansion terms in the kernel's low-rank representation. Empirical results verify the theoretical results. Chapter 4 presents PBBFMM3D, which is a parallel implementation of the fast multipole method (FMM) for evaluating pair-wise particle interactions (matrix-vector product) in three dimensions. PBBFMM3D applies to all non-oscillatory smooth kernel functions and only requires the kernel evaluations at data points. It has O(N) complexity as opposed to O(N^2) complexity from a direct computation. We discuss several algorithmic improvements and performance optimizations, such as shared memory parallelism using OpenMP. We present convergence and scalability results, as well as applications including particle potential evaluations, which frequently occur in PDE-related simulations, and covariance matrix computations that are essential parts in parameter estimation techniques, e.g., Kriging and Kalman filtering. Chapter 5 presents DCN, which is designed for datasets with dense and sparse combined features and enables automatic and efficient feature learning. Feature engineering is the key to the success of prediction models; however, the process often requires manual feature engineering or exhaustive searching. DCN combines a deep neural network that learns complex but implicit feature interactions, with a novel cross network that is more efficient in learning certain explicit bounded-degree feature interactions. Our experimental results have demonstrated its superiority over the state-of-art algorithms on the click-through-rate prediction dataset and dense classification dataset, in terms of both model accuracy and memory usage.
Large Scale Convex Optimization
DOWNLOAD
Author : Ernest K. Ryu
language : en
Publisher: Cambridge University Press
Release Date : 2022-12-01
Large Scale Convex Optimization written by Ernest K. Ryu and has been published by Cambridge University Press this book supported file pdf, txt, epub, kindle and other format this book has been release on 2022-12-01 with Mathematics categories.
Starting from where a first course in convex optimization leaves off, this text presents a unified analysis of first-order optimization methods – including parallel-distributed algorithms – through the abstraction of monotone operators. With the increased computational power and availability of big data over the past decade, applied disciplines have demanded that larger and larger optimization problems be solved. This text covers the first-order convex optimization methods that are uniquely effective at solving these large-scale optimization problems. Readers will have the opportunity to construct and analyze many well-known classical and modern algorithms using monotone operators, and walk away with a solid understanding of the diverse optimization algorithms. Graduate students and researchers in mathematical optimization, operations research, electrical engineering, statistics, and computer science will appreciate this concise introduction to the theory of convex optimization algorithms.
Mathematical Theories Of Machine Learning Theory And Applications
DOWNLOAD
Author : Bin Shi
language : en
Publisher: Springer
Release Date : 2019-06-12
Mathematical Theories Of Machine Learning Theory And Applications written by Bin Shi and has been published by Springer this book supported file pdf, txt, epub, kindle and other format this book has been release on 2019-06-12 with Technology & Engineering categories.
This book studies mathematical theories of machine learning. The first part of the book explores the optimality and adaptivity of choosing step sizes of gradient descent for escaping strict saddle points in non-convex optimization problems. In the second part, the authors propose algorithms to find local minima in nonconvex optimization and to obtain global minima in some degree from the Newton Second Law without friction. In the third part, the authors study the problem of subspace clustering with noisy and missing data, which is a problem well-motivated by practical applications data subject to stochastic Gaussian noise and/or incomplete data with uniformly missing entries. In the last part, the authors introduce an novel VAR model with Elastic-Net regularization and its equivalent Bayesian model allowing for both a stable sparsity and a group selection.
Stochastic Optimization For Large Scale Machine Learning
DOWNLOAD
Author : Vinod Kumar Chauhan
language : en
Publisher: CRC Press
Release Date : 2021-11-18
Stochastic Optimization For Large Scale Machine Learning written by Vinod Kumar Chauhan and has been published by CRC Press this book supported file pdf, txt, epub, kindle and other format this book has been release on 2021-11-18 with Computers categories.
Advancements in the technology and availability of data sources have led to the `Big Data' era. Working with large data offers the potential to uncover more fine-grained patterns and take timely and accurate decisions, but it also creates a lot of challenges such as slow training and scalability of machine learning models. One of the major challenges in machine learning is to develop efficient and scalable learning algorithms, i.e., optimization techniques to solve large scale learning problems. Stochastic Optimization for Large-scale Machine Learning identifies different areas of improvement and recent research directions to tackle the challenge. Developed optimisation techniques are also explored to improve machine learning algorithms based on data access and on first and second order optimisation methods. Key Features: Bridges machine learning and Optimisation. Bridges theory and practice in machine learning. Identifies key research areas and recent research directions to solve large-scale machine learning problems. Develops optimisation techniques to improve machine learning algorithms for big data problems. The book will be a valuable reference to practitioners and researchers as well as students in the field of machine learning.
Mathematical Analysis Of Machine Learning Algorithms
DOWNLOAD
Author : Tong Zhang
language : en
Publisher: Cambridge University Press
Release Date : 2023-07-31
Mathematical Analysis Of Machine Learning Algorithms written by Tong Zhang and has been published by Cambridge University Press this book supported file pdf, txt, epub, kindle and other format this book has been release on 2023-07-31 with Computers categories.
The mathematical theory of machine learning not only explains the current algorithms but can also motivate principled approaches for the future. This self-contained textbook introduces students and researchers of AI to the main mathematical techniques used to analyze machine learning algorithms, with motivations and applications. Topics covered include the analysis of supervised learning algorithms in the iid setting, the analysis of neural networks (e.g. neural tangent kernel and mean-field analysis), and the analysis of machine learning algorithms in the sequential decision setting (e.g. online learning, bandit problems, and reinforcement learning). Students will learn the basic mathematical tools used in the theoretical analysis of these machine learning problems and how to apply them to the analysis of various concrete algorithms. This textbook is perfect for readers who have some background knowledge of basic machine learning methods, but want to gain sufficient technical knowledge to understand research papers in theoretical machine learning.
Sparse Learning Under Regularization Framework
DOWNLOAD
Author : Haiqin Yang
language : en
Publisher: LAP Lambert Academic Publishing
Release Date : 2011-04
Sparse Learning Under Regularization Framework written by Haiqin Yang and has been published by LAP Lambert Academic Publishing this book supported file pdf, txt, epub, kindle and other format this book has been release on 2011-04 with categories.
Regularization is a dominant theme in machine learning and statistics due to its prominent ability in providing an intuitive and principled tool for learning from high-dimensional data. As large-scale learning applications become popular, developing efficient algorithms and parsimonious models become promising and necessary for these applications. Aiming at solving large-scale learning problems, this book tackles the key research problems ranging from feature selection to learning with mixed unlabeled data and learning data similarity representation. More specifically, we focus on the problems in three areas: online learning, semi-supervised learning, and multiple kernel learning. The proposed models can be applied in various applications, including marketing analysis, bioinformatics, pattern recognition, etc.
Improved Classification Rates For Localized Algorithms Under Margin Conditions
DOWNLOAD
Author : Ingrid Karin Blaschzyk
language : en
Publisher: Springer Nature
Release Date : 2020-03-18
Improved Classification Rates For Localized Algorithms Under Margin Conditions written by Ingrid Karin Blaschzyk and has been published by Springer Nature this book supported file pdf, txt, epub, kindle and other format this book has been release on 2020-03-18 with Mathematics categories.
Support vector machines (SVMs) are one of the most successful algorithms on small and medium-sized data sets, but on large-scale data sets their training and predictions become computationally infeasible. The author considers a spatially defined data chunking method for large-scale learning problems, leading to so-called localized SVMs, and implements an in-depth mathematical analysis with theoretical guarantees, which in particular include classification rates. The statistical analysis relies on a new and simple partitioning based technique and takes well-known margin conditions into account that describe the behavior of the data-generating distribution. It turns out that the rates outperform known rates of several other learning algorithms under suitable sets of assumptions. From a practical point of view, the author shows that a common training and validation procedure achieves the theoretical rates adaptively, that is, without knowing the margin parameters in advance.
Machine Learning And Knowledge Discovery In Databases
DOWNLOAD
Author : Frank Hutter
language : en
Publisher: Springer Nature
Release Date : 2021-02-24
Machine Learning And Knowledge Discovery In Databases written by Frank Hutter and has been published by Springer Nature this book supported file pdf, txt, epub, kindle and other format this book has been release on 2021-02-24 with Computers categories.
The 5-volume proceedings, LNAI 12457 until 12461 constitutes the refereed proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2020, which was held during September 14-18, 2020. The conference was planned to take place in Ghent, Belgium, but had to change to an online format due to the COVID-19 pandemic. The 232 full papers and 10 demo papers presented in this volume were carefully reviewed and selected for inclusion in the proceedings. The volumes are organized in topical sections as follows: Part I: Pattern Mining; clustering; privacy and fairness; (social) network analysis and computational social science; dimensionality reduction and autoencoders; domain adaptation; sketching, sampling, and binary projections; graphical models and causality; (spatio-) temporal data and recurrent neural networks; collaborative filtering and matrix completion. Part II: deep learning optimization and theory; active learning; adversarial learning; federated learning; Kernel methods and online learning; partial label learning; reinforcement learning; transfer and multi-task learning; Bayesian optimization and few-shot learning. Part III: Combinatorial optimization; large-scale optimization and differential privacy; boosting and ensemble methods; Bayesian methods; architecture of neural networks; graph neural networks; Gaussian processes; computer vision and image processing; natural language processing; bioinformatics. Part IV: applied data science: recommendation; applied data science: anomaly detection; applied data science: Web mining; applied data science: transportation; applied data science: activity recognition; applied data science: hardware and manufacturing; applied data science: spatiotemporal data. Part V: applied data science: social good; applied data science: healthcare; applied data science: e-commerce and finance; applied data science: computational social science; applied data science: sports; demo track.
Kernel Based Algorithms For Mining Huge Data Sets
DOWNLOAD
Author : Te-Ming Huang
language : en
Publisher: Springer Science & Business Media
Release Date : 2006-03-02
Kernel Based Algorithms For Mining Huge Data Sets written by Te-Ming Huang and has been published by Springer Science & Business Media this book supported file pdf, txt, epub, kindle and other format this book has been release on 2006-03-02 with Computers categories.
This is the first book treating the fields of supervised, semi-supervised and unsupervised machine learning collectively. The book presents both the theory and the algorithms for mining huge data sets using support vector machines (SVMs) in an iterative way. It demonstrates how kernel based SVMs can be used for dimensionality reduction and shows the similarities and differences between the two most popular unsupervised techniques.
Provably Efficient Methods For Large Scale Learning
DOWNLOAD
Author : Shuo Yang (Ph. D.)
language : en
Publisher:
Release Date : 2023
Provably Efficient Methods For Large Scale Learning written by Shuo Yang (Ph. D.) and has been published by this book supported file pdf, txt, epub, kindle and other format this book has been release on 2023 with categories.
The scale of machine learning problems grows rapidly in recent years and calls for efficient methods. In this dissertation, we propose simple and efficient methods for various large-scale learning problems. We start with a standard supervised learning problem of solving quadratic regression. In Chapter 2, we show that by utilizing the quadratic structure and a novel gradient estimation algorithm, we can solve sparse quadratic regression with sub-quadratic time complexity and near-optimal sample complexity. We then move to online learning problems. In Chapter 3, we identify a weak assumption and theoretically prove that the standard UCB algorithm efficiently learns from inconsistent human preferences with nearly optimal regret; in Chapter 4 we propose an approximate maximum inner product search data structure for adaptive queries and present two efficient algorithms that achieve sublinear time complexity for linear bandits, which is especially desirable for extremely large and slowly changing action sets. In Chapter 5, we study how to efficiently use privileged features with deep learning models. We present an efficient learning algorithm to exploit privileged features that are not available during testing time. We conduct comprehensive empirical evaluations and present rigorous analysis for linear models to build theoretical insights. It provides a general algorithmic paradigm that can be integrated with many other machine learning methods