China
Africa
Explore chapters and articles related to this topic
Penalized Regression Models
Published in Taylor Arnold, Michael Kane, Bryan W. Lewis, A Computational Approach to Statistical Learning, 2019
Taylor Arnold, Michael Kane, Bryan W. Lewis
Coordinate descent is a general purpose convex optimization algorithm particularly well-suited to solving the elastic net equation. Coordinate descent successively minimizes the objective function along each variable. In every step all but one variable is held constant and a value for the variable of interest is chosen to minimize the constrained problem. This process is applied iteratively over all of the variables until the algorithm converges. As we will see, each univariate subproblem closely resembles the orthogonal solution derived in Section 7.3.
Regularization Techniques for MR Image Reconstruction
Published in Joseph Suresh Paul, Raji Susan Mathew, Regularized Image Reconstruction in Parallel MRI with MATLAB®, 2019
Joseph Suresh Paul, Raji Susan Mathew
The general implementation of this method includes choosing some coordinate i and finding the optimal value of xi, given the remaining parameters. This procedure is repeated until some termination criterion is met. A coordinate descent algorithm for L(x)≜‖Ax−b‖22 called the shooting algorithm was proposed [53]. In this algorithm, the coordinate is chosen by repeatedly passing through the variables in order, and the optimal value is found using a simple analytic solution, obtained by setting xi to 0, and xi←xi−∇isf(xi)/∇ii2L(x). Although the coordinate descent algorithms are simple to implement, they are more effective if the variables have largely independent effects on the objective. The performance of the algorithm is degraded as the variables become more dependent. These methods may require so many coordinate updates that implementation becomes impractical.
Coordinate Gradient Descent Methods
Published in Yulei Wu, Fei Hu, Geyong Min, Albert Y. Zomaya, Big Data and Computational Intelligence in Networking, 2017
In the last decade we can observe an increasing interest into optimization problems of a very big size coming from networks, machine learning, the Internet, telecommunications, and traffic applications. In these applications, even the evaluation of the objective function at some point needs substantial computational efforts. Such problems require simple and scalable optimization algorithms. One of the most popular methods that fit into these requirements is the coordinate descent. However, due to its low efficiency on big problems and lack of convergence results, it has not received much attention until recently. This attention has returned due to the recent theoretical results and applications in networks [9], machine learning [8, 23], compressed sensing [24], protein loop closure [25], optimal control [3, 26] and optimization [27]. More specifically, for coordinate minimization convergence rates were given in BerTsi:89 and recently new variants and extensions were presented and analyzed e.g., in RazHon:13. The first global convergence rate for a randomized coordinate gradient descent method on smooth convex optimization was given in a seminal paper [13]. The generalization of this convergence rate analysis to composite convex optimization has been extensively studied, under different assumptions on the objective function, in Refs. [14, 15, 28]. Composite nonconvex optimization has been studied in PatNec:15, and ℓ0 regularized convex optimization has been studied in Refs. [26, 30]. Convergence rate results for randomized variants of coordinate gradient descent methods for linearly constrained convex optimization were given in Refs. [31, 32] and for linearly constrained nonconvex optimization were given in PatNec:15. For coordinate gradient descent methods based on cyclic coordinate search bounds on the rate of convergence were given recently in BecTet:12 and based on greedy coordinate search (e.g., Gauss–Southwell rule) bounds on the convergence rate were given in Refs. [12, 27]. Accelerated variants, based on the Nesterov fast gradient method [19], have also been proposed in the literature, see e.g., [13, 33, 34].
Control analysis and design via randomised coordinate polynomial minimisation
Published in International Journal of Control, 2022
Giuseppe C. Calafiore, Carlo Novara, Corrado Possieri
There hence appears to exist a need for reliable numerical methods that can tackle constrained polynomial optimisation problems of realistic size and provide: (a) optimal or sub-optimal solutions in reasonable time, and (b) some form of theoretical guarantee of convergence to the global optimum. A class of numerical methods that gained popularity in recent years, especially in the context of machine learning, is that of coordinate descent methods, see Wright (2015) for a recent survey. Coordinate descent algorithms solve optimisation problems by successively performing exact or approximate minimisations along coordinate directions.
A generic coordinate descent solver for non-smooth convex optimisation
Published in Optimization Methods and Software, 2021
The algorithm we implemented is described in [15] and can be downloaded on https://bitbucket.org/ofercoq/cd_solver. The present paper focuses on important implementation details about residual updates and dual variable duplication. The novelty of our code is that it allows a generic treatment of these algorithmic steps and includes a modelling interface in Python for the definition of the optimisation problem. Note that unlike most coordinate descent implementations, it can deal with non-separable non-smooth objectives and linear constraints.
Split Regularized Regression
Published in Technometrics, 2020
Anthony-Alexander Christidis, Laks Lakshmanan, Ezequiel Smucler, Ruben Zamar
Based on the previous discussion, we propose a computing algorithm based on coordinate descent. Coordinate descent has proven to be efficient for solving regularized least-squares problems, see, for example, Friedman, Hastie, and Tibshirani (2010). The following result is the main building block of this algorithm.