The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
$\ell_1$ Trend Filtering
Abstract
The problem of estimating underlying trends in time series data arises in a variety of disciplines. In this paper we propose a variation on Hodrick–Prescott (H-P) filtering, a widely used method for trend estimation. The proposed $\ell_1$ trend filtering method substitutes a sum of absolute values (i.e., $\ell_1$ norm) for the sum of squares used in H-P filtering to penalize variations in the estimated trend. The $\ell_1$ trend filtering method produces trend estimates that are piecewise linear, and therefore it is well suited to analyzing time series with an underlying piecewise linear trend. The kinks, knots, or changes in slope of the estimated trend can be interpreted as abrupt changes or events in the underlying dynamics of the time series. Using specialized interior-point methods, $\ell_1$ trend filtering can be carried out with not much more effort than H-P filtering; in particular, the number of arithmetic operations required grows linearly with the number of data points. We describe the method and some of its basic properties and give some illustrative examples. We show how the method is related to $\ell_1$ regularization-based methods in sparse signal recovery and feature selection, and we list some extensions of the basic method.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Self-citationRegression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Donoho, D. L. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2006.871582 · Cited by 23,994
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Donoho, D. L. (2004) · Cited by 16,703
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704
Candes, E. J., Romberg, J., & Tao, T. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.862083 · Cited by 15,639
(2021) · A First Course in Statistical Programming with R · doi:10.1017/9781108993456.011 · Cited by 13,724
Koenker, R., & Bassett, G. (1978) · Econometrica · doi:10.2307/1913643 · Cited by 13,316
Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 11,726
Breitung, J., & Hamilton, J. D. (1995) · Contemporary Sociology · doi:10.2307/2076916 · Cited by 10,966
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Multivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,000
Using SeDuMi 1.02, A Matlab toolbox for optimization over symmetric cones
Sturm, J. F. (1999) · Optimization Methods and Software · doi:10.1080/10556789908805766 · Cited by 7,377
Postwar U.S. Business Cycles: An Empirical Investigation
Hodrick, R. J., & Prescott, E. C. (1997) · Journal of Money, Credit and Banking · doi:10.2307/2953682 · Cited by 7,349
Model Selection and Estimation in Regression with Grouped Variables
Yuan, M., & Lin, Y. (2006) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00532.x · Cited by 7,261
Decoding by Linear Programming
Candes, E. J., & Tao, T. (2005) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.858979 · Cited by 7,121
Stable signal recovery from incomplete and inaccurate measurements
Candès, E. J., Romberg, J. K., & Tao, T. (2006) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20124 · Cited by 7,036
Permutation tests for joinpoint regression with applications to cancer rates
Kim, H.-J., Fay, M. P., Feuer, E. J., & Midthune, D. N. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000215)19:3<335::aid-sim336>3.0.co;2-z · Cited by 6,318
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (2001) · SIAM Review · doi:10.1137/s003614450037906x · Cited by 5,085
Spline Models for Observational Data.
Muller, H.-G., & Wahba, G. (1991) · Journal of the American Statistical Association · doi:10.2307/2290434 · Cited by 4,928
Enhancing Sparsity by Reweighted ℓ 1 Minimization
Candès, E. J., Wakin, M. B., & Boyd, S. P. (2008) · Journal of Fourier Analysis and Applications · doi:10.1007/s00041-008-9045-x · Cited by 4,863
Self-citationForecasting, Structural Time Series Models and the Kalman Filter
Fildes, R., Harvey, A. C., West, M., & Harrison, J. (1991) · The Journal of the Operational Research Society · doi:10.2307/2583225 · Cited by 4,682
Hastie, T., & Tibshirani, R. (2014) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat03141 · Cited by 3,156
Measuring Business Cycles: Approximate Band-Pass Filters for Economic Time Series
Baxter, M., & King, R. G. (1999) · Review of Economics and Statistics · doi:10.1162/003465399558454 · Cited by 2,827
Sparsity and Smoothness Via the Fused Lasso
Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., & Knight, K. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00490.x · Cited by 2,724
Primal-Dual Interior-Point Methods
Wright, S. J. (1997) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611971453 · Cited by 2,391
Methods for Trend Estimation from Summarized Dose-Response Data, with Applications to Meta-Analysis
Greenland, S., & Longnecker, M. P. (1992) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116237 · Cited by 2,353
Analysis of Financial Time Series
(2006) · Technometrics · doi:10.1198/tech.2006.s405 · Cited by 2,348
The Theory and Practice of Revenue Management
Talluri, K. T., & Van Ryzin, G. J. (2004) · International Series in Operations Research & Management Science · doi:10.1007/b139000 · Cited by 2,243
Reinsch, C. H. (1967) · Numerische Mathematik · doi:10.1007/bf02162161 · Cited by 2,236
Stable recovery of sparse overcomplete representations in the presence of noise
Donoho, D. L., Elad, M., & Temlyakov, V. N. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.860430 · Cited by 2,183
Christiano, L. J., & Fitzgerald, T. J. (2003) · International Economic Review · doi:10.1111/1468-2354.t01-1-00076 · Cited by 1,786
Wavelet Shrinkage: Asymptopia?
Donoho, D. L., Johnstone, I. M., Kerkyacharian, G., & Picard, D. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02032.x · Cited by 1,749
Order Restricted Statistical Inference.
Santner, T., Robertson, T., Wright, F. T., & Dykstra, R. L. (1990) · Journal of the American Statistical Association · doi:10.2307/2289813 · Cited by 1,586
Just relax: convex programming methods for identifying sparse signals in noise
Tropp, J. A. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.864420 · Cited by 1,407
Sensor Selection via Convex Optimization
Joshi, S., & Boyd, S. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.2007095 · Cited by 1,311
Self-citationNonlinear Time Series: Nonparametric and Parametric Methods
Fan, J., & Yao, Q. (2003) · Springer eBooks · doi:10.1007/b97702 · Cited by 1,198
Understanding Why Crime Fell in the 1990s: Four Factors that Explain the Decline and Six that Do Not
Levitt, S. D. (2004) · Journal of Economic Perspectives · doi:10.1257/089533004773563485 · Cited by 1,152
The Estimation of the Parameters of a Linear Regression System Obeying Two Separate Regimes
Quandt, R. E. (1958) · Journal of the American Statistical Association · doi:10.1080/01621459.1958.10501484 · Cited by 1,107
Image decomposition via the combination of sparse representations and a variational approach
Starck, J.-L., Elad, M., & Donoho, D. L. (2005) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.852206 · Cited by 987
Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA)
Elad, M., Starck, J.-L., Querre, P., & Donoho, D. L. (2005) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2005.03.005 · Cited by 904
A new approach to variable selection in least squares problems
Osborne, M. (2000) · IMA Journal of Numerical Analysis · doi:10.1093/imanum/20.3.389 · Cited by 859
Statistical Inference under Order Restrictions (The Theory and Application of Isotonic Regression)
Vincze, I., Barlow, R. E., Bartholomew, D. J., Bremner, J. M., & Brunk, H. D. (1973) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1402630 · Cited by 842
L 1-Regularization Path Algorithm for Generalized Linear Models
Park, M. Y., & Hastie, T. (2007) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2007.00607.x · Cited by 838
Robust Modeling With Erratic Data
Claerbout, J. F., & Muir, F. (1973) · Geophysics · doi:10.1190/1.1440378 · Cited by 833
Convex Analysis and Nonlinear Optimization
Borwein, J. M., & Lewis, A. S. (2006) · CMS books in mathematics · doi:10.1007/978-0-387-31256-9 · Cited by 780
A Direct Formulation for Sparse PCA Using Semidefinite Programming
d'Aspremont, A., El Ghaoui, L., Jordan, M. I., & Lanckriet, G. R. G. (2007) · SIAM Review · doi:10.1137/050645506 · Cited by 678
KOENKER, R., NG, P., & PORTNOY, S. (1994) · Biometrika · doi:10.1093/biomet/81.4.673 · Cited by 635
Nonparametric Regression and Spline Smoothing
Bakir, S. T. (2000) · Technometrics · doi:10.1080/00401706.2000.10486068 · Cited by 614
SEGMENTING TIME SERIES: A SURVEY AND NOVEL APPROACH
Keogh, E., Chu, S., Hart, D., & Pazzani, M. (2004) · Series in Machine Perception and Artificial Intelligence · doi:10.1142/9789812565402_0001 · Cited by 600
An Interior-Point Method for Large-Scale '1-Regularized Logistic Regression
Koh, K., Kim, S.-J., & Boyd, S. (2007) · Cited by 549
Self-citationFitting Segmented Regression Models by Grid Search
Lerman, P. M. (1980) · Applied Statistics · doi:10.2307/2346413 · Cited by 501
Piecewise linear regularized solution paths
Rosset, S., & Zhu, J. (2007) · The Annals of Statistics · doi:10.1214/009053606000001370 · Cited by 481
4 Two Illustrations of the Quantity Theory of Money
(2012) · Collected Papers on Monetary Theory · doi:10.4159/harvard.9780674067851.c4 · Cited by 479
Chen, S., & Donoho, D. L. (1994) · Cited by 467
On the approximation of curves by line segments using dynamic programming
Bellman, R. (1961) · Communications of the ACM · doi:10.1145/366573.366611 · Cited by 444
Deconvolution with the l 1 norm
Taylor, H. L., Banks, S. C., & McCoy, J. F. (1979) · Geophysics · doi:10.1190/1.1440921 · Cited by 436
Fitting Segmented Curves Whose Join Points Have to Be Estimated
Hudson, D. J. (1966) · Journal of the American Statistical Association · doi:10.1080/01621459.1966.10482198 · Cited by 426
Portfolio optimization with linear and fixed transaction costs
Lobo, M. S., Fazel, M., & Boyd, S. (2007) · Annals of Operations Research · doi:10.1007/s10479-006-0145-1 · Cited by 350
Self-citationEfficientL 1 regularized logistic regression
Lee, S.-I., Lee, H., Abbeel, P., & Ng, A. Y. (2006) · Cited by 285
Levy, S., & Fullagar, P. K. (1981) · Geophysics · doi:10.1190/1.1441261 · Cited by 258
Statistical inference under order restrictions : the theory and application of isotonic regression
Barlow, R. E. (1972) · Cited by 234
Forward stagewise regression and the monotone lasso
Hastie, T. J., Taylor, J., Tibshirani, R. J., & Walther, G. (2007) · Electronic Journal of Statistics · doi:10.1214/07-ejs004 · Cited by 212
On Asymptotic Distribution Theory in Segmented Regression Problems-- Identified Case
Feder, P. I. (1975) · The Annals of Statistics · doi:10.1214/aos/1176342999 · Cited by 210
Forward Stagewise Regression and the Monotone Lasso
Hastie, T. J., Taylor, J., Tibshirani, R. J., & Walther, G. (2006) · Cited by 192
McGee, V. E., & Carleton, W. T. (1970) · Journal of the American Statistical Association · doi:10.2307/2284278 · Cited by 187
Curve Fitting by Segmented Straight Lines
Bellman, R., & Roth, R. (1969) · Journal of the American Statistical Association · doi:10.1080/01621459.1969.10501038 · Cited by 170
The Log Likelihood Ratio in Segmented Regression
Feder, P. I. (1975) · The Annals of Statistics · doi:10.1214/aos/1176343000 · Cited by 165
A Bayesian Time Series Model of Multiple Structural Changes in Level, Trend, and Variance
Wang, J., & Zivot, E. W. (2000) · Journal of Business and Economic Statistics · doi:10.1080/07350015.2000.10524878 · Cited by 139
Covariance selection for nonchordal graphs via chordal embedding
Dahl, J., Vandenberghe, L., & Roychowdhury, V. (2008) · Optimization methods & software · doi:10.1080/10556780802102693 · Cited by 122
Approximation of curves by line segments
Stone, H. (1961) · Mathematics of Computation · doi:10.1090/s0025-5718-1961-0119390-6 · Cited by 104
Straight Lines with a Change-Point: A Bayesian Analysis of Some Renal Transplant Data
Smith, A. F. M., & Cook, D. G. (1980) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2986304 · Cited by 82
High-Dimensional Graphical Model Selection Using $\ell_1$-Regularized Logistic RegressionPreprint
Ravikumar, P., Wainwright, M. J., & Lafferty, J. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0804.4202 · Cited by 66
Guthery, S. B. (1974) · Journal of the American Statistical Association · doi:10.1080/01621459.1974.10480233 · Cited by 54
Jump process for the trend estimation of time series
Zhao, S., & Wei, G. (2002) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(02)00125-1 · Cited by 20
Cited by
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Self-citationBoyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,677
Self-citationEnhancing Sparsity by Reweighted ℓ 1 Minimization
Candès, E. J., Wakin, M. B., & Boyd, S. P. (2008) · Journal of Fourier Analysis and Applications · doi:10.1007/s00041-008-9045-x · Cited by 4,863
Self-citationDeep learning for short-term traffic flow prediction
Polson, N. G., & Sokolov, V. O. (2017) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2017.02.024 · Cited by 1,015
The solution path of the generalized lasso
Tibshirani, R. J., & Taylor, J. (2011) · The Annals of Statistics · doi:10.1214/11-aos878 · Cited by 553
Metadata
- Authors
- Kim, S.-J., Koh, K., Boyd, S., & Gorinevsky, D.
- Venue
- SIAM Review
- Published
- 2009-05
- Volume
- 51
- Issue
- 2
- Pages
- 339-360
- ISSN
- 0036-1445 (ISSN-L)
- 1095-7200
- OpenAlex
- W2089700840
- Cited by
- 615
- References
- 77
- FWCI
- 12.30
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.