In this talk, I will demonstrate the usefulness of greedy algorithms both in highly persistent time series and in big, distributed computing architecture. First, we propose a greedy-based algorithm, FHTD, for consistent variable selection of the high-dimensional unit-root ARX model, in which a fully general but unknown unit-root structure is allowed. Second, for estimating a multi-task linear regression with feature-distributed (or vertically partitioned) data, we employ the two-stage relaxed greedy algorithm (TSRGA). Because of its low communication complexity, which does not scale with the ambient dimension, TSRGA is computationally attractive in this setup. In both cases, the key theoretical ingredient is to characterize the rate of convergence along the iteration path. Finally, the methods are shown to outperform commonly-used benchmarks when applied to real-world economic data.