On the Unreasonable Effectiveness of the Greedy Algorithm: Greedy Adapts to Sharpness

Part of Proceedings of the International Conference on Machine Learning 1 pre-proceedings (ICML 2020)

Bibtex »Metadata »Paper »Supplemental »

Bibtek download is not availble in the pre-proceeding


Authors

Sebastian Pokutta, Mohit Singh, Alfredo Torrico

Abstract

<p>Submodular maximization has been widely studied over the past decades, mostly because of its numerous applications in real-world problems. It is well known that the standard greedy algorithm guarantees a worst-case approximation factor of 1 − 1/e when maximizing a monotone submodular function under a cardinality constraint. However, empirical studies show that its performance is substantially better in practice. This raises a natural question of explaining this improved performance of the greedy algorithm. In this work, we define sharpness for submodular functions as a candidate explanation for this phenomenon. The sharpness criterion is inspired by the concept of strong convexity in convex optimization. We show that the greedy algorithm provably performs better as the sharpness of the submodular function increases. This improvement ties closely to the faster convergence rates of the first order methods for strongly convex functions. Finally, we perform a computational study to empirically support our theoretical results and show that sharpness explains the greedy performance better than other justifications in the literature.</p>