Graphical lasso is a popular method for learning the structure of an undirected graphical model, which is based on an l₁ regularization technique. This method aims to find the conditional independence between the entries of a random vector by learning the sparsity pattern of the inverse correlation matrix from a limited number of samples. Graphical lasso is computationally expensive for large-scale problems. A numerically-cheap heuristic method for finding a graphical model is to simply threshold the sample correlation matrix. Recently, we have observed that the computationally-heavy graphical lasso and the simple thresholding method would produce the same solution for functional MRI data, electrical circuit data and many random systems, provided that a sparse graph is sought. The objective of this work is to develop a rigorous mathematical foundation for that observation. More precisely, we systematically study the relationship between these two methods by introducing the notions of sign-consistent and inverse-consistent matrices. We prove that thresholding and graphical lasso are equivalent if: (i) a certain matrix formed based on the sample correlation matrix is both sign-consistent and inverse-consistent, (ii) the gap between the largest thresholded and the smallest un-thresholded entries of the sample correlation matrix is not too small. We demonstrate the above conditions on acyclic graphs. These conditions are expected to be satisfied for sufficiently sparse graphical models.