We develop an order-N method for calculating the phonon transport in mesoscopic systems using the real space Kubo approach in the harmonic approximation. The advantage of our method over the Green function method is that we can compute the phonon transport properties of quasi-one dimensional structures as wide and as long as the experimentally relevant sample sizes within much shorter computation times. As a test case of this method, we calculate the elastic phonon mean free paths (MFP) of isotopically disordered carbon nanotubes (CNT). Our MFP values are in excellent agreement with the ones obtained from the Green function approach except for the very low energies. For such low energies, we use the analytic expression derived from perturbation theory. We apply the method to edge disordered graphene nanoribbons (GNR). We show that the phonon thermal conductance is reduced by more than two orders of magnitude due to edge disorder.