And the fact the other companies can says something else. Github, Basecamp, and even the very site you're on, Medium, are famously remote driven and managed and have been for years.
Amazon is a prime [sic] example of what productively looks like taken to an extreme. High employee dissatisfaction, which corresponds to high employee burnout and turnover, which in turn leads to higher costs in hiring and training, and even back to low productivity as new hires take time to come up to speed.
Frankly, the world needs to wake up to the fact that "productivity" is only one metric, and often a rather nebulous one at that.