Agree to a point... and disagree. Your final OrderView example is hitting right at the limit of complexity where I'd start considering a view model specific to the view in question.
The view's "logic" is embedded within the view's body, which makes the logic hard to understand and in turn expands the size of the view to the point where the view's structure is also hard to understand.
Might take a peek at the following and see how that applies to the view in question.
https://medium.com/better-programming/writing-testable-views-in-swiftui-ba28baec6a32
Love you're using Factory, but opening up DI in that fashion pretty much kills one of your earlier objections, that of not having access to the environment/environmentObjects.
Like your focus on services, but when apps get to a certain level of complexity there often needs to be a mediator between the services AND the view that consumes them, since different views often have different presentations of the same data.
An OrderService is great. But that app might have an order entry screen, an order summary screen, a past orders screen, and more, each with their own different needs and requirements. That's the role of a View Model. To mediate between the Model and and the View.
Which brings up the last point. I also think you're missing out on what the Model is in MVVM (or MVC, for that matter). A "Model" from an architectural perspective isn't just data, but reflects the data layer--the data model and the services needed to manage it.