Hi there
I mainly fly the Zibo 737-700U/800X/900U and the more complicated LES SAAB 340 and always set up the flight management computer with a full flight plan, with no ATC input at all. Personally I don't believe that the X-Plane 11 ATC system is very good anyway, so avoid it completely!
Consequently, No matter where I start from, I do always end up at the correct destination and this includes flights in GA aircraft such as the Baron or Cessna.
Have you tried a route just by using a flight plan on auto pilot and switching off ATC?
Could you please provide one example of a flight plan you've used where you've also ended up in the wrong destination, plus the aircraft & navigation system used? I will try it my end to see what happens using autopilot?
If you're not using the autopilot, do you fly the route manually, VOR to VOR, waypoint to waypoint?
Regards