I've recently got a new iMac with an i7 and 8Gb graphics card. I updated to High Sierra yesterday. X-Plane 11 (using the latest beta) seems just fine. No floating menus, and my plane starts up where I ask it to be.
I wonder what's happening. Maybe it's worth reinstalling the operating system (without wiping your data and programs, of course!).
Also, before trying that, it may be worth cleaning it all up with the latest version of Onyx. Worth a try. Clean out the caches, rebuild the launch services, etc. May not help, but you never know.