Had this problem for months when I had XP 10. Now since I got 11, I seemed to have solved the problem but I had to keep trying different drivers.
Go into your Nvidia Control Panel and set Power Management to Max Performance for XP
Turn Threaded Optimization OFF
Check if your GPU is overheating as well
Mine was so I downloaded EVGA Precision X (freeware tool) and downclocked my card (GTX 970 SSC ACX 2.0 4 GB VRAM) and raised the fan speed to 50% so my card is always getting cooled-since then, no issues
Hope this helps