I would say NO it is never too cold to bow hunt. I have hunted in -22 below straight temps (not windshield). The deer are forced to move in this stuff. I suggest getting all dressed up like you were if you were going to sit in a stand and practice shooting with all those clothes on. It does change how you shoot with all those layers on, not to mention thicker gloves, facemask etc. The biggest problem people have is they practice all Summer with a t-shirt on and no gloves or facemask. Then the time come sin the Winter with all these clothes on, gloves and face mask, they draw back and it feels real funky when you try to find your anchor point.
I still draw back every time I go out just so I can get accustomed to the “different’ feeling. 
Will the cold effect the performance of your bow??? Probably to some degree, but never enough where you should not hunt! 
I have killed 4 deer at or near below zero temps with opps at many more. This bad boy in my avatar was one of them! 