No. physics is generalized to algebra, you don’t need to know a lot of math to learn physics. Having more math will allow you to do more complicated problems and understand concepts the way it was discovered, but it isn’t limited to those who know calculus.