His famous drawing of the figure proportionally placed in the circle and square shows his application of mathematics in his art.
For Leonardo, then, mathematics meant essentially geometry. He received advanced training from Luca Pacioli at the Milanese court. Mathematics appealed to him because it is "the only science which contains within itself its own proof."
"Geometry provided the basis of the science of painting." The perspective construction of Italian Renaissance painting applied a lot of geometric principles.
When it came to optics. Leonardo was fascinated with the human eye and the process of vision.
He devised many experiments to describe and verify the phenomena.
He says, on the circular diffusion of light, "Just as the stone thrown into the water becomes the center and cause of various circles, and the sound made in the air spreads itself out in circles, so every body placed within the luminous air spreads itself out in circles and fills the surrounding parts with an infinite number of images itself."
No doubt, Leonardo was familiar with the ancient sources of the doctrine of light diffusion. His achievement was that of subjecting such inherited theories to the empirical proof of experimentation. He though, never had the mathematic sophistication to formulate a abstract principle defining the phenomena he observed. This is what separated him as a scientist to others such as Galileo and Kepler.