Often, the familiar hides the deepest mysteries. This is certainly the case with light, one of the take-it-for-granted physical phenomena that surrounds us in everyday life. We wake up to it, we turn it on and off, rarely thinking of what it really is. A good thing, for even the greatest physicists pause before talking about the nature of light.
In Opticks, the treatise on light Isaac Newton first published in 1704, he defends his belief that light is made of small "parts," asking: "Are not rays of light very small bodies emitted from shining substances?" In the same book, he maintains that light propagates at finite speed: "by an argument taken from the equations of the times of the eclipses of Jupiter's satellites, it seems that light is propagated in time, spending in its passage from the Sun to us about seven minutes of time." (Newton was referring to Ole Roemer's 1676 measurement of the speed of light from the eclipse of Io, one of Jupiter's moons.)
To Newton's atomic theory of light, there was the very successful wave theory, which explained typical light behavior such as refraction (the change in the direction of propagation of light when it goes from one medium to another, such as air to water, due to a change in wavelength) and diffraction (phenomena caused when waves hit obstacles, such as when a wave passes through slits).
In 1865, James Clerk Maxwell published his Dynamical Theory of the Electromagnetic Field, where he described light as undulations of electromagnetic fields. The theory beautifully joined together two apparently disconnected forces of Nature, electricity and magnetism, as manifestations of the electromagnetic field. An electric charge has a field around it, representing the spatial manifestation of its presence: for a point charge, the strength of the electric field decreases with the square of the distance, just as gravity does. Magnetic fields are generated when electric charges are accelerated. Thus, electricity and magnetism are unified through motion.
Picture a cork bobbing up and down on a pool. Circular waves will propagate outwards. These waves transport the energy of the cork's motion outwards, through the water. Substitute now the cork for a small electric charge, and the waves propagating concentrically are electromagnetic waves. If the cork moves up and down fast, the waves are closely spaced: they have small wavelength and high frequencies. If the cork moves slowly, the waves have long wavelength and low frequencies. With light, we can also think of long and short wavelengths. What we call visible light is a small window of all possible "light" waves, which stretch from the long wavelength radio waves (nothing to do with the sound waves coming out of a radio!) to the shorter wavelength ultraviolet, X-rays and gamma rays.
Quite naturally, Maxwell and others wondered what medium gave support to light waves. Since they could see light from far away stars, this medium had to be transparent. Also, since light could have such high frequencies, the medium (known as "luminiferous ether") had to be millions of times more rigid than steel. But it also had to be massless and without any viscosity, or it would affect the planets.
In spite of all this very weird and unusual properties, almost every physicist at the time embraced the ether as a bizarre necessity. How else to explain light's mysterious propagation?
In 1905, the 26-year-old Einstein published two papers that redefined light completely. (There were two other papers that year, one on Brownian motion that made molecules a reality and the other deriving the E=mc2 formula, quite a year for young Albert!) In the first one, he proposed that light can indeed be understood as being made of tiny corpuscles, which were later called photons. So, Newton and Maxwell were both right: Light is particle and wave. In the second paper, the one on the special theory of relativity, Einstein showed that light is just different than anything else: contrary to other waves, light waves do not need a material medium to propagate. They can propagate in total emptiness.
This dual behavior of light, wave and particle, was then extended to all forms of matter. In a sense, light and matter are neither wave-like nor corpuscular, being some kind of hybrid of the two. Only when we probe them with our experiments, which are set up to examine one of the two properties but not both at once, does light "become" particle or wave. So, the observer has a key role in determining the physical nature of light (and of matter) as she sets her experimental apparatus.
But even if matter also shares the wave-particle duality, light stands out as being the only entity that we know without mass. How, you may ask, can something exist without having mass? In modern physics, it's best to think of energy as being more fundamental than mass. And massless photons have energy in an amount directly proportional to their frequency: the higher the frequency, the higher the energy of the photon.
That nothing can travel faster or at the same speed of light has to do with it not having mass. As Einstein showed in his fourth 1905 paper, the mass of an object increases with speed: if it could reach the speed of light, its mass would become infinitely large. In other words, nothing can travel faster or at light speed. And why is that? We have no clue. As we also have no clue why the speed of light has the value that it does. Theories proposing that the speed of light may have changed over the life of the cosmos have met with little experimental support.
With light, I'm sure Einstein would agree with Daniel J. Boorstin's quote that "the greatest obstacle to discovery is not ignorance, but the illusion of knowledge." Even though we know so much, we still are far from knowing enough.