The squares marked A and B are the same shade of gray, yet they appear different. By joining the squares marked A and B with two vertical stripes of the same shade of gray, it becomes apparent that both squares are the same....
The visual system is not very good at being a physical light meter, but that is not its purpose. The important task is to break the image information down into meaningful components, and thereby perceive the nature of the objects in view.
More proof @ Edward H. Adelson's Checkershadow Illusion page.