There’s the old question: “Do you see that color the same way I see it in my mind?” To some extend this will remain a philosophical question involving the swapping of hypothetical consciousnesses between hypothetical brains. But here’s what we do know.
Three reasons why people looking at one thing can see something differently
1) Cone sensitivity differs between people
Cones are the cells on your retina sensing light in one of three color ranges. These ranges overlap, with peak sensitivity in either red, green or violet. The sensitivity for one wavelength depends on lighting.
Sensitivity can also differ greatly from person to person. Extreme examples of these differences we know as color vision deficiency, better known as color blindness. That can occur as mild as not being able to discern certain hues under bad lighting to seeing no colors at all. So clearly there are big and small interpersonal differences!
2) Vision changes constantly
Even an individual can perceive one color differently depending on the context. You know, when you come out of a dark cellar into the sunlight and everything looks harsh, almost black and white? Or when you take off your tinted sun glasses and everything again has a color cast, but in the opposite hue? That’s your brain trying to make sense of your eyes’ signals, recalibrating to new circumstances.
Time doesn’t even have to be involved. Only the color of the environment can already change what colors we see:
2色法によるイチゴの錯視。この画像はすべてシアン色（青緑色）の画素でできているが、イチゴは赤く見える。— Akiyoshi Kitaoka (@AkiyoshiKitaoka) February 28, 2017
Strawberries appear to be reddish, though the pixels are not. pic.twitter.com/Ginyhf61F7
In lab experiments on (sidenote: The effect causing people to think something like crime or terrorism occurs more often, whereas in reality it’s less prevalent. A whole other topic, about which I recommend this wonderful episode of the You Are Not So Smart podcast in which I first heard about it. ) researchers got people to classify colors as blue that they called purple just minutes earlier. They did this by asking the participants to identify blue in a long series of colors shown one after an other. At the start blues occurred frequently, but less so over time, making participants redefine their idea of blue just like that.
3) Culture defines how we see color
There’s a whole cultural processing layer defining what colors we’re able to see. The Colors episode of the Radiolab podcast made a convincing argument that in many civilizations, the color blue made a late entrance in people’s perception. In Homer’s Greece for instance, they didn’t have a word for it and likely weren’t able to see it. Instead, Homer describes the sea as wine-colored.
Depending on where children grow up, they learn different words to label colors. For instance, in Russia there is no word for just blue. There are two instead. One is голубой, pronounced goluboy, and is light and greenish compared to the other, синий (siniy).
Now it turns out that people who grow up speaking Russian, are faster in identifying different shades of blue in the range between those two. Another example: the Himba people who have a lot of words for green, find it easy to pick the odd one out of this series:
But then they find it difficult to pick out the cyan swatch that to me is obviously different:
Quite recently a statistical analysis by Noga Zaslavsky et. al. of color labeling across languages showed that the Russians and Himba people aren’t special cases. All over the world, people are more sensitive to seeing differences in colors in ranges where their languages have more qualifiers for them.