Synaesthesia- try it.

A few years ago I went to a Smithsonian exhibition on the subject of Synaesthesia- combining the process of 2 or more senses-
In this case it was on Visual Art and Music as one. This is not a new topic as Sir Isaac Newton and others have postulated on the connection between the two for centuries.
Some folks have even gone so far as to assign a color for every note of a Bach partita to come up with a painting.

I tend to look at it much more in terms of colors and Key centers.
For example-
the key of C major- no flats or sharps corresponds with a white/yellow to me. While the key with the most Gb major would be the complement-A dark purple.

Since there is no direct scientific correlation established yet, why not try a completely random experiment.
Try closing your eyes at the piano- hit some notes and see what feelings and sounds it evokes.

One of the reasons I associate dark colors with more flats and sharps, is because it brings a dark feeling to play in some of those keys.

See how it works for you.