Can I control the chart colors in my report?

I have an Urgency field with the options Red, Yellow, and Green. I created a report that displays as a pie graph using this field. But, the colors are wrong. The "yellow" group is shown in blue, the "green" group is showed in purple, and the "red" group is often orange. It might be my imagination, but I also thing the colors change based on how many red/yellow/green items exist. Is there a way to control the colors?