Flags are potent symbols, and mean different things to different people.
The Confederate flag to some might be a symbol of racial segregation and slavery, and is often used by white supremacists to convey this meaning.
It could also be read as a symbol of independence. To many people it doesn't mean much beyond a slight association with rebellion, making it occasionally fashionable amongst the ignorant.
The flag in itself is not racist, but is so commonly viewed as a symbol of racism that you have to be careful about using it - not because of the risk of offending somebody, but for the risk of somebody getting in your face about it. Given the historical context it is probably a good idea to steer clear of it - much like wearing a Union Jack (British flag) in certain former colonies of the British Empire.