Apply a different palette object to the application instance breaks the colors

I was making a theme system, and I faced a problem where when I was applying a new palette object to the application instance, the colors would break and everything goes near white.

After 2 hours of testing, turns out all I had to do is modifying the original palette object that I applied to the application instance from the start, then apply that modified palette back, it then worked like a charm!