The XOR operator allows us to only flip the bit we are interested in. With a mask that isolates the 5th bit, we can keep the other bits isolate since:
0 ^ 1 = 1
0 ^ 0 = 0
This means as long as there are zeros in the other bit positions, we can keep them unaffected. The 1 bit always result with a flip - just as we are looking for:
1 ^ 1 = 0
1 ^ 0 = 1