Cross entropy
In information theory, the cross entropy between two probability distributions ![p](/Images/godic/202501/07/83878c91171338902e0fe0fb97a8c47a0953.png")
and ![q](/Images/godic/202501/07/7694f4a66316e53c8cdd9d9954bd611d0953.png")
over the same underlying set of events measures the average number of bits needed to identify an event drawn from the set, if a coding scheme is used that is optimized for an "unnatural" probability distribution , rather than the "true" distribution .