This is another example of the problem with our new fascination with labels and redefining every concept we have known over the years. From what I have read, when "POC" is used, it is in a discussion in which the user is expressing desires to improve the life situations, or remove restrictions, of POC. It is simply cumbersome to say "Blacks, Latinos, Asians, Indigenous Americans, Indians, or any other non-white person...". And to say "non-white" is going to offend somebody. Some writers say we shouldn't use "Black", while others say "African American" is offensive. The new trend is that "Hispanic" is no longer acceptable, preferring Latino/Latina, but some people are complaining about that. No matter what labels are used, some people are going to be offended.
It would be nice if we could just say "people". But the people who are raising racial issues the loudest are non-white people, not all Black. If you can say "Yes, All White People are Racist", then "people of color" is certainly no more offensive.