The controversy is about trans women, and not much about trans men. Basicly, women get some special privileges, and they need some of those privileges for their safety. Trans women need very similar privileges for their safety, and for various reasons some women feel that some ways to share those privileges reduces their own safety.
Neither the pro-Trans activists nor the womens-rights activists see room to compromise, and the more they fight the higher they build their walls. Some polling etc indicates that the trans activists are way ahead in the Green Party, but both sides are about equal in the general population.
Obviously the Green Party should try to get everybody what they need, without it keeping others from having what they need. This should not be a controversy at all among us, but it came to us from the far left and spilled into us.