The sex binary is a form of hierarchy and oppression. Sex, in of itself, are categories made by other people to impose ideas of gender onto a person based on something as simple as their genitals. Penises and vaginas are not inherently gendered. They exist in a state of objectness until the person they are attached to determine what their gender is. Penises are not inherently male and vaginas are not inherently female. This goes against the basic biology 101 that many people are taught, but bear with me.
Sex is much more complicated than just penises and vaginas. Sex is a combination of primary sex characteristics (penises and vaginas), chromosomes, hormones, secondary sex characteristics (breasts, body hair, etc), and several other categories. Basically, sex is not something as simple as penis = male and vagina = female. While this is generally true, this is not always the case.
There are XY cis women who exist, XX cis men, cis men who have vaginas (due to childhood mutilation), and various other forms of intersex people. These people bring light to the outdated idea that sex is based on genitals, just as trans* people do. Trans* people challenge the very notion of the sex binary.