It's more than not having a child.
One thing that stuck out to me in watching family members go through infertility of one type or another was that there are many more layers to the grief than I would have known. People often post pictures of their babies on social media or tease acquaintances about why they don't have kids yet. It's important for us to fully understand what these 1 in 8 couples might experience when we do so.
Obviously, for people who desperately want kids, there's the surface level of sadness based on not having them. Added to this is how entrenched this pain is in any future planning -- whether lifestyle, location, financial... any decision you make depends on whether or not you have kids. Even just picturing your future, setting goals, or simply daydreaming is now tainted with loss. You may spend years without a definite answer in one direction or another.
On top of that, it's a physiological problem that can cause feelings of self-blame and inadequacy. Many people feel that they've let their partner down, or their family (if their family members frequently vocalize their desire for more grandchildren, nieces/nephews, etc). You could get lost in the world of over-analyzing everything you've put into your body, worrying about every chemical you've been exposed to, all the caffeine you drink, how often you put a laptop on your lap, etc.
It's not like you can go somewhere to forget about it, either. There are pregnant people and babies everywhere, and every sighting can be triggering. Even things like daycare centers, playgrounds, kids' sections of clothing and book stores...
Then there's the process itself. You may need frequent treatments and doctor visits, which can be incredibly expensive and physically taxing on your body. Some medications require frequent ultrasounds, which require missing work, but the timing is based on your unpredictable body, so you can't request it off ahead of time.
Hopefully people can begin to grasp why "well, there's always adoption!" isn't an adequate tonic to this particular pain.