I grew up in smaller towns. Not small in the grand scheme of things, but the town I spent most of my childhood in was about 20,000, and the city I went to university in was about 100,000. These are the sorts of places you drive a car to go somewhere, because traffic isn't a big deal 99% of the time(and because you may need to go to the next town over for something), and because they're not big enough to have good transit options, but too big to want to just walk.
Two years ago, I moved to Toronto. I'd always lived close enough to the city that I knew my way around it, but this was my first time living there. And it was surprisingly disorienting. Obviously the traffic sucks, and obviously there's a lot more people, but I expected that going in, and that wasn't what did it.
It took me a couple months to figure out why I felt so uneasy about moving around - the problem wasn't the density. The problem was the lack of density. It sounds bizarre to say that the biggest city in the country, and on paper probably the densest, lacked for density. But it does, and the reason why it does is interesting. See, I don't care how many people are around me. I don't care how big the suburbs are, or how many schools are around, or what have you. I care about my travel times. When I was living with my parents, 100km out of the city, getting to downtown Toronto took me about an hour. Now that I live in Toronto, getting to downtown Toronto takes me...well, almost an hour. It's a more comfortable trip in some ways, because I get to read on the subway, but it's not really much closer than it was before.
It used to be that just about everything I cared about was within 10-15 minutes of me, and now 10 minutes gets me almost nowhere. There's more restaurants in that circle, but I don't eat out much. There's more people, but I don't know them. I used to be able to cross town in the space of time it takes me to get to the subway. So as a result, the actual feel of it is almost a wasteland compared to what I was used to. It's a wasteland full of people and interesting things and nice shops, but it's like the ghost of civilization for my purposes - it looks the same, but it doesn't actually have any effect on me.
I don't live downtown, and it might change some things if I did, but I don't think so. Downtown means that a huge selection is close to you, but it cuts you off from everything more than a short distance away. I know people who refer to Bloor (which is about a 20 minute walk north of downtown) as "the wall", in the Game of Thrones sense of the term. So while the hustle and bustle is a warm cozy embrace, 90% of the city might as well not exist. I like that 90% - there's lots of interesting things there. But for a downtowner, getting to them is an ordeal. I have friends who are more likely to visit Montreal than Martin Grove Road, and I can't even really blame them. With the fact that both cities have airports fairly close to downtown, it may even be faster.
The funny thing is, I do like the city. One of the ways that having a lot of people around helps is that you can build strong subcultures more easily. My board gaming group gets like 50 people out every week - that'd never have happened back home. The bustling background does have bits and pieces that pop into my view sometimes, and I'm glad of them. Heck, even if I am going downtown, the fact that I don't need to park there is a relief. But as much as I like the city in a lot of ways, I just wish it was as dense as a small town.
(Urban planners of my acquaintance, I'm sorry for making your heads explode)