Should I worry about this?

I probably fit with about 90% of what you said, and I don't have a big problem with it. As long as you set your priorities and you have something to keep yourself amused, which it sounds like both boxes are checked, you needn't worry about it.