h2g2 The Hitchhiker's Guide to the Galaxy: Earth Edition

Hi,
I've been dating an old highschool friend for the last 2 months or so. He and I have a wonderful time together, but he's been out of town a lot, and hasn't called too much lately. Other than that, he's been wonderful...attentive, generous, considerate, etc.
He was divorced about 5 years ago, and just ended a 2 year relationship with another woman about 4 months before we started dating. Apparently, the "other woman" wanted a serious relationship..wanted to move in etc. She told him to S*** or get off the pot. He chose to end the relationship with her. He and I had 3 or 4 dates when he emailed me, stating that, "although our dates have been fun, I just don't think we have that much in common". After some discussion, where he claimed that he thought I wanted a serious relationship, and he wasn't ready for that, I assured him that I was cool with "casual". THEN we had sex, which was wonderful, but he hasn't been in town more than a few hours in the last few weeks...only calling twice in 3 weeks or so.
The thing that bothers me is this: he doesn't want me to come to his house (even though it's just 3 miles away, and he's shown me where it is), because he doesn't want his kids to be upset by "another woman". I can understand his reluctance to introduce his children to me if our relationship is not serious, but I found out that they are 17 and 22 years old! It seems to me that he is keeping his private life too private, and that his daughter and son are of the age where they shouldn't be affected by their father's dating relationships. Or, is it that he's just "Not that into me?"
I would love to hear feedback on this. Thanks, in advance.
lalalalalola