Stop messing around with emotions and doing it the hard way: Go to the court, have them make him take responsibility for himself and his decisions. My boyfriends dad is like this too: always wanted to be part of his life but always had an excuse to try to not pay support.
Go to the courts, get it in writing, get him to start fulfilling his responsibilities financially, and then talk about visitation.