My dog hacks, is this something I should worry about?

He doesn't do it all the time, but he will hack up flem, it's not like a cough and I'm really not sure if it is something I should worry about. We got him from someone that didn't take care of him very well, we took him to the vet when we first got him 5 months ago and tested him for heartworms and got all his shots. It doesn't even seem like it bothers him, but it is kinda gross. We call him a Llama because he hack's all the time, sometimes giving him water will make him stop, he always has water available but we will give him cold water because he will drink it when put in front of him unlike regular water, but somtimes it won't make him stop it just slows it down some. Should I be worried? He is otherwise a very happy go lucky dog, with lots of energy.