It is really easy to set up your voicemail with a message that gives specific times that you return calls. When you pick up the phone to answer it, just because it rings, you are interrupting whatever your current task at hand is. How long will it take you to recover from that call? Is it worth answering the phone every time it rings?
Make sure your staff knows how to answer the phone and take messages for you. Let them know when you do not want to be disturbed. If you do not have staff to handle this task for you set up your voicemail message in a manner that lets people know when you check your messages. Some people even choose to have their voicemail say a better method of contacting them like text or email.