You need to do a simple thing. Go to /etc/vncserver/vncservers.conf and edit the file with one of your favouite editors. My file looks like this:
VNCSERVERARGS=”-geometry 1920×1068 -depth 16 -BlacklistTimeout 0″
So the BlacklistTimeout 0 says that it should not blacklist ip’s when they try to connect. It is such a nasty feature.