A site to site VPN is what you are looking for.
Assuming House A with the server(s) is on the 192.168.1.0/24 subnet, and house B is on 192.168.2.0/24, both the VPN endpoints (i.e. the routers) would know about the other subnet, and would route traffic to it.
So, if your computer in house B was 192.168.2.10, and tried to access the server on 192.168.1.15, the router of house B would forward that traffic to router A, which would send it to the server, and vice versa.
Not many ISP routers will support this. You could use a small PC running OpenSSL or Sophos UTM (both free, at least for home use) at each end, or you could buy a router for each end that supports IPSec or SSL site to site. An example would be most of the Draytek series.
You could set up the VPN tunnelling rules to only send traffic for a particular IP through the VPN, so that all the house B devices can access at house A is the server, but the server at house A would be able to access everything at house B, it has to be this way as you can't have one way traffic.
So, people in house A in theory could do malicious things to house B devices by logging on to the server.
Edited by nemeth782 (Tue 11-Mar-14 09:03:15)