Destroying network instances
There is a counterpart to Network.Instantiate : Network.Destroy. The simple advice is, whenever possible, use Destroy NOT Network.Destroy.
Using Destroy() means that you let a client destroy its local copy of a network object in due time, not in the middle of doing something else.
Do not cache network instances (as a rule, do not cache anything until you are truly desperate. And one more thing: don't optimise). This is a recipe for creating intermittent bugs.
When do we know that a player has disconnected?
Any object can process OnPlayerDisconnected(Network player)
They're coming to get you Barbara!
As soon as you get this to work, you will notice that opening new clients causes logged out players to return from hell. This is all thanks to buffered RPCs. What does it mean for an RPC to be "buffered"? Well, it means that any buffered RPCs will hit every client you open, in play order.