Your Pagerank is calculated only based on your incoming links (backlinks).
When you link to external sites, you help other sites increase their PageRank, but you do not lose your page PageRank.
You can see the formula on the Wikipedia Pagerank page.
However, if you link to sites that are penalized by Google, you will also get penalized. You can safely link to other sites (even if they are penalized) if you add nofollow to that link.