When the government needs money that it doesn't have, it sells treasury bonds. Think of it like going to the bank to get an interest only mortgage for your home.
For example, lets say the government wants to borrow $100,000. They can issue a 30 year treasury bond at (X) interest rate. If it is 4%, they will pay the investor $4,000 per year for 30 years. If it is 6%, they will pay the investor $6,000 per year for 30 years. At the end of the 30 years, the investor then receives his initial $100,000 back.
Treasury bonds can be issued for 1 month, 3 month, 6 month, 1 year, 2 year, 5 year, 10 year, or 30 year terms. An investor usually receives a higher interest rate the longer they are willing to let the government borrow their money. For example, a 10 year treasury bond, if you bought it today, would pay you 3.5% interest. A 30 year bond would pay you 4.5%.
Bonds are valued by their demand in the free market. If there is a low demand for bonds, interest rates will go higher to entice buyers into the market. If rates go up, that means the bond is losing value. Here's why:
Lets say you bought $100,000 worth of 10 year treasury bonds at 5%. This means that every year the government pays you $5,000.
What if the rates on 10 year treasury bonds rose to 10%? That means it would only take $50,000 worth of 10 year treasury bonds to receive the same $5,000 every year.
So if you want to sell your bonds, you will only receive $50,000 in the market. The rates doubled, so the bonds lost half their value.
The following chart shows a long term look at 10 year treasury bond interest rates:

This past fall we saw a spectacular grand finale to a 30 year bull market in treasury bonds. Many people believe the yields will stay low forever. I think they will be surprised at where we go from here.