There are two popular methods of making something go really fast - jets and rockets. They are often confused. This entry explains the differences in their principles of operation.
As with most human technologies, nature got there first. Certain fungi spread spores using something similar to a rocket engine, while squid, cuttlefish and octopi use jet propulsion to move through water.
Humans invented solid fuelled rockets in China almost a thousand years ago, while the first practical jet engine was invented in England by Frank Whittle in the 20th Century.
As is the case with many technologies, the first application of both principles was in war. Rockets were used for delivering explosives over great distances, whereas jets were used to allow fighter planes to fly much faster than was possible using propellers.
Newton's Third Law
Newton's Third Law of motion states that for every action, there is an equal and opposite reaction. You can demonstrate this using a child's swing and an object such as a heavy jacket. Sit on the swing, holding the jacket, and keep your feet off the ground. Throw the jacket forwards. You will move backwards. The action of moving the jacket forward had an equal and opposite reaction of moving you (and the swing) backwards. (You didn't move as far as the coat because you weigh more than the coat does. Also, some of the energy was lost to air resistance, friction on the bearings of the swing etc, but the principle is what matters.)
A jet engine requires a medium in which to work. In the case of an aeroplane, the medium is the air. In the case of a squid, the medium is the water.
The engine works by drawing in the medium at the front, compressing it and accelerating it, then forcing it out at the back at a much higher speed. A squid does this by squeezing with its muscles, but jet engines require fuel, combustion and complex arrangements of turbines1.
The acceleration of the air or water in one direction causes an equal and opposite reaction, namely an acceleration of the engine or the squid in the opposite direction.
Going back to the swing analogy, the air in front of the jet engine is simply a continuous supply of jackets to throw out at the back. A jet engine therefore absolutely requires the presence of a medium which it can accelerate in order to work.
A rocket is in principle simpler than a jet engine, which may explain why rockets were invented so much earlier.
Basically, a rocket burns fuel and throws the combustion products out behind it. This produces the familiar 'equal and opposite reaction' and accelerates the rocket forward until the fuel runs out. And that's it.
In the swing analogy, a rocket engine has all the jackets it needs on the swing already - it needs no external supply. This on-board supply is sometimes called the reaction mass. Since a rocket can carry as much reaction mass as it needs, it can work perfectly well in the vacuum of space.
Jets vs Rockets
A jet uses the combustion of a small quantity of fuel to accelerate a large quantity of air. To get the same reaction, a rocket must combust a huge quantity of fuel, which also needs to be extremely high in energy. Petroleum products are fine for cars, and even jets, but rockets such as the space shuttle burn exotic mixtures mainly composed of hydrogen and oxygen. Because of these quantities, rockets are terribly inefficient compared to jets, which explains why Concorde has no competition from rocket airliners.
On the other hand, a rocket can generate huge accelerations which would be impossible to achieve with jets (which is why they're used in things like ejector seats), and they can work in the vacuum of space, which is obviously why they're used on spacecraft. They're also in principle much simpler to build and maintain, with fewer moving parts. This obviously accounts for the cheapness of fireworks, which are rockets, not jets.
Jets then Rockets
One method which has been proposed to take advantage of the different benefits of rockets and jets is the space-plane. This is a spacecraft which takes off from the ground like a aeroplane, under jet power. When the plane reaches an altitude where there's not enough air to sustain combustion in the jet engine2, rockets take over. Several designs have been produced, but so far no working examples have flown into space.
The big problem with a rocket as a vehicle is that it must carry all the fuel it will ever need. So in order to go to the Moon and back, you don't just need enough fuel to get your ship there and back; you in fact need enough fuel to get you and enough fuel to get home to the Moon and back. This soon adds up until the vast majority of the fuel on board is actually used up accelerating fuel tanks.
It gets worse. Fuel needs oxygen to support its combustion, and there is none in space. Oxygen must therefore be carried on the rocket as well, increasing the weight still further. Oxygen is usually carried in liquid form at very high pressure, which brings on a host of other problems.
It is commonly believed that jets or rockets work by 'pushing against' the air behind them. This is not true. Air, if pushed against, moves out of the way. This would also not account for the fact that rockets work in the vacuum of space.
It is also commonly believed that jets work in space. Also not true, since a jet requires a medium to accelerate, and in space, there is no sufficiently dense medium present. Although space contains about one hydrogen atom per cubic centimetre, an engine has been designed that might take advantage of this - see the Entry on the Bussard Ramjet.
NASA rather confusingly refers to the devices which are used to make minor adjustments to the position of spacecraft as attitude jets. This is because they fire jets of compressed gas through small nozzles into the vacuum of space. However, they should not be confused with jet engines as used on aircraft. Despite the name, they are in principle rockets. A slight difference here is that the reaction mass is used directly, like a jacket on a swing, and no combustion takes place. This is very inefficient, but the jets are only required for making small rotational adjustments, and the simplicity keeps the devices light and reliable.
Another common misconception is that rockets work by 'pushing against' their exhaust products in the vacuum of space. This is equally untrue. The exhaust products are themselves getting out of the way at great speed - more-or-less the speed of the engine itself - so there's no point trying to push against them.
Jets use a little fuel to accelerate a lot of air (or other medium) and thus, indirectly, themselves. If there's no medium, they don't work.
Rockets use a lot of fuel to accelerate themselves directly. If there's no air, no problem.