The idea of remaining safe while attacking from a distance has been and always will be a major driving force when it comes to inventing weapons. Basic devices such as the slingshot, spear, bow and arrow and even the boomerang were all developed to make it easier to kill the enemy while remaining as far away from danger as possible.
This need to attack from a distance, coupled with the invention of increasingly complex metalwork techniques, led to the invention of ever more complex devices for launching projectiles; in around 1500 AD mortars became popular as a siege weapon. A tube of metal from three to five feet in length, weighing several hundred pounds, would be placed on the ground and aimed into the air. Mortar shells would then be dropped into the tube and propelled upwards by an explosive charge. The mortar operator had relatively little control over where the shell landed, but despite this mortars increased in popularity and are still used today. A small mortar can easily be moved and operated by two people, and a small, high-quality shell can have the destructive force of a stick of dynamite: easily enough to destroy a small vehicle. The destructive power of the shells increases with size, of course - the larger the shell the greater the power - and mortars come in a range of sizes to reflect this. There is a trade-off between cost, weight and effectiveness. The mortar's usefulness as a weapon is limited by the lack of an accurate aiming mechanism and its relatively short range: Because the mortar shell has to go up before it can come down, much of its speed is dedicated to getting it far enough up into the air -- so against a ground target, it has limited reach.
You can solve the range problem by improving the path of the projectile - a straight line from launching device to target is far more efficient. Enter the rocket: A rocket makes a sensible choice, because it can easily be launched from a tube device and is capable of travelling for at least a mile. In fact, a large rocket can travel up to twenty miles. The equipment required for such rockets is not unlike that used in launching mortars. Essentially, all that is needed is a set of tubes mounted on a platform or towed by a small vehicle. While this is useful, in this form it is way too bulky to replace the easily deployed mortar.
Combine certain elements of these two weapons and you've got the basics behind a rocket-propelled grenade. A small rocket is mounted on a tube, which is then aimed and launched. It may not travel as far as a larger rocket - only a few hundred feet - but it is more portable, can be aimed directly at the target, and will still do the same damage as a stick of dynamite upon impact.