The first armies were small bands of men equipped with clubs and spears, led by tribal chiefs. Large organized armies first appeared in ancient Egypt and the Middle East about 1500 B.C. or earlier. As civilizations grew in the valleys of the Tigris and Euphrates rivers in Asia, and in the Nile Valley of Africa, ancient armies also started developing. The Babylonians had built a regular army of spear throwers and archers as early as 3200 B.C. In about 2500 B.C., the Sumerians used the first war chariots. These chariots were driven by small wild asses and, later, by horses. Organized armies that were equipped with spears and battering rams were first formed by Assyrians in the 8th century B.C.

One of the largest and most powerful ancient armies was that of the Persian Empire. Cyrus the Great of Persia was the mightiest conqueror of the period. Cyrus extended the Persian Empire to include most of southwestern Asia in the 300's B.C. It had perhaps 500,000 men in time of war. This force was made up of mass formations of infantry supported by charioteers. After being supreme for centuries, the Persian army was finally defeated by the smaller but more cleverly led Greek army under Alexander the Great. The first known military supply system was organized by Alexander the Great of Macedonia. The key to this army's success was its basic tactical unit, the phalanx. The ancient Greeks were the first to introduce phalanx, which was probably the first important tactical formation in history, in the 7th century B.C. Phalanx was primarily a defensive formation, where soldiers with spears and heavy armor stood from 4 to 50 rows deep in a solid rectangle. Although it could neither move rapidly nor attack across rough ground but it was strong enough to withstand the shock of a cavalry charge.

After the decline of Greece in the third century B.C., the Roman army rose to dominate the ancient world. Under the Roman republic, all young men were trained for military service and subjected to rigid discipline. The Romans developed the legion, during the early 4th century B.C. The legion was a tactical unit that was easier to maneuver than the Greek phalanx. Under Julius Caesar the Roman legions conquered most of western Europe, built roads and bridges. Consisting of three lines of small phalanxes called maniples or, later, cohorts, the legion is a rectangular formation that had greater flexibility than the phalanx. The Romans, who were also skilled military engineers, conquered Carthage in 146 B.C. and built a great empire overseas.

Totally secluded from the rest of the world, the China's civilization grew up from the empires of Europe and western Asia. It was long before the A.D. 14th century that the Chinese had knowledge of gunpowder, when Europeans began to use it in guns. As the Chinese did not use gunpowder in artillery, except possibly in rockets, it did not become an important weapon of war for them.

Ancient armies had no artillery, in the modern sense, but they employed "engines of war" that could throw huge stones, arrows, or pots of burning tar.

In the Battle of Adrianople (378 A.D.), mounted barbarian warriors defeated Roman legions, marking the decline of infantry as the decisive factor in battle. After the fall of Rome in the fifth century A.D., organized armies virtually ceased to exist in Europe. In the Middle Ages, military relationship was the basis of Western Europe's society. In this kind of a society, a king ruled the society supported by the nobles who had political, economic, judicial, and military power. Throughout the Middle Ages armies were small, poorly organized, and poorly equipped. The armored knight on horseback, wielding lance, armor and broadsword, was for centuries the dominant military force. Castles were built as defensive positions by local barons.

To minimize the cost of maintaining medieval armies, some rulers opted for mercenaries who were hired soldiers to fight for their nations. Some of the mercenaries were specialized in siege weaponry, such as the stone-throwing weapons called catapults, while others used crossbows or long spears known as pikes. The Swiss pikemen were the best-known mercenaries of the late Middle Ages, who used to form large phalanxes to prevent heavily armored, mounted knights from overrunning them. The company, which still forms the basic unit of most armies, was the Swiss mercenaries, formed groups with hired soldiers from England, Germany, and Italy.

Each feudal lord had his following of knights, squires, pikemen, and bowmen, but there was little organization; leadership depended on example and persuasion. The commander of a medieval army customarily divided it into three battles corresponding to right wing, center, and left wing. This type of organization was better than none, but it did not provide the commander with much control over his forces in combat.

The Ottoman Empire, which ruled what are now Turkey, the Middle East, Greece, and most of southeastern Europe were the first to establish the first full-time professional national army since the Roman Empire, during the 14th century. Mostly slave soldiers taken from the Balkan provinces, these soldiers, were known as Janissaries. On the battlefield they were much feared as they were loyal only to the sultan, who ruled the Ottoman Empire, and to their fellow Janissaries.The Western European nations maintained a permanent army walking on the lines of the Ottoman example. A permanent army, usually known as the Royal Army or King's Army was maintained by majority of the European nations by 1500. The German independent states were the first to develop permanent quartermaster organizations to find quarters and food for their armies.