As it turns out, a hardwood is not necessarily a harder material (more dense) and a softwood is not necessarily a softer material (less dense). For example, balsa wood is one of the lightest, least dense woods there is, and it's considered a hardwood.
The distinction between hardwood and softwood actually has to do with plant reproduction. All trees reproduce by producing seeds, but the seed structure varies. Hardwood trees are angiosperms, plants that produce seeds with some sort of covering. This might be a fruit, such as an apple, or a hard shell, such as an acorn.
Softwoods, on the other hand, are gymnosperms. These plants let seeds fall to the ground as is, with no covering. Pine trees, which grow seeds in hard cones, fall into this category. In conifers like pines, these seeds are released into the wind once they mature. This spreads the plant's seed over a wider area.
For the most part, angiosperm trees lose their leaves during cold weather while gymnosperm trees keep their leaves all year round. So, it's also accurate to say evergreens are softwoods and deciduous trees are hardwoods.
The hardwood/softwood terminology does make some sense. Evergreens do tend to be less dense than deciduous trees, and therefore easier to cut, while most hardwoods tend to be more dense, and therefore sturdier. But, as the classification of balsa wood demonstrates, there is no minimum weight requirement to become a hardwood.
Here are some interesting links: