A cave is a place where you can find many resources or it's just a little room. Sometimes you find them on the surface, others are buried underground. These are the easy ways to find them (else than finding them by just digging rock)
Surface Cave Entrance
Usually, when you go exploring the surface, you might find cave entrances. Watch out: Monsters may be at the entrance. This method is known as Cherrypicking, but you can also use a good stategy called strip mining.
Common entrance for a cavern.
Common Mob(s) in the cavern (Spiders as this picture shows)
Resource veins
As caves have a lot of resources, you can enter a cave by digging resource veins.
Dirt/Gravel veins
Caves have dirt/gravel veins on them depending of how big they are. A big cave has many dirt/gravel veins and a short cave might have only one of these. Even through you find no cave by digging these, you would had created a cave-like room or it maybe means that a cave is nearby.
Example of a Gravel vein to a entrance into a cave.
Dungeons
Dungeons are places where monsters generate and there is loot on them. Most dungeons are connected with caves. However, dungeons are quite rare and are not an easy way to find caves as most are found in a pre-existing cave and very few are found on the surface.
(More detail on Dungeons can be found here)
Ambience
Another way to find caves is to try and find spots where you here strange noises that indicate dark places. if you were to hear a zombie, spider, skeleton or slime you should dig to it because you might find a whole new cave system!
(More detail on Ambience can be found here)
The Button
Pressing the F3 key will show a mob's ID. This will show the location of a mob in a cave (when not on Peaceful); digging toward the mob will likely reveal a tunnel system where the mob was.