Venison's Latin root "venatio" refers to "hunting game", thus venison could mean the meat of any animal hunted down. The word was still being used in this context in the 19th century as "kangaroo venison".
Due to a ban on commoners hunting deer in certain parts of the medieval world, venison is often regarded as a meat for nobility only.
"Hares are thought to nourish melancholy, yet they are eaten as venison, both roasted and boiled." Fynes Moryson, An Itinerary, 1617.