I played him quite often in the jungle, and his clear speed is truly amazing. Generally, I would use him if my lanes can provide the CC I'm lacking as the slow can be missed, his post 6 ganks are actually quite strong in my opinion. If I jungle him, I usually go a bit more tankier than I would in lane, Frozen Mallet is something I almost always take on my jungle Zed.
I didn't say I would rush the Frozen Mallet, I usually get the Phage first and wait a bit until I upgrade it. I usually get the "standard" items such as Brutalizer, Last Whisper and upgraded boots in between. I did not have a problem damagewise so far, as 1 ulti on their carry would generally kill him if all my other spells landed.
The time when I finish my Frozen Mallet would generally be around the 30 minute mark, the time can vary though.