Quality of Experience (QoE) based handoffs in heterogeneous access networks (HAN) necessitates accurate QoE estimation and prediction. The current approaches to QoE-aware handoffs are limited. These approaches either lack the availability of underlying probing mechanism or lack the availability of QoE estimation and prediction mechanism. In this paper, we propose, develop and validate a novel method for QoE estimation and prediction using passive probing mechanisms. Our method is based on hidden Markov models and multi-homed mobility management protocol. Using extensive simulations and experimental studies, we show that our method achieves QoE estimation accuracy of 100% and prediction accuracy of 97% in HAN without using additional probe packets for QoE estimation and prediction.