U-Net and other U-shaped architectures have achieved significant success in image deconvolution tasks. However, challenges have emerged, as these methods might generate unrealistic artifacts or hallucinations, which can interfere with analysis in safety-critical scenarios. This paper intro- duces a novel approach for quantifying and comprehend- ing hallucination artifacts to ensure trustworthy computer vision models. Our method, termed the Conformal Halluci- nation Estimation Metric (CHEM), is applicable to any im- age reconstruction model, enabling efficient identification and quantification of hallucination artifacts. It offers two key advantages: it leverages wavelet and shearlet represen- tations to efficiently extract hallucinations of image features and uses conformalized quantile regression to assess hallu- cination levels in a distribution-free manner. Furthermore, from an approximation theoretical perspective, we explore the reasons why U-shaped networks are prone to hallucina- tions. We test the proposed approach on the CANDELS as- tronomical image dataset with models such as U-Net, Swin- UNet, and Learnlets, and provide new perspectives on hal- lucination from different aspects in deep learning-based im- age processing.