結構相似性

結構相似性的基本觀念為自然影像是高度結構化的[1]，亦即在自然影像中相鄰像素之間有很強的關聯性，而這樣的關聯性承載了場景中物體的結構資訊。人類視覺系統在觀看影像時已經很習慣抽取這樣的結構性資訊。因此，在設計影像品質衡量指標用以衡量影像失真程度時，結構性失真的衡量是很重要的一環。

給定兩個信號${\displaystyle \mathbf {x} }$和${\displaystyle \mathbf {y} }$，兩者的結構相似性定義為：

${\displaystyle {\text{SSIM}}(\mathbf {x} ,\mathbf {y} )=[l(\mathbf {x} ,\mathbf {y} )]^{\alpha }[c(\mathbf {x} ,\mathbf {y} )]^{\beta }[s(\mathbf {x} ,\mathbf {y} )]^{\gamma }}$，

${\displaystyle l(\mathbf {x} ,\mathbf {y} )={\frac {2\mu _{x}\mu _{y}+C_{1}}{\mu _{x}^{2}+\mu _{y}^{2}+C_{1}}}}$，${\displaystyle c(\mathbf {x} ,\mathbf {y} )={\frac {2\sigma _{x}\sigma _{y}+C_{2}}{\sigma _{x}^{2}+\sigma _{y}^{2}+C_{2}}}}$，${\displaystyle s(\mathbf {x} ,\mathbf {y} )={\frac {\sigma _{xy}+C_{3}}{\sigma _{x}\sigma _{y}+C_{3}}}}$。

其中，${\displaystyle l(\mathbf {x} ,\mathbf {y} )}$比較${\displaystyle \mathbf {x} }$和${\displaystyle \mathbf {y} }$的亮度，${\displaystyle c(\mathbf {x} ,\mathbf {y} )}$比較${\displaystyle \mathbf {x} }$和${\displaystyle \mathbf {y} }$的對比度，${\displaystyle s(\mathbf {x} ,\mathbf {y} )}$比較${\displaystyle \mathbf {x} }$和${\displaystyle \mathbf {y} }$的結構（structure），${\displaystyle \alpha >0}$，${\displaystyle \beta >0}$，${\displaystyle \gamma >0}$，為調整${\displaystyle l(\mathbf {x} ,\mathbf {y} )}$、${\displaystyle c(\mathbf {x} ,\mathbf {y} )}$、${\displaystyle s(\mathbf {x} ,\mathbf {y} )}$相對重要性的參數，${\displaystyle \mu _{x}}$及${\displaystyle \mu _{y}}$、${\displaystyle \sigma _{x}}$及${\displaystyle \sigma _{y}}$分別為${\displaystyle \mathbf {x} }$和${\displaystyle \mathbf {y} }$的平均值和標準差，${\displaystyle \sigma _{xy}}$為${\displaystyle \mathbf {x} }$和${\displaystyle \mathbf {y} }$的协方差，${\displaystyle C_{1}}$、${\displaystyle C_{2}}$、${\displaystyle C_{3}}$皆為常數，用以維持${\displaystyle l(\mathbf {x} ,\mathbf {y} )}$、${\displaystyle c(\mathbf {x} ,\mathbf {y} )}$、${\displaystyle s(\mathbf {x} ,\mathbf {y} )}$的穩定。

結構相似性指標的值越大，代表兩個信號的相似性越高。

試想，若使用全等的兩張圖片去做SSIM運算，也就是說${\displaystyle \mu _{x}=\mu _{y}}$且${\displaystyle \sigma _{x}=\sigma _{y}}$，如此一來

${\displaystyle {\text{SSIM}}(\mathbf {x} ,\mathbf {x} )={\frac {2\mu _{x}^{2}+C_{1}}{\mu _{x}^{2}+\mu _{x}^{2}+C_{1}}}\times {\frac {2\sigma _{x}^{2}+C_{2}}{\sigma _{x}^{2}+\sigma _{x}^{2}+C_{2}}}\times {\frac {\sigma _{xx}+C_{3}}{\sigma _{x}\sigma _{x}+C_{3}}}=1}$

結構相似性指標具有下列性質：

• 對稱性

結構相似性指標是對稱的，亦即

${\displaystyle {\text{SSIM}}(\mathbf {x} ,\mathbf {y} )={\text{SSIM}}(\mathbf {y} ,\mathbf {x} )}$。

• 有上下界

結構相似性指標的範圍為−1到1。當衡量的兩個信號完全相同時，結構相似性指標的值為1。

實際使用時，簡化起見，一般會將參數設為${\displaystyle \alpha =\beta =\gamma =1}$及${\displaystyle C_{3}=C_{2}/2}$，得到：

${\displaystyle {\text{SSIM}}(\mathbf {x} ,\mathbf {y} )={\frac {(2\mu _{x}\mu _{y}+C_{1})(2\sigma _{xy}+C_{2})}{(\mu _{x}^{2}+\mu _{y}^{2}+C_{1})(\sigma _{x}^{2}+\sigma _{y}^{2}+C_{2})}}}$。

在計算兩張影像的結構相似性指標時，會開一個局部性的視窗，一般為${\displaystyle N}$×${\displaystyle N}$的小區塊，計算出視窗內信號的結構相似性指標，每次以像素為單位移動視窗，直到整張影像每個位置的局部結構相似性指標都計算完畢。將全部的局部結構相似性指標平均起來即為兩張影像的結構相似性指標。

結構相似性指標因其簡單而有效，近年來廣泛被使用在影像與視訊處理的相關應用上，像是影像壓縮[4]、影像浮水印[5]、無線視訊串流[6]、核磁共振成像[7]等等。

結構相似性指標有其限制，對於影像出現位移、縮放、旋轉（皆屬於非結構性的失真）的情況無法有效的運作。為解決此問題，另已發展出在小波域進行運算的結構相似性指標，稱作複小波結構相似性指標[8]（英文：complex wavelet SSIM，CW-SSIM）。

• http://www.ece.uwaterloo.ca/~z70wang/research/ssim/

1. Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli, "Image quality assessment: from error visibility to structural similairty," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600−612, Apr. 2004.
2. Zhou Wang and Alan C. Bovik, "Mean squared error: Love it or leave it? - A new look at signal fidelity measures," IEEE Signal Processing Magazine, vol. 26, no. 1, pp 98−117, Jan. 2009.
3. H.R. Sheikh, M.F. Sabir, and A.C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms," IEEE Transactions on Image Processing, vol.15, no.11, pp.3440−3451, Nov. 2006.
4. T. Richter, K. J. Kim, "A MS-SSIM optimal JPEG 2000 encoder," in Proc. Data Compression Conf., pp.401−410, Mar. 2009.
5. A. M. Alattar, E. T. Lin, and M. U. Celik, "Digital watermarking of low bit-rate advanced simple profile MPEG-4 compressed video," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 8, pp. 787−800, Aug. 2003.
6. V. Vukadinovi and G. Karlsson, "Trade-offs in bit-rate allocation for wireless video streaming," in Proc. ACM Int. Symp. Modeling, Analysis, and Simulation of Wireless and Mobile Systems, Quebec, Canada, 2005, pp. 349−353
7. S. A. Reinsberg, S. J. Doran, E. M. Charles-Edwards, and M. O. Leach, "A complete distortion correction for MR images: II. Rectification of static-field inhomogeneities by similarity-based profile mapping," Phys. Med. Biol., vol. 50, no. 11, pp. 2651−2661, June 2005.
8. Z. Wang and E. P. Simoncelli, "Translation insensitive image similarity in complex wavelet domain," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, pp. 573−576, Mar. 2005.