통계/확률론 및 수리통계학 (10) 썸네일형 리스트형 Measure theoretic description of change-of-variables 1. SettingsTwo measurable spaces:\[(X,\mathcal{A},\mu)\quad\text{and}\quad (Y,\mathcal{B},\mu_T)\]Consider a transform \(T\) such that:\[T:X\rightarrow Y\]where T is a measurable map. i.e., \(T^{-1}(B)\in \mathcal{A}\quad \forall B\in\mathcal{B}\).\(T\) induces a pushforward measure \(\mu_T\):\[\mu_T(B)=\mu(T^{-1}(B))\]2. Change-of-variablesLet \(\nu\) and \(\mu_T\) are measures defined on \(Y\).. Measure theoretic description of KL divergence In this writing, I consider the KL divergence with measure theoretic approach. Suppose \(P\) and \(Q\) are probability measures on a measurable space \((\Omega, \mathcal{F})\), and \(P\ll Q\). That is, \(P\) is absolutely continuous with respect to \(Q\). (Or, equivalently, \(Q\) dominates \(P\)) KL divergence is defined as below:\[\begin{aligned} \mathcal{D}_{KL}(P||Q)&:=E_P\left[\log \frac{dP}.. Metric vs distance measure Metric은 symmetry를 만족해야함. d(x,y)=d(y,x) Distance measure는 symmetry를 만드시 만족할 필요 없음. ex) KL-divergence. 그러면 \(Metric \subset Distance \: Measure\)인가? Metric은 distance measure의 특수한 경우가 되는건가?? ------------------------------------------------------------------------ (2024.01.31 수정) "Although the KL divergence measures the “distance” between two distributions, it is not a distance measure. This is beca.. Is a pdf a probability measure? No. Conditions of a Probability Measure (\(P\)) 1) \(P\) must return results in \([0,1]\) ... Conditions of a PDF (\(f\)) 1) \(\int_{-\infty}^\infty f(x)dx=1\) ... Differences Pdf는 output이 1을 넘을 수 있음. 그러나 probability measure는 output이 [0,1]에 있어야 함. 또한 pdf는 리만-스틸체스 적분을 사용. Probability measure는 르벡적분을 사용. 따라서 pdf는 probability measure가 아님. 의미를 보면 다음과 같음. \(f: \mathbb{R} \rightarrow \mathbb{R}\) \(P: .. Covariance matrix의 다양한 이름들 & Autocorrelation matrix Covariance matrix $$Cov(\mathbf{x})=\mathbb{E}[(\mathbf{x}-\mathbb{E}[\mathbf{x}])(\mathbf{x}-\mathbb{E}[\mathbf{x}])^T]$$ Covariance matrix = auto-covariance matrix = variance-covariance matrix = variance matrix = dispersion matrix 진짜 헷갈린다. 각자 다 다른 것 같이 보이지만 모두 covariance matrix를 의미한다. Autocorrelation matrix $$\mathbf{R}_{xx}=\mathbb{E}[\mathbf{x}\mathbf{x}^T]$$ 확률 P와 기대값의 부등식 \[P(X \ge \varepsilon) \le \frac{\mathbb{E}(X)}{\varepsilon} \] 를 증명한다. 르벡적분을 이용하면, \[P(X \ge \varepsilon)=\int_{[X \ge \varepsilon]}dP\] \[=\int_{\Omega}\mathbb{1}_{[X \ge \varepsilon]}dP\] \[\le\int_{\Omega}\frac{X}{\varepsilon}\mathbb{1}_{[X \ge \varepsilon]}dP\] \[\le\int_{\Omega}\frac{X}{\varepsilon}dP\] 기대값의 정의에 의해 \[=\frac{\mathbb{E}(X)}{\varepsilon}\] Limsup Liminf Intution \(\limsup\)과 \(\liminf\)를 직관적으로 접근하는 법 \({A_n}\)이 있으면 \[S_1=A_1 \cap A_2 \cap A_3 \cap \cdots \subset A_1 \subset A_1 \cup A_2 \cup A_3\cup \cdots = B_1\] \[S_2=\quad A_2 \cap A_3 \cap \cdots \subset A_2 \subset \quad \cup A_2 \cup A_3\cup \cdots = B_2\] \[S_3=\quad \quad A_3 \cap \cdots \subset A_3 \subset \quad \quad A_3\cup \cdots = B_3\] 이렇게 하면 당연히 \(S_n\)은 increasing set이고 \(B_n\)은 decreasi.. Maximum A Posteriori (MAP) $$\hat{m}=\arg\max_i{P_{m|\overrightarrow{R}}}(i|\overrightarrow{r})=\arg\max_i P_{\overrightarrow{R}|m}(\overrightarrow{r}|i)P_m(i)=\arg\max_i P_{\overrightarrow{R}|m}(\overrightarrow{r}|i)$$ 마지막은 상수라고 가정하면 제거가능 ex) \(\frac{1}{M}\) 이전 1 2 다음