Table of Contents

1. Mean

1.1. Intuition

  • The mean is defined relative to the notion of the total of a specific problem.
  • A mean is a single value such that, when replaced all the data with the mean, it would calculate to the same total.
    • This is the idea of the
  • The 1.3 and its variations are the solution to the optimization problem of the (statistical) variation.

1.2. Minimum and Maximum

1.3. Arithmetic Mean

  • Often simply, Mean
  • \[ m_1 = \frac{x_1 + x_2 + \cdots + x_n}{n} \]

1.4. Geometric Mean

1.4.1. Properties

  • It is equal to the arithmetic-harmonic mean defined by the limit of the sequnces \(a_i\) and \(g_i\):
    • \[ a_0 = x, h_0 = y\\ a_{n+1} = \mathrm{AM}(a_n, h_n), h_{n+1} = \mathrm{HM}(a_n,h_n) \]

1.5. Spherical Mean

1.5.1. Definition

  • For a continuous function \(u\colon U \to \mathbb{F}\), with \(U\) being the open subset of the Euclidean space \(\mathbb{R}^n\) and \(\mathbb{F}\) being either the real or complex number,
  • The spherical mean over the sphere of radius \(r\) centered at \(x\) is defined by:
    • \[ \frac{1}{\omega_{n-1}(r)}\int_{\partial B(x,r)} u(y)\mathrm{d}^{\wedge n-1}y \]
      • where \(B(x,r)\subset U\), \(\mathrm{d}^{n-1}y\) is the spherical measure, and \(\omega_{n-1}(r)\) is the size of the hypersurface of \((n-1)\)-sphere.
  • The spherical mean is often denoted as:
    • \[ \int\limits_{\partial B(x,r)}\!\!\!\!\!\!\!\!\!-\ u(y)\,\mathrm{d}S(y) \]
    • This notation is also used for the sometimes.

1.6. Arithmetic-Geometric Mean

  • AGM

1.6.1. Definition

  • It is the limit of the sequnces \(a_i\) and \(g_i\):
    • \[ a_0 = x, g_0 = y\\ a_{n+1} = \mathrm{AM}(a_n, g_n), g_{n+1} = \mathrm{GM}(a_n,g_n) \]

1.7. Quadratic Mean

  • Root Mean Square(RMS)

1.8. Relations

  • \[ \rm min(\mathbf{x}) \le HM(\mathbf{x}) \le GM(\mathbf{x}) \le LM(\mathbf{x}) \le AM(\mathbf{x}) \le QM(\mathbf{x}) \le CM(\mathbf{x}) \le max(\mathbf{x}) \]
  • Equality holds if and only if all the variables are equal.
  • QM_AM_GM_HM_inequality_visual_proof.svg
  • \[ \mathrm{AM}(a,b)\cdot \mathrm{HM}(a,b) = \mathrm{GM}(a,b) \]
  • \[ \mathrm{GM}(\mathrm{AM}(a,b), \mathrm{HM}(a,b)) = \mathrm{GM}(a,b) \]
  • \[ \mathrm{AM}(\mathrm{HM}(a,b), \mathrm{CM}(a,b)) = \mathrm{AM}(a,b) \]

1.9. Elementary Symmetric Mean

  • For a sequence of nonnegative real numbers \((a_i)_{i=1}^n\), the elementary symmetric means \(S_k\) are given by:
    • \[ S_k = \frac{e_k}{\binom{n}{k}}. \]
    • The numerator is the ((6659b4b7-3e70-4daf-869a-ca654cccfb1b)), and the denominator is the number of such polynomials

1.10. Newton's Inequalities

1.10.1. Statement

  • \[ S_{k-1}S_{k+1} \le S_k^2 \]
  • with equality if and only if all the numbers \(a_i\) are equal.

1.11. Maclaurin's Inequality

1.11.1. Statement

  • \[ S_1 \ge \sqrt{S_2} \ge \sqrt[3]{S_3} \ge \cdots \ge \sqrt[n]{S_n} \]
  • with equality if and only if all the \(a_i\) are equal.
  • The case \(n=2\) is already known as the inequality of arithmetic and geometric mean.

1.12. Bernoulli's Inequality

1.12.1. Statement

  • \((1+x)^r \ge 1+rx\)
    • for every integer \(r\ge 1\) and real number \(x\ge -1\), with strict inequality if \(x\neq 0\land r\ge 2\). logseq.order-list-type:: number
    • for every integer \(r\ge 0\) and real number \(x\ge -2\). logseq.order-list-type:: number
    • for every even integer \(r\ge 0\) and real number \(x\). logseq.order-list-type:: number
    • for every real number \(r\ge 1\) and \(x\ge -1\), with strict inequality if \(x\neq 0\land r\neq 1\). logseq.order-list-type:: number
    • for every real number \(0\le r\le 1\) and \(x\ge -1\). logseq.order-list-type:: number

1.13. Harmonic Mean

  • See

1.14. Pythagorean Mean

1.14.1. Para-Axioms

  • First-Order Homogeneity: \(\mathrm{M}(bx_1, \dots, bx_n) = b\mathrm{M}(x_1,\dots, x_n)\)
  • Total Symmetry: \(\mathrm{M}(\dots, x_i,\dots,x_j,\dots) = \mathrm{M}(\dots, x_j,\dots,x_i,\dots)\)
  • Monotonicity (in all variables): \(a\le b \implies \mathrm{M}(a,x_2,\dots,x_n) \le \mathrm{M}(b,x_2,\dots, x_n)\)
  • Idempotence: \(\forall x, \mathrm{M}(x,x,\dots,x) = x\)

1.15. Contraharmonic Mean

  • Complementary to the 1.13

1.15.1. Definition

  • For positive real numbers \(x_1, \dots, x_n\),
  • \[ \mathrm{C}(x_1, \dots,x_n) = \frac{\frac1n(x_1^2 + \cdots + x_n^2)}{\frac1n(x_1+\cdots + x_n)} \]

1.15.2. Properties

  • \[ \mathrm{AM}(\mathrm{HM}(a,b), \mathrm{CM}(a,b)) = \mathrm{AM}(a,b) \]

1.16. Fréchet Mean

1.16.1. Fréchet Variance

1.16.2. Definition

  • Karcher Means
    • \[ m = \operatorname*{arg\,min}_{p\in M}\sum_{i=1}^Nd(p,x_i)^2 \]
  • If there is a unique \(m\) that strictly minimizes \(\Psi\), then it is Fréchet mean.

1.17. Generalized Mean

  • Power Mean

1.17.1. Definition

  • For a nonzero real number \(p\), and positive real numbers \(x_1,\dots, x_n\), the generalized mean with exponent \(p\) is:
    • \[ m_p(x_{i\in I}) = \left(\frac{\sum_{i\in I}x_i^p}{|I|}\right)^{\frac{1}{p}} \]

1.17.2. Special Cases

  • \(m_1\) is ((669af589-9d1a-498b-8b4d-8e08c5c5fdef)) .
  • \(m_0\) is ((66dca57f-a7a3-417b-a6ae-3b54cd7da93f)) via limit.
  • \(m_{-1}\) is ((66dca57f-34a6-4478-8f8d-03f47d04331e)) .
  • \(m_{-\infty}\) and \(m_\infty\) is ((66dcb9d4-93c1-4d54-ac45-52ccca7d48ad)) .
  • \(m_2\) is ((66dcba0b-b633-48b8-845b-090e661e37f8)) , or root mean square.

1.17.3. General Mean Inequality

  • If \(p

1.18. Quasi-Arithmetic Mean

  • Generalized \(f\)-Mean, Kolmogorov-Najumo-de Finetti Mean, Kolmogorov Mean

1.18.1. Definition

  • For a injective continuous function \(f\colon I\to \mathbb{R}\) with an interval \(I\),
    • \[ M_f(\mathbf{x}) = f^{-1}\left(\frac{1}{n}\sum_{i=1}^nf(x_i)\right) \]

1.18.2. LogSumExp

  • RealSoftMax(LSE), Multivariable Softplus
  • Smooth approximation to the maximum function
  • \[ \mathrm{LSE}(x_1,\dots,x_n) := \log(\exp(x_1),\dots,\exp(x_n)) \]

1.18.3. Jensen's Inequality

  • See ((66cc6b5d-334f-4190-8d3a-7c3a8b422295))

1.18.4. Special Cases

  • \(f\ \text{identity}\): ((669af589-9d1a-498b-8b4d-8e08c5c5fdef))

1.19. Heronian Mean

1.19.1. Definition

  • \[ H = \frac{1}{3}(a + \sqrt{ab} + b) \]

1.19.2. Properties

  • The volume of a frustum is the product of the height and the Heronian mean of areas of the opposing parallel faces.

1.20. Chisini Mean

  • Substitution Mean

1.20.1. Definition

  • A function of \(n\) variables give rises to a Chisini mean \(M\), if for every vector \((x_1,\dots, x_n)\) there exists a unique \(M\) such that:
    • \[ f(M, M,\cdots, M) = f(x_1, x_2, \dots, x_n). \]

1.20.2. Special Cases

  • \(f\ \text{summation}\): ((669af589-9d1a-498b-8b4d-8e08c5c5fdef))
  • \(f\ \text{product}\): ((66dca57f-a7a3-417b-a6ae-3b54cd7da93f))
  • \(f\ \text{reciprocal summation}\): ((66dca57f-34a6-4478-8f8d-03f47d04331e))
  • \(f\ \text{summation after squaring}\): ((66dcba0b-b633-48b8-845b-090e661e37f8))
  • \(f\ \text{summation after exponentiation}\): ((66dca57f-d49e-4c0a-a5f4-b50e38508148))
  • \(f\ \text{summation after filtering with a function}\): ((66dca57f-5dab-4280-bc28-26e8dc8f5fcc))
  • \(f\ \text{volume of a frustum in terms of the areas of the bases}\): ((66dca57f-0868-4ab4-9e7d-dc2ad49b99f3))

1.21. Lehmer Mean

1.21.1. Definition

  • \[ L_p(\mathbf{x}) = \frac{\sum_{k=1}^nx_k^p}{\sum_{k=1}^nx_k^{p-1}} \]

1.21.2. Weighted Lehmer Mean

  • \[ L_{p,w}(\mathbf{x}) = \frac{\sum_{k=1}^nw_kx_k^p}{\sum_{k=1}^nw_kx_k^{p-1}} \]

1.21.3. Special Cases

  • \(L_0\) is the ((66dca57f-34a6-4478-8f8d-03f47d04331e))
  • \(L_{1/2}((x_1, x_2))\) is the ((66dca57f-a7a3-417b-a6ae-3b54cd7da93f))
  • \(L_1\) is the ((669af589-9d1a-498b-8b4d-8e08c5c5fdef))
  • \(L_2\) is the ((66dcbac1-0b9b-4411-9798-1fc461e54ddb))
  • \(\lim_{p\to -\infty}L_p\) and \(\lim_{p\to \infty}L_p\) is the ((66dcb9d4-93c1-4d54-ac45-52ccca7d48ad))

1.22. Heinz Mean

1.22.1. Definition

  • For two non-negative number \(a, b\),
  • \[ \mathrm{H}_x(a,b) = \frac{a^xb^{1-x} + a^{1-x}b^x}{2} \]
    • with \(0\le x\le 1/2\).

1.22.2. Properties

  • It interpolates between the arithmetic (\(x=0\)) and geometric \((x=1/2)\) mean.

1.23. a-Mean

1.23.1. Definition

  • For any real vector \(a = (a_1,\dots, a_n)\), the \(\mathbf{a}\)-mean \([a]\) of positive real numbers are:
    • \[ [a] := \frac{1}{n!}\sum_{\sigma\in S_n}x_{\sigma(1)}^{a_1}\cdots x_{\sigma(n)}^{a_n} \]

1.23.2. Special Cases

  • \(a = (1,0,\dots, 0)\) is the ((669af589-9d1a-498b-8b4d-8e08c5c5fdef))
  • \(a = (1/n,\dots,1/n)\) is the ((66dca57f-a7a3-417b-a6ae-3b54cd7da93f))
  • \(a = (x,1-x)\) is the ((66dca57f-4d2d-4ee6-97c0-c41b8ad948a3))
  • \(a = (-1, 0, \dots, 0)\) is the ((66dca57f-34a6-4478-8f8d-03f47d04331e))

1.23.3. Muirhead's Inequality

1.23.3.1. Statement
  • \[ [a] \le [b] \iff a=Pb \]
    • where \(P\) is a ((66dc1af6-8421-4123-b5ea-d80978cb7140)) .
    • The equality holds if and only if \(a=b\) or all \(x_i\) are equal.

1.24. Logarithmic Mean

1.24.1. Definition

  • \[ \lim_{(\xi,\eta)\to (x,y)} \frac{\eta - \xi}{\ln\eta - \ln\xi} \]

1.24.2. Properties

  • \(\mathrm{GM} \le \mathrm{LM} \le \mathrm{AM}\)

1.25. Identric Mean

  • For two positive real numbers \(x,y\):
  • \[ I(x,y) := \frac{1}{e}\cdot \lim_{(\xi,\eta)\to (x,y)}\left(\frac{\xi^\xi}{\eta^\eta}\right)^{\frac{1}{\xi-\eta}} = \lim_{(\xi,\eta)\to (x,y)}\exp\left(\frac{\xi\ln\xi - \eta\ln\eta}{\xi-\eta}-1\right) \]
  • Consider the function \(x\mapsto x\ln x\), is is finding the slope of the secant and applying the inverse of the derivative.

1.26. Stolarsky Mean

1.26.1. Definition

  • For \(0

1.26.2. Properties

  • \[ S_p(a,b)=f'^{-1}\left(\frac{f(b)-f(a)}{b-a}\right) \]
    • where \(f(x)=x^p\). Here, \(S_p(a,b)\) is guaranteed to be in \((a,b)\) by Mean value theorem.
    • Calculate the average rate of change of \(x^p\) on the interval \((a,b)\) and invert it into the value that has the same instantaneous rate of change.
  • \[ S_2(a,b)=\frac{a+b}{2}. \]
  • \[ S_{-1}(a,b) = \sqrt{ab}\, . \]

1.26.3. Special Cases

  • \(S_{-\infty}\) and \(S_\infty\) is the ((66dcb9d4-93c1-4d54-ac45-52ccca7d48ad))
  • \(S_{-1}\) is the ((66dca57f-a7a3-417b-a6ae-3b54cd7da93f))
  • \(S_0\) is the ((66dca57f-6961-4efb-9d5b-808a38ed866f))
  • \(S_{1/2}\) is the power mean with exponent \(1/2\)
  • \(S_1\) is the ((66dcb019-55ef-4069-8bd7-12fac861c708))
  • \(S_2\) is the ((669af589-9d1a-498b-8b4d-8e08c5c5fdef))
  • \(S_3 = \mathrm{QM}(x,y,\mathrm{GM}(x,y))\)

1.27. Circular Mean

1.27.1. Definition

  • \[ \bar{\alpha} = \operatorname{atan2}\left(\frac{1}{n}\sum_{j=1}^n\sin\alpha_j, \frac{1}{n}\sum_{j=1}^n\cos\alpha_j\right) \]
  • Infinite

1.28. Cesáro Summation

  • Cesáro Mean, Cesáro Limit

1.28.1. Definition

  • For a sequence \((a_n)_{n=1}^\infty\), and the partial sum \(s_k\) of the first \(k\) terms,
  • The sequence is called Cesáro summable, if the arithmetic mean of its first \(n\) partial sums tends to a finite number:
    • \[ \lim_{n\to \infty} \frac{1}{n}\sum_{k=1}^ns_k = A < \infty \]

1.29. Hölder Summation

  • For a sequence \((a_n)_{n=1}^\infty\), \(H_n^0 := s_n\), and \(H_n^{k+1}\) is defined to be the partial arithmetic mean of the first \(n\) terms of the \(H_n^k\).
  • If the limit \[ \lim_{n\to \infty}H_n^k \] exists for some \(k\), it is called the Hölder sum, or the \((H,k)\) sum of the series.
  • The series is called Hölder summable if the following limit exists:
    • \[ \lim_{n\to \infty, k\to \infty} H_n^k \]

2. Variance

2.1. Covariance Matrix

3. Penalized Least Squares Criterion

Given datapoints \( \{(x_i, y_i)\}_i \), peanlized least squares (PLS) looks for the the function \( \hat{f} \) within a Hilbert space that fits the data the most: \[ \hat{f} := \min_{f\in \mathcal{H}} \left[ \frac{1}{n}\sum_i(y_i - f(x_i))^{2} + P(\| f\|^2) \right]. \]

3.1. Kimeldorf-Wahba Representer Theorem

The solution to the PLS can be given in terms of \( K \)-function \[ \hat{f}(x) = \sum_{i=1}^n \beta_i K(x,x_i) \] where \( K \)-function is the reproducing kernels in the reproducing kernel Hilbert space (RKHS), that satisfies \[ \langle f, K(\cdot, x_i)\rangle = f(x_i). \]

4. Reference

Created: 2025-05-06 Tue 23:35