From Section 3 we know that \((S, \cdot)\) does not have any exponential distributions. But the question of whether the partial order graph \((S, \subseteq)\) (or the covering graph \((S, \upa)\)) has constant rate distributions is open (at least for me). In this section, we give some partial results.
Recall from Section 1 that the \(\sigma\)-algebra associated with \((S, \subseteq)\) is the reference \(\sigma\)-algebra \(\ms P(S)\). From a result in Section 1.3, the graph is stochastic, so that the reliability function of a probability measure on \(\ms P(S)\) uniquely determines the measure. The following result shows how to recover the reliability function from the density function.
Let \(X\) be a random variable in \(S\) with probability density function \(f\), and let \(F\) denote the reliability function of \(X\) for \((S, \subseteq)\) so that \[F(x) = \sum_{y \in x S} f(y) =\sum_{x \subseteq y} f(y), \quad x \in S\] If \(\sum_{x \in S} F(x) = \E\left[2^{\#(X)}\right] \lt \infty\) then \[f(x) = \sum_{x \subseteq y} (-1)^{\#(y) - \#(x)} F(y), \quad x \in S\]
This follows immediately from the Möbius inversion result in Section 1.4 and the Möbius function and path function given in Section 1. The fact that \(\sum_{x \in S} F(x) = \E\left[2^{\#(X)}\right]\) follows from the basic moment result.
Recall that the positive semigroup \((S, \subseteq)\) is a uniform; in the notation used before, if \(x \subseteq y\) then \(d(x, y) = \#(y) - \#(x)\), the length of a path from \(x\) to \(y\) in \((S, \upa)\).
For \(n \in \N\), let
So \(\{A_n: n \in \N\}\) is the partition of \(S\) induced by the function \(\#\) from \(S\) to \(\N\), and \(\{A_n(x): n \in \N\}\) is a partition of \(\{y \in S: x \subseteq y\}\). The following result restates proposition .
Let \(F\) denote the reliability function of \(X\) for \((S, \subseteq)\). Then \[f(x) = \sum_{n = 0}^\infty (-1)^n \sum_{y \in A_n(x)} F(y), \quad x \in S\]
For the covering relation \(\upa\), note that \(x \uparrow y\) if and only if \(x \subset y\) and \(\#(y) = \#(x) + 1\). Equivalently \(y = x \cup \{i\}\) for some \(i \notin x\). More generally, let \(\uparrow^n\) denote the \(n\)-fold composition power of \(\uparrow\) for \(n \in \N\), so that \(x \uparrow^n y\) if and only if \(y \in A_n(x)\). (Note that \(\upa^0\) is the equality relation \(=\) so that \(x \upa^0 y\) if and only if \(y = x\).) The main point of our next discussion is to show that if there exists a distribution with constant rate for \((S, \uparrow)\) then the distribution has constant rate for \((S, \uparrow^n)\) for each \(n \in \N_+\) and the distribution has constant rate for \((S, \subseteq)\).
As before, let \(X\) be a random variable in \(S\) with probability density function \(f\), and let \(F\) denote the reliability function of \(X\) for the partial order graph \((S, \subseteq)\) so that \(F(x) = \P(x \subseteq X)\) for \(x \in S\). For \(n \in \N\), let \(G_n\) denote the reliability function of \(X\) for the graph \((S, \upa^n)\), so that \(G_n(x) = \P[X \in A_n(x)]\) for \(x \in S\). Of course, \(G_0 = f\).
Note that \(F = \sum_{n = 0}^\infty G_n\).
The following result gives a recursive relationship between \(G_{n + 1}\) and \(G_n\) for \(n \in \N\).
Let \(n \in \N\). Then \[G_{n + 1}(x) = \frac{1}{n + 1} \sum_{i \notin x} G_n(x \cup \{i\}), \quad x \in S\]
Note that \[G_{n + 1}(x) = \sum_{y \in A_{n + 1}(x)} f(y) = \frac{1}{n + 1} \sum_{i \notin x} \sum_{z \in A_n(x \cup \{i\})} f(z) = \frac{1}{n + 1} \sum_{i \notin x} G_n(x \cup \{i\}), \quad x \in S\] The factor \(1 / (n + 1)\) is present because we have counted every set \(y \in A_{n + 1}(x)\) exactly \(n + 1\) times in the middle double sum: we could interchange \(i \notin x\) with any of the \(n + 1\) elements in a set \(z \in A_n(x \cup \{i\})\) without changing the subset \(y\).
Suppose that \(X\) has constant rate \(\beta \in (0, \infty)\) for \((S, \uparrow)\). Then
The proofs follow easily from .
We give some examples of distributions that are not constant rate, but are nonetheless interesting. These models are not mutually exclusive.
Suppose that \(X\) is a random variable in \(S\) with the property that \(i \in X\) with probability \(p_i\), independently over \(i \in \N\). We assume that \(\prod_{i \in \N_+} p_i = 0\) so that \(X\) is finite with probability 1.
Parts (a) and (b) follow from the definition of the model. For part (c), note that if \(x \subseteq y\) then \(y^c \subseteq x^c\) and hence \[r(y) = \prod_{i \in y^c} (1 - p_i) \ge \prod_{i \in x^c} (1 - p_i) = r(x)\] For part (d), note that \[F_1(x) = \sum_{j \in x^c} f(x \cup \{j\}) = \sum_{j \in x^c} \left(\prod_{i \in x} p_i\right) p_j \left(\prod_{i \in x^c} (1 - p_i)\right) \frac{1}{1 - p_j}, \quad x \in S\]
Note that the expression \(p_j / (1 - p_j)\) in parts (d) and (e) of example is the odds ratios for the event \(\{j \in X\}\), where \(j \in x^c\).
Suppose that we pick a random population size \(N \in \N\) with probability density function \(g\). Given \(N = n\), we put \(i \in X\) independently with probability \(p\) for \(i \in \{1, \ldots, n\}\).
Suppose that we generalize the model in example so that, given \(N = n\), we have \(i \in X\) with probability \(p_i\) independently for \(i \in \{1, \ldots, n\}\)
Suppose that \(X\) is a random variable in \(S\) with the property that for \(j \in \N_+\) the events \(\{1 \in X\}, \{2 \in X\}, \ldots, \{j - 1 \in X\}\) are conditionally independent with \[\P(i \in X \mid j \in X) = p_{ij}, \quad i \in \{1, \ldots j - 1\}\] Let \(g(i) = \P(i \in X)\) for \(i \in \N_+\), and let \(a(x) = x - \{\max(x)\}\) and \(b(x) = \{1, \ldots, \max(x) - 1\} - a(x)\) for \(x \in S\)
The defining property in example generalizes the Markov property
that we saw in Section 3 for almost exponential distributions on \((S, \cdot)\).
Suppose that \(N\) is distributed on \(\N\) with probability density function \(g\) so that \(g(n) = \P(N = n)\) for \(n \in \N\). Suppose that \(U\) is a random variable in \(\N_+\) with probability density function given by \(p_i = \P(U = i)\) for \(i \in \N_+\). Finally suppose that \(\bs{U} = (U_1, U_2, \ldots)\) is a sequence of independent copies of \(U\), independent of \(N\). Now define \(X\) in \(S\) by \[X = \{U_1\} \{U_2\} \cdots \{U_N\}\] The probability density function \(f\) of \(X\) is given as follows: if \(i_1, i_2, \ldots, i_n \in \N_+\) with \(i_1 \lt i_2 \lt \cdots \lt i_n\), then \[f(\{i_1, i_2, \ldots, i_n\}) = g(n) p_{i_1}(p_{i_2} + p_{i_2 - 1})(p_{i_3} + p_{i_3 - 1} + p_{i_3 - 2}) \cdots (p_{i_n} + p_{i_n - 1} + \cdots + p_{i_n - n + 1})\]
By construction, \(N = \#(X)\). For \(x \in S\) with \(\#(x) = n \in \N_+\), \(X = x\) if and only if \(N = n\) and \(\{U_1\} \{U_2\} \cdots \{U_n\}\) is one of the \(n!\) factorings of \(x\) into singletons. So in particular, \[f(\emptyset) = \P(N = 0) = g(0)\] For \(i \in \N_+\), \[f(\{i\}) = g(1) \P(U_1 = i) = g(1) p_i\] For \(i, \, j \in \N_+\) with \(i \lt j\), \[f(\{i, j\}) = g(2)(p_j p_i + p_i p_{j - 1}) = g(2) p_i (p_j + p_{j - 1})\] Continuing in this way gives the result.
We are particularly interested in the case where \(N = \#(X)\) has a Poisson distribution for reasons that will be clear in proposition .
In the model of example , suppose that \(N\) has the Poisson distribution with parameter \(\lambda \in (0, \infty)\). Then \(X\) does not have constant rate for \((S, \subseteq)\).
Let \(F\) denote the reliability function of \(X\) for \((S, \subseteq)\). Note first that \(F(\emptyset) = 1\) so if \(X\) has constant rate then the rate constant must be \(g(0)\). Note next that \(1 \in X\) if and only if \(N = n\) for some \(n \in \N_+\) and \(U_k = 1\) for some \(k \in \{1, 2, \ldots, n\}\). So \[\P(1 \in X) = F(\{1\}) = \sum_{n = 0}^\infty g(n) [1 - (1 - p_1)^n] = (1 - p_0) - \sum_{n = 1}^\infty g(n) (1 - p_1)^n\] Since \(N\) has the Poisson distribution with parameter \(\lambda \in (0, \infty)\) this reduces to \begin{align*} F(1) &= (1 - e^{-\lambda}) - e^{-\lambda} \sum_{n = 1}^\infty \frac{\lambda^n}{n!} (1 - p_1)^n \\ &= (1 - e^{-\lambda}) - e^{-\lambda} [e^{\lambda(1 - p_1)} - 1] = 1 - e^{\lambda p_1} \end{align*} So if \(X\) has constant rate (which must be \(e^{-\lambda}\)) then \[\lambda e^{-\lambda} p_1 = e^{-\lambda}(1 - e^{-\lambda p_1})\] or equivalently \(\lambda p_1 + e^{-\lambda p_1} = 1\), which is impossible since \(\lambda \in (0, \infty)\) and \(p_1 \in (0, 1)\).
Lets return now to the general case where \(X\) has probability density function \(f\) and has reliability function \(F\) for \((S, \subseteq)\).
If \(X\) has constant rate \(\alpha \in (0, 1)\) for \((S, \subseteq)\) then \[ f(x) = \frac{1}{\alpha} \sum_{n = 0}^\infty (-1)^n \sum_{y \in A_n(x)} f(y), \quad x \in S \]
In proposition showed above that if \(X\) has constant rate for \((S, \upa)\) then \(X\) also has constant rate for \((S, \subseteq)\). We do not know if the converse is true. However, we also know that if \(X\) has constant rate for \((S, \upa)\) then \(U = \#(X)\) has a Poisson distribution. Our next result shows that this is also true if \(X\) has constant rate for \((S, \subseteq)\).
Suppose that \(X\) has constant rate \(\alpha \in (0, 1)\). Then \[ \alpha \P(U = k) = \E\left[(-1)^{U+k} \binom{U}{k}\right], \quad k \in \N \] Hence \(U\) has the Poisson distribution with parameter \(-\ln \alpha\).
Let \(f\) denote the probability density function of \(X\), as above. From , \[\P(U = k) = \sum_{x \in A_k} f(x) = \frac{1}{\alpha} \sum_{n = 0}^\infty (-1)^n \sum_{x \in A_k} \sum_{y \in A_n(x)} f(y)\] The last two sums are over all \(x, y \in S\) with \(\#(y) = n + k\), and \(x \subseteq y\). Interchanging the order of summation gives \begin{align*} \P(U = k) &= \frac{1}{\alpha} \sum_{n = 0}^\infty (-1)^n \sum_{y \in A_{n+k}} \sum \{f(y): x \in A_k, \, x \subseteq y\} \\ &= \frac{1}{\alpha} \sum_{n = 0}^\infty (-1)^n \sum_{y \in A_{n + k}} \binom{n + k}{k} f(y)\\ &= \frac{1}{\alpha} \sum_{n = 0}^\infty (-1)^n \binom{n + k}{k} \P(U = n + k) \end{align*} Equivalently (with the substitution \(j = n + k\)), \[\alpha \P(U = k) = \sum_{j = k}^\infty (-1)^{j - k} \binom{j}{k} \P(U = j)\] With the usual convention on binomial coefficients, that is \(\binom{a}{b} = 0\) if \(b \lt 0\) or \(b \gt a\), we have \[\alpha \P(U = k) = \sum_{j = 0}^\infty (-1)^{j + k} \binom{j}{k} \P(U = j) = \E\left[(-1)^{k + U} \binom{U}{k} \right]\] The fact that \(U\) has a Poisson distribution with parameter \(-\ln \alpha\) now follows from the characterizations in Section 6.
Our last discussion makes a bit of progress on the existence question. For \(k \in \N\), let \(g_k\) denote the conditional probability density function of \(X\) given that \(U = k\), so that \[g_k(x) = \P(X = x \mid U = k), \quad x \in A_k\]
Suppose that \(U\) has the Poisson distribution with parameter \(\lambda\). Then the condition for \(X\) to have constant rate \(e^{-\lambda}\) is \[ e^{-\lambda} \frac{g_k(x)}{k!} = \sum_{n = 0}^\infty \frac{(-\lambda)^n}{(n + k)!} \sum_{y \in A_n(x)} g_{n + k}(y), \quad k \in \N, \, x \in A_k \]
The next result gives a condition that is sufficient (but not necessary) for the condition in .
The condition in holds if \[ \sum_{y \in A_n(x)} g_{n + k}(y) = \binom{n + k}{k} g_k(x), \quad k, \, n \in \N, \; x \in A_k \]
Suppose that the equation above holds holds. Then for \(x \in A_k\), \begin{align*} \sum_{n=0}^\infty \frac{(-\lambda)^n}{(n + k)!} \sum_{y \in A_n(x)} g_{n + k}(y) &= \sum_{n = 0}^\infty \frac{(-\lambda)^n}{(n + k)!} \frac{(n + k)!}{n! k!} g_k(x) \\ &= \frac{g_k(x)}{k!} \sum_{n = 0}^\infty \frac{(-\lambda)^n}{n!} = e^{-\lambda} \frac{g_k(x)}{k!} \end{align*} and thus the equation in holds.
So we have \[ g_k(x) = \frac{1}{\binom{n + k}{k}} \sum_{y \in A_n(x)} g_{n + k}(y), \quad k, \, n \in \N, \; x \in A_k \] which has the following interpretation:
Suppose that \(n, \, k \in \N\) and that \(Y\) is a random variable taking values in \(A_{n + k}\) with probability density function \(g_{n+k}\). Let \(X\) be a randomly chosen subset of \(Y\) of size \(k\). Then the density function \(g_k\) of \(X\) satisfies the condition in .
The precise meaning of the hypothesis is that given \(Y = y\) where \(y \in A_{n + k}\), the conditional distribution of \(X\) is uniform on the collection of subsets of \(y\) of size \(k\). Thus, for \(x \in A_k\), \[g_k(x) = \P(X = x) = \sum_{y \in A_{n + k}(x)} \P(Y = y) \P(X = x \mid Y = y) = \frac{1}{\binom{n + k}{k}} \sum_{y \in A_{n + k}(x)} g_n(y)\]
In particular, the condition in is consistent. The density function of \(Y\) itself corresponds to \(k = n\)