This document discusses universal lossless source coding for independent piecewise identically distributed (i.p.i.d.) sources. It proves that Bayes codes, which are universal source codes, minimize the mean redundancy asymptotically for i.p.i.d. sources where the number of stationary segments increases to infinity as the data size increases. Specifically, it shows that the mean redundancy of the Bayes codes does not depend on the coding probability and equals the Bayes redundancy asymptotically, demonstrating that the Bayes codes optimize compression in this scenario.
The document proposes a new multiuser detection algorithm for CDMA based on the belief propagation algorithm. It develops a factor graph that represents the posterior distribution of information bits in a way that reduces the number of cycles compared to previous factor graphs. Numerical simulations show the proposed belief propagation algorithm outperforms the parallel interference canceller detector, especially for signature sequences that produce factor graphs with fewer cycles and weaker cross-correlations between users. The approach shows promise for application to other communication systems.
The document proposes a new method for document classification with small training data. It discusses previous methods that estimate parameters for prior distributions either using fixed values or estimating data. The new proposed method estimates parameters for prior distributions as a weighted combination of estimating data and training data. Experiments show the new method achieves higher accuracy than previous methods, especially with small training data sizes.
The document proposes reducing the computational complexity of message passing algorithms like belief propagation (BP) and concave-convex procedure (CCCP) for multiuser detection in CDMA systems. It does this by changing the factor graph structure used to represent the detection problem from a fully connected graph (Factor Graph I) to a sparsely connected graph (Factor Graph II). Simulation results show the proposed CCCP detector for the new factor graph achieves near optimal performance with lower complexity than existing approaches.
The document discusses methods for statistical spelling correction based on decision theory. It defines the spelling correction problem and previous approaches. It then presents two new methods: 1) Evaluating the error rate per sentence using Bayes optimal and approximate algorithms based on dynamic programming. 2) Evaluating the error rate per word also using Bayes optimal and approximate algorithms like the BCJR algorithm. The goal is to minimize error rates with computational efficiency for real-world applications.
This document analyzes the asymptotic properties of expected cumulative logarithmic loss in Bayesian estimation when models are nested and when there is misspecification. The main theorem states that if the true distribution does not belong to the model class, the asymptotic loss per symbol goes to the Kullback-Leibler divergence between the true and model distributions, rather than 0. If the true distribution does belong to the model class, the results reduce to previous studies. The proof is separated into two parts and relies on a lemma showing posterior concentration at the true model.
This document proposes a unified framework for approximating the optimal key estimation of stream ciphers using probabilistic inference. It formulates the key estimation problem as determining the secret key that maximizes the joint probability based on the observed keystream. An approximation algorithm called the sum-product algorithm is used to efficiently compute approximate marginal probabilities on a factor graph representing the cipher structure. Preprocessing techniques can reduce the complexity of the sum-product algorithm when applied to ciphers using linear feedback shift registers.
This document proposes a linear programming (LP) based approach for solving maximum a posteriori (MAP) estimation problems on factor graphs that contain multiple-degree non-indicator functions. It presents an existing LP method for problems with single-degree functions, then introduces a transformation to handle multiple-degree functions by introducing auxiliary variables. This allows applying the existing LP method. As an example, it applies this to maximum likelihood decoding for the Gaussian multiple access channel. Simulation results demonstrate the LP approach decodes correctly with polynomial complexity.
The document proposes a method to calculate the theoretical throughput limit of type-I hybrid selective-repeat ARQ with a finite receiver buffer using Markov decision processes. The authors model the problem as an MDP and develop an algorithm to compute the maximum expected utility and throughput limit by applying dynamic programming. Simulation results show the throughput of previous methods approaches the proposed theoretical limit with increasing buffer size.
The document proposes a new multiuser detection algorithm for CDMA based on the belief propagation algorithm. It develops a factor graph that represents the posterior distribution of information bits in a way that reduces the number of cycles compared to previous factor graphs. Numerical simulations show the proposed belief propagation algorithm outperforms the parallel interference canceller detector, especially for signature sequences that produce factor graphs with fewer cycles and weaker cross-correlations between users. The approach shows promise for application to other communication systems.
The document proposes a new method for document classification with small training data. It discusses previous methods that estimate parameters for prior distributions either using fixed values or estimating data. The new proposed method estimates parameters for prior distributions as a weighted combination of estimating data and training data. Experiments show the new method achieves higher accuracy than previous methods, especially with small training data sizes.
The document proposes reducing the computational complexity of message passing algorithms like belief propagation (BP) and concave-convex procedure (CCCP) for multiuser detection in CDMA systems. It does this by changing the factor graph structure used to represent the detection problem from a fully connected graph (Factor Graph I) to a sparsely connected graph (Factor Graph II). Simulation results show the proposed CCCP detector for the new factor graph achieves near optimal performance with lower complexity than existing approaches.
The document discusses methods for statistical spelling correction based on decision theory. It defines the spelling correction problem and previous approaches. It then presents two new methods: 1) Evaluating the error rate per sentence using Bayes optimal and approximate algorithms based on dynamic programming. 2) Evaluating the error rate per word also using Bayes optimal and approximate algorithms like the BCJR algorithm. The goal is to minimize error rates with computational efficiency for real-world applications.
This document analyzes the asymptotic properties of expected cumulative logarithmic loss in Bayesian estimation when models are nested and when there is misspecification. The main theorem states that if the true distribution does not belong to the model class, the asymptotic loss per symbol goes to the Kullback-Leibler divergence between the true and model distributions, rather than 0. If the true distribution does belong to the model class, the results reduce to previous studies. The proof is separated into two parts and relies on a lemma showing posterior concentration at the true model.
This document proposes a unified framework for approximating the optimal key estimation of stream ciphers using probabilistic inference. It formulates the key estimation problem as determining the secret key that maximizes the joint probability based on the observed keystream. An approximation algorithm called the sum-product algorithm is used to efficiently compute approximate marginal probabilities on a factor graph representing the cipher structure. Preprocessing techniques can reduce the complexity of the sum-product algorithm when applied to ciphers using linear feedback shift registers.
This document proposes a linear programming (LP) based approach for solving maximum a posteriori (MAP) estimation problems on factor graphs that contain multiple-degree non-indicator functions. It presents an existing LP method for problems with single-degree functions, then introduces a transformation to handle multiple-degree functions by introducing auxiliary variables. This allows applying the existing LP method. As an example, it applies this to maximum likelihood decoding for the Gaussian multiple access channel. Simulation results demonstrate the LP approach decodes correctly with polynomial complexity.
The document proposes a method to calculate the theoretical throughput limit of type-I hybrid selective-repeat ARQ with a finite receiver buffer using Markov decision processes. The authors model the problem as an MDP and develop an algorithm to compute the maximum expected utility and throughput limit by applying dynamic programming. Simulation results show the throughput of previous methods approaches the proposed theoretical limit with increasing buffer size.