�ݺ�ߣ

1Vijaya Laxmi, Dept. of EEE, BIT, Mesra

Syllabus
Module 1: Introduction: Probability models in Electrical engineering. Basic concepts of Probability
theory. Random experiments. Axioms of probability. Conditional probability. Independence of
events. Sequential experiments. (4)
Module 2: Random Variables: Definition. Classification. Cumulative distribution function. Probability
density function. Functions of Random Variables. Expected values. Moments. Variance and
Standard deviation. Markov and Chebyshev inequalities. Testing a fit of a distribution to data.
Transform methods: Characteristic function; Probability generating function; Laplace transform of
the pdf. Transformation of random variable. (5)
Module 3: Multiple Random Variables: Vector random variables. Pairs of random variables.
Independence of random variables. Conditional probability and conditional expectation. Multiple
random variables. Functions of several random variables. Expected value of function of random
variables. Jointly Gaussian random variables. (4)
Module 4: Sums of Random Variables and Long-term averages: Sums of random variables: Mean;
Variance; pdf of sum of random variables. Sample mean and law of large numbers. Central LimitVariance; pdf of sum of random variables. Sample mean and law of large numbers. Central Limit
theorem. Minimum mean square error filtering: Estimating a random variable with a constant;
stored data wiener filter; Real time wiener filter. (4)
Module 5: Random Processes: Definition. Specification: Joint distribution of time samples; Mean;
Autocorrelation and Autocovariance functions. Discrete random processes: iid random processes;
sum processes: Binomial counting and Random Walk processes. Continuous-time random
processes: Poisson processes; Processes derived from Poisson processes; Wiener process and
Brownian Motion. Stationarity. Time Averaging and Ergodicity.
(10)
Module 6: Analysis and Processing of Random signals: Power spectral Density: Continuous and discrete;
Power spectral density as a time average. Response of Linear Systems to random signals. Amplitude
modulation by random signals. Optimum Linear systems. Kalman Filter. Estimating the Power
spectral density. White noise. (5)
Module 7: Markov Chains: Markov processes. Discrete-time Markov Chains. Continuous-time Markov
Chains. Time reversed Markov Chains. (4)

Books:
Text Book
1. Probability and random Processes for Electrical Engineering- A. Leon-Garcia
Reference Books
2. Probabaility, Random Variables and Stochastic Processes- A. Papoulis & S. U. Pillai.
3. Random Signals- K. Sam Shanmugan & A.M Breipohl.3. Random Signals- K. Sam Shanmugan & A.M Breipohl.

Course Objectives
This course enables the students :
• To describe and classify different types of random variables,
random processes, probability density function and
cumulative distribution function
• To estimate statistical properties of random variables and
random processes such as expected value, variance, standardrandom processes such as expected value, variance, standard
deviation and correlation functions.
• To evaluate autocorrelation functions for given power spectral
density, correlate the mean square error of any system with
the correlation functions and analyse the response of linear
system to random inputs.
• To design real time wiener filter, stored data Wiener filter and
Kalman filter for any system.

Course Outcomes
After the completion of this course, students will be able to:
• Enumerate properties of probability density function,
cumulative distribution function, correlation functions and
power spectral density of a random process
• Describe different types of random variable and random
processesprocesses
• Calculate expected value, variance, standard deviation and
correlation functions of a random variable and random
process.
• Analyse the response of linear system to random inputs.
• Design a Wiener and Kalman filter for a system and compare
with classical filters

MODULE 1

Probabilistic models in Electrical and Computer
Engineering
Mathematical models are used as tools in analysis and design.
This is based on certain factors:
• Based on making choices from various alternatives.
• Choices are based on criteria such that cost, reliability and
performanceperformance
• Decisions are made based on estimates that are obtained
using different models.

Mathematical model
• This is used when observational phenomenon has
measurable properties.
• It consists of a set of assumptions about how a system or
physical process works.
• These assumptions are stated in the form of mathematical• These assumptions are stated in the form of mathematical
relations involving the important parameters and variables of
the system.
• Experiments carried out determines ‘givens’ in the
mathematical relations and solution allows to predict the
measurements.

Deterministic model
• An experiment carried out determine the exact outcome of
the experiment and the solution of the set of mathematical
equations specifies the exact outcome of the experiment.
• Example: Circuit theory

Probabilistic models
• Random experiment: In this case the outcome varies in an
unpredictable manner, when the experiment is repeated
under the same conditions.
• Deterministic models are not appropriate for random
experiment.experiment.
• Example: selection of a ball from a box containing three balls
labeled 0,1 and 2.
• In this case, the outcome of the experiment is a number from
a set .
Where S is the sample space.
 2,1,0S

Outcome of Random experiment

Statistical Regularity
• This states that the averages obtained in long
sequences of repetitions (trials) of random
experiment consistently yield approximately the
same value.
Let N (n), N (n), N (n) are the number of times outcomes are• Let N0(n), N1(n), N2(n) are the number of times outcomes are
balls 0, 1 and 2

Relative frequency
• The relative frequency is given by,
• Statistical regularity means, fk(n) varies less and less
   
trialsofnumbertheisnwhere
n
nN
nf k
k
,

• Statistical regularity means, fk(n) varies less and less
about a constant value as n is made large
  koutcomeofyprobabilittheispnf kk
n


lim

• The relative frequencies fk(n) converges to 1/3 as
• If one more ball is placed on the box numbered as 0
Probability of outcome 0 is 2/4
Probability of outcome 1 & 2 is ¼
• Hence, the conditions under which a random experiment is
n
• Hence, the conditions under which a random experiment is
performed, determine the probability of outcome of an
experiment.

Properties of Relative Frequencies
• Suppose for a random experiment, there are K
possible outcomes, the sample space is given by
 KS ....,,2,1,0
....,,3,2,11)(0.2
....,,3,2,1)(0.1

 k
Kkfornf
KkfornnN
 
  1.4
.3
....,,3,2,11)(0.2
1
1







K
k
k
K
k
k
k
nf
nnN
Kkfornf
Property 3 shows that the sum of number of occurances of all
Possible outcomes must be n.
Property 4 shows that the sum of all relative frequencies equals n,
and is obtained by dividing both sides of property 3 by n. 15Vijaya Laxmi, Dept. of EEE, BIT, Mesra

• Suppose the event is ‘even numbered ball selected’
• Then
• Relative frequency,
     nNnNnNE 20 
         nfnf
n
nNnN
nfE 20
20



• So, we can say that if ‘A and B occurs’ then C be the event
• And
n
     
     nfnfnf
nNnNnN
BAC
BAC



Axioms on Probability
It supposes
• A random experiment has been defined and a set S of all
possible outcomes has been identified
• A class of subsets of S called events has been specified
• Each event a has been assigned a number P[A]• Each event a has been assigned a number P[A]
 
 
     BorAPBPAP
eouslysimuloccurcannotBandAIf
SP
AP



tan.3
1.2
10.1

Building a Probabilistic Model
This involves
• Defining the random experiment
• Specifying the set S of all possible outcomes and events of interest
• Specifying a probability assignment from which probabilities of all events
of interest can be computed
• Ex. Telephone conversation to determine whether a speaker is currently
speaking or silent. An speaker in general remains active (speaking) for 1/3
of time and remain silent for 2/3 (listening to others)
This problem is similar to selecting a ball from a box containing two
white balls (silent) and one black ball (active).

Problem
• Find the probability that 24 speakers out of 48
independent speakers are active simultaneously.
• Solution:• Solution:
The problem is same as finding the probability of
selecting 24 black balls in 48 independent trials in
the previous experiment.

Example
• A communication system is required to transmit 48
simultaneous conversations from city A to city B.
• The speech is first converted into voltage signals which are
digitized and bundled into packets of information that
corresponds to 10ms signals.corresponds to 10ms signals.

Stochastic processes
• Random experiments which has outcome as a function of time or
space
Examples:
• Speech recognition system: speech signal in converted to voltage
waveforms
• Image processing system: the intensity of image varies over a• Image processing system: the intensity of image varies over a
rectangular region
• Queuing system: the number of customers varies with time
• Arrival of telephone calls at an exchange
• Arrival of a customer to a service station
• Breakdown of a component of some system
• Number of printing errors in a book

BASIC CONCEPTS OF PROBABILITY THEORY
• RANDOM EXPERIMENT:
A random experiment is an experiment in which the
outcome varies in an unpredictable fashion, when the
experiment is repeated under the same conditions.
• SAMPLE SPACE:
The sample space S of a random experiment is defined as
the set of all possible outcomes.

Types of sample space
• Discrete sample space:
The sample space is countable that means its
outcome has one to one correspondence with
the positive integers.the positive integers.
Ex. Tossing of coin or fair die.
• Continuous sample space:
The sample space is not countable.
Ex: Picking up a number between 0 and 1.

EVENT
• EVENT: The event occurs if and only if the outcome
of the experiment ξ is in the subset.
Ex: Select a ball from box numbered from 1 to 50.
• Certain event:
It consists of all outcomes hence always occurs.It consists of all outcomes hence always occurs.
• Null event/impossible event):
It contains no outcome and it never occur.
• Elementary event:
Event from a discrete sample space that consists of a
single outcome.

PROBABILITY:
It is the measure of uncertainty of occurrence of an event or
chance of probability of occurrence of an event.
THE AXIOMS OF PROBABILITY:
Axiom 1: 0 ≤ P [A]
Axiom 2: P[S] =1Axiom 2: P[S] =1
Axiom 3:
For any no. of mutual exclusively events A1, A2, A3 …An. in the
class C.
P (A1 Ụ A2 Ụ A3………………… Ụ An)=
P(A1)+P(A2)+P(A3)+……..P(An).

•Theorem 1:
If there are two events A1<A2 then.
P (A1) ≤ P (A2). And P(A2-A1)=P(A2)-P(A1).
•Theorem 2:
The probability of an event is lies between 0 ≤P (A) ≤1.
•Theorem 3:
P (Ф) =0.
Theorems on Probability
P (Ф) =0.
•Theorem 4:
P (A′) =1-P (A).
•Theorem 5:
If there are ‘n’ no of mutually exclusively events
P(A)=P(A1)+P(A2)+P(A3)+……..P(An)=1.
•Theorem 6:
P (A Ụ B) =P (A) +P (B)-P (A∩B)

Example
• Choose the experiment tossing a die the sample space
S={1,2,3,4,5,6}, find the probability of P(2 Ụ 5).
• Solution:
P(1)=P(2)=P(3)=P(4)=P(5)=P(6)= 1/6
P (2) =1/6, P (5) =1/6P (2) =1/6, P (5) =1/6
P (2 Ụ 5). =P (2) +P (5)-P (2∩5)
= 1/6+1/6=2/6.
Because(P (2∩5) =0)

CONDITIONAL PROBABILITY
• If A and B are two events A, B, such that P (A)>0, the
conditional probability of B of A is given by
   
 AP
BAP
ABP

|   0APprovided
 AP

Example
• Tossing a die and the event is odd and is less then ‘4’.
• Solution:
P(1)=P(2)=P(3)=P(4)=P(5)=P(6)= 1/6
P (A) = P (1) +P (3) +P (5) =3/6.
P (B) = P (1) +P (2) +P (3) =3/6P (B) = P (1) +P (2) +P (3) =3/6
P (A∩B) = P (1) +P (3) =2/6.
   
 AP
BAP
ABP

|
3/2
6/3
6/2


INDEPENDENT PROBABILITY
• If ‘A ‘and ‘B’ are two independent events, the
conditional probability of ‘B’ given ‘A’ is
   
 
   
 
 BP
AP
BPAP
AP
BAP
ABP 

| 
   
 BP
APAP
ABP |

�ݺ�ߣ

Module 1 sp

Recommended

More Related Content

Similar to Module 1 sp (20)

More from Vijaya79 (7)

Recently uploaded (20)

Module 1 sp