Digital Communication #1: Information Theory

OBJECTIVES:

✔ To define Information Theory
✔ To explain Information Capacity
✔ To solve problems related to Information Theory

KEY TERMS:

🔑 Bandwidth is the range of frequencies within a band. It is one of the most significant limitations to the performance of a communication system.
🔑 Noise is any undesirable energy that causes distortion to signal.
🔑 Signal-to-Noise Ratio (S/N) is the ratio between the signal and the noise. The greater the value of signal to noise ratio, the better the performance of the system. This ratio is unitless.

Source

UNITS:

⚠ bps (bit per second) for the Information Capacity
⚠ Hz (Hertz) for the Bandwidth
⚠ sec (seconds) for Transmission Time

KEY FORMULAS:

🔑 S/N decibels = (20log) S/N ➡ for VOLTAGE
🔑 S/N decibels = (10log) S/N ➡ for POWER
🔑 S/N = Signal/Noise

---

Source

💡What is Information?

Information is what conveyed or represented by a particular arrangement or sequence of things. – Dictionary

We give and take informations almost every minute of the day. These are basically the facts we provide and obtain. We exchange ideas through sign languages or chitchats. Even now that you are reading this, we are already communicating. But in the world of electronic communication, it is defined as the knowledge that is received and communicated through the electronic communication system. When I was a kid, I used to play Mom-Dad-Baby game with my friends where we act as if we are a family; we cook fake foods using leaves, soil and water. It was way happier playing outdoor games but nowadays parents chose to make their children play with PCs. There are also these parents who have children who are not yet a year old but already own tablets. I am not saying that it is a bad parenting, but it is way different from what children used to be. The relationship between the kid and the PC is already bound with communication.

🔓 Do you know my professor once told me that the term we use for knowledge is INTELLIGENCE

Enough of the fuss and let us get things started. Our world nowadays revolves a lot with the innovations and achievements of men of science. This field of study is so broad to understand in one-time-study only. So let us understand it piece by piece. Almost all of us own a PC right? And you probably wonder why every time you send message, there is a designated capacity or limit of words, numbers or characters that can be used. If you are a type of person that does not care about it at all I want you to keep on reading because later on you might get curious; and if you are the other type of person who is as curious as I am then stay tuned and let us figure things out together.

---

💡What are Noise and Bandwidth?

Have you ever noticed that sometimes, when you are watching anything in a television and it suddenly get distorted? That is not because your TV is old or what, it is because of a signal that interferes the signal of out television. That type of unwanted signal is what we call Noise. These interferences are present all the time. It can be categorized on either a Correlated Noise or Uncorrelated Noise. When you say it is Correlated Noise then the interference is only present when there is a signal on your device. On the other hand, in an Uncorrelated Noise the undesirable signal is present all the time whether there is a signal or not. The range of frequency that your signal is containing informations is what we call Bandwidth. These two factors, the Noise and the Bandwidth, are main problem or limitation on the performance of our communication system. These are basically the limitations on why when we text or send message to someone and is too long, we need to type it in more than one go and send separately because as the bandwidth increases, the noise increases as well. If the communication system allows us to use a huge bandwidth there will be a great range of frequency that will get distorted and lost along the way. Those are the reasons why your message must be resend from time to time or the message cannot be received on time. So whenever your girlfriend/boyfriend have not texted you yet, she/he might be suffering from these factors. I am just saying. Haha.
Let us dig deeper. How can we calculate the Capacity of Information we can use? What are the factors that affect it? What are the laws that concerns with the Information Theory?

🔓 What are the sizes of Information?

Bit is composed of 2 elements
Nibble is composed of 4 elements
Byte is composed of 8 elements
Word is composed of 16 elements

---

💡What is Information Theory?

Information Theory is highly theoretical study of the efficient use of bandwidth to propagate information through electronic communication system.

This theory is used to determine the numbers of independent symbols that can be carried through the system in a given time which is known as Information Capacity. In addition, it is measurement of how much information is able to propagate through the system. So this is the capacity of how many characters, in a text message, you can type in a one-go; or the informations you provide and obtain in a communication system. It is expressed with the number of bits transmitted in a second or known as bits per second

🔓 Information Capacity can be solved using Either Hartley’s Law or Shannon’s Law.

Hartley’s Law🔥

In Bell Telephone Laboratories on 1928, an information theorist named Ralph V. R. Hartley proved that the information capacity is directly proportional to the bandwidth.

I = Bt

I = Information Capacity
B =Bandwidth
t = Transmission time

EXAMPLE:

Solve the information capacity for a communication channel with a bandwidth of 2.7kHz in 9.9 seconds

Given:

I = ?
B = 2.7 kilo Hertz
t = 9.9 seconds

Substituting the given informations to the formula manually:

I = (2.7kHz)(9.9s)
I = 26.73 kilo bps

Information Capacity is equal to 26.73kbps.

Shannon’s Limit of Information Capacity 🔥

Claude Elwood Shannon was an American who is known as the father of information theory. He is also from Bell Telephone Laboratories. He published a paper in 1948 where he stated a relation of information capacity to the bandwidth and the signal-to-noise ratio.

I = 3.32Blog(1+S/N)

I = Information Capacity
B = Bandwidth
S/N = Signal-to-Noise ratio

EXAMPLE:

For a channel capacity for a signal power of 200 Watts, noise power of 10 Watts and a bandwidth of 2 kilo Hertz, what is the information capacity?

Given:

I = ?
S = 200 Watts
N = 10 Watts
B = 2 kilo Hertz

Substituting the given informations to the formula:

I = 3.32(2kHz)log(1+S/N)
S/N = (200W)/(10W)
S/N = 20
I = 3.32(2kHz)log(1+20)
I = 8.779 kilo bits per second

Information Capacity is equal to 8.779kbps.

BIBLIOGRAPHY

Book: Wyne Tomasi. “Electronic Communication Systems”. Pages 347 and 348.

Yours Truly,
SISSY

I am a part of @steemitfamilyph. Join us! Follow - Upvote - Resteem - Comment