ELE637 - FUNDAMENTALS of INFORMATION THEORY
| Course Name | Code | Semester | Theory (hours/week) |
Application (hours/week) |
Credit | ECTS |
|---|---|---|---|---|---|---|
| FUNDAMENTALS of INFORMATION THEORY | ELE637 | Any Semester/Year | 3 | 0 | 3 | 8 |
| Prequisites | ||||||
| Course language | Turkish | |||||
| Course type | Elective | |||||
| Mode of Delivery | Face-to-Face | |||||
| Learning and teaching strategies | Lecture Question and Answer Problem Solving | |||||
| Instructor (s) | Department Faculty | |||||
| Course objective | The objective of the course is to introduce ? the notion of entropy and information ? the fundamental limits of data compression ? the fundamental limits of data transmission systems. | |||||
| Learning outcomes |
| |||||
| Course Content | Introduction, review of probability, Entropy, relative entropy, mutual information, inequalities, The asymptotic equipartition property, Data compression, Channel capacity, Differential entropy, the Gaussian channel, Network information theory. | |||||
| References | Elements of Information Theory, Cover and Thomas, Wiley Interscience Gallager, "Claude E. Shannon: A Retrospective on His Life, Work, and Impact", IEEE Trans. Inform. Theory, vol.47, no.7, Nov. 2001 Wyner, "Fundamental Limits in Information Theory", Proc. of the IEEE, vol.69, no.2, Feb. 1981 Verdu, "Fifty Years of Shannon Theory", IEEE Trans. Inform. Theory, vol.44, no.6, Oct. 1998 | |||||
Course outline weekly
| Weeks | Topics |
|---|---|
| Week 1 | Review of probability theory, entropy |
| Week 2 | Relative entropy and mutual information |
| Week 3 | Jensen?s inequality and its consequences |
| Week 4 | Asymptotic equipartition property |
| Week 5 | Data compression and Kraft inequality |
| Week 6 | Optimal codes, Huffman codes |
| Week 7 | Shannon-Fano-Elias coding |
| Week 8 | Midterm Exam |
| Week 9 | Channel capacity examples |
| Week 10 | Channel coding theorem |
| Week 11 | Fano?s inequality and the converse to the coding theorem |
| Week 12 | Differential entropy |
| Week 13 | Gaussian channel |
| Week 14 | Network information theory |
| Week 15 | Final exam |
| Week 16 | Final exam |
Assesment methods
| Course activities | Number | Percentage |
|---|---|---|
| Attendance | 0 | 0 |
| Laboratory | 0 | 0 |
| Application | 0 | 0 |
| Field activities | 0 | 0 |
| Specific practical training | 0 | 0 |
| Assignments | 1 | 5 |
| Presentation | 0 | 0 |
| Project | 0 | 0 |
| Seminar | 1 | 5 |
| Midterms | 1 | 40 |
| Final exam | 1 | 50 |
| Total | 100 | |
| Percentage of semester activities contributing grade succes | 0 | 50 |
| Percentage of final exam contributing grade succes | 0 | 50 |
| Total | 100 | |
WORKLOAD AND ECTS CALCULATION
| Activities | Number | Duration (hour) | Total Work Load |
|---|---|---|---|
| Course Duration (x14) | 14 | 3 | 42 |
| Laboratory | 0 | 0 | 0 |
| Application | 0 | 0 | 0 |
| Specific practical training | 0 | 0 | 0 |
| Field activities | 0 | 0 | 0 |
| Study Hours Out of Class (Preliminary work, reinforcement, ect) | 14 | 8 | 112 |
| Presentation / Seminar Preparation | 0 | 0 | 0 |
| Project | 1 | 25 | 25 |
| Homework assignment | 1 | 5 | 5 |
| Midterms (Study duration) | 0 | 0 | 0 |
| Final Exam (Study duration) | 1 | 25 | 25 |
| Total Workload | 31 | 66 | 209 |
Matrix Of The Course Learning Outcomes Versus Program Outcomes
| D.9. Key Learning Outcomes | Contrubition level* | ||||
|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | |
| 1. Has general and detailed knowledge in certain areas of Electrical and Electronics Engineering in addition to the required fundamental knowledge. | X | ||||
| 2. Solves complex engineering problems which require high level of analysis and synthesis skills using theoretical and experimental knowledge in mathematics, sciences and Electrical and Electronics Engineering. | X | ||||
| 3. Follows and interprets scientific literature and uses them efficiently for the solution of engineering problems. | X | ||||
| 4. Designs and runs research projects, analyzes and interprets the results. | X | ||||
| 5. Designs, plans, and manages high level research projects; leads multidiciplinary projects. | X | ||||
| 6. Produces novel solutions for problems. | X | ||||
| 7. Can analyze and interpret complex or missing data and use this skill in multidiciplinary projects. | X | ||||
| 8. Follows technological developments, improves him/herself , easily adapts to new conditions. | X | ||||
| 9. Is aware of ethical, social and environmental impacts of his/her work. | X | ||||
| 10. Can present his/her ideas and works in written and oral form effectively; uses English effectively | X | ||||
*1 Lowest, 2 Low, 3 Average, 4 High, 5 Highest