Information Theory Assignments

1^st Week

• Read Oishi (textbook) Chapter 1

• Mathematica Exercize: set union, intersection, complement (difference)

  Submit on A4 paper at the next class; don't forget to include your name and student id at the top. Refresh: Refer to Toyoizumi-sensei's useful notes from Computer Literacy lectures for reminders about Mathematica.

  Use Mathematica to calculate set union, intersection, and complement (difference) using digits from your student ID as sample data.

  Hint: Use something like list = IntegerDigits[ToExpression[StringDrop[$UserName,1]]] to generate sample data. Use something like lists = Partition[list, 4, 3] to break that list into two lists. Use Intersection[] (or ESC inter ESC, where "ESC" is the "Escape" key), Union[] (or ESC un ESC), and Complement[]. (Use this sample session as a model if you get stuck. [You can save this file through your browser (right-click or control+click in Firefox to "Save Link As...", right-click in Safari to "Download Linked File As...", right-click in Explorer to "Download Link to Disk", etc.) to your file system, then open in Mathematica.])

  Extra credit

  What is the set of (unique) digits in your student ID? That is, the answer will have the set of the digits in your student number, with each digit appearing only once. For instance, the set of digits in "s1181234" is {1,2,3,4,8} (and not {1,1,1,2,3,4,8}). (Hint: convert from a list to a set by monadically (using a single argument) invoking one of the set operations on a list.)

  Mathematica Exercize: average & variance

  Write Mathematica functions to calculate mean (average) and variance. Use the digits of your student ID as sample data, as you did on the previous assignment. Print out onto A4 paper and attach to the first part of the assignment. Hint: Use the functions Apply[Plus, data] and Length[data], along with arithmetic functions - (subtraction) and ^ (exponentiation), to calculate these two values. Experiment with the Mathematica palettes for expression entry (of, for example, fractions and powers). (Use this sample session or this method for calculating Pi from random numbers if you need reminders about Mathematica syntax.)

• Bring in Y1,000 to buy Chinese Rings puzzle.

2^nd Week

• Do Problem 1.3 (p. 14 of Oishi, repeated on p. 19 of course notes). Note that E(X)² is equivalent to E²(X) (but not E(X²)). Hand in solution on A4 paper.
• Read Oishi 2.1-4
• Play with "Patience Puzzle" (a.k.a. "Chinese Rings"). (Double extra credit if you can demonstrate its solution.)

3^rd Week

• Read Oishi 2.5-8

4^th Week

• Play with "Patience Puzzle." (Extra credit if you can demonstrate its solution.)
• Study for 1^st midterm exam (open book, open notes, calculator and dictionary okay).

5^th Week

• Do Oishi problems 2.2, 2.5. (Don't forget to record your name and student number at the top of the sheet.)
• Do the problem on p. 48 of the lecture notes, corresponding to Oishi 2.6: Draw code trees for C₇ and C₈, calculate average code length, and confirm the Kraft Inequality.
• Solve the Patience Puzzle! Demonstrate its solution to get credit. (Explore the course reference links for hints if you need.)

6^th Week

• (No new assignment.)
• Solve the Patience Puzzle! Demonstrate its solution to get credit. (Explore the course reference links for hints if you need.)

7^th Week

• Read Oishi sections 3.1-3.6.
• On an A4 piece of paper,
  • Do the exercizes on p. 64 of the lecture notes. Hand in on A4 paper. (Don't forget to write your name and student ID at the top.)
• Study for 2^nd midterm exam.

8^th Week

• Read Oishi sections 3.6-3.9
• Hand in on A4 paper [don't forget to write your name and student id]
  • Do problem on p. 82 of the course notes (linked to Section 3.8, p. 53, of Oishi): make a Huffman code based on probabilities (16/32, 8/32, 4/32, 4/32). What happens when accumulated probabilities that equal some other probability are sorted above the previous value?
  • Calculate entropy H(1/3, 1/3, 1/3). Use of of Mathematica, calculator, etc. encouraged. (Don't forget to use Log[2, whatever] in Mma to get binary logs.)
  • Calculate H(1/3, 1/3, 1/6, 1/6)

9^th Week

• Read Oishi Chapter 4.

10^th Week

• Mathematica Exercize: Markov processes
  Use Mathematica to calculate steady state (eigenvector) of a Markov process (reflecting a matrix made from your student ID):

  • Use the sample session as a model. (Right- or control-click to download.)
  • Replace the sample student ID at the top with your own.
  • Compile the sample code to make a unique transition matrix modeling an ergodic Markov process and find its eigenvector both iteratively and analytically. (Reminder about Mma: Edit>Select All to select the entire file, then Shift+Return to execute.)
  • Save the compiled Mathematica file as concatenation of your student ID and "-Markov" with the Mathematica file type extension ".nb" (for example, "s1181234-Markov.nb").
  • Email it as an attachment to TA Prabath at m5141110@u-aizu.ac.jp by Monday morning.

11^th Week

• (No assignment; Happy New Year!)

12^th Week

• Do exercize in section 4.5.2.1 of notes (p. 101, '11-'12 edition).
• Read Oishi 5.

13^th Week

• Read Oishi 6.

14^th Week

• Read the Appendix in Oishi (about Analog/Digital analysis).
• Determine the 6^th roots of 1, expressed in polar (degrees and radians) and rectangular (complex) notation. Sketch these roots on a unit circle. (submitted on A4 paper)
• Study for final exam