A Probability and Entropy Calculator

Table of Contents:

• Introduction:
• The Language:
• The Task: Probability and Entropy
• The Programs:
  • version 1.0
  • version 1.2
• Conclusions:
• Code: version 1.2
• References:

Introduction:

During my studies in CSC 551 Systems Theory, I came across an interesting use for my concurrent studies with the Visual Basic Windows Construction Language. I made several attempts to build a program that would calculate the conditional probabilities and entropy given a joint matrix probability matrix. Since the language and GUI constructs of the Visual Basic Language were new to me - I did not succeed to get my program to compute to the level I intended, yet I was able to calculate probabilities of a 4 x 4 matrix; in order to go further with this concept I will need to implement control arrays - which are unique to the Visual Basic Language. Yet, the program does exemplify the concept I was seeking; to create an GUI that was both friendly to use and capable of acting as a research tool - to a small degree I made some advancement towards this goal.

RETURN TO TOP OF PAGE

The Language:

Visual Basic is QBasic's and GWBasic younger relative; it uses similar language syntax with these earlier versions of Basic. What makes the VB Language so unique and powerful is the easy interface for building MS Windows applications; the interface is like a cross between a paint program and any programming editor. WIMP (Windows Icon Menu Pointer) constructs are pre-built as software objects - they can be arranged, coded, and linked with variables to the code that the programmer writes.

RETURN TO TOP OF PAGE

The Task: Probability and Entropy

Given a joint probability matrix:

where the sum of all the matrix values add up to 1, the probability of any event will be (xnym / Sum of all xnym pairs). The self information of an event x can be given by: ; and the average information in a set of events, x = {x,...xn} is:

My problem was to code an easy interface to input an n by n matrix and calculate normal and conditional probability matrices. Footnote1 .

The Programs:

Each program took about 5 hours each to code - I was limited by my inexperience with the Basic Language; I found that I could not push the Language to its real power -without the use of control arrays and multiple windows. Also, I was limited to the computing environment of VB. Perhaps using the Dynamic Link Library (DLL) functions I might be able to connect the programs created with VB to a more powerful calculating Kernel - Mathematica, for example.

RETURN TO TOP OF PAGE

version 1.0

My first VB. program looked good, ran fine, but gave the wrong calculations - my ignorance of the problem I was trying to solve was apparent from the output. I used scroll bars to select the values for the 2 x 2 joint probability matrix - I tried several other input methods; none worked. One drawback to this version was that matrix values were not limited to summing to one - which perplexed me at first because I was getting probability values greater than one. I experimented using color to indicate when the values in the original matrix summed too one.

RETURN TO TOP OF PAGE

version 1.2

My second attempt was more ambitious: solution of the probabilities and entropy values of a 4 x 4 matrix - I used a parent object to build the initial matrix and then pasted them to the form Footnote2 I was building; this simplified the program's construction. In order to avoid the errors from finding the log of zero I used the following subroutine to calculate the log base 2 of X:

Static Function Log2 (X)
	If X = 0 Then 
		Log2 = 0
	Else 
		Log2 = Log(X) / Log(2)
	End 

The program started calculating when the calculate button was pushed; in fact, all of the code that ran the calculations was "behind" this event. After coding three 4 X 4 matrices and two entropy matrices I realized that I had run out of room on the screen; my workspace was to small! One way of fixing this would be to allow multiple windows be dedicated to either probabilities or entropy. I settled for demonstrating to myself that yes, VB. could be used to build an interface for mathematical calculations.

RETURN TO TOP OF PAGE

Conclusions:

Using the object driven Language Visual Basic I made a ( trivial ) computing tool to calculate probabilities and entropy; in doing this I became familiar with the problem I was solving and the Language I was using. Because a matrix calculation tool needs to be utilized to handle n by m matrices, the code to calculate the answers needs to dynamically controlled - implementing control arrays and new windows could help do this. Even with the Language's limitations, applications could be linked with to a more powerful Kernel to handle high volume computations.

RETURN TO TOP OF PAGE

Code: version 1.2

Sub inbar_Change ()
End Sub

Sub Command3_Click ()
End
End Sub

Sub Command2_Click ()
vx1y1 = Val(x1y1.text)
vx1y2 = Val(x1y2.text)
vx1y3 = Val(x1y3.text)
vx1y4 = Val(x1y4.text)
vx2y1 = Val(x2y1.text)
vx2y2 = Val(x2y2.text)
vx2y3 = Val(x2y3.text)
vx2y4 = Val(x2y4.text)
vx3y1 = Val(x3y1.text)
vx3y2 = Val(x3y2.text)
vx3y3 = Val(x3y3.text)
vx3y4 = Val(x3y4.text)
vx4y1 = Val(x4y1.text)
vx4y2 = Val(x4y2.text)
vx4y3 = Val(x4y3.text)
vx4y4 = Val(x4y4.text)
vintotal1 = vx1y1 + vx1y2 + vx1y3 + vx1y4
vintotal2 = vx2y1 + vx2y2 + vx2y3 + vx2y4
vintotal3 = vx3y1 + vx3y2 + vx3y3 + vx3y4
vintotal4 = vx4y1 + vx4y2 + vx4y3 + vx4y4
vintotal = vintotal1 + vintotal2 + vintotal3 +
vintotal4
End Sub
 
Sub Command1_Click ()
vx1y1 = Val(x1y1.text)
vx1y2 = Val(x1y2.text)
vx1y3 = Val(x1y3.text)
vx1y4 = Val(x1y4.text)
vx2y1 = Val(x2y1.text)
vx2y2 = Val(x2y2.text)
vx2y3 = Val(x2y3.text)
vx2y4 = Val(x2y4.text)
vx3y1 = Val(x3y1.text)
vx3y2 = Val(x3y2.text)
vx3y3 = Val(x3y3.text)
vx3y4 = Val(x3y4.text)
vx4y1 = Val(x4y1.text)
vx4y2 = Val(x4y2.text)
vx4y3 = Val(x4y3.text)
vx4y4 = Val(x4y4.text)
vintotal1 = vx1y1 + vx1y2 + vx1y3 + vx1y4
vintotal2 = vx2y1 + vx2y2 + vx2y3 + vx2y4
vintotal3 = vx3y1 + vx3y2 + vx3y3 + vx3y4
vintotal4 = vx4y1 + vx4y2 + vx4y3 + vx4y4
vintotal = vintotal1 + vintotal2 + vintotal3 +
vintotal4
vpx1 = vx1y1 + vx1y2 + vx1y3 + vx1y4
vpx2 = vx2y1 + vx2y2 + vx2y3 + vx2y4
vpx3 = vx3y1 + vx3y2 + vx3y3 + vx3y4
vpx4 = vx4y1 + vx4y2 + vx4y3 + vx4y4
vpy1 = vx1y1 + vx2y1 + vx3y1 + vx4y1
vpy2 = vx1y2 + vx2y2 + vx3y2 + vx4y2
vpy3 = vx1y3 + vx2y3 + vx3y3 + vx4y3
vpy4 = vx1y4 + vx2y4 + vx3y4 + vx4y4
px1.text = Format$(vpx1, "0.000")
px2.text = Format$(vpx2, "0.000")
px3.text = Format$(vpx3, "0.000")
px4.text = Format$(vpx4, "0.000")
py1.text = Format$(vpy1, "0.000")
py2.text = Format$(vpy2, "0.000")
py3.text = Format$(vpy3, "0.000")
py4.text = Format$(vpy4, "0.000")

If vpx1 = 0 Then
	vpy1x1 = 0
	vpy2x1 = 0
	vpy3x1 = 0
	vpy4x1 = 0
Else
	vpy1x1 = vx1y1 / vpx1
	vpy2x1 = vx1y2 / vpx1
	vpy3x1 = vx1y3 / vpx1
	vpy4x1 = vx1y4 / vpx1
End If py1x1.text = Format$(vpy1x1, "0.000")

py2x1.text = Format$(vpy2x1, "0.000")
py3x1.text = Format$(vpy3x1, "0.000")
py4x1.text = Format$(vpy4x1, "0.000")

If vpx2 = 0 Then
	vpy1x2 = 0
	vpy2x2 = 0
	vpy3x2 = 0
	vpy4x2 = 0
Else
	vpy1x2 = vx2y1 / vpx2
	vpy2x2 = vx2y2 / vpx2
	vpy3x2 = vx2y3 / vpx2
	vpy4x2 = vx2y4 / vpx2
End If

py1x2.text = Format$(vpy1x2, "0.000")
py2x2.text = Format$(vpy2x2, "0.000")
py3x2.text = Format$(vpy3x2, "0.000")
py4x2.text = Format$(vpy4x2, "0.000")

If vpx3 = 0 Then
	vpy1x3 = 0
	vpy2x3 = 0	
	vpy3x3 = 0
	vpy4x3 = 0
Else
	vpy1x3 = vx3y1 / vpx3
	vpy2x3 = vx3y2 / vpx3
	vpy3x3 = vx3y3 / vpx3
	vpy4x3 = vx3y4 / vpx3
End If

py1x3.text = Format$(vpy1x3, "0.000")
py2x3.text = Format$(vpy2x3, "0.000")
py3x3.text = Format$(vpy3x3, "0.000")
py4x3.text = Format$(vpy4x3, "0.000")

If vpx4 = 0 Then
	vpy1x4 = 0
	vpy2x4 = 0
	vpy3x4 = 0
	vpy4x4 = 0
Else
	vpy1x4 = vx4y1 / vpx4
	vpy2x4 = vx4y2 / vpx4
	vpy3x4 = vx4y3 / vpx4
	vpy4x4 = vx4y4 / vpx4
End If

py1x4.text = Format$(vpy1x4, "0.000")
py2x4.text = Format$(vpy2x4, "0.000")
py3x4.text = Format$(vpy3x4, "0.000")
py4x4.text = Format$(vpy4x4, "0.000")

If vpy1 = 0 Then
	vpx1x1 = 0
	vpx2x1 = 0
	vpx3x1 = 0
	vpx4x1 = 0
Else
	vpx1y1 = vx1y1 / vpy1
	vpx2y1 = vx1y2 / vpy1
	vpx3y1 = vx1y3 / vpy1
	vpx4y1 = vx1y4 / vpy1
End If

px1y1.text = Format$(vpx1y1, "0.000")
px2y1.text = Format$(vpx2y1, "0.000")
px3y1.text = Format$(vpx3y1, "0.000")
px4y1.text = Format$(vpx4y1, "0.000")

If vpy2 = 0 Then
	vpx1y2 = 0
	vpx2y2 = 0
	vpx3y2 = 0
	vpx4y2 = 0
Else
	vpx1y2 = vx2y1 / vpy2
	vpx2y2 = vx2y2 / vpy2
	vpx3y2 = vx2y3 / vpy2
	vpx4y2 = vx2y4 / vpy2
End If

px1y2.text = Format$(vpx1y2, "0.000")
px2y2.text = Format$(vpx2y2, "0.000")
px3y2.text = Format$(vpx3y2, "0.000")
px4y2.text = Format$(vpx4y2, "0.000")

If vpy3 = 0 Then
	vpx1y3 = 0
	vpx2y3 = 0
	vpx3y3 = 0
	vpx4y3 = 0
Else
	vpx1y3 = vx3y1 / vpy3
	vpx2y3 = vx3y2 / vpy3
	vpx3y3 = vx3y3 / vpy3
	vpx4y3 = vx3y4 / vpy3
End If

px1y3.text = Format$(vpx1y3, "0.000")
px2y3.text = Format$(vpx2y3, "0.000")
px3y3.text = Format$(vpx3y3, "0.000")
px4y3.text = Format$(vpx4y3, "0.000")

If vpy4 = 0 Then
	vpx1y4 = 0
	vpx2y4 = 0
	vpx3y4 = 0
	vpx4y4 = 0
Else
	vpx1y4 = vx4y1 / vpy4
	vpx2y4 = vx4y2 / vpy4
	vpx3y4 = vx4y3 / vpy4
	vpx4y4 = vx4y4 / vpy4
End If

px1y4.text = Format$(vpx1y4, "0.000")
px2y4.text = Format$(vpx2y4, "0.000")
px3y4.text = Format$(vpx3y4, "0.000")
px4y4.text = Format$(vpx4y4, "0.000")
vhx1 = Abs(vpx1 * Log2(vpx1))
vhx2 = Abs(vpx2 * Log2(vpx2))
vhx3 = Abs(vpx3 * Log2(vpx3))
vhx4 = Abs(vpx4 * Log2(vpx4))
vhy1 = Abs(vpy1 * Log2(vpy1))
vhy2 = Abs(vpy2 * Log2(vpy2))
vhy3 = Abs(vpy3 * Log2(vpy3))
vhy4 = Abs(vpy4 * Log2(vpy4))
hx1.text = Format$(vhx1, "#0.000")
hx2.text = Format$(vhx2, "#0.000")
hx3.text = Format$(vhx3, "#0.000")
hx4.text = Format$(vhx4, "#0.000")
hy1.text = Format$(vhy1, "#0.000")
hy2.text = Format$(vhy2, "#0.000")
hy3.text = Format$(vhy3, "#0.000")
hy4.text = Format$(vhy4, "#0.000")
End Sub

RETURN TO TOP OF PAGE

References:

RETURN TO TOP OF PAGE