Introduction to Tensorflow

In this article we will learn to do basic computations with Tensorflow. We will go beyond basic Hello World. We will learn to do matrix operations and a few other things.

We will assume that you have installed Python, Tensorflow and Numpy. All code samples assume that you have imported the packages as follows:

import numpy as np
import tensorflow as tf

Computational Graph

In Tensorflow much of the code you write goes into building a computational graph. You can then execute that graph over and over again. I personally think this is a very powerful concept and makes neural network programming easier. For example you can run a minimization operation repeatedly until a certain accuracy is reached. And because we do all the computation from a graph, TF can distribute the workload to one or more GPUs and multiple machines.

Placeholder

A basic building block of a graph is input variables. In TF we call them placeholders. A placeholder is a read only variable that serves as input to the graph.

One of the most basic operations you can do in TF is add two real numbers. Let’s see how we can build a graph using input placeholders and operations.

a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
graph = tf.add(a, b)

#Now execute the graph with 
#actual values for a and b
sess = tf.Session()
result = sess.run(graph, {a:10.0, b:20.0})

#Will print 30.0
print result  

#Execute the graph again with 
#different input values
result = sess.run(graph, {a:45.0, b:30.0})

#Will print 75.0
print result  

The second argument to Session.run() is a dictionary. The element names in the dictionary must match the placeholder python variable names. The method executes the graph and returns the output.

Matrix Computation

Well, that was easy, wasn’t it? Now let’s add two matrices. To define a matrix placeholder we need to provide its shape in addition to the data type. Below we add two 3×2 matrices:

X = tf.placeholder(tf.float32, [3, 2])
Y = tf.placeholder(tf.float32, [3, 2])
graph = tf.add(X, Y)

xArr = np.array([
    [1, 2],
    [3, 4],
    [5, 6]
])

yArr = np.array([
    [7, 8],
    [9, 0],
    [1, 2]
])

sess = tf.Session()

res = sess.run(graph, {X: xArr, Y: yArr})

print res

Multiplying two matrices is very similar.

X = tf.placeholder(tf.float32, [4, 2])
Y = tf.placeholder(tf.float32, [2, 4])
graph = tf.matmul(X, Y)

xArr = np.array([
    [1, 2],
    [3, 4],
    [5, 6],
    [7, 8]
])
yArr = np.array([
    [1, 2, 3, 4],
    [5, 6, 7, 8]
])

sess = tf.Session()

res = sess.run(graph, {X: xArr, Y: yArr})

print res

Variable

Variables carry values just like placeholders. But there are key differences. The value of a variable can be updated during the execution of a graph. If you execute the graph repeatedly a variable will maintain its state between executions. In contrast, the value of a placeholder is set at the beginning of a graph execution (when you call Session.run()) and its value is not expected to change.

In the example below we will create a graph that increments the value of a simple variable by one.

increment = tf.placeholder(tf.float32)
stateVar = tf.Variable(0.0)
updatedValue = tf.add(stateVar, increment)
graph = tf.assign(stateVar, updatedValue)

sess = tf.Session()

#initialize all variables
sess.run(tf.global_variables_initializer())

for _ in range(5):
    sess.run(graph, {increment: 1})
    #Print out the value of the variable
    print "State variable value: ", sess.run(stateVar)

This will print out:

State variable value:  1.0
State variable value:  2.0
State variable value:  3.0
State variable value:  4.0
State variable value:  5.0

A few things to note about variables:

  • Even though we specified that 0.0 is the initial value of stateVar, we still had to execute tf.global_variables_initializer() to initialize the variables. This may seem unnecessary. But in real life you will usually initialize variables with random values which require additional computation. As a result the initialization process itself needs execution of a graph.
  • To read the value of a variable you need to execute the variable as a graph. We are doing that here using sess.run(stateVar).

Constant

A constant is like a placeholder. But its value is set at definition time. You can not supply a different value for different executions of a graph. Below is the same code as above. Except, increment has been converted to a constant.

increment = tf.constant(1.0)
stateVar = tf.Variable(0.0)
updatedValue = tf.add(stateVar, increment)
graph = tf.assign(stateVar, updatedValue)

sess = tf.Session()

#initialize all variables
sess.run(tf.global_variables_initializer())

for _ in range(5):
    sess.run(graph)
    #Print out the value of the variable
    print "State variable value: ", sess.run(stateVar)
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s