Keras: installation with Tensorflow and opencv

$ mkvirtualenv keras_tf
$ workon keras_tf

$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.12.1-cp27-none-linux_x86_64.whl

#$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-0.12.0rc2-py2-none-any.whl
wheel name should come from
linklink

$ pip install --upgrade $TF_BINARY_URL

$ pip install numpy scipy

$ pip install scikit-learn

$ pip install pillow

$ pip install h5py

$ pip install keras Before we get too far we should check the contents of our keras.json  configuration file. You can find this file in ~/.keras/keras.json . $gedit ~/.keras/keras.json . add  "image_dim_ordering": "tf" in the file and file contents should look lik {     "image_dim_ordering": "tf",     "epsilon": 1e-07,     "floatx": "float32",     "backend": "tensorflow" } You might be wondering what exactly image_dim_ordering  controls. Using TensorFlow, images are represented as NumPy arrays with the shape (height, width, depth), where the depth is the number of channels in the image. However, if you are using Theano, images are instead assumed to be represented as (depth, height, width). Find CV2.so $ cd / $ sudo find . -name '*cv2.so*' ./Users/adrianrosebrock/.virtualenvs/cv/lib/python2.7/site-packages/cv2.so ./Users/adrianrosebrock/.virtualenvs/gurus/lib/python2.7/site-packages/cv2.so ./Users/adrianrosebrock/.virtualenvs/keras_th/lib/python2.7/site-packages/cv2.so ./usr/local/lib/python2.7/site-packages/cv2.so and copy that to virtual environment $ cd ~/.virtualenvs/keras_tf/lib/python2.7/site-packages/ $ ln -s /usr/local/lib/python2.7/site-packages/cv2.so cv2.so $ cd ~ ------------------------------------------------ References Content taken from http://www.pyimagesearch.com/2016/11/14/installing-keras-with-tensorflow-backend/

Keras: Hyper-parameters optimization

link

http://machinelearningmastery.com/grid-search-hyperparameters-deep-learning-models-python-keras/

http://www.pyimagesearch.com/2016/08/15/how-to-tune-hyperparameters-with-python-and-scikit-learn/

and simple image-base neural network
http://www.pyimagesearch.com/2016/09/26/a-simple-neural-network-with-python-and-keras/

Keras: IRIS dataset example

Important Point

we need to convert classes that look like

setosa
versicolor
setosa
virginica
...

to a table that looks like

setosa versicolor virginica
     1          0         0
     0          1         0
     1          0         0
     0          0         1

import numpy as np
from keras.models import Sequential
from keras.layers import Dense,Dropout,Activation,Flatten
from keras.layers import Convolution2D,MaxPooling2D
from keras.utils import np_utils
from keras.datasets import mnist
from matplotlib import pyplot as plt
import keras.backend as K
import pandas as pd
from sklearn.cross_validation import train_test_split
#import keras.backend as K

def f1_score(y_true, y_pred):

    tp = K.sum(K.round(K.clip(y_true * y_pred, 0,1)))
    predicted_p = K.sum(K.round(K.clip(y_pred,0,1)))
    possible_p = K.sum(K.round(K.clip(y_true,0,1)))

    p = tp/(predicted_p + K.epsilon())
    r = tp/(possible_p + K.epsilon())
    beta  = 1
    bb = beta**2
    fbeta_score = (1 + bb) * (p * r)/(bb * p + r + K.epsilon())

    return fbeta_score
def prec(y_true, y_pred):

    tp = K.sum(K.round(K.clip(y_true * y_pred, 0,1)))
    predicted_p = K.sum(K.round(K.clip(y_pred,0,1)))
    possible_p = K.sum(K.round(K.clip(y_true,0,1)))

    p = tp/(predicted_p + K.epsilon())
  

    return p
def recall(y_true, y_pred):

    tp = K.sum(K.round(K.clip(y_true * y_pred, 0,1)))
    predicted_p = K.sum(K.round(K.clip(y_pred,0,1)))
    possible_p = K.sum(K.round(K.clip(y_true,0,1)))

    p = tp/(predicted_p + K.epsilon())
    r = tp/(possible_p + K.epsilon())
 

    return r

def one_hot_encode_object_array(arr):
    '''One hot encode a numpy array of objects (e.g. strings)'''
    uniques, ids = np.unique(arr, return_inverse=True)
    return np_utils.to_categorical(ids, len(uniques))


seed = 7
np.random.seed(seed)

dataframe = pd.read_csv("iris.csv",header=None)
dataset = dataframe.values

X = dataset[:,0:4].astype(float)
Y = dataset[:,4]

train_X, test_X, train_y, test_y = train_test_split(X, Y, train_size=0.5, random_state=1)
#print X
Y_train = one_hot_encode_object_array(train_y)
Y_test = one_hot_encode_object_array(test_y)
#print Y_test


model = Sequential()
model.add(Dense(16, input_shape=(4,)))
model.add(Activation('sigmoid'))
model.add(Dense(3))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy',f1_score,prec, recall])
model.fit(train_X,Y_train, verbose=1, batch_size=1, nb_epoch=100)
score = model.evaluate(test_X, Y_test,  verbose =0)

print "score is "
print score

-----------------------------------------
we got accuracy of 100%


References
Tutorial link
Code improvement link

Keras: Epoch vs Batch_size

Information got from the link

In the neural network terminology:

• one epoch = one forward pass and one backward pass of all the training examples
• batch size = the number of training examples in one forward/backward pass. The higher the batch size, the more memory space you'll need.
• number of iterations = number of passes, each pass using [batch size] number of examples. To be clear, one pass = one forward pass + one backward pass (we do not count the forward pass and backward pass as two different passes).

Example: if you have 1000 training examples, and your batch size is 500, then it will take 2 iterations to complete 1 epoch.

Keras 2.0: Precision, recall

import keras.backend as K

def f1_score(y_true, y_pred):

    # Count positive samples.
    #c1 = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
    #c2 = K.sum(K.round(K.clip(y_pred, 0, 1)))
    #c3 = K.sum(K.round(K.clip(y_true, 0, 1)))

    true_positives = K.sum(K.round(K.clip(y_true * y_pred , 0, 1)))

   predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))
   possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
    p = true_positives / (predicted_positives + K.epsilon())

r = true_positives / (possible_positives + K.epsilon()) beta = 1 # fmeasure bb = beta**2 fbeta_score = (1 + bb) * (p * r) / (bb * p + r + K.epsilon())

      return fbeta_score 

model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy', f1_score])