Keras: Hyperparameter optimization example 1

import numpy
from sklearn.model_selection import GridSearchCV
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasClassifier
import pandas as pd
import numpy as np
import tensorflow as tf
from sklearn.cross_validation import train_test_split

from keras.utils import np_utils
def one_hot_encode_object_array(arr):
    '''One hot encode a numpy array of objects (e.g. strings)'''
    uniques, ids = np.unique(arr, return_inverse=True)
    return np_utils.to_categorical(ids, len(uniques))
def create_model():
    model = Sequential()
    model.add(Dense(12, input_dim=4, activation='relu'))
    model.add(Dense(3, activation='sigmoid'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model

seed = 7
numpy.random.seed(seed)
dataframe = pd.read_csv("iris.csv",header=None)
dataset = dataframe.values

X = dataset[:,0:4].astype(float)
Y = dataset[:,4]

train_X, test_X, train_y, test_y = train_test_split(X, Y, train_size=0.5, random_state=1)
#print X
Y_train = one_hot_encode_object_array(train_y)
Y_test = one_hot_encode_object_array(test_y)
#Y = one_hot_encode_object_array(Y1)
#print Y
model = KerasClassifier(build_fn=create_model, verbose=0)
batch_size = [10,20,40]
epochs =[10,50,100]
param_grid= dict(batch_size=batch_size, epochs=epochs)
grid = GridSearchCV(estimator=model, param_grid=param_grid)
grid_result=grid.fit(train_X,Y_train)

print("Best:%f using %s" %(grid_result.best_score_, grid_result.best_params_))
means = grid_result.cv_results_['mean_test_score']
stds = grid_result.cv_results_['std_test_score']
params = grid_result.cv_results_['params']
for mean, stdev, param in zip(means, stds,params):
    print("%f (%f) with: %r" % (mean, stdev, param))


-----------------------------------------------------------------------------
Results
Best:0.826667 using {'epochs': 100, 'batch_size': 10}
0.266667 (0.018856) with: {'epochs': 10, 'batch_size': 10}
0.640000 (0.198662) with: {'epochs': 50, 'batch_size': 10}
0.826667 (0.099778) with: {'epochs': 100, 'batch_size': 10}
0.413333 (0.049889) with: {'epochs': 10, 'batch_size': 20}
0.506667 (0.082192) with: {'epochs': 50, 'batch_size': 20}
0.773333 (0.147271) with: {'epochs': 100, 'batch_size': 20}
0.266667 (0.018856) with: {'epochs': 10, 'batch_size': 40}
0.320000 (0.299333) with: {'epochs': 50, 'batch_size': 40}
0.746667 (0.131993) with: {'epochs': 100, 'batch_size': 40}
 

Keras: Optimizing number of hidden layers

link to a helpful tutorial

http://machinelearningmastery.com/exploratory-configuration-multilayer-perceptron-network-time-series-forecasting/

hadoop: Container is running beyond virtual memory limits

From the error message, you can see that you're using more virtual memory than your current limit of 1.0gb. This can be resolved in two ways:

Disable Virtual Memory Limit Checking

YARN will simply ignore the limit; in order to do this, add this to your yarn-site.xml:

<property>
  <name>yarn.nodemanager.vmem-check-enabled</name>
  <value>false</value>
  <description>Whether virtual memory limits will be enforced for containers.</description>
</property>

The default for this setting is true.

Increase Virtual Memory to Physical Memory Ratio

In your yarn-site.xml change this to a higher value than is currently set

<property>
  <name>yarn.nodemanager.vmem-pmem-ratio</name>
  <value>5</value>
  <description>Ratio between virtual memory to physical memory when setting memory limits for containers. Container allocations are expressed in terms of physical memory, and virtual memory usage is allowed to exceed this allocation by this ratio.</description>
</property>

The default is 2.1

You could also increase the amount of physical memory you allocate to a container.

Make sure you don't forget to restart yarn after you change the config.

Reference to the original link

hadoop: hadoop auxservice: mapreduce_shuffle does not exist

Please use this in yarn-site.xml; when you set the framework to use as yarn, it starts to look for these values.

<configuration>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
</configuration>

Keras 2: Dense call update

init in api 1 is converted to kernel_initializer

Keras: saving models for future use

http://machinelearningmastery.com/save-load-keras-deep-learning-models/

python: read all files in the directory and copy the text in one file

from os import listdir
from os.path import isfile, join
mypath ="./puzzles/p"
onlyfiles = [f for f in listdir(mypath) if isfile(join(mypath,f))]
#text = []
for file in onlyfiles:
    with open(mypath + "/" + file,'r') as f:
        text = f.readlines()
        #text = [l for l in text if "ROW" in l]
        with open("out.txt","a") as f1:
            f1.writelines(text)

Keras: installation with Tensorflow and opencv

$ mkvirtualenv keras_tf
$ workon keras_tf

$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.12.1-cp27-none-linux_x86_64.whl

#$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-0.12.0rc2-py2-none-any.whl
wheel name should come from
linklink

$ pip install --upgrade $TF_BINARY_URL

$ pip install numpy scipy

$ pip install scikit-learn

$ pip install pillow

$ pip install h5py

$ pip install keras Before we get too far we should check the contents of our keras.json  configuration file. You can find this file in ~/.keras/keras.json . $gedit ~/.keras/keras.json . add  "image_dim_ordering": "tf" in the file and file contents should look lik {     "image_dim_ordering": "tf",     "epsilon": 1e-07,     "floatx": "float32",     "backend": "tensorflow" } You might be wondering what exactly image_dim_ordering  controls. Using TensorFlow, images are represented as NumPy arrays with the shape (height, width, depth), where the depth is the number of channels in the image. However, if you are using Theano, images are instead assumed to be represented as (depth, height, width). Find CV2.so $ cd / $ sudo find . -name '*cv2.so*' ./Users/adrianrosebrock/.virtualenvs/cv/lib/python2.7/site-packages/cv2.so ./Users/adrianrosebrock/.virtualenvs/gurus/lib/python2.7/site-packages/cv2.so ./Users/adrianrosebrock/.virtualenvs/keras_th/lib/python2.7/site-packages/cv2.so ./usr/local/lib/python2.7/site-packages/cv2.so and copy that to virtual environment $ cd ~/.virtualenvs/keras_tf/lib/python2.7/site-packages/ $ ln -s /usr/local/lib/python2.7/site-packages/cv2.so cv2.so $ cd ~ ------------------------------------------------ References Content taken from http://www.pyimagesearch.com/2016/11/14/installing-keras-with-tensorflow-backend/