Wednesday, November 29, 2017

cakephp3, use contain() and filter the result by associated table's field

There is contain() method in cakephp. This method is useful and is often used. But if you are using hasMany or belongsToMany in the table files, and if you try to filter the result in accordance with the associated field in where, you get failed.

$this->find() // Suppose this fetches records from Articles
    ->contain(['Comments'])  // Suppose Articles has many Comments
    ->where(['Comments.type' => 1]); //Error!


Articles Table
idname
1test article

Comments Table
idnamearticle_id
1comment1
2comment1
3comment1

But if the relationship is hasOne or belongsTo, you do not get any error. You can filter the result by associated table's field.

Articles Table
idnamecomment_id
1test article1

Comments Table
idname
1comment
2comment
3comment

This is because, if the tables are hasMany or belongsToMany, separate SQL statements are executed as follows:

# Comments doesn't exist in the first SQL statement,
# so you can not specify fields of Comments to filter Articles.
SELECT * FROM articles;
SELECT * FROM comments WHERE article_id IN (1, 2, 3, 4, 5); 

If you want to filter the Articles records in accordance with Comments records, use a closure to give conditions to the other SQL statement:

$query = $articles->find()->contain([
    'Comments' => function (\Cake\ORM\Query $q) {
       return $q->where(['Comments.type' => 1]);
    }
]);

So that you will get records of Articles where Comments.type is 1.
(You can also use matching() instead of contain().)

document of cakephp about this


Sunday, November 26, 2017

Use find('list')

Contents

CakePHP
1. Setting up CakePHP3 with CentOS

2. Bake and Migration


4. Add a calendar page

5. Use "contain"

6. Use find('list')

find('list')


In cakephp3, find('list') is also very useful. It is used like this:
// In a controller or table method.
$products = $this->Products; //Or use table registry.
$query = $products->find('list');
$data = $query->toArray();

// Data now looks like:
// id of the record => "name" field of the table.
$data = [
    1 => 'product name 1',
    2 => 'product name 2',
];

The array data can be used as options of a HTML drop down select box:
<?php echo $this->Form->input('products', ['type'=>'select', 'options'=>$data]); ?>

If you want other field than 'name' for the value, you can specify what field should be the value:
// In a controller or table method.
$query = $articles->find('list', [
    'keyField' => 'id',
    'valueField' => 'description'
]);
$data = $query->toArray();

// Data now looks like
$data = [
    '1' => 'desctiption 1',
    '2' => 'desctiption 2',
];

Relevant cakephp cookbook:
https://book.cakephp.org/3.0/en/orm/retrieving-data-and-resultsets.html#finding-key-value-pairs

Sunday, November 19, 2017

How to start Unity programming with C#

In this post, suppose we are using Windows OS (8.1).

1. Install Unity


Download Unity from here: https://store.unity.com/
Personal version is free, so choose the personal version. When you get the installer, start installing Unity.

2. Open Unity And Add A Cube


You will see the Unity's window:


Right click on the "Untiled*" pane. Select cube from the menu.

Cube is added in the scene.


Now click "Main Camera" from the left pane. Then click "Inspector" from the right side.

Change the position from the Inspector. I changed it to "x:0, y:1, z:-3". It will change the default position of the Main Camera.

Now we will check if the position was really changed. Click the play button from the upper menu.


You can see the Main Camera's position is right in front of the cube, which is same as the position that we wrote in the inspector.

Now exit from the play mode by pressing the play button.


3. Apply Gravity To The Object


Click the cube and click "add component".

Click "Physics".

Then click "Rigid body".

Rigid body component is added to the cube.

Change the position (to make the cube float in the air).
Position: x0, y3, z0.

Run the code by pressing the play button.

The cube will fall by the gravity (and fall forever because there is no ground).

Now add plane under the cube and set the position of the plane x:0, y:0, z:0.

Run the code by pressing the play button. The cube will fall on the plane now.

To apply gravity and make objects collide with each other:

Falling objects need the following:
    1.Collider
    2. Rigidbody
Ground that receives the falling objects needs the following:
    1. Collider that fits the form of the ground.

You can find the collider in the inspector of the cube and the plane. Box and Plane have the collider by default, so we didn't add collider manually this time.
"Box collider" is the collider



4. Add Prefab


Drag and drop the cube object to Assets. You get a prefab of cube on the Assets view.

Or you can create prefab from the Assets view too. Right click on the Assets and click "Create" > "Prefab".

We get a Prefab on the Assets view. Drag and drop the game object on the Prefab.

Anyway, after creating the prefab of the cube, create an empty game object.
Right click on the left pane > "Create Empty"

Then you get an empty game object.

 And click the empty game object and from its inspector, click "add component".

Scroll down the menu and click "New Script" and name it "CubeControl". Then click "Create and Add".

Maybe automatically MonoDevelop or VisualStudio or both of them will be opened. If not opened, just double-click the script from the Assets view:


As you see on the code, Start() function is used for initialization. Update() is called per frame.
using System.Collections; using System.Collections.Generic; using UnityEngine; public class CubeControl : MonoBehaviour { // Use this for initialization void Start () { } // Update is called once per frame void Update () { } }

Change the script like this:
using System.Collections; using System.Collections.Generic; using UnityEngine; public class CubeControl: MonoBehaviour { public GameObject prefab; // Use this for initialization void Start () { } // Update is called once per frame void Update () { if (Input.GetKey(KeyCode.Z)) { GameObject.Instantiate(prefab); } } }

Then see again the Unity view and click the empty game cube. From the inspector of the empty game object, you can see that a "prefab" item is created.


Drag and drop the prefab of the cube to the "prefab" item of the CubeControl of the empty game object.


You will see that the cube prefab is attached to the CubeControl as its prefab.


Now look at the script again... According to the script, as long as you press "Z", the prefab is instantiated.
using System.Collections; using System.Collections.Generic; using UnityEngine; public class CubeControl: MonoBehaviour { public GameObject prefab; // Use this for initialization void Start () { } // Update is called once per frame void Update () { if (Input.GetKey(KeyCode.Z)) { GameObject.Instantiate(prefab); } } }

Now check if it really works.

As long as you press "z" key on the keyboard... the cube prefab is instantiated..!


If you change "GetKey" to "GetKeyDown", the cube prefab is instantiated "every time" you press z on the keyboard.
using System.Collections; using System.Collections.Generic; using UnityEngine; public class CubeControl: MonoBehaviour { public GameObject prefab; // Use this for initialization void Start () { } // Update is called once per frame void Update () { if (Input.GetKeyDown(KeyCode.Z))
{ GameObject.Instantiate(prefab); } } }

5. Move the main camera


Now create a script for the main camera. Click on "Add Component" of the main camera and add "New Script". The name is "CameraControl".




Open the script and change the code like this:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class CameraControl : MonoBehaviour {
// Use this for initialization
void Start () {
     
    }

// Update is called once per frame
void Update () {
        if (Input.GetKey(KeyCode.UpArrow))
        {
            this.transform.Translate(Vector3.forward * Time.deltaTime * 2);
        }
        if (Input.GetKey(KeyCode.RightArrow))
        {
this.transform.Rotate(Vector3.up, Time.deltaTime*50);
        }
        if (Input.GetKey(KeyCode.LeftArrow))
        {
this.transform.Rotate(Vector3.up, -Time.deltaTime*50);
        }
        if (Input.GetKey(KeyCode.DownArrow))
        {
            this.transform.Translate( -1 * Vector3.forward * Time.deltaTime * 2);
        }
    }
}

And run the project:

Your main camera moves (by pressing up arrow, down arrow, right arrow, left arrow)!


Saturday, November 18, 2017

Improved version of dog vs cat of Keras

I improved a little using adam and selu. I am not sure if it actually is better, but seems like it is working better. Original version of dog vs cat is here.


from __future__ import print_function

import numpy as np
import keras
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras.layers.noise import AlphaDropout
from keras.preprocessing.image import ImageDataGenerator
from keras.models import model_from_json
from keras.layers import Dense, Dropout, Activation
from keras.preprocessing.text import Tokenizer
import keras.backend.tensorflow_backend as KTF
import tensorflow as tf
import os.path

f_log = './log'
f_model = './model/dogvscat'
model_yaml = 'dogvscat_model.yaml'
model_filename = 'dogvscat_model.json'
weights_filename = 'dogvscat_model_weights.hdf5'

batch_size = 200
epochs = 2
nb_validation_samples = 25000

print('Building model...')


if os.path.isfile(os.path.join(f_model,model_filename)):
    print('Saved parameters found. I will use this file...')
    json_string = open(os.path.join(f_model, model_filename)).read()
    model = model_from_json(json_string)
    model.summary()
    model.compile(loss='categorical_crossentropy',
                optimizer='adam',
                metrics=['accuracy'])
    model.load_weights(os.path.join(f_model,weights_filename))
else:
    print('Saved parameters Not found. Creating new model...')
    model = Sequential()
    model.add(Conv2D(32, (3, 3), input_shape=(128, 128, 3)))
    model.add(Activation('selu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))

    model.add(Conv2D(64, (3, 3)))
    model.add(Activation('selu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))

    model.add(Flatten())
    model.add(Dense(64, kernel_initializer='lecun_normal'))
    model.add(Activation('selu'))
    model.add(AlphaDropout(0.1))
    model.add(Dense(2))
    model.add(Activation('softmax'))

    model.summary()

    model.compile(loss='categorical_crossentropy',
                optimizer='adam',
                metrics=['accuracy'])

train_datagen = ImageDataGenerator(
    rescale=1.0 / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1.0 / 255)

train_generator = train_datagen.flow_from_directory(
    'data/train',
    target_size=(128, 128),
    batch_size=batch_size,
    class_mode='categorical',
    shuffle=True)

validation_generator = test_datagen.flow_from_directory(
    'data/validation',
    target_size=(128, 128),
    batch_size=batch_size,
    class_mode='categorical',
    shuffle=True)

tb_cb = keras.callbacks.TensorBoard(log_dir=f_log, histogram_freq=0)
cp_cb = keras.callbacks.ModelCheckpoint(filepath = os.path.join(f_model,weights_filename), monitor='val_loss', verbose=1, save_best_only=True, mode='auto')
cbks = [tb_cb, cp_cb]

history = model.fit_generator(
    train_generator,
    steps_per_epoch=np.ceil(nb_validation_samples/batch_size),
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=np.ceil(nb_validation_samples/batch_size),
    callbacks=cbks
    )

score = model.evaluate_generator(validation_generator, nb_validation_samples)

print('')
print('Test score:', score[0])
print('Test accuracy:', score[1])

json_string = model.to_json()
open(os.path.join(f_model,model_filename), 'w').write(json_string)
yaml_string = model.to_yaml()
open(os.path.join(f_model,model_yaml), 'w').write(yaml_string)
print('save weights')
model.save_weights(os.path.join(f_model,weights_filename))

Tuesday, November 14, 2017

Install Keras on Linux


At first, install your python3.6.
$ sudo apt-get update
$ sudo apt-get install python3.6

Now pip3.
$ sudo apt-get install  python3-pip

Update pip3.
$ python3.6 -m pip install -U pip

Install setuptools.
$ python3.6 -m pip install setuptools

Install tensorflow.
If your PC doesn't have GPU:
$ python3.6 -m pip install tensorflow
If your PC has nvidia GPU:
$ python3.6 -m pip install tensorflow-gpu
(If your PC has nvidia GPU, you need also cuda. get the deb file from here: https://developer.nvidia.com/cuda-80-ga2-download-archive
and install cuda8.0 like $ sudo apt-get install cuda-8.0)

Install keras.
$ python3.6 -m pip install keras

Some basic packages for python3.6.
$ python3.6 -m pip install h5py
$ python3.6 -m pip install matplotlib
$ python3.6 -m pip install pillow
$ python3.6 -m pip install pandas

Use python to run keras programs.
Python3.6
$ python3.6 xxx.py

Sunday, November 12, 2017

Installation of Linux mint

I am using windows 8.1 for the OS. In this post, I will explain how to install Linux mint to PC.

1. Go to the official website of Linux mint.

2. Download the ISO file of Linux mint (the latest version is Cinnamon as of November 2017).

3. Prepare a DVD-R.

4. Download and install ImgBurn use it to burn the ISO of Linux mint to the DVD-R. This burnt DVD-R is called a live CD.

5. Wait until burning the ISO is finished. When it is finished, you hear weird music.

6. Make sure your PC is connected with LAN cable and is connected to the power supply.
Restart your PC and, before Windows logo is shown, press F2 many times to go to BIOS setup. (Depending on the manufacturer of your PC, the key to go to BIOS setup might be different. If pressing F2 doesn't work, please check what key you need to press.)

7. Go to the "Boot" section. Press F5/F6 to change the priority. Make the "CD-ROM drive" highest.


8. Restart the PC. Linux mint in the live CD will be started. If you want to install Linux mint, click the install icon on the desktop.

9. After installing Linux mint, change the priority on the BIOS setup and make "Hard drive" highest.


And it's done! :)

Use Keras on Windows

Install Python3 (64bit)


At first, we need to install Python3 to use Keras. Keras can be used with Python3 64 bit version, not with 32 bit version. So please make sure your PC is 64 bit and install Python3 64 bit version.

Access to the Python's website:
https://www.python.org/

Python's website

Select Downloads -> WIndows

Choose "Windows x86-64 web-based installer" of  python 3.6.
(Because, as of November 2017, tensorflow supports only 3.5 and 3.6.)

Run the installer. Make sure to check "Add Python to PATH"

Now follow the instruction of the installer and install the Python3 for 64bit version.

After the installation, on the command prompt, check if the installation went successfully.
python -V

Python 3.6.3

If the version is displayed, the installation is success. If not, it's fail. We need to re-try to install Python3 (maybe, in that case, python is not added to PATH).

Install Keras 


Run the command prompt as administrator (just in case).

If you install Python3, you can use pip also on the command prompt.
pip -V

pip 9.0.1 from c:\users\jack\appdata\local\programs\python\python36\lib\site-pack
ages (python 3.6)

Now we will install Keras using pip. Run the following command:
pip install Keras

Install Tesorflow:
(if your PC doesn't have GPU):
pip install --upgrade tensorflow
(if your PC has GPU):
pip install --upgrade tensorflow-gpu

Other packages too:
pip install matplotlib
pip install h5py
pip install pillow

Create a text file and rename it as "test.py". (If the extension isn't shown, change the setting to show the extension at first)

Copy and paste the following in the test.py:
#!/usr/bin/env python
# -*- coding: UTF-8 -*-

from keras.datasets import mnist
from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.utils import np_utils

# Get the default MNIST data
(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(60000, 784) / 255
X_test = X_test.reshape(10000, 784) / 255

y_train = np_utils.to_categorical(y_train)
y_test = np_utils.to_categorical(y_test)

# Network
model = Sequential([
        Dense(512, input_shape=(784,)),
        Activation('sigmoid'),
        Dense(10),
        Activation('softmax')
    ])

# Compile
model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])

# Learning
model.fit(X_train, y_train, batch_size=200, verbose=1, epochs=20, validation_split=0.1)

# Forecast
score = model.evaluate(X_test, y_test, verbose=1)
print('test accuracy : ', score[1])

Then save it.

Go to the directory on command prompt. For example:
cd C:\Users\Jack\Documents\workplace\New folder

Run the test.py:
python test.py

If the machine learning of MNIST started, the set up is success!!

Now you can create & run programs of Keras with your Windows PC!!









Saturday, November 11, 2017

How to use DL4J for Windows

I am using Windows8.1 for my OS. The official quick start guide is here.
Suppose we are using Windows 8.1.

JDK


At first, we will install JDK. Make sure you will install (or have installed already?) Java8. As of November 2017, DL4J doesn't work under Java9.
Install from here: http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html


Then set the path for Java home. Open the control panel and choose the "System and Security" --> "System".
Click the Advanced tab and the button Environment Variables.



Click New... at the System Variables or User variables.
Use JDK_HOME as the variable name and "C:\Program Files\Java\(your jdk folder)" as the path.

If you have installed wrong version of Java (like java9) and want to change the version, download the right version of Java and change the path of JAVA_HOME to the new JDK's path. If this doesn't work and if you are using windows8, check if there is "C:\ProgramData\Oracle\Java\javapath" in the "Path" in System Variables. If there is, delete "C:\ProgramData\Oracle\Java\javapath" from the path. Otherwise the Java version doesn't change on the command prompt.

Check if JDK is installed successfully by running a command "java -version" on command prompt.
java -version

java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)

Maven


Then install Maven.
Visit the official site of Maven: http://maven.apache.org/download.cgi
And install Maven (like "apache-maven-3.5.2-bin.zip") from there.

After the installation, add both M2_HOME and MAVEN_HOME variables in the Windows environment, and point it to your Maven folder.


Maven is a project management tool, which is aimed for Java projects.
Available commands of Maven:
install
clean
compile
package
validate
test-compile
test
integration-test
verify
deploy

Check if Mave is installed successfully by running the following command.
mvn -version

Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T16:58:1
3+09:00)
Maven home: C:\Program Files\Apache\maven\bin\..
Java version: 1.8.0_151, vendor: Oracle Corporation
Java home: C:\Program Files\Java\jdk1_8\jre
Default locale: en_GB, platform encoding: MS932
OS name: "windows 8.1", version: "6.3", arch: "amd64", family: "windows"
'cmd' is not recognized as an internal or external command,
operable program or batch file.

Git


Download the latest Git for Windows installer from here: https://git-for-windows.github.io/
Then install git with the installer.

After the installation, check if the git works correctly on the command prompt.
git --version

git version 2.7.4


Install DL4J


Create a workplace folder and name it as "dl4j".

Then move to the directory on the command prompt.
cd C:\Users\Jack\Documents\dl4j

Install dl4j from git.
git clone https://github.com/deeplearning4j/dl4j-examples.git
cd dl4j-examples/
mvn clean install

Intellij


Install free community version of Intellij from here: https://www.jetbrains.com/idea/download/#section=windows

Open intellij and choose "import project".


Choose the folder of dl4j-examples which we downloaded just now and click "OK".


Choose "Maven" and click "Next".


Just click "Next" without touching anything.

Tick "open jfx" and click "Next".


Tick the URL of dl4j and click "Next".



Tick the SDK and click "Next".


Click "Finish".


Wait until the loading is finished.


Choose an example which you want to run and click on it to open the source. Then right-click on the class name. A menu will be shown. Click "Run" from the menu. Then the deep learning will be started.

I will try LenetMnistExample:

The result of LenetMnistExample:
==========================Scores========================================
 # of classes:    10
 Accuracy:        0.9746
 Precision:       0.9748
 Recall:          0.9745
 F1 Score:        0.9744
Precision, recall & F1: macro-averaged (equally weighted avg. of 10 classes)
========================================================================
o.d.e.c.LenetMnistExample - ****************Example finished********************