Saturday, June 23, 2018

Useful Linux commands (CemtOS7)

These commands are for CentOS 7.

Apache error log

$ less /var/log/httpd/error_log

Nignx error log

$ less /var/log/nginx/error.log

See mysql/mariadb error log

$ less /var/log/mysql/error.log

php-fpm error log

$ less /var/log/php-fpm/www-error.log
And google the error message. Usually google knows how to deal with the error.

Edit Apache conf file

$ sudo vi /etc/httpd/conf/httpd.conf

Edit Nignx conf file

$ sudo vi /etc/nginx/conf.d/default.conf

Permission

Check permission
$ ls -la
Change permission
$ sudo chmod 755 /var/www/html/index.py
Make something executable
$ sudo chmod +x /var/www/html/index.py
Change owner
$ sudo chown apache:apache /var/www/html/index.py

SELinux

When SELinux is enabled, make something executable (recursively).
$ sudo chcon -R -t httpd_sys_script_exec_t /var/www/html/index.py
When SELinux is enabled, make something readable (recursively).
$ sudo chcon -R -t httpd_sys_content_t /var/www/html/index.py
Check context
$ ls -Z
Disable SELinux.
$ sudo setenforce 0
Enable SELinux.
$ sudo setenforce 1
Check SELinux status (enabled or disabled).
$ sudo getenforce

Returned value: Enforcing (enabled) / Permissive (disabled)

Directories

Change currecnt directory
$ cd /vagrant
Go up one directory
$ cd ..
Go down to a directory
$ cd ./some_folder
Check current directory
$ pwd

Symbolic link

Symbolic link is a kind of file that is similar to the shortcut in Windows. Create a symbolic link "python3.6" that refers to /bin/python_three:
$ sudo ln -s /bin/python3.6 /bin/python_three
Or:
$ sudo ln -s /usr/bin/python3.6 /usr/bin/python_three
And python_three can be used as another name of python3.6 command. 

Create symbolic link from apache's html to vagrant's shared folder. This can be used to sync the two folders (Both folders "html" and "vagrant" always have same contents inside after running these commands):
$ sudo rm -rf /var/www/html
$ sudo ln -fs /vagrant /var/www/html 
Delete a symbolic link
$ sudo unlink /bin/python_three
Or just remove the link like removing a file
$ sudo rm -f /bin/python_three 

Network

Check status of ports and established connections
$ sudo netstat -an 
Search port 80 from the result
$ sudo netstat -an | grep 80
("| grep something") can be used to search for any text data. For example,
$ sudo yum list
shows a list of packages. But if you use grep
$ sudo yum list | grep php
Stop the processes that are using port 3306 and 8080.
$ sudo kill -9 $(sudo lsof -t -i:3306)
$ sudo kill -9 $(sudo lsof -t -i:8080)

For Ubuntu

Upgrade Ubuntu major version.
$ sudo do-release-upgrade

MariaDB/Postgresql

Run SQL from sql file.
// postgresql
$ psql -U [postgres username] -d [db name] -a -f [path to the .sql file]
// mariadb $
sudo mysql -u [your user name of mariaDB] -p[your password of mariaDB] $ source [path to the .sql file]

Sunday, June 17, 2018

How to use Ansible with public key/private key

Prerequisites

  • I'm using windows8 and using virtual CentOS 7.3 in windows local as Ansible's host machine. If you don't have Linux or Mac, use virtual machine like I do. But in that case, don't forget to set the host's IP adress to 198.168.33.11 because duplicate IP causes an error.
  • As such, we will use two virtual machines in Windows.
    1. Control node (CentOS7.3, IP:192.168.33.10).
    2. Managed nodes (Hosts) (CentOS7.3, IP:192.169.33.11).
See here for virtual machines.

Install Ansible

At first, install python3.6 from IUS repository in Control node.
$ sudo rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
$ sudo yum install -y https://centos7.iuscommunity.org/ius-release.rpm
$ sudo yum install -y python36u python36u-libs python36u-devel python36u-pip
And install ansible.
$ python3.6 -m pip install ansible
$ sudo yum install -y sshpass 
And check if you have installed it successfully.
$ ansible --version
ansible 2.5.5
  config file = None
  configured module search path = ['/home/vagrant/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.6.5 (default, Feb  6 2018, 10:57:32) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
We installed ansible from pip. So we don't have /etc/ansible/ansible.cfg. You can create the cfg file by yourself. The default file can be found in their github. See their documentation too.

Add hosts information

Run the following to create hosts file in Control node:
$ sudo mkdir /etc/ansible/
$ sudo vi /etc/ansible/hosts
Write as follows:
[mynodes]
192.168.33.11
And save and close it by pressing :wq.
Also run this command too:
$ sudo vi /etc/ansible/ansible.cfg
Then write as follows:
[defaults]
host_key_checking = true #different from before
Then save and close it.

Get a private/public key

If you want a pair of private/public key, run the following to get the key pair in Control node:
$ cd ~/.ssh
$ ssh-keygen

Generating public/private rsa key pair.
Enter file in which to save the key (~/.ssh/id_rsa): # Just press enter key.
Enter passphrase (empty for no passphrase): # Just press enter key.
Enter same passphrase again: # Just press enter key.
Your identification has been saved in /home/vagrant/.ssh/id_rsa.
Your public key has been saved in /home/vagrant/.ssh/id_rsa.pub.
The key fingerprint is:
6f:9d:ed:39:06:75:70:9b:ff:c0:71:51:ff:00:8a:dc vagrant@localhost.localdomain
The key's randomart image is:
+--[ RSA 2048]----+
|            .   o|
|       . o . ...o|
|        o E   .o=|
|              o++|
|        S    o +o|
|         . ..oo .|
|          o o....|
|         .   .o..|
|             .o. |
+-----------------+

$ ls .ssh
And you will get these files: authorized_keys, id_rsa, id_rsa.pub, known_hosts... Check the key:
$ ls ~/.ssh
$ cat ~/.ssh/id_rsa.pub

#Copy this! 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5MZHya9h3LwYux7Va+hA73M0T6JnjpUoLw/0jk073x3JWOu5pIPhlA+zxOUTu5A+5PRkkQH9EYK7Y+OlnW45FZppnRtZeS38SAJxsaM0WvfQorRJKDeZdIvb8TLEJIiFA9rJo98/U1fmqGY3KBMUTnYalGTj1rLmenjUpVi9Z8BKlxAfX5xojN7ZyhcFbp9gZUKg3USUTz+q6UsAT8tq4ZqIPz2f52+RpPumSf+cVxuqv5o4dsBxGQfyRGHEEPb4QYWyjdn6EzdNbjec4iqZ6XYVDu1nar+pCLU8Bz0MJoCe38tae2GeuvJgW5cgKJ6ZWk1GqWaEgLaJpwZqbuHMD vagrant@localhost.localdomain
Copy the long random letters. And paste it in the Managed nodes.
$ sudo vi ~/.ssh/authorized_keys
Like this:

then save and close it by :wq. (Other than using copy and paste, you can use also "scp" command and transfer the id_rsa.pub file to the hosts machine. It's difficult but worth checking.)

Try the ansible command

We will try the ansible command. But please note that you can't use sudo command in this case. Because the public key was not for the root. The public key was created for a user "vagrant". If you use sudo, you will be recognized as "root". So the command will be rejected by the Managed nodes.
Run this command in Control node:
$ ansible 192.168.33.10 -m ping
If you see this, you successfully did everything:

Example

Save the following as "test.yaml" in the vagrant shared folder /vagrant in Control node:
- hosts: mynodes
  tasks:
    - name: Update yum
      become: yes
      become_method: sudo
      yum: 
        name: '*' 
        state: latest
    - name: Install apache
      become: yes
      become_method: sudo
      yum:
        pkg: httpd
        state: latest
        update_cache: true
Then run the following command:
$ ansible-playbook /vagrant/test.yaml -u vagrant
"u" option specifies which user should be used. For other options, look at this.
The developement environment will be automatically created in the hosts machine by ansible.

You can explicitly specify where the hosts file is:
$ ansible-playbook /vagrant/php.yaml -u vagrant -i /etc/ansible/hosts
If the hosts file is somewhere else.


Use module

This is an example of using file module:
- hosts: mynodes
  tasks:
  - name: Make a test file. Directory too if not exist.
    file: # File module here!! 
      path: /tmp/test
      owner: vagrant
      group: vagrant 
      state: directory
      mode: 0755
See here for the detail of File module. You can easily use modules in playbook.

Run the playbook.
$ ansible-playbook /vagrant/test2.yml -u vagrant -i /etc/ansible/hosts
Result:

You can see the directory /tmp/test is created in managed nodes (i.e. mynodes):

If you want to run the commands with super user priviledge:

- hosts: mynodes
  tasks:
  - name: Make a test file. Directory too if not exist.
    become: yes
    become_method: sudo
    file:
      path: /etc/test
      owner: vagrant
      group: vagrant 
      state: directory
      mode: 0755

Develop modules on your own

You can develop your module on your own. See the below page for the detail: https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html

Roles

Now we know how to write a playbook. But what if we make a lot of tasks? The playbook would be too long and impossible to reuse for future solutions. This is where we must use roles and separate the tasks into smaller files.
This is an example of a directory when we use Roles.
test.yml
roles/
   apache2/
     tasks/
The tasks directory s mandatory, but we can use the following directories too other than tasks:
test.yml
roles/
   apache2/
     files/
     templates/
     tasks/
     handlers/
     vars/
     defaults/
     meta/
  • tasks: contains the main list of tasks to be executed by the role.
  • handlers: contains handlers, which may be used by this role or even anywhere outside this role.
  • defaults: default variables for the role (see Using Variables for more information).
  • vars: other variables for the role (see Using Variables for more information).
  • files: contains files which can be deployed via this role.
  • templates: contains templates which can be deployed via this role.
  • meta: defines some meta data for this role. See below for more details.
cited from ansible page.
We will use only tasks in this post.

Example

At first, create roles and apache directory and create necessary files in Control node:
$ mkdir -p /vagrant/roles/apache/tasks
$ touch /vagrant/test3.yml
$ touch /vagrant/roles/apache/tasks/main.yml
$ touch /vagrant/roles/apache/tasks/debian.yml
$ touch /vagrant/roles/apache/tasks/redhat.yml
Then write as follows respectively:
test3.yml
- hosts: mynodes
  become: yes
  become_method: sudo
  remote_user: "vagrant"
  roles:
    - apache
main.yml
# you can have any generic tasks directly in main.yml
- name: get the date
  shell: "date"
  register: date

# you can directly write non-generic tasks here too.
#- name: Install apache2 (RedHat).
#  yum: name=httpd
#  when: "ansible_os_family == 'RedHat'"
 
#- name: Install apache2 (Debian).
#  apt: name=apache2
#  when: "ansible_os_family == 'Debian'"

# or include files
- include: debian.yml
  when: ansible_os_family == 'Debian'

- include: redhat.yml
  when: ansible_os_family == 'RedHat'

debian.yml
- name: install apache
  apt:
    state: latest
    update_cache: true
    name:
      - httpd
redhat.yml
- name: install apache
  yum:
    state: latest
    update_cache: true
    name:
      - httpd

Run the example

Run the example:
$ ansible-playbook /vagrant/test3.yml -u vagrant -i /etc/ansible/hosts
Result:









Saturday, June 16, 2018

How to use Docker toolbox

Prerequisites

  • I'm using Windows8.

Install Docker toolbox for Windows

Get an installer for Docker toolbox from here: https://docs.docker.com/toolbox/toolbox_install_windows/



Install Docker toolbox with the installer. Please don't forget to tick git for Windows if you don't have it. If you don't know what it is, tick it. If you already have virtualbox, you can untick virtualbox. ##Use Docker toolbox Double-click on the shortcut icon of Docker quick start terminal:


If you run it for the first time, it creates a virtual machine used for Docker engine on the Virtualbox. After that, you will login to the virtual machine.


This message means a virtual machine called "default" is ready at IP:192.168.99.100. The IP address varies depending on the environment. From this virtual machine, you use Docker. To check if it really works, run this command on it:
$ docker run hello-world
If you see this, you successfully did it:
docker run hello-world

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
ca4f61b1923c: Pull complete
Digest: sha256:ca0eeb6fb05351dfc8759c20733c91def84cb8007aa89a5bf606bc8b315b9fc7
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.
...

Commands of Docker

To create a virtual machine called "dev":
$ docker-machine create --driver virtualbox dev
To check all machines and their status:
$ docker-machine ls
To stop the virtual machine called dev:
$ docker-machine stop dev
To start the virtual machine called dev:
$ docker-machine start dev
To remove the virtual machine called dev:
$ docker-machine rm dev

Access the Docker via SSH connection

Check the IP to connect on the Docker toolbox:
$ docker-machine ip
192.168.99.100
You can use this IP to connect from Putty via SSH connection. The user name is "docker" the password is "tcuser".


Shared folder

"C:\Users" in host windows machine is shared to "/c/Users" in guest machine by default.


Stop the Docker's machine

You can stop the default machine by
$ docker-machine stop default