feed.xml

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <title>Hello, World!</title>
    <link href="http://terryoy.github.io/feed.xml" rel="self" />
    <link href="http://terryoy.github.io/" />
    <updated>2020-02-01T10:09:37+08:00</updated>
    <id>http://terryoy.github.io/</id>
    <entry>
        <title type="html"><![CDATA[How to Write a Shell Script?]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2019/07/how-to-write-a-shell-script-.html"/>
        <published>2019-07-01T18:11:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2019/07/how-to-write-a-shell-script-.html</id>
        <category scheme="http://terryoy.github.io/tag/#shell" term="shell" label="shell" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>Finnally, I&#39;ve done it! Wow, it took me 3 hours to explore the ability of bash scripts.</p>
<p>I was trying to write a shell script to help me create this blog article, in an interactive way. </p>
<p>The blog articles are organized in a directory structure: <code>_content/{year}/{article-name-slug.md}</code>,  and it also has a template to write the meta info for this article, which is like:</p>

<pre><code># How to Write a Shell Script?
- date: 2019-07-01 18:11
- tags: shell
- category: guides
-------</code></pre>
<p>So the basic idea is like this:</p>

<ul>
<li>I want to create the page where I could simply input the title, tags, and category</li>
<li>the date time is generated by the script</li>
<li>the file path is determined by the information I input and the date</li>
<li>the file name is slugified</li>
<li>all the information is filled in the template</li>
<li>use vim to open the file and filled the initial content with the template</li>
</ul>
<p>All the functions I needed and searched are listed below:</p>

<ul>
<li>prompt info and read values from user input. using <code>echo</code> and <code>read</code> command</li>
<li>list the tags and categories. using <code>ls -p</code>, <code>tr</code>, <code>sort</code>, and <code>column</code></li>
<li>making the steps as functions</li>
<li>fill template file with environmental variables. using <code>cat</code>, <code>echo</code> and <code>eval</code></li>
<li>creatign temporary file to hold the generated content from the template. using <code>$RANDOM</code> env variable for a random number.</li>
<li>fill the vi editing content after entering the editor. using <code>vi &lt;filename&gt; -c &quot;read &lt;tmpfile&gt;&quot;</code></li>
</ul>
<p>So the final example of the shell script can be found <a href="https://github.com/terryoy/terryoy.github.io/blob/master/_newpage.sh">here</a></p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Setup ARM Development Environment with QEMU]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2018/04/setup_arm_development.html"/>
        <published>2018-04-21T23:26:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2018/04/setup_arm_development.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <category scheme="http://terryoy.github.io/tag/#qemu" term="qemu" label="qemu" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <h3 id="toc_0">1. Clone the kernel repository</h3>
<div class="highlight"><pre><span></span>$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

<span class="c1"># or just download the package from the mirror</span>
$ wget https://mirror.tuna.tsinghua.edu.cn/kernel/v4.x/linux-4.4.1.tar.xz
</pre></div>

<h3 id="toc_1">2. Install the Cross Compiling Toolchain</h3>
<div class="highlight"><pre><span></span>$ sudo apt-get install gcc-arm-linux-gnueabi

<span class="c1"># another option is &quot;arm-linux-gnueabihf&quot; but we use &quot;gcc-arm-gnueabi&quot; here</span>
</pre></div>

<h3 id="toc_2">3. Make the first compile</h3>
<p>We&#39;re emulating the vexpress Cortex A9 for demo.</p>
<div class="highlight"><pre><span></span>$ <span class="nb">export</span> <span class="nv">ARCH</span><span class="o">=</span>arm  
$ <span class="nb">export</span> <span class="nv">CROSS_COMPILE</span><span class="o">=</span>arm-linux-gnueabi-  
$ make vexpress_defconfig  
$ make zImage -j8  
$ make modules -j8  
$ make dtbs
</pre></div>

<h3 id="toc_3">4. Install QEMU</h3>
<div class="highlight"><pre><span></span>$ sudo apt-get install qemu
</pre></div>

<h3 id="toc_4">5. Make Root File System</h3>

<h3 id="toc_5">5.1 busybox</h3>
<div class="highlight"><pre><span></span>$ wget http://www.busybox.net/downloads/busybox-1.25.1.tar.bz2  
$ tar xvf busybox-1.25.1.tar.bz2
$ make defconfig  
$ make <span class="nv">CROSS_COMPILE</span><span class="o">=</span>arm-linux-gnueabi-  
$ make install <span class="nv">CROSS_COMPILE</span><span class="o">=</span>arm-linux-gnueabi-
</pre></div>
<p>Then you find the executables in <code>_install</code> folder. Next we start to create the rootfs.</p>

<h3 id="toc_6">5.2 rootfs</h3>
<div class="highlight"><pre><span></span>$ sudo mkdir rootfs
$ sudo mkdir rootfs/lib 

<span class="c1"># copy busybox to rootfs</span>
$ sudo cp _install/* -r rootfs/

<span class="c1"># copy arm libs to lib</span>
<span class="c1"># sudo cp -P /usr/arm-linux-gnueabi/lib/* rootfs/lib/</span>
</pre></div>

<h3 id="toc_7">5.3 Create 4 tty devices</h3>
<div class="highlight"><pre><span></span>$ sudo mkdir -p rootfs/dev
$ sudo mknod rootfs/dev/tty1 c <span class="m">4</span> <span class="m">1</span>  
$ sudo mknod rootfs/dev/tty2 c <span class="m">4</span> <span class="m">2</span>  
$ sudo mknod rootfs/dev/tty3 c <span class="m">4</span> <span class="m">3</span>  
$ sudo mknod rootfs/dev/tty4 c <span class="m">4</span> <span class="m">4</span>
</pre></div>
<p>You can also create other folders in the image. (Reference: <a href="https://learningfromyoublog.wordpress.com/2016/04/05/131/">learningfromyoublog.wordpress.com/2016/04/05/131/</a>)</p>

<h3 id="toc_8">5.4 Make the Image</h3>
<div class="highlight"><pre><span></span><span class="c1"># creat an empty image</span>
$ dd <span class="k">if</span><span class="o">=</span>/dev/zero <span class="nv">of</span><span class="o">=</span>a9rootfs.ext3 <span class="nv">bs</span><span class="o">=</span>1M <span class="nv">count</span><span class="o">=</span><span class="m">32</span>  

<span class="c1"># format to ext3</span>
$ mkfs.ext3 a9rootfs.ext3  

<span class="c1"># copy files into the image</span>
$ sudo mkdir tmpfs  
$ sudo mount -t ext3 a9rootfs.ext3 tmpfs/ -o loop  
$ sudo cp -r rootfs/*  tmpfs/  
$ sudo umount tmpfs
</pre></div>

<h3 id="toc_9">6. Start QEMU with the Image</h3>
<div class="highlight"><pre><span></span><span class="c1"># open in current console</span>
$ qemu-system-arm -M vexpress-a9 -m 512M -dtb extra_folder/vexpress-v2p-ca9.dtb -kernel extra_folder/zImage -nographic -append <span class="s2">&quot;root=/dev/mmcblk0 rw console=ttyAMA0&quot;</span> -sd a9rootfs.ext3 

<span class="c1"># open in new window(maybe GUI)</span>
$ qemu-system-arm -M vexpress-a9 -m 512M -dtb extra_folder/vexpress-v2p-ca9.dtb -kernel extra_folder/zImage -append <span class="s2">&quot;root=/dev/mmcblk0 rw&quot;</span> -sd a9rootfs.ext3
</pre></div>
<p>Now you enter the console of the emulator system. Press <code>Ctrl+A C</code> to exit to <code>(qemu)</code> console, or <code>Ctrl+A X</code> to exit..</p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Learn to Use CMake to Compile Code]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2018/01/learn-to-use-cmake.html"/>
        <published>2018-01-12T14:41:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2018/01/learn-to-use-cmake.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <category scheme="http://terryoy.github.io/tag/#programming" term="programming" label="programming" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>I&#39;m trying to use the Linux environment to develop C/C++ programs, but never really get on the path.</p>
<p>It&#39;s not the language that I don&#39;t understand, but the environment, the toolchain and how to configure a project in real life is my question. Linux have many programs developed in C/C++, but compiling them myself always out of my scope, which is blocking me from deeper understanding of Linux.</p>
<p>So here I&#39;m reading a walkthrough to help myself making a C/C++ project work.</p>
<p><strong>CMake</strong> is said to be an OS and compiler independent build system. So you first write CMake configurations in any source directories, and then it can generate a native build environment that will compile source code, create libraries, generate wrappers and build executables in arbitrary combinations.</p>

<h3 id="toc_0">0. Build Essential</h3>
<p>First of all, if you want to compile anything in Linux, you should install the <code>build-essential</code> package first.</p>
<div class="highlight"><pre><span></span>$ sudo apt install build-essential
</pre></div>
<p>It contains all the common packages to build Debian packages, such as: g++, gcc, hurd, libc, dpkg, make, etc.</p>
<p>Next step let&#39;s install the CMake package:</p>
<div class="highlight"><pre><span></span>$ sudo apt install cmake
</pre></div>

<h3 id="toc_1">1. A Hello World and a CMakeLists.txt</h3>
<p>The minimal demo has two files. </p>
<p>A <code>hello.cpp</code> source file:</p>
<div class="highlight"><pre><span></span><span class="cp">#include</span> <span class="cpf">&lt;iostream&gt;</span><span class="cp"></span>

<span class="k">using</span> <span class="k">namespace</span> <span class="n">std</span><span class="p">;</span>

<span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="kt">void</span><span class="p">)</span> <span class="p">{</span>
     <span class="n">cout</span> <span class="o">&lt;&lt;</span> <span class="s">&quot;Hello World&quot;</span> <span class="o">&lt;&lt;</span> <span class="n">endl</span><span class="p">;</span>
     <span class="k">return</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
<span class="p">}</span>
</pre></div>
<p>A <code>CMakeLists.txt</code> config file:</p>

<pre><code># Specify the minimum version for CMake

cmake_minimum_required(VERSION 2.8)

# Project's name
project(hello)

# Set the output folder where your program will be created
set(CMAKE_BINARY_DIR ${CMAKE_SOURCE_DIR}/bin)
set(EXECUTABLE_OUTPUT_PATH ${CMAKE_BINARY_DIR})
set(LIBRARY_OUTPUT_PATH ${CMAKE_BINARY_DIR})

# The following folder will be included
include_directories(&quot;${PROJECT_SOURCE_DIR}&quot;)

# Compile the program to hello
add_executable(hello ${PROJECT_SOURCE_DIR}/hello.cpp)</code></pre>
<p>The three lines <code>cmake_minimum_required</code>, <code>project(hello)</code>, and <code>add_executable</code> are essential.</p>
<p>Run below two commands to compile the exectuatble:</p>
<div class="highlight"><pre><span></span><span class="c1"># Generate CMake configurations</span>
$ cmake -H. -Bbuild

<span class="c1"># Build the executable</span>
$ cmake --build build -- -j3
</pre></div>

<h3 id="toc_2">References</h3>
<p><a href="https://tuannguyen68.gitbooks.io/learning-cmake-a-beginner-s-guide/content/chap1/chap1.html">Learning CMake: A Beginner&#39;s Guide</a>
<a href="https://cmake.org/cmake-tutorial/">CMake Official Tutorial</a></p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Networking in Docker]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2018/01/networking-in-docker.html"/>
        <published>2018-01-12T11:41:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2018/01/networking-in-docker.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <category scheme="http://terryoy.github.io/tag/#docker" term="docker" label="docker" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>After installed an Ubuntu container in my docker, now I&#39;m trying to learn the networking tools in Ubuntu and Docker.</p>

<ol>
<li>Networking Tools in Ubuntu Again</li>
</ol>
<div class="highlight"><pre><span></span><span class="c1"># package for ifconfig(which is too old)</span>
$ apt install net-tools

<span class="c1"># newer package for networking</span>
$ apt install iproute2

<span class="c1"># ping command</span>
$ apt install iputils-ping

<span class="c1"># check network interface</span>
$ ifconfig
eth0      Link encap:Ethernet  HWaddr <span class="m">02</span>:42:ac:11:00:02  
          inet addr:172.17.0.2  Bcast:0.0.0.0  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5956 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3345 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:8376715 <span class="o">(</span><span class="m">8</span>.3 MB<span class="o">)</span>  TX bytes:186204 <span class="o">(</span><span class="m">186</span>.2 KB<span class="o">)</span>
</pre></div>

<ol>
<li>Docker Commands for Networking</li>
</ol>
<p>There are three types of networks in docker, and the <code>bridge</code> network is by default presented in all docker instances.</p>
<div class="highlight"><pre><span></span><span class="c1"># List networks in Docker</span>
$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
a6bf46c2bc44        bridge              bridge              <span class="nb">local</span>
e3826e52e7e6        host                host                <span class="nb">local</span>
08a134ed472c        none                null                <span class="nb">local</span>

<span class="c1"># Check docker&#39;s network detail</span>
$ docker network inspect bridge
<span class="o">[</span>
    <span class="o">{</span>
        <span class="s2">&quot;Name&quot;</span>: <span class="s2">&quot;bridge&quot;</span>,
        <span class="s2">&quot;Id&quot;</span>: <span class="s2">&quot;a6bf46c2bc44dc16523ac28edd5524fccef79b779ae52602c868001763cd21c4&quot;</span>,
        <span class="s2">&quot;Created&quot;</span>: <span class="s2">&quot;2018-01-09T11:36:15.168104899Z&quot;</span>,
        <span class="s2">&quot;Scope&quot;</span>: <span class="s2">&quot;local&quot;</span>,
        <span class="s2">&quot;Driver&quot;</span>: <span class="s2">&quot;bridge&quot;</span>,
        <span class="s2">&quot;EnableIPv6&quot;</span>: false,
        <span class="s2">&quot;IPAM&quot;</span>: <span class="o">{</span>
            <span class="s2">&quot;Driver&quot;</span>: <span class="s2">&quot;default&quot;</span>,
            <span class="s2">&quot;Options&quot;</span>: null,
            <span class="s2">&quot;Config&quot;</span>: <span class="o">[</span>
                <span class="o">{</span>
                    <span class="s2">&quot;Subnet&quot;</span>: <span class="s2">&quot;172.17.0.0/16&quot;</span>,
                    <span class="s2">&quot;Gateway&quot;</span>: <span class="s2">&quot;172.17.0.1&quot;</span>
                <span class="o">}</span>
            <span class="o">]</span>
        <span class="o">}</span>,
        <span class="s2">&quot;Internal&quot;</span>: false,
        <span class="s2">&quot;Attachable&quot;</span>: false,
        <span class="s2">&quot;Ingress&quot;</span>: false,
        <span class="s2">&quot;ConfigFrom&quot;</span>: <span class="o">{</span>
            <span class="s2">&quot;Network&quot;</span>: <span class="s2">&quot;&quot;</span>
        <span class="o">}</span>,
        <span class="s2">&quot;ConfigOnly&quot;</span>: false,
        <span class="s2">&quot;Containers&quot;</span>: <span class="o">{</span>
            <span class="s2">&quot;cfc178841a7940b6cbf43c8e0dbd7fb6672af3b3e8a9020632c75352685ec685&quot;</span>: <span class="o">{</span>
                <span class="s2">&quot;Name&quot;</span>: <span class="s2">&quot;ubuntu&quot;</span>,
                <span class="s2">&quot;EndpointID&quot;</span>: <span class="s2">&quot;8c14adc4a61eacdc8bd2b261d2af4f17c7a966d9fcfb8c61b5f9284fa2eded45&quot;</span>,
                <span class="s2">&quot;MacAddress&quot;</span>: <span class="s2">&quot;02:42:ac:11:00:02&quot;</span>,
                <span class="s2">&quot;IPv4Address&quot;</span>: <span class="s2">&quot;172.17.0.2/16&quot;</span>,
                <span class="s2">&quot;IPv6Address&quot;</span>: <span class="s2">&quot;&quot;</span>
            <span class="o">}</span>
        <span class="o">}</span>,
        <span class="s2">&quot;Options&quot;</span>: <span class="o">{</span>
            <span class="s2">&quot;com.docker.network.bridge.default_bridge&quot;</span>: <span class="s2">&quot;true&quot;</span>,
            <span class="s2">&quot;com.docker.network.bridge.enable_icc&quot;</span>: <span class="s2">&quot;true&quot;</span>,
            <span class="s2">&quot;com.docker.network.bridge.enable_ip_masquerade&quot;</span>: <span class="s2">&quot;true&quot;</span>,
            <span class="s2">&quot;com.docker.network.bridge.host_binding_ipv4&quot;</span>: <span class="s2">&quot;0.0.0.0&quot;</span>,
            <span class="s2">&quot;com.docker.network.bridge.name&quot;</span>: <span class="s2">&quot;docker0&quot;</span>,
            <span class="s2">&quot;com.docker.network.driver.mtu&quot;</span>: <span class="s2">&quot;1500&quot;</span>
        <span class="o">}</span>,
        <span class="s2">&quot;Labels&quot;</span>: <span class="o">{}</span>
    <span class="o">}</span>
<span class="o">]</span>
</pre></div>

<h3 id="toc_0">3. Internal Networking</h3>
<p>By default, the container is connected within a local network(172.17.0.1/16) bridged to the host machine, host can access the net gateway ip <code>172.17.0.1</code>. The contianers should be able to access each other in the local network provided by this net gateway. When you use the command <code>docker network inspect bridge</code>, you could see the list of containers and their IPs assigned.</p>

<h4 id="toc_1">3.1 Accessing container&#39;s service by port mapping</h4>
<p>Some containers as Nginx provide a service through a local port, in which case you could use port forwarding by your local IP. For example, you can check this port forwarding by <code>docker port</code> command or just listing the containers:</p>
<div class="highlight"><pre><span></span><span class="c1"># list ports for container</span>
$ docker port nginx
<span class="m">80</span>/tcp -&gt; <span class="m">0</span>.0.0.0:5000

<span class="c1"># list containers</span>
$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                   PORTS                  NAMES
cfc178841a79        ubuntu              <span class="s2">&quot;/bin/bash&quot;</span>              <span class="m">3</span> weeks ago         Up <span class="m">4</span> hours                                      ubuntu
0200f61f5d65        nginx               <span class="s2">&quot;nginx -g &#39;daemon of…&quot;</span>   <span class="m">2</span> months ago        Up <span class="m">3</span> minutes             <span class="m">0</span>.0.0.0:5000-&gt;80/tcp   webserver
</pre></div>
<p>Now you can see there is a port mapping from <code>host:5000</code> to <code>container:80</code>.</p>
<p>By default, no port is opened for a container to host, you need to specify the port mapping at <code>docker run</code> command with <code>-p</code>.</p>

<pre><code># `-d` is when you needed it run as a daemon.
$ docker run nginx -d -p 5000:80 --name nginx nginx</code></pre>
<p>To open a port mapping for an existing container is very tricky. It is often said that you need to create another container instance based on the current just to open a port.</p>

<h4 id="toc_2">3.2 Creating a Network</h4>
<p>Now let&#39;s try something advanced. Let&#39;s create the different types of network one by one.</p>

<h5 id="toc_3">3.2.1 Bridge Network</h5>
<p>By default, the network you created will be a <code>bridged</code> network. You can add container to the network later. It creates a subnet and open only specific ports to the host to access service in the subnet, like NAT.</p>
<div class="highlight"><pre><span></span>$ docker network create gamezone
$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
0fe8bfe5222d        gamezone            bridge              <span class="nb">local</span>

<span class="c1"># if you want to remove the bridge network</span>
$ docker network rm gamezone
</pre></div>
<p>Now let&#39;s create an Ubuntu container called &ldquo;gamecenter&rdquo; which connects to the gamezone network and maps port 5000 to the docker host&#39;s 15000.</p>
<div class="highlight"><pre><span></span>$ docker create --name gamecenter <span class="se">\</span>
        --network gamezone <span class="se">\</span>
        --publish <span class="m">15000</span>:5000 <span class="se">\</span>
        -t -i ubuntu /bin/bash

$ docker container ls -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS                     NAMES
8cf4e293aae9        ubuntu              <span class="s2">&quot;/bin/bash&quot;</span>              <span class="m">10</span> minutes ago      Up <span class="m">10</span> minutes               <span class="m">0</span>.0.0.0:15000-&gt;5000/tcp   gamecenter

$ docker network inspect gamezone
<span class="o">[</span>
    <span class="o">{</span>
        <span class="s2">&quot;Name&quot;</span>: <span class="s2">&quot;gamezone&quot;</span>,
        <span class="s2">&quot;Id&quot;</span>: <span class="s2">&quot;9b6417528a0e4a37ab9818e185a8046af06b3a9ae0eba40a2e494087676b70c9&quot;</span>,
        <span class="s2">&quot;Created&quot;</span>: <span class="s2">&quot;2018-06-09T15:24:43.6288314Z&quot;</span>,
        <span class="s2">&quot;Scope&quot;</span>: <span class="s2">&quot;local&quot;</span>,
        <span class="s2">&quot;Driver&quot;</span>: <span class="s2">&quot;bridge&quot;</span>,
        <span class="s2">&quot;EnableIPv6&quot;</span>: false,
        <span class="s2">&quot;IPAM&quot;</span>: <span class="o">{</span>
            <span class="s2">&quot;Driver&quot;</span>: <span class="s2">&quot;default&quot;</span>,
            <span class="s2">&quot;Options&quot;</span>: <span class="o">{}</span>,
            <span class="s2">&quot;Config&quot;</span>: <span class="o">[</span>
                <span class="o">{</span>
                    <span class="s2">&quot;Subnet&quot;</span>: <span class="s2">&quot;172.18.0.0/16&quot;</span>,
                    <span class="s2">&quot;Gateway&quot;</span>: <span class="s2">&quot;172.18.0.1&quot;</span>
                <span class="o">}</span>
            <span class="o">]</span>
        <span class="o">}</span>,
        <span class="s2">&quot;Internal&quot;</span>: false,
        <span class="s2">&quot;Attachable&quot;</span>: false,
        <span class="s2">&quot;Ingress&quot;</span>: false,
        <span class="s2">&quot;ConfigFrom&quot;</span>: <span class="o">{</span>
            <span class="s2">&quot;Network&quot;</span>: <span class="s2">&quot;&quot;</span>
        <span class="o">}</span>,
        <span class="s2">&quot;ConfigOnly&quot;</span>: false,
        <span class="s2">&quot;Containers&quot;</span>: <span class="o">{</span>
            <span class="s2">&quot;8cf4e293aae932693f6769eea08665306485b17018999d8725f14b659e51c919&quot;</span>: <span class="o">{</span>
                <span class="s2">&quot;Name&quot;</span>: <span class="s2">&quot;gamecenter&quot;</span>,
                <span class="s2">&quot;EndpointID&quot;</span>: <span class="s2">&quot;6ca3d79c633520dbee752ab6b462404aa8507f8eb9f2b2cf840de2d3322d27bb&quot;</span>,
                <span class="s2">&quot;MacAddress&quot;</span>: <span class="s2">&quot;02:42:ac:12:00:02&quot;</span>,
                <span class="s2">&quot;IPv4Address&quot;</span>: <span class="s2">&quot;172.18.0.2/16&quot;</span>,
                <span class="s2">&quot;IPv6Address&quot;</span>: <span class="s2">&quot;&quot;</span>
            <span class="o">}</span>
        <span class="o">}</span>,
        <span class="s2">&quot;Options&quot;</span>: <span class="o">{}</span>,
        <span class="s2">&quot;Labels&quot;</span>: <span class="o">{}</span>
    <span class="o">}</span>
<span class="o">]</span>

$ docker start gamecenter
$ docker attach gamecenter
$ ifconfig
eth0      Link encap:Ethernet  HWaddr <span class="m">02</span>:42:ac:12:00:02  
          inet addr:172.18.0.2  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:20872 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8684 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:28266802 <span class="o">(</span><span class="m">28</span>.2 MB<span class="o">)</span>  TX bytes:476241 <span class="o">(</span><span class="m">476</span>.2 KB<span class="o">)</span>
</pre></div>
<p>You can notice that the container &ldquo;gamecenter&rdquo; is running in the subnet of <code>172.18.0.1</code> currently. And we have open a port <code>15000</code> on the host, to map to gamecenter&#39;s port <code>5000</code>.</p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Setup Arm Development Environment]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2017/12/setup_arm_development.html"/>
        <published>2017-12-31T00:00:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2017/12/setup_arm_development.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <category scheme="http://terryoy.github.io/tag/#misc" term="misc" label="misc" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <h3 id="toc_0">1. Clone the kernel repository</h3>
<div class="highlight"><pre><span></span>$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

<span class="c1"># or just download the package from the mirror</span>
$ wget https://mirror.tuna.tsinghua.edu.cn/kernel/v4.x/linux-4.4.1.tar.xz
</pre></div>

<h3 id="toc_1">2. Install the Cross Compiling Toolchain</h3>
<div class="highlight"><pre><span></span>$ sudo apt-get install gcc-arm-linux-gnueabi

<span class="c1"># another option is &quot;arm-linux-gnueabihf&quot; but we use &quot;gcc-arm-gnueabi&quot; here</span>
</pre></div>

<h3 id="toc_2">3. Make the first compile</h3>
<p>We&#39;re emulating the vexpress Cortex A9 for demo.</p>
<div class="highlight"><pre><span></span>$ <span class="nb">export</span> <span class="nv">ARCH</span><span class="o">=</span>arm  
$ <span class="nb">export</span> <span class="nv">CROSS_COMPILE</span><span class="o">=</span>arm-linux-gnueabi-  
$ make vexpress_defconfig  
$ make zImage -j8  
$ make modules -j8  
$ make dtbs
</pre></div>

<h3 id="toc_3">4. Install QEMU</h3>
<div class="highlight"><pre><span></span>$ sudo apt-get install qemu
</pre></div>

<h3 id="toc_4">5. Make Root File System</h3>

<h3 id="toc_5">5.1 busybox</h3>
<div class="highlight"><pre><span></span>$ wget http://www.busybox.net/downloads/busybox-1.25.1.tar.bz2  
$ tar xvf busybox-1.25.1.tar.bz2
$ make defconfig  
$ make <span class="nv">CROSS_COMPILE</span><span class="o">=</span>arm-linux-gnueabi-  
$ make install <span class="nv">CROSS_COMPILE</span><span class="o">=</span>arm-linux-gnueabi-
</pre></div>
<p>Then you find the executables in <code>_install</code> folder. Next we start to create the rootfs.</p>

<h3 id="toc_6">5.2 rootfs</h3>
<div class="highlight"><pre><span></span>$ sudo mkdir rootfs
$ sudo mkdir rootfs/lib 

<span class="c1"># copy busybox to rootfs</span>
$ sudo cp _install/* -r rootfs/

<span class="c1"># copy arm libs to lib</span>
<span class="c1"># sudo cp -P /usr/arm-linux-gnueabi/lib/* rootfs/lib/</span>
</pre></div>

<h3 id="toc_7">5.3 Create 4 tty devices</h3>
<div class="highlight"><pre><span></span>$ sudo mkdir -p rootfs/dev
$ sudo mknod rootfs/dev/tty1 c <span class="m">4</span> <span class="m">1</span>  
$ sudo mknod rootfs/dev/tty2 c <span class="m">4</span> <span class="m">2</span>  
$ sudo mknod rootfs/dev/tty3 c <span class="m">4</span> <span class="m">3</span>  
$ sudo mknod rootfs/dev/tty4 c <span class="m">4</span> <span class="m">4</span>
</pre></div>
<p>You can also create other folders in the image. (Reference: <a href="https://learningfromyoublog.wordpress.com/2016/04/05/131/">learningfromyoublog.wordpress.com/2016/04/05/131/</a>)</p>

<h3 id="toc_8">5.4 Make the Image</h3>
<div class="highlight"><pre><span></span><span class="c1"># creat an empty image</span>
$ dd <span class="k">if</span><span class="o">=</span>/dev/zero <span class="nv">of</span><span class="o">=</span>a9rootfs.ext3 <span class="nv">bs</span><span class="o">=</span>1M <span class="nv">count</span><span class="o">=</span><span class="m">32</span>  

<span class="c1"># format to ext3</span>
$ mkfs.ext3 a9rootfs.ext3  

<span class="c1"># copy files into the image</span>
$ sudo mkdir tmpfs  
$ sudo mount -t ext3 a9rootfs.ext3 tmpfs/ -o loop  
$ sudo cp -r rootfs/*  tmpfs/  
$ sudo umount tmpfs
</pre></div>

<h3 id="toc_9">6. Start QEMU with the Image</h3>
<div class="highlight"><pre><span></span><span class="c1"># open in current console</span>
$ qemu-system-arm -M vexpress-a9 -m 512M -dtb extra_folder/vexpress-v2p-ca9.dtb -kernel extra_folder/zImage -nographic -append <span class="s2">&quot;root=/dev/mmcblk0 rw console=ttyAMA0&quot;</span> -sd a9rootfs.ext3 

<span class="c1"># open in new window(maybe GUI)</span>
$ qemu-system-arm -M vexpress-a9 -m 512M -dtb extra_folder/vexpress-v2p-ca9.dtb -kernel extra_folder/zImage -append <span class="s2">&quot;root=/dev/mmcblk0 rw&quot;</span> -sd a9rootfs.ext3
</pre></div>
<p>Now you enter the console of the emulator system. Press <code>Ctrl+A C</code> to exit to <code>(qemu)</code> console, or <code>Ctrl+A X</code> to exit..</p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Exploring Ubuntu From Fundamentals(1)]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2017/12/get-to-know-the-core-ubuntu.html"/>
        <published>2017-12-22T15:49:00+08:00</published>
        <updated>2020-02-01T10:09:37+08:00</updated>
        <id>http://terryoy.github.io/2017/12/get-to-know-the-core-ubuntu.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>Recently I have installed docker on my mac. It&#39;s such a wonderful virtualized environment that I don&#39;t need to install a whole system image on VirtualBox to explore in Ubuntu. </p>
<p>The docker image of Ubuntu is somehow essential subset of packages which give you the system environment. It doesn&#39;t even have <code>vi</code> or the <code>lsb_release</code> command for you to read the distro version. So I want to take some notes while I&#39;m exploring the fundamentals in Ubuntu.</p>

<h3 id="toc_0">1. System Info</h3>
<div class="highlight"><pre><span></span><span class="c1"># Check Ubuntu version</span>
$ cat /etc/issue
$ cat /etc/issue.net
Ubuntu <span class="m">16</span>.04.3 LTS

<span class="c1"># More information of Ubuntu</span>
$ cat /etc/os-release
<span class="nv">NAME</span><span class="o">=</span><span class="s2">&quot;Ubuntu&quot;</span>
<span class="nv">VERSION</span><span class="o">=</span><span class="s2">&quot;16.04.3 LTS (Xenial Xerus)&quot;</span>
<span class="nv">ID</span><span class="o">=</span>ubuntu
<span class="nv">ID_LIKE</span><span class="o">=</span>debian
<span class="nv">PRETTY_NAME</span><span class="o">=</span><span class="s2">&quot;Ubuntu 16.04.3 LTS&quot;</span>
<span class="nv">VERSION_ID</span><span class="o">=</span><span class="s2">&quot;16.04&quot;</span>
<span class="nv">HOME_URL</span><span class="o">=</span><span class="s2">&quot;http://www.ubuntu.com/&quot;</span>
<span class="nv">SUPPORT_URL</span><span class="o">=</span><span class="s2">&quot;http://help.ubuntu.com/&quot;</span>
<span class="nv">BUG_REPORT_URL</span><span class="o">=</span><span class="s2">&quot;http://bugs.launchpad.net/ubuntu/&quot;</span>
<span class="nv">VERSION_CODENAME</span><span class="o">=</span>xenial
<span class="nv">UBUNTU_CODENAME</span><span class="o">=</span>xenial

<span class="c1"># System Services</span>
$ service --status-all
 <span class="o">[</span> - <span class="o">]</span>  bootmisc.sh
 <span class="o">[</span> - <span class="o">]</span>  checkfs.sh
 <span class="o">[</span> - <span class="o">]</span>  checkroot-bootclean.sh
 <span class="o">[</span> - <span class="o">]</span>  checkroot.sh
 <span class="o">[</span> - <span class="o">]</span>  hostname.sh
 <span class="o">[</span> ? <span class="o">]</span>  hwclock.sh
 <span class="o">[</span> - <span class="o">]</span>  killprocs
 <span class="o">[</span> - <span class="o">]</span>  mountall-bootclean.sh
 <span class="o">[</span> - <span class="o">]</span>  mountall.sh
 <span class="o">[</span> - <span class="o">]</span>  mountdevsubfs.sh
 <span class="o">[</span> - <span class="o">]</span>  mountkernfs.sh
 <span class="o">[</span> - <span class="o">]</span>  mountnfs-bootclean.sh
 <span class="o">[</span> - <span class="o">]</span>  mountnfs.sh
 <span class="o">[</span> ? <span class="o">]</span>  ondemand
 <span class="o">[</span> - <span class="o">]</span>  procps
 <span class="o">[</span> - <span class="o">]</span>  rc.local
 <span class="o">[</span> - <span class="o">]</span>  sendsigs
 <span class="o">[</span> - <span class="o">]</span>  umountfs
 <span class="o">[</span> - <span class="o">]</span>  umountnfs.sh
 <span class="o">[</span> - <span class="o">]</span>  umountroot
 <span class="o">[</span> - <span class="o">]</span>  urandom

<span class="c1"># Supported Shells</span>
$ cat /etc/shells
/bin/sh
/bin/dash
/bin/bash
/bin/rbash
</pre></div>
<p>I have install <code>vim</code> and <code>man-db</code> to acquire the ability of text editing and manual reading. Now let&#39;s check what packages we have installed.</p>
<div class="highlight"><pre><span></span>$ apt list --installed

<span class="c1"># or if you just want the count</span>
$ apt list --installed <span class="p">|</span> grep installed <span class="p">|</span> wc -l 

<span class="c1"># or if you want to see a description of each pacakge</span>
$ dpkg-query -l
</pre></div>
<p>Now I can see that the essential packages to run my system has only <strong>116</strong> packages and <strong>62</strong> of them are libs.</p>

<h3 id="toc_1">2. Reading Manuals</h3>
<p>The most important documentation tool in Linux is <code>man</code>, another is <code>info</code> from package &ldquo;texinfo&rdquo;. And there are also something you can find in <code>/usr/share/doc</code>, although many packages just use it for change logs and copyright, but the readme part is sometimes helpful information.</p>
<p>The <code>man</code> command is in <code>man-db</code> package, and <code>info</code> is in <code>info</code> package.</p>

<h4 id="toc_2">2.1 Man pages</h4>
<p>Man pages are grouped the documents of packages into 8(or 9) sections.</p>

<ul>
<li>1 Executable programs or shell commands</li>
<li>2 System calls (functions provided by the kernel)</li>
<li>3 Library calls (functions within program libraries)</li>
<li>4 Special files (usually found in /dev)</li>
<li>5 File formats and conventions eg /etc/passwd</li>
<li>6 Games</li>
<li>7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)</li>
<li>8 System administration commands (usually only for root)</li>
<li>9 Kernel routines [Non standard]</li>
</ul>
<p>The numbers of sections you see here, are used to marked the pages when cross reference in man pages. For example, you can see &ldquo;man(7)&rdquo;, &ldquo;mandb(8)&rdquo;, etc. Sometimes there will be a same keyword in different sections. You can view all man pages of the same keyword, or specify which section you want to see, or search man pages by keyword.</p>
<div class="highlight"><pre><span></span><span class="c1"># view man page for man</span>
$ man man

<span class="c1"># view all &quot;intro&quot; in every section</span>
$ man -a intro

<span class="c1"># view &quot;intro&quot; in section 3</span>
$ man -s <span class="m">3</span> intro

<span class="c1"># list all man pages</span>
$ man -k .

<span class="c1"># list all man pages in section 5</span>
$ man -s <span class="m">5</span> -k .

<span class="c1"># list all man pages in section 5 match keyword &#39;systemd&#39;</span>
$ man -s <span class="m">5</span> -k systemd
</pre></div>

<h4 id="toc_3">2.2 Info pages</h4>
<p>This kind of document is written in <a href="https://www.gnu.org/software/texinfo/">texinfo</a>, supported in  Emacs, generated by <code>makeinfo</code>;</p>
<p>The texinfo format has node structure, and can export to different format such as HTML, etc. The <code>info</code> command more navigation methods to the document.</p>
<p>Read <a href="http://www.troubleshooters.com/linux/info.htm">here</a> about the keyboard interface.</p>

<h4 id="toc_4">2.3 Dcoumentation paths</h4>

<ul>
<li>/usr/share/man, folder for man pages of pacakges. They&#39;re separated into different sections man(1-9) according the pacakge categories.</li>
<li>/usr/share/info, folder for info pages, which is meant for providing more detail information, and it&#39;s generally used in GNU projects, but man pages are in much greater favour.</li>
<li>/usr/share/doc, folder for holding other document or refernces. Most packages contains only changelog and copyright info, some provided a README file, and some package provides templates or configuration files.</li>
</ul>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Getting Laravel Started]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2017/11/laravel-get-started.html"/>
        <published>2017-11-24T10:37:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2017/11/laravel-get-started.html</id>
        <category scheme="http://terryoy.github.io/tag/#php" term="php" label="php" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>After I found that PHP7.1 is in-the-box with OSX, it motivates me to start learning PHP(a.k.a. &ldquo;the best language in the world&rdquo; :P) and give some try to small projects. What I got from my friend is that <a href="https://laravel.com/">Laravel</a> is the quite popular web framework for PHP. So now I setting up the development and deployment environment to get started.</p>

<h3 id="toc_0">1.Installing dependencies</h3>
<p>On my MBP desktop, it&#39;s recommended to use <a href="https://laravel.com/docs/5.5/valet">Valet</a> if you want just a small additional tools to run it, and connect to existing databases on the machine. Otherwise it would be good to use a virtualbox image <a href="https://laravel.com/docs/5.5/homestead">Homestead</a> to have an all-in-one setup for development.</p>
<p>While my Ubuntu 16 server, I would like to install dependencies by hand, so I could get familiar with what it takes to run Laravel. Now here&#39;s what I tried.</p>
<div class="highlight"><pre><span></span>$ sudo apt install php7.0 php7.0-cli php7.0-zip php7.0-mbstring php7.0-mbstring php7.0-xml
</pre></div>
<p>Then we need to install <a href="https://getcomposer.org">Composer</a> to install other php packages. It&#39;s like the <code>pip</code> to Python and the <code>npm</code> to NodeJS.</p>
<div class="highlight"><pre><span></span><span class="c1"># install to $HOME/bin</span>
$ <span class="nb">cd</span> ~
$ wget https://getcomposer.org/installer -o composer-setup.php
$ php composer-setup.php --install-dir<span class="o">=</span>bin --filename<span class="o">=</span>composer


<span class="c1"># (alternatively) You could try the one-line command bellow for default installation</span>
$ curl -s https://getcomposer.org/installer <span class="p">|</span> php
</pre></div>
<p>Then try the composer command to see if you have it.</p>
<div class="highlight"><pre><span></span>$ composer
</pre></div>

<h3 id="toc_1">2.Installing Laravel Valet</h3>
<p><strong>Valet</strong> is a lightweight solution for hosting Laravel in development, and only avaiable on OSX. It uses brew to install php7.1 and other dependencies, but first you need Composer to install Laravel Valet. So install the composer using the script above first.</p>
<p>After composer is installed, use it to download the Valet package.</p>
<div class="highlight"><pre><span></span>$ composer global require laravel/valet
</pre></div>
<p>After composer install the package, you should note about where it is installed. Because you need to set the $PATH environment variables to enable the command from the packages. On OSX, the executable programs are in <code>~/.composer/vendor/bin</code>, and on Ubuntu, the path might be <code>~/.config/composer/vendor/bin</code>.</p>
<p>Then you can use valet commands to manage your development environment.</p>

<h3 id="toc_2">3.Install Laravel</h3>
<p>Installing Laravel is similar to Valet. Use <code>composer</code> to download and install the package globally.</p>
<div class="highlight"><pre><span></span>$ composer global require <span class="s2">&quot;laravel/installer&quot;</span>
</pre></div>
<p>Then you can use <code>laravel</code> command to create a new project:</p>
<div class="highlight"><pre><span></span>$ laravel new myproject
</pre></div>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Basic Usage for Docker]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2017/11/docker-basic.html"/>
        <published>2017-11-06T16:12:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2017/11/docker-basic.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <category scheme="http://terryoy.github.io/tag/#docker" term="docker" label="docker" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>Just to keep some notes when started to use docker for my development projects.</p>

<h3 id="toc_0">0. Get Started</h3>
<p>For my mac, download the .dmg file from official web site. Then it will run a program with a task icon in the top bar.</p>
<p>After the program has been initialized, you can open a terminal and try a few commands:</p>
<div class="highlight"><pre><span></span><span class="c1"># check versions</span>
$ docker --version
$ docker-compose --version
$ docker-machine --version

<span class="c1"># Current images</span>
$ docker images

<span class="c1"># Running instances</span>
$ docker ps
</pre></div>

<h3 id="toc_1">1. First Instance</h3>
<p>The docker official Hello World image is small enough, which you can try as your first instance.</p>
<div class="highlight"><pre><span></span>$ docker run hello-world
</pre></div>
<p>The image does not exist locally currently, but docker will continue to try downloading it and then run it.</p>
<p>Then, you wil see the image locally.</p>
<div class="highlight"><pre><span></span>$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
hello-world         latest              725dcfab7d63        <span class="m">2</span> days ago          <span class="m">1</span>.84kB
</pre></div>
<p>However, the container for this image is not visible in the list because <strong>the process has existed</strong>. You need to use a <code>-a</code> parameter to see it:</p>
<div class="highlight"><pre><span></span>$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                     PORTS               NAMES
c32eac048fda        hello-world         <span class="s2">&quot;/hello&quot;</span>            <span class="m">5</span> minutes ago       Exited <span class="o">(</span><span class="m">0</span><span class="o">)</span> <span class="m">5</span> minutes ago                       cranky_shaw
</pre></div>

<h3 id="toc_2">2. Server Instance</h3>
<p>The Hello World container doesn&#39;t do anything. If you want to try something interesting, try the <a href="https://docs.docker.com/docker-for-mac/#explore-the-application-and-run-examples">nginx image</a> instead.</p>
<div class="highlight"><pre><span></span>$ docker run -d -p <span class="m">8001</span>:80 --name webserver1 nginx
</pre></div>
<p>I choose a different port for the server, just to show how the port mapping is handled. When you check the instance with <code>docker ps</code>, you can see the image name, instance name, and the port mapping and get the meaning. When the container is up, you can access <code>http://localhost:8001/</code> to see the nginx home page.</p>
<div class="highlight"><pre><span></span>$ docker ps 
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS                  NAMES
6f6ba1d3f285        nginx               <span class="s2">&quot;nginx -g &#39;daemon ...&quot;</span>   About a minute ago   Up About a minute   <span class="m">0</span>.0.0.0:8001-&gt;80/tcp   webserver1

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
nginx               latest              40960efd7b8f        <span class="m">38</span> hours ago        108MB
hello-world         latest              725dcfab7d63        <span class="m">2</span> days ago          <span class="m">1</span>.84kB
</pre></div>
<p>A good point is that you can see how much disk usage every image uses. Now you can stop the service, or remove the unused resources by commands.</p>
<div class="highlight"><pre><span></span><span class="c1"># container life cycle methods</span>
$ docker stop webserver1
$ docker start webserver1
$ docker restart webserver1


<span class="c1"># remove a container</span>
$ docker rm webserver1

<span class="c1"># remove an image</span>
$ docker rmi hello-world
</pre></div>
<p>For server instance like Ubuntu, you need an interactive shell to work with it, so the command will be a little bit different:</p>

<pre><code># start a new container named 'ubuntu'
$ docker run --name ubuntu -ti ubuntu-core

# run an existing container
$ docker container start -i ubuntu

# attach console to a server if it has started without interactive environment
$ docker container start ubuntu
$ docker attach ubuntu

# If you want to detach from a server without stopping it
# use `ctrl-p ctrl-q` key sequence</code></pre>
<p>You may wonder the file size of each containers, you can checkt it by <code>docker ps -s</code>. You can see a ubuntu core only uses very small space with the minimal setup.</p>
<div class="highlight"><pre><span></span>$ docker ps -s
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                  NAMES               SIZE
070a6e845653        ubuntu              <span class="s2">&quot;/bin/bash&quot;</span>              About an hour ago   Up About an hour                           ubuntu              <span class="m">96</span>.7MB <span class="o">(</span>virtual 219MB<span class="o">)</span>
0200f61f5d65        nginx               <span class="s2">&quot;nginx -g &#39;daemon ...&quot;</span>   <span class="m">2</span> hours ago         Up <span class="m">2</span> hours          <span class="m">0</span>.0.0.0:5000-&gt;80/tcp   webserver           2B <span class="o">(</span>virtual 108MB<span class="o">)</span>
</pre></div>

<h3 id="toc_3">3. Other Settings</h3>
<p>There are some Docker preferences also mentioned in the Get Started guide, which I should make some notes here:</p>

<ul>
<li><strong>Disk Image Location</strong>. on Mac, it is stored in a file called &ldquo;Docker.qcow2&rdquo; somewhere in ~/Library, and all the containers are store within this image. You can move it somewhere else.</li>
<li><strong>Experimental features</strong>. It is not recommended for production environment, then I wonder why it is turned on by default on Mac.</li>
<li>*<em>Registry, Configuration, and Certificates for the Daemon *</em>, use it when you needed.</li>
</ul>

<h4 id="toc_4">3.1 Download Images by Tags</h4>
<div class="highlight"><pre><span></span><span class="c1"># Download images/repos from registry</span>
$ docker pull &lt;image&gt;:&lt;tag&gt;

<span class="c1"># Example of getting a core ubuntu system</span>
$ docker pull ubuntu:xenial
$ docker run --name ubuntu -ti ubuntu
</pre></div>

<h3 id="toc_5">4. Docker for Mac vs Docker Toolbox</h3>
<p>Docker Toolbox is also installed by the .dmg installation. The programs include <code>docker-compose</code> and <code>docker-machine</code>.</p>

<ul>
<li>Docker for Mac is a Mac native  application, you get only one VM, and it is managed by Docker for Mac. The VM is used with a lightweight solution called HyperKit.</li>
<li>Using Dokcker Toolbox, you can set up one or more VM and manage them.</li>
</ul>

<h3 id="toc_6">5. Docker Architecture</h3>
<p>The Docker architecture can split into 3 parts:</p>

<ul>
<li>Docket Client, the <code>docker</code> cli tools for user to interact with docker daemon with Docker API.</li>
<li>Docker Host, the service daemon <code>dockerd</code> listen for Docker API request and perform all kinds of management tasks about images, containers, networks, and volumns.</li>
<li>Docker Registry, a cloud service which stores Docker images, such as Docker Hub and Docker Cloud(both are public registries).</li>
</ul>

<h4 id="toc_7">5.1 Docker Objects</h4>

<ul>
<li>Image, a read-only template with instructions for creating Docker container.</li>
<li>Container, an instance of an image, with network, file system, etc. attached. You can create, delete, stop, resume containers.</li>
<li>Service, allow you to scale containers across Docker daemons.</li>
</ul>

<h4 id="toc_8">5.2 The Underlying technology</h4>

<ul>
<li>Namespaces. When you run a container, Docker create a set of namespaces for the container, which provide a layer of isolation. The namespaces includes:

<ul>
<li><code>pid</code> for process</li>
<li><code>net</code> for network interfaces</li>
<li><code>ipc</code> for interprocess communication</li>
<li><code>mnt</code> for file system</li>
<li><code>uts</code> for kernel and version identifiers (Unix Timeshare System)</li>
</ul></li>
<li>Control Groups. A <code>cgroup</code> limits an application to a specific set of resources, and allow Docker Engine to share hardware resources to containers with optionally limits and constraints.</li>
<li>Container Format. A combination of namespaces, control groups, and UnionFS, packed into a Wrapper. The default format is <code>libcontainer</code>.</li>
</ul>
<p>Refs:
<a href="https://docs.docker.com/engine/docker-overview/#docker-architecture">Docker Architecture</a>
<a href="https://store.docker.com/">Docker Store</a> is a market where you can distribute your images.</p>

<h3 id="toc_9">6. Developing Apps with Docker</h3>
<p>It&#39;s easy to setup Docker to deploy with your app. All you need to do is to add a <code>Dockerfile</code> in your source folder and then build the image. The official example demonstrate how a small flask app is built with Docker.</p>

<h4 id="toc_10">6.1 First time image</h4>
<p>I summarize the steps as below:</p>

<ul>
<li>Go to docker <a href="https://hub.docker.com/_/python/">hub</a> to find a target python image. It already has all kinds of  Dockerfile template for you to copy.</li>
<li>Create a local copy of the Docker file. Make sure you understand the template and knows what to modify according to your app.</li>
<li>Write a flask <code>app.py</code> and a <code>requirements.txt</code> as usual.</li>
<li>Build the docker image and then run with it.</li>
</ul>
<div class="highlight"><pre><span></span><span class="c1"># prepare the source files(...skipped here)</span>
$ ls
Dockerfile    app.py      requirements.txt

<span class="c1"># build the image</span>
$ docker build -t slim-flask .

<span class="c1"># create and run the container</span>
$ docker run -p <span class="m">4000</span>:80 slim-flask
 * Running on http://0.0.0.0:80/ <span class="o">(</span>Press CTRL+C to quit<span class="o">)</span>
</pre></div>
<p>The result will be:</p>

<ul>
<li>Only a Dockerfile is added to your source, and others are still remain</li>
<li>A Python docker image is downloaded, which takes the size of 156MB or 691MB depends on if you choose the &ldquo;slim&rdquo; version.</li>
<li>A flask image for your app is genenrated, which contains the content from the original python image, the pip installed packages from requirements, and your source files. So the size is slightly bigger than the python image.</li>
</ul>
<div class="highlight"><pre><span></span>$ docker images 
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
slim-flask          latest              3032f935f40c        <span class="m">10</span> seconds ago      166MB
python              slim                a79297999298        <span class="m">45</span> hours ago        156MB
</pre></div>

<h4 id="toc_11">6.2 Create image from a container</h4>
<p>However, sometimes you want to make configurations inside a container and want to save it as an image. You can do that:</p>
<div class="highlight"><pre><span></span>$ docker container commit <span class="o">[</span>options<span class="o">]</span> CONTAINER <span class="o">[</span>repository:<span class="o">[</span>TAG<span class="o">]]</span>

<span class="c1"># example</span>
$ docker container commit ubuntu ubuntu-image
</pre></div>

<h4 id="toc_12">6.3 Publish docker image</h4>
<p>Use it when it is needed.</p>
<div class="highlight"><pre><span></span>$ docker login             <span class="c1"># Log in this CLI session using your Docker credentials</span>
$ docker tag &lt;image&gt; username/repository:tag  <span class="c1"># Tag &lt;image&gt; for upload to registry</span>
$ docker push username/repository:tag            <span class="c1"># Upload tagged image to registry</span>
$ docker run username/repository:tag                   <span class="c1"># Run image from a registry</span>
</pre></div>

<h3 id="toc_13">7. Docker mirrors</h3>

<ul>
<li>Docker(Official), <code>--registry-mirror=https://registry.docker-cn.com</code></li>
<li>Netease, <a href="http://hub-mirror.c.163.com">hub-mirror.c.163.com</a></li>
<li>USTC, <a href="https://docker.mirrors.ustc.edu.cn">docker.mirrors.ustc.edu.cn</a></li>
<li>Daocloud &amp; Alicloud, need registration, and Alicloud needs an dev platform account.</li>
</ul>
<p><a href="https://ieevee.com/tech/2016/09/28/docker-mirror.html">source</a></p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Python Function Decorators and The Use in Django]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2017/03/python-decorators-and-use-in-django.html"/>
        <published>2017-03-26T16:26:00+08:00</published>
        <updated>2020-02-01T10:09:37+08:00</updated>
        <id>http://terryoy.github.io/2017/03/python-decorators-and-use-in-django.html</id>
        <category scheme="http://terryoy.github.io/tag/#python" term="python" label="python" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <h3 id="toc_0">1. Basic of functions</h3>

<ul>
<li>Functions can be assign</li>
<li>Functions can be defined inside a function</li>
<li>Functions can be passed as a parameter and return as a returning value</li>
<li>Inner functions have access to the enclosing scope</li>
</ul>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">&#39;World&#39;</span><span class="p">):</span>
   <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Hello, </span><span class="si">{0}</span><span class="s1">!&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">name</span><span class="p">)))</span>

<span class="c1"># assign</span>
<span class="n">bar</span> <span class="o">=</span> <span class="n">foo</span>
<span class="n">bar</span><span class="p">()</span>
<span class="c1"># =&gt; Hello, world!</span>


<span class="c1"># Use in parameter and return as result</span>
<span class="k">def</span> <span class="nf">greetings</span><span class="p">(</span><span class="n">call</span><span class="p">):</span>
    <span class="n">welcome</span> <span class="o">=</span> <span class="s1">&#39;Welcome back!&#39;</span> <span class="c1"># the enclosing scope to &quot;greeting_to&quot;</span>

    <span class="k">def</span> <span class="nf">greeting_to</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
        <span class="n">call</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
        <span class="nb">print</span><span class="p">(</span><span class="n">welcome</span><span class="p">)</span>

    <span class="k">return</span> <span class="n">greeting_to</span>

<span class="n">greet</span> <span class="o">=</span> <span class="n">greetings</span><span class="p">(</span><span class="n">bar</span><span class="p">)</span> <span class="c1"># set bar as a greeting function</span>
<span class="n">greet</span><span class="p">(</span><span class="s1">&#39;Terry&#39;</span><span class="p">)</span> <span class="c1"># use the return function to generate greetings</span>
<span class="c1"># =&gt; Hello, Terry!</span>
<span class="c1"># =&gt; Welcome, back!</span>
</pre></div>

<h3 id="toc_1">2. Decorators</h3>
<p>A <strong>Function Decorator</strong> is a wrapper to an existing function, in which you can do some pre-process to the parameter or post-process to the returning value of the existing funciton.</p>
<p>The function decorator in Python must follows the below rules:</p>

<ul>
<li>It is a function that accept a function parameter(like saying <em>&ldquo;a decorator to which function&rdquo;</em>)</li>
<li>The decorator function must define and return a function as a result, so that the client gets something act similar to the existing function(act just like a function wrapper). </li>
<li>Do the customizing things inside the wrapper function, and call the existing function eventually</li>
</ul>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">greeting_decorator</span><span class="p">(</span><span class="n">call</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
        <span class="n">call</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Isn&#39;t it powerful?&quot;</span><span class="p">)</span>

    <span class="k">return</span> <span class="n">wrapper</span>


<span class="c1"># The decorator syntax</span>
<span class="nd">@greeting_decorator</span>
<span class="k">def</span> <span class="nf">greet</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
    <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Hello, </span><span class="si">{0}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">name</span><span class="p">))</span>

<span class="n">greet</span><span class="p">(</span><span class="s1">&#39;Terry&#39;</span><span class="p">)</span> <span class="c1"># decorated function though the function name is the same as defined</span>
<span class="c1"># =&gt; Hello, Terry </span>
<span class="c1"># =&gt; Isn&#39;t it powerful?</span>
</pre></div>
<p>Decorators can be chained, and can also accept extra parameters when defining the customization to the function. However, decorators that accept parameters need to add another wrapper to the simple decorator.</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">room</span><span class="p">(</span><span class="n">room_name</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">decorator_wrapper</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
        <span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
            <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Enter room: </span><span class="si">{0}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">room_name</span><span class="p">))</span>
            <span class="n">func</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>

        <span class="k">return</span> <span class="n">wrapper</span>
    <span class="k">return</span> <span class="n">decorator_wrapper</span>

<span class="k">def</span> <span class="nf">leave_on_greet</span><span class="p">(</span><span class="n">greet_func</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">greeting_wrapper</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
        <span class="n">greet_func</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Thanks, I&#39;m leaving&quot;</span><span class="p">)</span>

    <span class="k">return</span> <span class="n">greeting_wrapper</span>


<span class="nd">@room</span><span class="p">(</span><span class="s1">&#39;Matrix&#39;</span><span class="p">)</span>
<span class="nd">@leave_on_greet</span>
<span class="k">def</span> <span class="nf">greet</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
    <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Hello, </span><span class="si">{0}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">name</span><span class="p">))</span>


<span class="n">greet</span><span class="p">(</span><span class="s1">&#39;Terry&#39;</span><span class="p">)</span>
<span class="c1"># =&gt; Enter room: Matrix</span>
<span class="c1"># =&gt; Hello, Terry</span>
<span class="c1"># =&gt; Thanks, I&#39;m leaving</span>
</pre></div>

<h3 id="toc_2">3. Example: using decorator in Django views</h3>
<p>It is very common that you want to write decorators for the request in views.py. For example, Django itself provides <a href="https://docs.djangoproject.com/en/1.10/topics/http/decorators/">a list of decorators</a> that you can use in certain scenarios, such as restricting HTTP methods, or cache controls. There are also other examples that could be consider: logging requests or checking auth tokens.</p>
<p>Here I write a very simple example that logs requests which a specified module name. We will have to use a new feature here that passes arguments between functions, because Django view methods can accept arguments defined in URL patterns.</p>
<div class="highlight"><pre><span></span><span class="c1"># a request log decorator which you can define the module</span>
<span class="k">def</span> <span class="nf">log_request</span><span class="p">(</span><span class="n">module</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">decorator_wrapper</span><span class="p">(</span><span class="n">view_func</span><span class="p">):</span>
        <span class="k">def</span> <span class="nf">func_wrapper</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
            <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;[</span><span class="si">{0}</span><span class="s1">] </span><span class="si">{1}</span><span class="s1"> </span><span class="si">{2}</span><span class="s1"> </span><span class="si">{3}</span><span class="s1">&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">module</span><span class="p">,</span> <span class="n">request</span><span class="o">.</span><span class="n">get_host</span><span class="p">(),</span> <span class="n">request</span><span class="o">.</span><span class="n">method</span><span class="p">,</span> <span class="n">request</span><span class="o">.</span><span class="n">get_full_path</span><span class="p">()))</span>
            <span class="k">return</span> <span class="n">view_func</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
        <span class="k">return</span> <span class="n">func_wrapper</span>
    <span class="k">return</span> <span class="n">decorator_wrapper</span>


<span class="c1"># use in views.py</span>
<span class="nd">@log_request</span><span class="p">(</span><span class="n">module</span><span class="o">=</span><span class="s2">&quot;Book&quot;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">book_detail</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">book_id</span><span class="p">):</span>
    <span class="k">return</span> <span class="n">HttpResponse</span><span class="p">(</span><span class="s1">&#39;Book info: </span><span class="si">{0}</span><span class="s1">&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">book_id</span><span class="p">))</span>


<span class="c1"># URL patterns</span>
<span class="n">urlpatterns</span> <span class="o">=</span> <span class="p">[</span>
    <span class="n">url</span><span class="p">(</span><span class="sa">r</span><span class="s1">&#39;^book/(?P&lt;book_id&gt;\d+/info$&#39;</span><span class="p">,</span> <span class="n">views</span><span class="o">.</span><span class="n">book_detail</span><span class="p">),</span>
    <span class="c1"># ...</span>
<span class="p">]</span>
</pre></div>
<p>Test in command line:</p>
<div class="highlight"><pre><span></span><span class="c1"># test in shell, and see the book id correctly returned</span>
$ curl http://localhost/book/1/info
Book info: <span class="m">1</span>

<span class="c1"># check the log in server console, everything is shown</span>
<span class="o">[</span>Book<span class="o">]</span> localhost:8000 GET /v1/book/1/info
</pre></div>

<h3 id="toc_3">Reference</h3>
<p>Reference: <a href="http://thecodeship.com/patterns/guide-to-python-function-decorators/">thecodeship.com/patterns/guide-to-python-function-decorators/</a></p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Usage for Redis on Linux]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2017/03/linux-redis-usage.html"/>
        <published>2017-03-10T22:20:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2017/03/linux-redis-usage.html</id>
        <category scheme="http://terryoy.github.io/tag/#database" term="database" label="database" />
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <h3 id="toc_0">0. Setup</h3>
<div class="highlight"><pre><span></span>$ sudo apt-get install redis-server

<span class="c1"># login redis</span>
$ redis-cli
</pre></div>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[How I fix my Ubuntu with iRST(Intel Rapid Start Technology) Raid]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2016/12/how-i-fix-ubuntu-boot-with-irst-raid.html"/>
        <published>2016-12-29T00:05:00+08:00</published>
        <updated>2020-02-01T10:09:37+08:00</updated>
        <id>http://terryoy.github.io/2016/12/how-i-fix-ubuntu-boot-with-irst-raid.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>I have bought a Haier X3 laptop with a 500GB hard disk and a 32GB SSD with which you could use to boot and cache disk rapidly for Windows via Intel&#39;s technology. However, I&#39;m a Linux fan, so I also need to install Ubuntu on my computer.</p>
<p>At first, the Windows 7 system was pre-installed. When I installed Ubuntu it failed at booting if the Windows 7 had enabled iRST. I cannot find any working solution to fix the Ubuntu problem at the time, so I just disabled Win 7&#39;s iRST, and then use the SSD as normal disk. Later I had a very good time with the Linux environment.</p>
<p>Recently, I&#39;m going to refresh the whole system and make both clean again. So I give a second try to enable Windows 7&#39;s iRST and make Ubuntu boot correctly, and I succeeded after two days hard work!</p>
<p>The root cause is that the raid management tool on Linux will assemble the partitions as RAID and then the <code>initramds</code> cannot find the partition with UUID. It fails like this:</p>
<div class="highlight"><pre><span></span>Gave up waiting <span class="k">for</span> root device. Common problems:
 - Boot args <span class="o">(</span>cat /proc/cmdline<span class="o">)</span>
   - Check <span class="nv">rootdelay</span><span class="o">=</span> <span class="o">(</span>did the system <span class="nb">wait</span> long enough?<span class="o">)</span>
   - Check <span class="nv">root</span><span class="o">=</span> <span class="o">(</span>did the system <span class="nb">wait</span> <span class="k">for</span> the right device?<span class="o">)</span>
 - Missing modules <span class="o">(</span>cat /proc/modules<span class="p">;</span> ls /dev<span class="o">)</span>
ALERT! /dev/disk/by-uuid/52152d36-9dc4-42a6-8be1-3966bf397b09 does not exist. Dropping to a shell!

BusyBox v1.22.1 <span class="o">(</span>Ubuntu <span class="m">1</span>:1.22.0-15ubuntu1<span class="o">)</span> built-in shell <span class="o">(</span>ash<span class="o">)</span>
Enter <span class="s1">&#39;help&#39;</span> <span class="k">for</span> a lost of built-in commands.
<span class="o">(</span>initramfs<span class="o">)</span>
</pre></div>
<p>I have searched on the internet about it, finding that it&#39;s related to the block id that generated for the partitions are missing:</p>
<div class="highlight"><pre><span></span><span class="o">(</span>initramfs<span class="o">)</span> blkid
/dev/sda: <span class="nv">TYPE</span><span class="o">=</span><span class="s2">&quot;isw_raid_member&quot;</span>
/dev/sdb: <span class="nv">TYPE</span><span class="o">=</span><span class="s2">&quot;isw_raid_member&quot;</span>

<span class="o">(</span>initramfs<span class="o">)</span> ls /dev/md*
md/  md126  md127
</pre></div>
<p>While the partitions of <code>/dev/sda/</code> are supposed to be recognized by UUIDs, but it is created as a software RAID device(md126), so the partitions of it cannot be accessed at boot time. I have tried to install Ubuntu via my usb disk again and again, also try to use <code>chroot</code> to update the grub record and the initramfs on the partiion, and I still don&#39;t have luck. After a long frustrating period, I suddenly come up the idea: why not just disable the raid array at boot time? </p>
<p>After I tried the below two commands in <code>initramfs</code>, it come back to normal boot after I exit the initramfs mode!</p>
<div class="highlight"><pre><span></span><span class="o">(</span>initramfs<span class="o">)</span> mdadm --stop --scan

<span class="o">(</span>initramfs<span class="o">)</span> blockdev --rereadpt /dev/sda
</pre></div>
<p>The first command scan all the RAID array entries and close them. The second command re-read the partitions with UUIDs so you get the UUID entries for initramfs to boot.</p>
<p>This time I have confident to make the solution permanent. First of all, I will need to use my Ubuntu usb boot disk to enter the shell(&#39;Go Back&#39; in the first step of the installation, then you find a choice for executing a shell) and mount the partition. (My target Ubuntu parition is /dev/sda3)</p>
<div class="highlight"><pre><span></span>$ mount /dev/sda3 /mnt
$ mount -t proc /proc /mnt/proc
$ mount -t sysfs /sys /mnt/sys
$ mount -o <span class="nb">bind</span> /dev /mnt/dev
$ chroot /mnt /bin/bash
</pre></div>
<p>This allow me to mount the partition just as I&#39;ve booted into it. Then I will update it&#39;s boot process(e.g. grub &amp; initramfs).</p>
<div class="highlight"><pre><span></span>$ grub-install /dev/sda
$ update-grub

$ vi /etc/initramfs-tools/script/local-top/mdadm
mdadm --stop --scan
blockdev --rereadpt /dev/sda

$ chmod +x /etc/initramfs-tools/script/local-top/mdadm
$ update-initramfs -u
</pre></div>
<p>Voilà! My Ubuntu&#39;s back!&hellip;wait, my Windows 7 has gone! It seems that GRUB has erase my Windows 7 boot entry. This doesn&#39;t stop me now. It&#39;s easy to get Windows back.</p>
<p>I tried to run <code>update-grub</code> again, but it doesn&#39;t recognize the Windows boot entry even though os-prober is enabled. So I need to add it manually:</p>
<div class="highlight"><pre><span></span><span class="c1"># check the Windows partition UUID</span>
$ blkid

$ vi /boot/grub/grub.cfg
<span class="c1"># (look for the section of &quot;/etc/grub.d/40_custom&quot; and add the content below)</span>
menuentry <span class="s2">&quot;Windows 7&quot;</span> <span class="o">{</span>
    insmod ntfs
    <span class="nb">set</span> <span class="nv">root</span><span class="o">=</span><span class="s1">&#39;(hd0,1)&#39;</span>
    search --no-floppy --fs-uuid --set 76B0EE43B0EE0987
    chainloader +1
<span class="o">}</span>
</pre></div>
<p>Then reboot. It is finally OK.</p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Working with Grub2 in Linux]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2016/10/working-with-grub2-linux.html"/>
        <published>2016-10-21T10:21:00+08:00</published>
        <updated>2020-02-01T10:09:37+08:00</updated>
        <id>http://terryoy.github.io/2016/10/working-with-grub2-linux.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p><strong>Grub2</strong>(GRUB: the GRand Unified Bootloader) is the main bootloader in Debian/Ubuntu distribution. I have always wanted to know the details to change or fix the booting process. So I did a research and make a note here.</p>

<h3 id="toc_0">1. Overview</h3>
<p>Normally we should use the util <code>update-grub</code> to update the grub2 boot entries. In grub(the older, not grub2), we need to update the boot menu file <code>/boot/grub/menu.lst</code> carefully to manage the entries. In this case, it could be easy to make mistakes, or inconvenient while editing the file and get required information at the same time. </p>
<p>In grub2 with <code>update-grub</code>, this process can be managed in updating a general setting file and a list of scripts that generate entries. These files are:</p>

<ul>
<li><code>/etc/default/grub</code>, general grub settings,</li>
<li><code>/etc/grub.d/*</code>, this directory contains a list of executable scripts and will be processed in order. You can add the files according to the convention:

<ul>
<li>&ldquo;00_*&rdquo;: It is reserved for 00_header.</li>
<li>&ldquo;10_*&rdquo;: Native boot entries.</li>
<li>&ldquo;20_*&rdquo;: Third party apps (e.g. memtest86+).</li>
</ul></li>
</ul>
<p>The number namespace in-between is configuratble by system installer and/or administrator. The order of other number will be reflected in the menu if you set 01_otheros, 11_otheros, etc.</p>
<p>For example, I have below config files in <code>/etc/init.d/</code> for a ubuntu installation:</p>

<pre><code>00_header
05_debian_theme
10_linux
20_linux_gen
30_os-prober
30_uefi_firmware
40_custom
41_custom</code></pre>

<h3 id="toc_1">2. Clean up Grub2 Menu</h3>
<p>After a release or a kernel upgrade, you may usually find unuseful entries in the boot menu, which are mainly for previous state recovery. So here let&#39;s see how to remove the unuseful entries.</p>
<p>First are the unnecessary kernel entries. They exist because you install more older kernel versions lying around on your machine. So you can check if there are more than one <code>linux-headers-&lt;version&gt;</code> entry and <code>linux-image-&lt;version&gt;</code> entry, then remove the unnecessary package via package manager.</p>
<p>Second, unwanted entries in <code>/etc/init.d/</code> such as memtest86+. Disabling them is very easy, just make them not executable then it&#39;s done.</p>
<div class="highlight"><pre><span></span>$ <span class="nb">cd</span> /etc/init.d/
$ sudo chmod -x 20_memtest86+
</pre></div>
<p>And the last thing is to update grub:</p>
<div class="highlight"><pre><span></span>$ sudo update-grub
</pre></div>
<p>Sometimes if you just want to edit some of the boot menu item, you could lookup the <code>menuentry</code> in the file and edit the content. For example, there is a <code>30_os-prober</code> file in <code>/etc/grub.d/</code>, which checks if there are other OS exists on local drives and create boot entries for them. You can see the similar content in each OS part and using <code>cat</code> and <code>menuentry</code> to add an entry to the grub config.</p>

<h3 id="toc_2">3. Grub2 Settings</h3>
<p>In the general setting file <code>/etc/default/grub</code>, there are some basic settings that you could change.</p>

<pre><code># (template of /etc/default/grub)
# 1. Default boot entry, can be the index or the title of the boot item; SAVEDEFAULT option let you save every last choice as next default
GRUB_DEFAULT=0
GRUB_SAVEDEFAULT=true

# 2. Go directly to the default boot entry unless you press 'shift' or 'esc' at startup. The &quot;QUIET&quot; option will display the count down of the hidden timeout before going to boot, which you can use a chance for a boot splash
GRUB_HIDDEN_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true

# 3. Grub background, just simply set the image path(supports '.png', '.tga', '.jpg' or '.jpeg')
GRUB_BACKGROUND='/path/to/image.jpg'</code></pre>
<p>Ref: (run as shell cmd) <code>info -f grub -s &#39;Simple configuration&#39;</code></p>

<h3 id="toc_3">4. Grub2 tools</h3>

<h4 id="toc_4">4.1 Grub Rescue</h4>
<p>Many times when grub boot failed, it will run into <code>grub-rescue</code> shell environment. It is expected that you fix something(usually try to recover the damaged boot record.) and set a bootable entry for grub to continue the boot process</p>
<p>Below is an example of a common process.</p>
<div class="highlight"><pre><span></span><span class="c1"># check available parition</span>
grub rescue&gt; ls
<span class="o">(</span>hd0<span class="o">)</span>,<span class="o">(</span>hd0,msdos3<span class="o">)</span>,<span class="o">(</span>hd0,msdos2<span class="o">)</span>,<span class="o">(</span>hd0,msdos1<span class="o">)</span>

<span class="c1"># check available grub boot record in a partition</span>
grub rescue&gt; ls <span class="o">(</span>hd0,msdosX<span class="o">)</span>/boot/grub

<span class="c1"># set boot entry</span>
grub rescue&gt; <span class="nb">set</span> <span class="nv">root</span><span class="o">=(</span>hd0,msdos3<span class="o">)</span>
grub rescue&gt; <span class="nb">set</span> <span class="nv">prefix</span><span class="o">=(</span>hd0,msdos3<span class="o">)</span>/boot/grub
grub rescue&gt; insmod /boot/grub/normal.mod

<span class="c1"># go to startup screen</span>
grub rescue&gt;normal

<span class="c1"># (is this the alternative way?)</span>
rescue&gt;linux /boot/vmlinuz-xxx-xxx <span class="nv">root</span><span class="o">=</span>/dev/sdax
rescue&gt;initrd /boot/initrd.img-xxx-xxx
rescue&gt;boot
</pre></div>

<h3 id="toc_5">Reference</h3>

<h1 id="toc_6"></h1>
<p>The GRUB manual: (run in linux shell) <code>info grub</code></p>
<p>Clean up Grub2 entries: <a href="http://www.howtogeek.com/howto/17787/clean-up-the-new-ubuntu-grub2-boot-menu/">www.howtogeek.com/howto/17787/clean-up-the-new-ubuntu-grub2-boot-menu/</a>. (There is also a link to an article for the old <code>grub</code>)</p>
<p>How to Configure Grub2 boot loader settings: <a href="http://www.howtogeek.com/196655/how-to-configure-the-grub2-boot-loaders-settings/">www.howtogeek.com/196655/how-to-configure-the-grub2-boot-loaders-settings/</a></p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Frontend Workflow Toolbox Summary]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2016/09/frontend-workflow-summary.html"/>
        <published>2016-09-18T23:45:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2016/09/frontend-workflow-summary.html</id>
        <category scheme="http://terryoy.github.io/tag/#frontend" term="frontend" label="frontend" />
        <category scheme="http://terryoy.github.io/tag/#study" term="study" label="study" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>I&#39;m working my job as a frontend developer currently. So I will do some research and write about the tools I&#39;ve been using to develop Javasript frontend projects.</p>

<h3 id="toc_0">1. CSS</h3>

<h3 id="toc_1">2. Javascript</h3>

<h4 id="toc_2">2.1 Browserify</h4>
<p>Uses the same module system as nodejs that can compile, build, and pack javascript modules in node-flavoured CommonJS way for browser. You can manage frontend dependencies with npm, using <code>require()</code> method to import, and &ldquo;module.exports&rdquo; to export, just like the way nodejs does.</p>
<p>When compiling, browserify will start with an entry file(e.g. &#39;./app.js&#39;), then find the whole <code>abstract syntax tree</code> of the file by static analysis, and pack all the related content into one files.</p>
<p>In order to make node modules more compatible with browser side, browserify also provides many <a href="https://github.com/substack/browserify-handbook#builtins">builtins modules</a> such as <code>events</code>, <code>os</code>, <code>querystring</code>, etc.</p>
<p>The best part of browserify is that providing the static analysis and compiling platform, you can also have customized transformers for files other than the standard JS. This enables compiling coffee script or other favors, and also other plugins such as code coverage, in a pipe streaming way. </p>
<p>In bundling, browserify has two functions: </p>

<ol>
<li><code>ignore</code>, making a stub definition for some methods only available for node-specific modules</li>
<li><code>exclude</code>, to separate files in different bundle, so don&#39;t pack the specific files</li>
</ol>
<p>Reference: <a href="https://github.com/substack/browserify-handbook#introduction">github.com/substack/browserify-handbook#introduction</a></p>

<h4 id="toc_3">2.2 Gulp</h4>
<p>If you want to do something quick and dirty, just checkout the Gulp Recipes and copy the the code you want. The Recipes have prepare many examples of small tasks for you to reference.</p>
<p>References:</p>

<ul>
<li>Gulp:  <a href="https://github.com/gulpjs/gulp">github.com/gulpjs/gulp</a></li>
<li>Gulp + Browserify Guide: <a href="https://www.viget.com/articles/gulp-browserify-starter-faq">www.viget.com/articles/gulp-browserify-starter-faq</a></li>
<li>Gulp Recipes: <a href="https://github.com/gulpjs/gulp/tree/master/docs/recipes">github.com/gulpjs/gulp/tree/master/docs/recipes</a></li>
<li>Principles of Gulp Plugin: <a href="https://github.com/gulpjs/gulp/blob/master/docs/writing-a-plugin/README.md">github.com/gulpjs/gulp/blob/master/docs/writing-a-plugin/README.md</a></li>
</ul>

<h4 id="toc_4">2.3 Babel</h4>
<p>The language transfrm platform and the cutting edge of ES(ECMAScript/Javascript) standard. It can transform ES6(ES2015), ES7(ES2016) scripts to compatible ES5. So you can use it as a <code>transformer</code> in browserify and compile the files you have written in ES6, etc. The official web site says the main features about it:</p>

<ul>
<li>Babel Presets,</li>
<li>Babel Polyfill, new globals such as Promise or new native methods. </li>
<li>JSX and Flow support</li>
</ul>
<p>References:</p>

<ul>
<li>ES6: <a href="https://github.com/rse/es6-features">github.com/rse/es6-features</a></li>
<li>Babel: <a href="http://babeljs.io/">babeljs.io/</a></li>
<li>Babel Plugins(Presets, Stage-X, and Transform Plugins): <a href="http://babeljs.io/docs/plugins/">babeljs.io/docs/plugins/</a> </li>
</ul>

<h4 id="toc_5">2.4  Webpack</h4>

<h3 id="toc_6">3. Others</h3>

<h4 id="toc_7">3.1 Coffee Script</h4>

<h4 id="toc_8">3,2 Type Script</h4>

<h3 id="toc_9">4. Vocabulary</h3>
<p>There are some terms in web frontend development that is not meaning obviously, so I collect some of them here.</p>

<ul>
<li><p><a href="https://en.wikipedia.org/wiki/Polyfill"><strong>Polyfill</strong></a> - a polyfill is cod e that implements a feature on web browsers that do <em>not</em> support it. It is a <em>shim</em> for a browser API.</p></li>
<li><p><a href="https://en.wikipedia.org/wiki/Shim_(computing)"><strong>Shim</strong></a> - A small library that transparently intercepts API calls. This term can be  commonly used to support an old API in a newer environment, or a new API in an older environment, or running programs on a different platform than they were developed for.</p></li>
<li><p><a href="https://en.wikipedia.org/wiki/Glob_(programming"><strong>glob</strong></a>) - A pattern that specifies a set of files with wildcard characters. It is originally a program <code>/etc/glob</code> that look for files by patterns, then it is used in a lof of libraries in many language. The original word is short for &#39;global command&#39;.</p></li>
</ul>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Put Linux into Old PC - (3) Localize the Language Environment]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2016/06/put-linux-into-old-pc-3-localization.html"/>
        <published>2016-06-28T21:03:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2016/06/put-linux-into-old-pc-3-localization.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <category scheme="http://terryoy.github.io/tag/#study" term="study" label="study" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <h3 id="toc_0">Part 3. Localize the language environment</h3>
<p>It is very easy to support the multi-language environment if you&#39;re installing the normal Linux desktop. However, it takes a few steps if you want to try the manual way. To make it more challenging, I would like to add Japanese support(the second foreign language I&#39;m going to learn) together with Chinese support which is my mother language.</p>
<p>The job to enable this multi-language environment includes three parts:</p>

<ul>
<li>Able to view the Asian characters,</li>
<li>Able to input the Asian characters,</li>
<li>The user interface are translated into the familiar language. (However, I might prefer an English environment just for working~)</li>
</ul>

<h4 id="toc_1">3.1 Enabling the Input Method</h4>
<p>Now I want to enable the input method at first, because it enable me to try enter other characters. There are several approaches to support multi-language input method, such as <code>SCIM</code>, <code>ibus</code>, and <code>uim</code>. The currently recommendation is <strong>ibus</strong>. There are several popular alternatives for both Japanese and Chinese input methods. <code>Anthy</code> and <code>mozc</code> are for Japanese, and <code>pinyin</code>, <code>sunpinyin</code> and <code>google-pinyin</code> are for Chinese(let&#39;s hope that Sogou Pinyin is joining this competition~). You can choose as you prefer, but I will first install only two basic choices.</p>
<div class="highlight"><pre><span></span><span class="c1"># Install Anthy and Pinyin</span>
$ sudo apt-get install ibus-anthy ibus-pinyin

<span class="c1"># Setup the configurations for the first time (you may need to manually add the input method in the enabled list)</span>
<span class="c1"># (for input method config)</span>
$ im-config
<span class="c1"># (for ibus config)</span>
$ ibus-setup
</pre></div>
<p>Be sure to change the method switching keyboard shortcuts to &ldquo;Ctrl+Space&rdquo;, otherwise you may not correctly enable it(by default, it uses &ldquo;Super+Space&rdquo; to switch between but I have no &ldquo;Super&rdquo; key on my keyboard).</p>
<p>After all the steps above, the ibus input method is still not enabled at start up, which means you need to manually start the daemon by executing &ldquo;ibus-setup&rdquo;. However, no doubt that I will fix this soon.</p>
<p>To enable things at X start up, you need to put commands in <code>~/.xprofile</code> or <code>~/.xinitrc</code>, depends on the windows manager requirements. <code>~/.xprofile</code> is often used before windows manager start up, while <code>~/.xinitrc</code> is used by &ldquo;initx&rdquo; and its frontend &ldquo;startx&rdquo; to initialize the X window manager(remember we put &ldquo;exec openbox-session&rdquo; in this file previously?). The <code>ibus-daemon</code> program starts the ibus daemon server, so if I want to start ibus when X starts, I will put it in the ~/.xinitrc file. <a href="#ref">[2]</a></p>
<div class="highlight"><pre><span></span>$ vi ~/.xinitrc
<span class="c1"># (add at the end of the file)</span>
ibus-daemon -drx
</pre></div>
<p>You can check out what the part of &ldquo;-drx&rdquo; means by <code>man ibus-daemon</code>. It&#39;s easy.</p>

<h4 id="toc_2">3.2 Testing the Input method</h4>
<p>There are a lot of applications that supports input method. Two I have tried, the browser and the terminal. Not every termimal emulator supports ibus input. Luckily, I have chosen <code>lxterminal</code> and it does the job.</p>
<p>The first and easy test is on the browser. I changed the keyboard shortcut for &ldquo;switching to next input method&rdquo; to Ctrl+Space, because the default one is Super+Space but my <code>Super</code> key isn&#39;t available at the time and <code>Ctrl</code> is more like my preference. </p>
<p>The Japanese font seems to be already available with the installation above, but the Chinese font isn&#39;t. So, by installing some extra Chinese fonts, you should be able to see most of the east asian characters.</p>
<div class="highlight"><pre><span></span><span class="c1"># Chinese Font</span>
$ sudo apt-get install fonts-wqy-microhei fonts-wqy-zenhei
</pre></div>
<p>The &ldquo;Typing Method&rdquo; in the Japanese input method is more like a switch from Pinyin to Wubi in Chinese. My Japanese keyboard has Hiragana characters written on the keys, so I can choose &ldquo;Kana&rdquo; as typing method(a little bit like Wubi, typing words in characters). If you choose &ldquo;Romanji&rdquo; as the typing method, then you should input the characters in a phonetic way.</p>

<h4 id="toc_3">3.3 Localizing the UI</h4>
<p>The English UI is good enough for me in the most circumstances. But still, I want to look into the details for changing the language of applicatoin interfaces.</p>

<h5 id="toc_4">Basic Locale Settings</h5>
<p>The system variable &ldquo;LANG=xx_YY.ZZZZ&rdquo; has defined the language settings in three ways: <code>language code</code>, <code>country code</code>, and <code>encoding</code>. (Notice the upper cases and the lower cases.) This variable affects the language interface of most of the applications. To do an experiement, try to set the variable as below, before entering the X window environment. You will see the menu of applications are changed to a different font, but the language is possibly still in English. This is because the application has no Chinese locale interface installed.</p>
<div class="highlight"><pre><span></span>$ <span class="nb">export</span> <span class="nv">LANG</span><span class="o">=</span>zh_CN.UTF-8
</pre></div>

<h3 id="toc_5">Reference</h3>
<p><a name="ref"></a>
[1] Debian i18n: <a href="https://www.debian.org/doc/manuals/debian-reference/ch08.en.html">www.debian.org/doc/manuals/debian-reference/ch08.en.html</a> <br/>
[2] IBus: <a href="https://wiki.archlinux.org/index.php/IBus">wiki.archlinux.org/index.php/IBus</a></p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[A Short Reference of Python Logging]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2016/05/short-ref-python-logging.html"/>
        <published>2016-05-26T19:55:00+08:00</published>
        <updated>2020-02-01T10:09:37+08:00</updated>
        <id>http://terryoy.github.io/2016/05/short-ref-python-logging.html</id>
        <category scheme="http://terryoy.github.io/tag/#python" term="python" label="python" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>I have for many times use the logging function, but never understand it completely. So I go through the document and make some notes, hoping it will help me use it more quickly in the future. </p>

<h3 id="toc_0">1. Basic Config</h3>
<p>If you want to use a programmable method other than a configuration file, the basicConfig() method is the general initializing method.</p>
<p>The most basic form is default log, which you don&#39;t need basicConfig(). It is using console output with WARNING level.</p>
<div class="highlight"><pre><span></span><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">logging</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">logging</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">&#39;hello&#39;</span><span class="p">)</span> <span class="c1"># no output</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">logging</span><span class="o">.</span><span class="n">warn</span><span class="p">(</span><span class="s1">&#39;world&#39;</span><span class="p">)</span>
<span class="n">world</span>
</pre></div>
<p>The basic config contains a list of below elements:</p>

<ul>
<li>  filename    - Using a <em>FileHandler</em> to output the log</li>
<li>  filemode    - file open mode(&#39;r&#39;, &#39;w&#39;, &#39;a&#39;), mainly used to choose append or write a new log file</li>
<li>  format      - A string for specifying the log output template, If you want to lookup a list of supported keywords, look for section &#39;LogRecord attributes&#39; in the python official document</li>
<li>  datefmt     - A specified date/time format.</li>
<li>  level       - set the <em>root</em> loglevel for the logger</li>
<li>  stream      - Specify a stream for the StreamHandler, for example, a buffer output stream or stdout. It will be ignored if &ldquo;filename&rdquo; is present.</li>
</ul>
<p>The logger can be initialized only once when basicConfig() is called. Then </p>
<div class="highlight"><pre><span></span><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">logging</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">logging</span><span class="o">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="n">filename</span><span class="o">=</span><span class="s1">&#39;program.log&#39;</span><span class="p">,</span> 
        <span class="n">filemode</span><span class="o">=</span><span class="s1">&#39;a&#39;</span><span class="p">,</span>
        <span class="nb">format</span><span class="o">=</span><span class="s2">&quot;</span><span class="si">%(asctime)-15s</span><span class="s2"> </span><span class="si">%(levelname)s</span><span class="s2"> [</span><span class="si">%(module)s</span><span class="s2">] </span><span class="si">%(message)s</span><span class="s2">&quot;</span><span class="p">,</span>
        <span class="n">datefmt</span><span class="o">=</span><span class="s2">&quot;%Y-%m-</span><span class="si">%d</span><span class="s2"> %H:%M:%S.</span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span>
        <span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
</pre></div>

<h3 id="toc_1">2. Configuration Object and the Modular Approach</h3>
<p>When choosing the Modular Approach of logging, you need to deal with 4 elements:</p>

<ul>
<li>  loggers     - the interface that application modules used to log things</li>
<li>  handlers    - send the log records (that loggers created) to the appropriate destinations</li>
<li>  filters     - provide a finer grained facility for determining which log records should be output</li>
<li>  formatters  - specify the layout of the log records in the final output</li>
</ul>

<h4 id="toc_2">2.1 Logger hierarchy</h4>
<p>The loggers used by all the modules are formed in a conceptual hierarchy by the naming with a separator(&#39;.&#39;). For example: &#39;abc.text&#39;, is the descendant of logger &#39;abc&#39;, while &#39;abc&#39; can be the parent of &#39;abc.text&#39;, &#39;abc.pdf&#39;, &#39;abc.image&#39;, etc. A good convention is to use loggers in a module sense, using in each <code>.py</code> as below:</p>
<div class="highlight"><pre><span></span><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
</pre></div>
<p>The root of all loggers is called the &ldquo;root&rdquo; logger, which prints the logger name as &ldquo;ROOT&rdquo; in output.
</p>

<h4 id="toc_3">2.2 Useful handlers</h4>
<p>There are some useful handlers in the section of python <em>Logging Howto</em> document. Some of them are listed as here:</p>

<ul>
<li>  StreamHandler   - to stream object (default stdin?)</li>
<li>  FileHandler     - to a disk file</li>
<li>  <strong>RotatingFileHandler</strong>         - from <em>BaseRotatingHandler</em>, send logs to files, rotating log file with a maximum file size.</li>
<li>  <strong>TimedRotatingFileHandler</strong>    - from <em>BaseRotatingHandler</em>, send logs to files, rotating log file at a certain timed intervals.</li>
<li>  SocketHandler/DatagramHandler   - send log messages to TCP/IP and UDP sockets</li>
<li>  SMTPHandler     - Send to a designated email address</li>
<li>  NullHandler     - Do nothing, it&#39;s used in development that supports logging with this mock</li>
</ul>
<p>This shows the variety of logging output scenarios, which you could look them up in the python doc.</p>

<h4 id="toc_4">2.3 The propagation of loggers</h4>
<p>Look at the flow of logging in the below diagram from <a href="https://docs.python.org/2/howto/logging.html#logging-advanced-tutorial">python&#39;s tutorial</a>,</p>
<p><img src="https://docs.python.org/2/_images/logging_flow.png" alt="logging flow"/></p>
<p>When a log record is send to the logger in the module, it will first check if its own filter(the filter of a logger) reject it, then pass to its handler; if propagation is set to true(by default), it will pass the log record to it&#39;s parent too, so the log record will bubble up till the root logger, and each logger will judge by their handler and filter to decide whether to output the log record. So we often setup a top level logger, and then configure a child logger only if needed.</p>

<h4 id="toc_5">2.4 Configuring Logging</h4>
<p>The most usual approaches are using <code>fileConfig()</code> and <code>dictConfig()</code>. With fileConfig() you can use a <strong>.conf</strong> file to load the settings (this approach is deprecated), and with dictConfig() you can use even wider range of persistence choices, such as JSON, python file, yaml, etc.</p>
<p>For example, I have written a small utils for command line interaction and also want to log the HTTP request details. So I defined two handlers: one for console output, another for file output so that I can review the details. The console output must be simple without unneccessary information, and the file output should contains all the time, module details for investigation. Here is my configuration using a python file. (The advantages for a python configuration is that you can also use expressions and comments.)</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span><span class="o">,</span> <span class="nn">logging.config</span>

<span class="n">config</span> <span class="o">=</span> <span class="p">{</span>
    <span class="s2">&quot;log_config&quot;</span><span class="p">:</span> <span class="p">{</span>
        <span class="s2">&quot;version&quot;</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
        <span class="s2">&quot;formatters&quot;</span><span class="p">:</span> <span class="p">{</span>
            <span class="s2">&quot;brief&quot;</span><span class="p">:</span> <span class="p">{</span>
                <span class="s2">&quot;format&quot;</span><span class="p">:</span> <span class="s2">&quot;</span><span class="si">%(message)s</span><span class="s2">&quot;</span><span class="p">,</span>
            <span class="p">},</span>
            <span class="s2">&quot;detail&quot;</span><span class="p">:</span> <span class="p">{</span>
                <span class="s2">&quot;format&quot;</span><span class="p">:</span> <span class="s2">&quot;</span><span class="si">%(asctime)-15s</span><span class="s2"> </span><span class="si">%(levelname)s</span><span class="s2"> [</span><span class="si">%(name)s</span><span class="s2">.</span><span class="si">%(funcName)s</span><span class="s2">] </span><span class="si">%(message)s</span><span class="s2">&quot;</span><span class="p">,</span>
                <span class="s2">&quot;datefmt&quot;</span><span class="p">:</span> <span class="s1">&#39;%Y-%m-</span><span class="si">%d</span><span class="s1"> %H:%M:%S&#39;</span><span class="p">,</span>
            <span class="p">},</span>
        <span class="p">},</span>
        <span class="s2">&quot;handlers&quot;</span><span class="p">:</span> <span class="p">{</span>
            <span class="s2">&quot;console&quot;</span><span class="p">:</span> <span class="p">{</span>
                <span class="s2">&quot;class&quot;</span><span class="p">:</span> <span class="s2">&quot;logging.StreamHandler&quot;</span><span class="p">,</span>
                <span class="s2">&quot;level&quot;</span><span class="p">:</span> <span class="s2">&quot;INFO&quot;</span><span class="p">,</span>
                <span class="s2">&quot;formatter&quot;</span><span class="p">:</span> <span class="s2">&quot;brief&quot;</span><span class="p">,</span>
            <span class="p">},</span>
            <span class="s2">&quot;file&quot;</span><span class="p">:</span> <span class="p">{</span>
                <span class="s2">&quot;class&quot;</span><span class="p">:</span> <span class="s2">&quot;logging.handlers.RotatingFileHandler&quot;</span><span class="p">,</span>
                <span class="s2">&quot;filename&quot;</span><span class="p">:</span> <span class="s2">&quot;dev.log&quot;</span><span class="p">,</span>
                <span class="s2">&quot;level&quot;</span><span class="p">:</span> <span class="s2">&quot;DEBUG&quot;</span><span class="p">,</span>
                <span class="s2">&quot;formatter&quot;</span><span class="p">:</span> <span class="s2">&quot;detail&quot;</span><span class="p">,</span>
            <span class="p">},</span>
        <span class="p">},</span>
        <span class="s2">&quot;root&quot;</span><span class="p">:</span> <span class="p">{</span>
            <span class="s2">&quot;handlers&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;console&quot;</span><span class="p">,</span> <span class="s2">&quot;file&quot;</span><span class="p">],</span>
            <span class="s2">&quot;level&quot;</span><span class="p">:</span> <span class="s2">&quot;DEBUG&quot;</span><span class="p">,</span>
        <span class="p">},</span>
        <span class="s2">&quot;loggers&quot;</span><span class="p">:</span> <span class="p">{</span>
            <span class="s2">&quot;requests&quot;</span><span class="p">:</span> <span class="p">{</span>
                <span class="s2">&quot;handlers&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;file&quot;</span><span class="p">],</span>
                <span class="s2">&quot;level&quot;</span><span class="p">:</span> <span class="s2">&quot;DEBUG&quot;</span><span class="p">,</span>
                <span class="s2">&quot;propagate&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
            <span class="p">}</span>
        <span class="p">},</span>
    <span class="p">},</span>
<span class="p">}</span>

<span class="n">logging</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">dictConfig</span><span class="p">(</span><span class="n">config</span><span class="p">[</span><span class="s2">&quot;log_config&quot;</span><span class="p">])</span>
</pre></div>
<p>There are two formatters: &ldquo;brief&rdquo; for simply output the message body, the root loggers is default logger for all the modules I write, and also the 3rd party libraries like python <code>requests</code>. Since all the info log should appear in both <strong>console</strong> and the <strong>file</strong>, I need to put both inthe root logger. However, to avoid the unneccssary debug log showing in console, I set the level <strong>INFO</strong> in the <strong>console</strong> handler. This enables the file logger logs everything while the console doesn&#39;t. </p>
<p>Next I discover that the library &ldquo;requests&rdquo; also have some &ldquo;INFO&rdquo; log which is unneccessary in console, so I will specificially make it disappear using the <strong>loggers</strong> config. The important thing here is to use the <strong>propagate</strong> feature. </p>
<p>The &ldquo;file&rdquo; logger wants the requests&#39; debug log, so I need to set the level to DEBUG. By default, it will propagate the log record to the &ldquo;root&rdquo; logger which make it appear to console. So I will use <code>propagate: False</code> to disable the propagation. Then the log records  will stay in the &ldquo;requests&rdquo; logger and will be invisible to the &ldquo;root&rdquo; logger.</p>
<p>If you&#39;re not sure what to config with, write a small example project to experiment the result.</p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Put Linux into Old PC - (2) Desktop Environment]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2016/05/put-linux-into-old-pc-2-desktop.html"/>
        <published>2016-05-18T09:47:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2016/05/put-linux-into-old-pc-2-desktop.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <category scheme="http://terryoy.github.io/tag/#study" term="study" label="study" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <h3 id="toc_0">Part 2. Setup a minimal desktop environment</h3>

<h4 id="toc_1">2.1 Xorg and Openbox</h4>
<p>To use a desktop environment, <strong>Xorg</strong> is the package you won&#39;t miss(for more information you can also check the <a href="htts://www.x.org/">x.org</a> and <a href="https://www.freedesktop.org/">freedesktop.org</a> web-sites. I choose <strong>Openbox</strong> as the window manager, which is one of the most lightweight window manager. It is the best showcase of minimalism, yet the functions shall be simple enough for customization. The window manager is not a desktop environment, it is just simply a manager to windows, their styles and behaviors. A complete desktop environment is not necessary, or maybe it&#39;s the chance that I could design my own.</p>
<div class="highlight"><pre><span></span>$ sudo apt-get install xorg
<span class="c1"># now you can enter the GUI environment by command &quot;startx&quot;, but you still need a window manager to do further things</span>
$ sudo apt-get install openbox obconf openbox-themes
<span class="c1"># the above packages and their dependencies will allow you to build and configure an Openbox system, along with choices of themes.</span>
</pre></div>
<p>What&#39;s interesting about &ldquo;startx&rdquo; is that, if you try it after you install xorg and before any Window Manager, you&#39;ll get a graphical shell environment with mouse available. The font is definitly different from what you had in the normal console. It is I think the essential GUI environment possible. And if you press Ctrl+D to exit the session, you get back to your original console environment, so the <strong>xorg-session</strong> is exited.</p>
<p>After installing the openbox package, it&#39;s not yet completed. You also need to add the a <code>~/.xinitrc</code> configuration so that it calls the openbox when you execute &ldquo;startx&rdquo;.</p>
<div class="highlight"><pre><span></span>$ <span class="nb">echo</span> <span class="s2">&quot;exec openbox-session&quot;</span> &gt; ~/.xinitrc
$ startx
</pre></div>
<p>Now you can see GUI environment by Openbox. Right click on the desktop and you get a start menu, you can start a terminal emulator or configure the openbox using <em>Obconf</em>.</p>
<p>If you see an error says cannot get access to ~/.Xauthority, it&#39;s possibly you ran it in root user. It doesn&#39;t need to be root user, and you should chown the file to yourself.</p>
<p>Before going to the next step, let&#39;s look for a better console terminal than the system default one(xterm), otherwise it&#39;s inconvenient to work on multiple sessions and copy&amp;paste text. The <code>lxterminal</code> is lightweight and good enough.</p>
<div class="highlight"><pre><span></span>$ sudo apt-get install lxterminal
</pre></div>
<p>Then you can try it out with the right-click menu and select &ldquo;Terminal emulator&rdquo;. You can check the key-bindings in LXterminal&#39;s Preferences menu.</p>
<p>Next we try to do more about the customization.</p>

<h4 id="toc_2">2.2 Openbox Configuration (menu.xml and rc.xml)</h4>
<p>Openbox has two main configuration files:</p>

<ul>
<li>rc.xml, which sets keybindings, desktop names and window behavior</li>
<li>menu.xml, which contains the instructions for the right-click menu.</li>
</ul>
<p>The configuration files can be found in <code>/etc/xdg/openbox/</code>, while you can have your user-specific version in <code>~/.config/openbox/</code>.</p>
<p>Reference for OpenBox: <a href="http://openbox.org/wiki/Help:Configuration">http://openbox.org/wiki/Help:Configuration</a></p>

<h5 id="toc_3">2.2.1 Desktop behaviors</h5>
<p>First we look into some settings in <strong>rc.xml</strong>. Most of the configuration items you can found them in the <code>Obconf</code> program, which is right in the desktop&#39;s right-click menu.</p>
<p>The configurations are mainly in below categories:</p>

<ul>
<li>Resistance, distances for handling window moving behavior, such as overlap and moving towards different desktops.</li>
<li>Focus, behaviors for switching window focus with mouse.</li>
<li>Placement, decides the position when a new window is displayed</li>
<li>Theme, this is where you can customize the visual design of the environment. (I will cover the Theme customization in the following section.)</li>
<li>Desktops, desktop behaviors</li>
<li>Resize, resizing and moving windows</li>
<li>Margins, the desktop margin settings(the boundaries where windows will not resize to&hellip;)</li>
<li>Applications, customize settings per application. You can also configure this with an GUI app <code>ObApp</code> <em>(will cover later)</em></li>
<li>Keyboard, keyboard shortcuts and behaviours</li>
<li>Mouse, mouse shortcuts and behaviours</li>
<li>Menu, right-click(system) menu behaviours. </li>
<li>Dock, dock-apps settings, it&#39;s kind of apps that will have docking behaviours</li>
</ul>
<p>The other configuration files is <strong>menu.xml</strong>. It defines the system menu content and is referred by the <em>rc.xml</em>. Since the content are quite obvious, I don&#39;t need to tell more about it. Check out the <a href="http://openbox.org/wiki/Help:Menus">wiki page</a> for customizing menus, notice it can also has &ldquo;Pipe menus&rdquo; where the menu content is genereted from the pipe of the command line results, which means you can programs to hack. ;-)</p>
<p>There are also two files under <code>/etc/xdg/openbox</code>. One is <strong>autostart</strong>, you can include some initializing scripts here when an Openbox X Session is started; the other file is <strong>environment</strong>, where you can set system wide variables for the Openbox session. Both of them can be user specific by creating files with the same name under <code>~/.config/openbox/</code>.</p>

<h5 id="toc_4">2.2.2 Openbox Themes</h5>
<p>You can create your own themes for better looking of the desktop environment. The file structure is like below:</p>

<pre><code>ThemesDirectory  (such as /usr/share/themes, ~/.local/share/themes or ~/.themes)
 |
 +-&gt; ThemeName  (This is the name of the theme, such as Clearlooks)
      |
      +-&gt; openbox-3  (This the type of the theme - it's for Openbox 3!)
           |
           |-&gt; themerc  (This is the main theme file, documented in this page)
           |
           |-&gt; max.xbm  (These are optional xbm masks for the titlebar buttons)
           |-&gt; close.xbm
           ...
           +-&gt; shade.xbm</code></pre>
<p>I have created a theme called &ldquo;LinuxInOldPC&rdquo; under &ldquo;~/.themes/&rdquo;, with the <code>openbox-3</code> folder and the default <code>themerc</code> file inside. The content template can be copied from the wiki page at the end of this section.</p>
<p>Wiki of Openbox Themes: <a href="http://openbox.org/wiki/Help:Themes">http://openbox.org/wiki/Help:Themes</a></p>

<h5 id="toc_5">2.2.3 Key-bindings</h5>
<p>Here is a small trick where I would love to have, to conveniently dock the active window to part of the screen.</p>
<p>Reference for Windows docking: <a href="https://gist.github.com/terryoy/5486156">https://gist.github.com/terryoy/5486156</a></p>
<p>Another OpenBox Guide: <a href="https://urukrama.wordpress.com/openbox-guide/](https://urukrama.wordpress.com/openbox-guide/)">urukrama.wordpress.com/openbox-guide/](urukrama.wordpress.com/openbox-guide/)</a></p>

<h4 id="toc_6">2.3 A simple web browser that do the trick</h4>
<p>After searching the apt-cache library for web browser, I discover there is a simple browser called <code>surf</code>, which uses WebKit/Gtk to render. I think this tool is well enough for browsing modern web sites, and also friendly to command line. So I pick it for the initial choice of a browser for me to search information online on this computer.</p>
<div class="highlight"><pre><span></span>$ sudo apt-get install surf
</pre></div>
<p><strong>Surf</strong> has no tab support, no menu no any distractions. If you want to navigate to urls, hit <strong>Ctrl-g</strong> to enter a new URL.</p>
<p>Reference for &ldquo;surf&rdquo;: <a href="http://surf.suckless.org/">http://surf.suckless.org/</a></p>
<p>Next I also give a try to Chromium, the open source version of Chrome. It seems this more sophisticated version of webkit browser also works fine under linux. The performance on loading static web site is quick. So I decided to use Chromium as my default currently.</p>
<p>But now we have a problem here, what if I want to change the default browser in OpenBox? Debian provides a program call <strong>update-alternatives</strong> so that you could set default program for specific purpose.</p>
<div class="highlight"><pre><span></span><span class="c1"># list current default program settings</span>
$ update-alternatives --get-selections

<span class="c1"># update a default program</span>
$ sudo update-alternatives --config x-www-browser
</pre></div>
<p>It will prompt you an option list of possible programs, so that you can easily decide which program to use.</p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Put Linux into Old PC - (1) Installation and Hardware]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2016/05/put-linux-into-old-pc-1-hardware.html"/>
        <published>2016-05-15T21:04:00+08:00</published>
        <updated>2020-02-01T10:09:36+08:00</updated>
        <id>http://terryoy.github.io/2016/05/put-linux-into-old-pc-1-hardware.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <category scheme="http://terryoy.github.io/tag/#study" term="study" label="study" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <h3 id="toc_0">0. How this series begin?</h3>
<p>I bought from Taobao a used Japanese IBM G40 PC, which has Pentium 4 (2.5GHz) processor, 512MB RAM and a 140GB hard disk (I believe the hardware spec has been changed by the used PC seller). Since the power consumption and the speed are not suitable for very modern applications, I decided to use it as an experiment machine for creating a minimal linux workspace. This series of blog posts will be the progress showing how I work on this machine to make it very friendly and efficient for daily used.</p>

<h3 id="toc_1">Part 1. System Setup</h3>
<p>I started to boot with the Ubuntu 16.04 LTS server image(flashed on a USB stick), the booting is a bit strange than previous versions, it shows a &ldquo;boot:&rdquo; prompt which need you to tells what to boot, but it&#39;s not difficult to find out how to boot into the installation mode.</p>
<p>The reason using Ubuntu latest version is because I want to keep up with the development of the software packages, Debian is in a very stable status, while the Ubuntu packages are active enough to try out new things. I format the whole disk and install Ubuntu on it. This is the starting point.</p>

<h4 id="toc_2">1.1 Problem 1: hibernation on lid close</h4>
<p>I&#39;m so exciting at the moment when I finish all the installation and reboot, because I finally get this project started! But the first problem come very quickly - <strong>when I try to lower the lid, the screen just go dark, and when I reopen it, the going hibernate process sticks in every few seconds</strong>. I think there is some problem with the led sensor, because it automatically goes into hibernate a few seconds later when I activate again. So I think it is related to the power options in system settings.</p>
<p>This problem leads me to know about the <strong>&ldquo;systemd&rdquo;</strong> program. Searching solutions on the internet, I find out the settings in systemd will handle the power related key event. It sets how system react when user press Power key, Hibernate Key, Lid close/open, etc.</p>
<div class="highlight"><pre><span></span>$ sudo vi /etc/systemd/logind.conf
<span class="c1"># --in the file, change below line</span>
<span class="c1">#HandleLidSwitch=suspend</span>
<span class="c1"># --to</span>
<span class="nv">HandleLidSwitch</span><span class="o">=</span>ignore

$ sudo service systemd-logind restart
</pre></div>
<p>This helps me to get over the hibernation problem at once. <strong>Systemd</strong> is a Linux system and service manager. There is similar programs like ubuntu&#39;s &ldquo;upstart&rdquo;, or Mac OS&#39;s &ldquo;launchd&rdquo;. You can search for more info about it. <strong>The man page of systemd(init) is definitely worth reading to understand the first process of the system.</strong></p>
<p>There are two other power related packages <code>acpi</code> and <code>acpid</code>. The Ubuntu document recommends that you can remove the packages if you do not have a laptop. Might read that later.</p>

<h4 id="toc_3">1.2 Problem 2: Japanese keyboard layout</h4>
<p>Although I set some Japanese keyboard layout in the installation, but it doesn&#39;t match all the keys with my Japanese IBM G40 keyboard. Apparently I need to switch the keyboard layout and do a few tests. The configuration program I used here is <code>dpkg-reconfigure</code>, which allows you to configure a package again after they&#39;re installed. The package will be &ldquo;keyboard-configuration&rdquo;.</p>
<div class="highlight"><pre><span></span><span class="c1"># choose a different keyboard mapping</span>
$ sudo dpkg-reconfigure keyboard-configuration
</pre></div>
<p>The above one already solved the problem. I changed the keyboard model to IBM ThinkPad T60/R60/T61/R61, then it says the layout of this keyboard varies in different country, so I&#39;m able to choose the &ldquo;Japanese&rdquo; as the country of the origin layout. </p>
<p>Some threads mentioned about the <code>console-setup</code> as I found out it is mainly for the encoding and the font set in the console, it might be related to the problem but not at this point. I may referred back if I try to work with Chinese character in the console.</p>
<div class="highlight"><pre><span></span><span class="c1"># This change some console character set options</span>
$ sudo dpkg-reconfigure console-setup
</pre></div>
<p>A little more on dpkg-reconfigure, if you use <code>man dpkg-recongure</code> to check its manual, you will find another related command called <code>debconf-show</code>. It shows you all the current configurations of the package. So before you go with &ldquo;dpkg-reconfigure&rdquo;, you can check the configuration first.</p>
<div class="highlight"><pre><span></span>$ sudo debconf-show keyboard-configuration
</pre></div>
<p>It shows how many configuration items of the package and their values. Check out the man page for &ldquo;keyboard-configuration&rdquo;, &ldquo;console-setup&rdquo;, &ldquo;dpkg-reconfigure&rdquo; and &ldquo;debconf-show&rdquo; to explore more.</p>

<h4 id="toc_4">1.3 Remapping the unused keys</h4>
<p>The Japanese keyboard on my laptop doesn&#39;t have a Win(or Super) key, but have some abandoned key for old Japanese input. So I kind of like to remap the key to be more useful, such as using one as Super key.</p>
<p>Reference for <strong>key remapping</strong>: <a href="http://askubuntu.com/questions/24916/how-do-i-remap-certain-keys-or-devices">askubuntu.com/questions/24916/how-do-i-remap-certain-keys-or-devices</a></p>

<h4 id="toc_5">1.4 Problem 3: WiFi Connection</h4>
<p>The original G40 doesn&#39;t have wireless network connection. Fortunately I have a spare USB wifi adapter, but I need to configure the WiFi manually via command line. The adapter I used is Ralink RT5370, which is already supported in the kernel, so I don&#39;t need to explicitly install a driver for it.</p>
<p>First I could use <code>lsusb</code> to check that device is recognized. Then I move on to updating the configuration in <code>/etc/network/interfaces</code>. Check out the man page of <strong>interfaces</strong>, and you will find the information of the keywords and syntax for configuring the network interface. For example:</p>

<ul>
<li>  <em>auto</em>, telling <code>ifup</code> to get this interface up automatically</li>
<li>  <em>iface</em>, define an interface (template) using different methods(inet, inet6, etc.)</li>
<li>  <em>allow-\</em>, allow the interface to be brought up by various sub-system(e.g. allow-hotplug, allow-auto, etc.)</li>
<li>  <em>pre-up</em>, <em>post-down</em>, commands to be execute before the interface is up or after it is down</li>
</ul>
<p>The configuration syntax is not difficult, so I added the below lines to enable the wifi adapter at system startup:</p>

<pre><code>auto wlan0
iface wlan0 inet dhcp
iface wlan0 inet6 auto</code></pre>
<p>Next I discover that my WiFi adapter is not called &ldquo;wlan0&rdquo;(you can check that by command <code>iwconfig</code>). Instead, it is called &ldquo;wlx5c63bf2a8b28&rdquo;, no wonder why I cannot bring it up when execute ifup. So now I need to change the name for it to make it more readable using <strong>udev</strong>. udev is responsible for which device gets which name. By the Systemd v197 standard of &ldquo;Predictable Network Interface Names&rdquo;, interfaces are prefixed with &ldquo;en&rdquo; for ethernet, &ldquo;wl&rdquo; for WLAN, and &ldquo;WW&rdquo; for WWAN. </p>
<div class="highlight"><pre><span></span><span class="c1"># you check the interface entires list first, and fine the MAC address we&#39;ll need in udev</span>
$ ip link
<span class="c1"># (or alternatively for wireless interfaces)</span>
$ iw dev

<span class="c1"># update udev configuration</span>
$ sudo vi /etc/udev/rules.d/10-network.rules
<span class="c1"># add below line and save</span>
<span class="nv">SUBSYSTEM</span><span class="o">==</span><span class="s2">&quot;net&quot;</span>, <span class="nv">ACTION</span><span class="o">==</span><span class="s2">&quot;add&quot;</span>, ATTR<span class="o">{</span>address<span class="o">}==</span><span class="s2">&quot;5c:63:bf:2a:8b:28&quot;</span>, <span class="nv">NAME</span><span class="o">=</span><span class="s2">&quot;wlan0&quot;</span>
</pre></div>
<p>The <strong>udev</strong> program is a dynamic device management software, it supplies the system softwares with device events, manage permissions of device nodes and may create additional symlinks in the &ldquo;/dev&rdquo; directory, or provide names to unpredictable device names from the kernel. The man page for <code>udev</code> is worth reading.</p>
<p>Reboot to test out if the device is named correctly. The device can be found in <code>/sys/class/net/</code>, with a symbolic link to the device&#39;s DEVPATH.</p>
<p>Next step is to setup the WPA2 authentication of the WiFi with my SSID and password. We&#39;ll need <code>wpasupplicant</code> package for that.</p>
<div class="highlight"><pre><span></span>$ sudo apt-get install wpasupplicant
$ sudo vi /etc/wpa_supplicant/example.conf
<span class="c1"># add the following contents:</span>
<span class="c1">#   ctrl_interface=/run/wpa_supplicant</span>
<span class="c1">#   update_config=1</span>
$ sudo wpa_passphrase &lt;SSID&gt; &lt;password&gt; &gt;&gt; /etc/wpa_supplicant/example.conf

<span class="c1"># to test the configuration: 1. start wpa_supplicant in the background, 2. use wpa_cli to interactive with the interface</span>
$ wpa_supplicant -i wlan0 -c /etc/wpa_supplicant/example.conf
<span class="c1"># (or alternatively, add &quot;-B&quot; parameter to the wpa_supplicant command to make it run as a daemon in the background, then use &quot;wpa_cli&quot; to work interactively)</span>
$ wpa_cli
&gt;scan
&gt;scan_results
<span class="c1"># now you see the hotspot scan result, which means the configuration work</span>

<span class="c1"># Go back to our network interface setup, we will add wpa_supplicant to it(the &quot;-D&quot; is to specify the driver to use)</span>
$ sudo vi /etc/network/interfaces
auto wlan0
iface wlan0 inet dhcp
  pre-up wpa_supplicant -B -Dwext -i wlan0 -c /etc/wpa_supplicant/example.conf
<span class="c1"># save and test the interface</span>
<span class="c1"># (PS, sometimes I forgot the &quot;-B&quot; parameter in the wpa_supplicant command, it will make the ifup job hang because it will run as a daemon in the foreground.)</span>
$ sudo ifdown wlan0
$ sudo ifup wlan0
<span class="c1"># make sure the DHCP client can get an IP, otherwise the network auto start process in boot up might hang for 5 minutes to get the network...</span>
</pre></div>
<p>Hard-coding the WiFi SSID and password in the configuration is not convenience in real environment, but so far in my experiement environment, it is OK to use it first. We will get back to the network manager later to make it more convenience to connect different WiFi network.</p>

<h4 id="toc_6">Problem 1.5 WiFi Connection Revisited</h4>
<p>Last night I has successfully connect the wifi adapter to my home&#39;s network, howvever some new issues come up:</p>

<ul>
<li>  The bandwidth is only 1Mb/s</li>
<li>  Not convenient to configure SSID and passphrase</li>
</ul>
<p>After checking the <a href="https://www.debian.org/doc/manuals/debian-reference/ch05.en.html">Debian&#39;s document</a>, the network setup using <em>ifupdown</em> approach is a bit outdate, and the modern way is to use NetworkManager(NM) or Wicd(wicd and associated packages). </p>
<p>(BTW, it is good to have the <em>debian-handbook</em> and <em>debian-reference</em> package installed in your local machine for any reference needed.)</p>
<p>At first, I try to look up a proper driver for the adapter. There is a package <a href="https://wiki.debian.org/rt2800usb">rt2800usb</a> to support Ralink 802.11n usb devices on Linux. However, I found out that Ubuntu has already installed the <code>linux-firmware</code> which includes the rt28xx driver, so I decided to check it later.</p>
<p>Now I try to install the <strong>network-manager</strong> first. It is a program in two parts: a root daemon handling activation and configuration of network interfaces, and a user interface that controls it. It is provided by gnome project so the GUI is by default for gnome environment. However, it also provides a command line tool call <code>nmcli</code> in the package, so I will try it first.</p>
<div class="highlight"><pre><span></span>$ sudo apt-get install network-manager
$ sudo service network-manager start
$ nmcli <span class="nb">help</span>
</pre></div>
<p>It will ignores the interfaces(except <em>lo</em>) in /etc/network/interfaces and use its own configuration, so comment out all leaving only <em>lo</em> in /etc/network/interfaces. Next. try a few commands for nmcli to check the network status.</p>
<div class="highlight"><pre><span></span><span class="c1"># list network devices</span>
$ nmcli device

<span class="c1"># list connections</span>
$ nmcli connection
</pre></div>
<p>now we can try to connect the wifi with the WiFi adapter</p>
<div class="highlight"><pre><span></span><span class="c1"># list the wifi hot spots</span>
$ nmcli device wifi list
<span class="c1"># or refresh the list if you don&#39;t get it</span>
$ nmcli device wifi rescan

<span class="c1"># connect your hot spot</span>
$ nmcli device wifi connect &lt;SSID<span class="p">|</span>BSSID&gt; password &lt;password&gt;
</pre></div>
<p>It&#39;s as simple as just one command line, and yet so powerful than what I expected. The network manager auto saves the WiFi connection in its database, and it can also be activated automatically when system starts up. Save my day!</p>
<p>Reference for <strong>nmcli</strong>: <a href="https://fedoraproject.org/wiki/Networking/CLI">https://fedoraproject.org/wiki/Networking/CLI</a></p>
<p>After checking the connection, the bandwidth, everything goes well. So I can stop my researching for the network solution now. Next let&#39;s work on the desktop environment.</p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Config your git repository to use SSH Key]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2016/04/git-ssh-config.html"/>
        <published>2016-04-17T16:11:00+08:00</published>
        <updated>2020-02-01T10:09:37+08:00</updated>
        <id>http://terryoy.github.io/2016/04/git-ssh-config.html</id>
        <category scheme="http://terryoy.github.io/tag/#shell" term="shell" label="shell" />
        <category scheme="http://terryoy.github.io/tag/#virtualbox" term="virtualbox" label="virtualbox" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>It is very time-consuming typing user and password every time when you do git pull/push to sync your code. Using the rsa ssh key for the authentication will save you a lot of time.</p>

<h3 id="toc_0">1. Generate SSH Key</h3>
<p>Just use the method I mentioned before in <a href="/2014/01/ssh-key-based-auth-server.html#toc_1">this article</a>, you will get a public/private key pair. Upload the public key to your github or other git hosting service.</p>

<h3 id="toc_1">2. Config your local SSH</h3>
<p>Create or open the file <code>~/.ssh/config</code> for ssh key config. The content will be similar as below:</p>

<pre><code>Host dev
 HostName dev.local
 IdentityFile ~/.ssh/dev_rsa
Host github.com
 HostName github.com
 IdentityFile ~/.ssh/github_rsa</code></pre>
<p>You can use a &ldquo;host&rdquo; alias for ssh connect instead of the full host name, this would be convenient if you might switch server host.</p>

<h3 id="toc_2">3. Choose the correct git url</h3>
<p>Usually public git hosting services use HTTPS or GIT protocol, and your private repository can use SSH protocol like in <a href="/2014/10/common-git-commands.html#toc_3">this article</a>.</p>

<pre><code>* Git URL: git@bitbucket.org:&lt;accountname&gt;/&lt;reponame&gt;.git  
* Use with alias: git@bitbucket:&lt;accountname&gt;/&lt;reponame&gt;.git</code></pre>

<h3 id="toc_3">4. No more ask of  the passphrase</h3>
<p>Initially if you use a ssh key with passphrase, you might need to enter the passphrase every time you use it. It will be kind of annoying although provide better security. However, sometimes I don&#39;t want it to be entered every time in my private trusted computer. So we need <code>ssh-agent</code> to handle that.</p>
<div class="highlight"><pre><span></span><span class="c1"># install the openssh-client package if you&#39;re on Debian/Ubuntu</span>
$ sudo apt-get install openssh-client

<span class="c1"># the ssh-add and ssh-agent tools will be available after the installation, but you need to start ssh-agent in your .bashrc script</span>
$ vi ~/.bashrc
<span class="nb">eval</span> <span class="k">$(</span>ssh-agent<span class="k">)</span>

<span class="c1"># add the ssh key to ssh-agent (the passphrase will be asked once)</span>
$ <span class="nb">cd</span> ~/.ssh
$ ssh-add -k gitub_rsa
</pre></div>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[Ubuntu Desktop Sharing to Mac OSX]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2016/02/ubuntu-desktop-sharing-to-osx.html"/>
        <published>2016-02-26T10:52:00+08:00</published>
        <updated>2020-02-01T10:09:37+08:00</updated>
        <id>http://terryoy.github.io/2016/02/ubuntu-desktop-sharing-to-osx.html</id>
        <category scheme="http://terryoy.github.io/tag/#linux" term="linux" label="linux" />
        <category scheme="http://terryoy.github.io/tag/#osx" term="osx" label="osx" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>I have a spare laptop rest in my house which I seldom use(I major use my macbook pro for work), but it&#39;s still good for developing projects and runs Linuxi Mint. So I want it to be available any time when I&#39;m at home so I could use it to continue my side project works. With ssh and VNC, I can setup both the command line and desktop environment online so I don&#39;t need to open the laptop, but use my macbook to access the environment. Then I have the convenience that the laptop can put anywhere in the house and clear my desktop.</p>
<p>I found <a href="http://www.fiz-ix.com/2012/12/ubuntu-to-mac-os-x-screen-sharing-with-vinagre/">this post</a> very useful to do the trick. It uses &ldquo;vino&rdquo; with easily setup steps, and you can get &ldquo;Vinagre&rdquo; VNC client on Linux ox,  but you need some notices about the access on OSX. So I write my process here.</p>

<h3 id="toc_0">1. Setup Vino(VNC for gnome)</h3>
<p>Just reference the post. </p>
<div class="highlight"><pre><span></span>$ sudo apt-get vino
<span class="c1"># the preferences is the same as &quot;Desktop Sharing&quot; preference panel in Linux Mint</span>
$ vino-preferences
</pre></div>
<p>You can set up the items just as the post said in vino-preferences, but on Linux Mint it lacks out the &ldquo;Advanced settings&rdquo; mentioned in the post. So you need &ldquo;dconf-editor&rdquo; to manually set it.</p>
<div class="highlight"><pre><span></span>$ sudo apt-get install dconf-editor
$ dconf-editor
</pre></div>
<p>Find the config item in dconf-editor with path <strong>&ldquo;desktop.gnome.remote-access&rdquo;</strong>. You need to uncheck the &ldquo;require encryption&rdquo; to support OSX, otherwise the authentication is unsupported on OSX.</p>

<h3 id="toc_1">2. Using Screen Sharing on Mac OSX</h3>
<p>Turn on the &ldquo;Spotlight&rdquo; with keyboard shortcut <code>cmd+K</code>, enter &ldquo;screen sharing&rdquo; and you can find the default desktop sharing client app on Mac OSX.</p>
<p>Type &ldquo;vnc://your_server_ip:5900&rdquo; and then you get the prompt to enter password for the access. That&#39;s it!</p>

<h3 id="toc_2">3. More</h3>
<p>The post also talked about using the <code>avahi</code> service to broadcast the desktop sharing service to OSX. It is an advanced option to try, I will update it if I have tried.</p>
]]>
        </content>
    </entry><entry>
        <title type="html"><![CDATA[VirtualBox Command Line Tricks]]></title>
        <author><name>terryoy</name></author>
        <link href="http://terryoy.github.io/2016/02/vbox-cmd-tools-tricks.html"/>
        <published>2016-02-17T11:01:00+08:00</published>
        <updated>2020-02-01T10:09:37+08:00</updated>
        <id>http://terryoy.github.io/2016/02/vbox-cmd-tools-tricks.html</id>
        <category scheme="http://terryoy.github.io/tag/#shell" term="shell" label="shell" />
        <category scheme="http://terryoy.github.io/tag/#virtualbox" term="virtualbox" label="virtualbox" />
        <content type="html" xml:base="http://terryoy.github.io/" xml:lang="en">
            <![CDATA[ <p>Sometimes we need to work with VirtualBox&#39;s command line tools to make things done. Here are a collections of tasks I encountered.</p>

<h3 id="toc_0">1. Converting an Linux .img format to a .vdi (VirtualBox disk)</h3>
<div class="highlight"><pre><span></span>$ VBoxManage convertfromraw mydisk.img mydisk.vdi --format vdi
</pre></div>

<h3 id="toc_1">2. Mount virtual disk on your ubuntu system</h3>
<p>In linux, we can use a the <code>qemu-nbd</code> and <code>kpartx</code> to mount a .vdi file on a path. The <em>qemu-nbd</em> tool can create a block device to enable it to be share via NBD(Network Block Device) protocol. Then, we will use <code>kpartx</code> to create device maps from the partition tables of the device, which makes the partition of the image mountable.</p>
<div class="highlight"><pre><span></span><span class="c1"># (prerequisites] install the dependencies</span>
$ sudo apt-get install qemu-utils kpartx

<span class="c1"># to mount the device</span>
$ sudo modprobe -nbd
$ sudo qemu-nbd -c /dev/nbd0 &lt;vdi_file&gt;
$ sudo kpartx -a /dev/nbd0
$ sudo mount /dev/mapper/nbd0p1 /mnt/&lt;mount_point&gt;

<span class="c1"># delete the device when no longer needed</span>
$ sudo umount /mnt/&lt;mount_point&gt;
$ sudo qemu-nbd -d /dev/nbd0
</pre></div>
<p>(&hellip;to be continued)</p>
]]>
        </content>
    </entry>
</feed>