Testing nonexistence in etcd v3+

In a project I use etcd as a persistent store of state information of the system. etcd is a great piece of software and suits my needs. Till today I tried to determin, how to code a transaction condition to test non existence.

Nothing is mentioned in the doc.

And it is suprisingly easy.

Just test version of the key/val. For nonexisting key it evaluates as equals to zero.

If you want to do a transaction testing existence of one key and non existence of the other you can use following  transaction:

#!/bin/sh

etcdctl del --prefix test-

etcdctl put test-xxx "ahoj"
etcdctl del test-yyy "lidi"

etcdctl txn -i <<__eof test-xxx="" ver=""> "0"
ver("test-yyy") = "0" 

put test-success yeah

put test-failure nay

__EOF

etcdctl get --prefix test-

Nginx Proxy With lets encrypt using Docker compose

I invested some work in managing my little hosted server more efficiently.
I use docker to run my services for a long time now. My budget for hosted machine is too small to start even single node Kubernetes which is my preferred way to manage containers. I ended up in using docker compose.

There is a little drawback in using compose. Every compose file creates it's own network by default and for my scenario, when I use the single server to host multiple domains through nginx proxy container it did not work as expected.

When multiple compose configurations are started, each has its own config and by default each runs in separate network.

Here is a simple trick which will do it.

 Create dedicated network prior starting it all and connect all the compose configurations to this network by default instead letting the compose create a network per configuration.

I start my reverse proxy containers using following compose:

version: '2'

services:

  nginx-proxy:
    image: jwilder/nginx-proxy
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "nginx-proxy-vhosts:/etc/nginx/vhost.d"
      - "nginx-proxy-html:/usr/share/nginx/html"
      - "/var/run/docker.sock:/tmp/docker.sock:ro"
      - "nginx-proxy-certs:/etc/nginx/certs"

  letsencrypt-nginx-proxy-companion:
    image: jrcs/letsencrypt-nginx-proxy-companion
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    volumes_from:
      - "nginx-proxy"

volumes:
  nginx-proxy-vhosts:
  nginx-proxy-html:
  nginx-proxy-certs:
 
networks:
   default:
     external:
        name: proxy      

The last lines starting with networks will do it.

If you want to expose the service through this proxy, add the same section at the end of compose file. Eg:

version: '2'

services:
  some.service.name:
    image: php:apache
    environment:
      VIRTUAL_HOST: your.own.domain
      LETSENCRYPT_HOST: your.own.domain
      LETSENCRYPT_EMAIL: your.own@email.xxx
    volumes:
      - "./html:/var/www/html"

networks:
  default:
    external:
      name: proxy




And that is all ... for today


IOTA IRI remote authentication

When installing IOT IRI atm you can start it with authentication enabled. As the IRI developers really like trytes and trits and their home made curl hash (which by the way was proved a bit broken), the authentication uses the curl  for hashing the password. No documentation for this feature so some code digging was necessary to find the answer on how to configure remote authentication.

The configuration option can look like this:

• in iri.ini
  REMOTE_AUTH =iota:DRDR...SRCJ
• in command line arguments
  --remote-auth iota: DRDR...SRCJ

Above config means when prompted for password, you enter username "iota" and password "password".

To generate passwor hash use following script:

const Curl=require("iota.lib.js/lib/crypto/curl/curl.js")

const a2t=require("iota.lib.js/lib/utils/asciiToTrytes.js")

const conv=require("iota.lib.js/lib/crypto/converter/converter.js")

const IOTA=require("iota.lib.js")

let password=process.argv[2]

if ( ! password ) {

console.log("Usage: node hash-password.js ascii-password-to-hash")

process.exit(1)

}

let pwdTrytes = a2t.toTrytes(password)

let pwdTrits = conv.trits(pwdTrytes)

let curl=new Curl()

curl.initialize()

curl.absorb(pwdTrits, 0, pwdTrits.length)

let out=[]

curl.squeeze(out, 0, Curl.HASH_LENGTH)

console.log(conv.trytes(out))

This example is part of iota-experiments.

Konverze S-JTSK (Křovák) do WSG84 (GSM) pomocí PROJ4.JS

Ač nejsem odborníkem, přes kartografii, projekce, občas mám potřebu konvertovat souřadnice dostupné od ČÚZK např. v rámci výměnného referenčního formátu exportovaného z RÚIAN. Dlouho jsem hledal nástroj, který by byl dobře dostupný s mými znalostmi a hlavně, který bych mohl zařadit do vlastních konverzních rutin. Cčková knihovna PROJ.4 je pro mě příliš komplikovaná pro moje občasné hrátky. Začal jsem tedy příkazovou řádkou konkrétně příkazem cs2cs a při nedávné další iteraci konvertoru jsem narazil na dlouho hledané.

Existuje port PROJ.4  projektu jako nativní javascript knihovna pro node.js - proj4.js.

Použití pro převod mezi systémy zmíněnými v titulku (S-JSTK do WSG84) je triviální.

Instalujeme modul:
npm install --save proj4

importujeme knihovnu
const proj4 = require('proj4')
definujeme projekce
proj4.defs("EPSG:4326","+proj=longlat +datum=WGS84 +no_defs");
proj4.defs("EPSG:5514","+proj=krovak +lat_0=49.5 +lon_0=24.83333333333333 +alpha=30.28813972222222 +k=0.9999 +x_0=0 +y_0=0 +ellps=bessel +towgs84=589,76,480,0,0,0,0 +units=m +no_defs");


a provedeme vlastní konverzi podle potřeby:
console.log(proj4("EPSG:5514","EPSG:4326",[-710056.96,-1116931.88]));

a kupupříkladu pro indexaci v Elasticsearch 

let gps=proj4("EPSG:5514","EPSG:4326",[-710056.96,-1116931.88])
let data={
   location: {
        lat:gps[1],
        lon:gps[0]
   }
}  

dash cam to timelapse

My dash cam produces AVI video file every 5 minutes. I decided to create timelapse from the videos on linux using ffmpeg. Following the inspiration
I created following script

for f in *.AVI; do echo "file $f" >> files.txt; done 

[a file called "files.txt" is created] 

#create scaled timelapse 
ffmpeg -f concat -i files.txt -an -vcodec libx264 -filter:v 'setpts=0.05*PTS,scale=640:480' dashcam.mov

Happlink (formerly Plug-Up) Security KEY on Fedora 23

After several hours I managd to get Plug-Up SecureKEY U2F token working on Fedora 23. Some Ubuntu ubased advisory recommends to add an udev rule in a form of [not working]:

SUBSYSTEM=="usb", ATTRS{idVendor}=="2581", ATTRS{idProduct}=="f1d0", MODE="0660", GROUP="[the right  group user is member of]"  

After many minutes of google-fu I managed to get it working. The magic spell is this:

# Happlink (formerly Plug-Up) Security KEY
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="2581", ATTRS{idProduct}=="f1d0", TAG+="uaccess"

and the spell should be written into /etc/udev/rules.d/10-sec-key.rules

then run udevadm trigger


Sources:

http://www.ha-obsession.net/2016/04/fedora-23-kde-using-gpg-yubikey4-pcscd.html

http://askubuntu.com/questions/674064/happlink-plug-up-fido-u2f-security-key-why-it-doesnt-work-immediately-in-ubuntu

Print images on POS printer

For some unimportant reason I have mini thermal bluetooth printer IMP006A (ordered from alibaba.com). It is reliable piece of HW which I spent some time playing with.

Here you can see the result. Except receipts the printer can be used to print images. Here I share the piece of java code that does the magic.

The image to be printed must be in PBM format max. 373 px width. During the printing process the image is rotated 90 degrees clockwise.

You can use ImageMagick convert utility to prepare the image for printing.

convert  saturn.png \
        -rotate -90 \
        -scale x373  \
        -dither FloydSteinberg \
        -colors 2 \
         image.pbm

Where saturn.png is the input image of any kind ImageMagick can read. 
image.pbm is the resulting pbm image ready to be printed. 

The image is 

• saturn.png - read from a file
• -rotate -90 - rotated 90 degrees CCW
• -scale x373 - scaled to full width of paper
• - dither ... -colors 2 - color are reduced to 2 (BW) using dithering to render shades of gray on BW device
• image.pbm - converson result is written to a file

Then if you have the code compiled just run:

java  printqr.PrintQR  image.pbm

And you will get image.prn containg a set of commands for the printer to print the image. Use linux command line to send the data to the printer.

Use USB to connect the printer to Linux box. When connected to linux box with recent distro (tested on Fedora 20), the printer will be accessible as /dev/usb/lp0. By default the device is accessible for root (and lp group members) only. Send the data to the printer usinf gollowing command:

sudo sh -c 'cat image.prn > /dev/usb.lp0'

And that's all. 

For illustration (click to enlarge):

Code:


package printqr;

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.Random;

/** Maximum image printed width (input height - the image is rotated) should be 373 px
  *
  * To prepare image for printing:
  * convert saturn.png  -rotate -90 -scale x373 -dither FloydSteinberg -colors 2 image.pbm
  *  or
  * convert saturn.png   -rotate -90 -scale x373 -dither Riemersma -colors 2 image.pbm
  *
  *
 * */

public class PrintQR {
    final static int[] wsChars=new int[] {' ','\n','\t'};
   
    public static void main(String[] args){
        FileInputStream fis=null;
        FileOutputStream fos=null;
        try {
           
            //prepare image name
            String inputImage="image.pbm";
            if (args.length>0)
                inputImage=args[0];
           
            if (! inputImage.endsWith(".pbm"))
                throw new IllegalArgumentException("Input image must be NetBPM PBM format with .pbm extension.");
           
            //open and read image
            fis=new FileInputStream(inputImage);
            //read header
            expectChar(fis,'P');           
            expectChar(fis,'4');
            expectWhitespace(fis);
            int width=readDecimal(fis);
            int height=readDecimal(fis);
           
            //read data
            int rowWidthBytes= width%8 == 0 ? width / 8 : width / 8 +1 ;
            int dataLength=rowWidthBytes*height;
            byte[] data=new byte[dataLength];
            int off=0;
            int remaining=dataLength;
            while (remaining>0){
                int cnt=fis.read(data, off, remaining);
                remaining-=cnt;
                off+=cnt;
            }
           
            System.out.printf("Loaded image %d x %d\n", width, height);
           
            fos=new FileOutputStream(inputImage.replace(".pbm", ".prn"));
           
            //reset printer
            fos.write(new byte[]{0x1b,0x40});
           
            //set linespacing 0
            fos.write(new byte[]{0x1b,0x33,0x00});
           
            //print image
            printBitmap24pinDoubleDensity(width, height, data, fos);
           
        }
        catch (Exception e){
            e.printStackTrace();
        }
        finally {
            if (fos!=null) try {fos.close();}catch (Exception e){};
            if (fis!=null) try {fos.close();}catch (Exception e){};
        }
    }

    /**
     * Sends image to the printer in the right format with the right commands
     * @param width
     * @param height
     * @param data
     * @param fos
     * @throws IOException
     */
    private static void printBitmap24pinDoubleDensity(int width, int height, byte[] data,
            FileOutputStream fos) throws IOException {
        int bWidth=width%8 == 0 ? width / 8 : width / 8 +1;
        int hLow=(height) & 0x000000ff;
        int hHigh= ( (height) >> 8 )& 0x000000ff;
       
        for (int x=0; x< bWidth; x+=3){
            // ESC * m nL nH d1... dk
            // m - 0x21 for 24 pin printer double density
            // nL/nH - low/high byte of number of "strokes", each stroke consists of 24 dots (8 bytes)
            // data should be (nL+nH*256)*3 bytes
            fos.write(new byte[]{0x1b,0x2a,0x21,(byte)hLow,(byte)hHigh});
           
            //get appropriate data from image - for the operation concider image rotated 90 deg CCW
            for (int y=height-1; y>=0; y--){
                fos.write(new byte[]{
                        get(data,x,y,bWidth, height),
                        get(data,x+1,y,bWidth, height),
                        get(data,x+2,y,bWidth, height),
                });
            }
            fos.write('\n');
        }
    }
   
    /**
     * Extracts byte of image data - the image is like this in mempory 
     *  ----------------X----------------------->
     * |          0        1        2        3        4        5
     * | 0 ........ ........ ........ ........ ........ ........
     * | 1 ........ ........ ........ ........ ........ ........
     * | 2 ........ ........ .####### ######## ........ ........
     * Y 3 ........ ........ .####### ####.... ........ ........
     * | 4 ........ ........ .######. ........ ........ ........
     * | 5 ........ ........ .#...... ........ ........ ........
     * | 6 ........ ........ .#...... ........ ........ ........
     * V 7 #.#.#.#. .#.#.#.# ....#... ........ ........ ........
     *
     * First triplet sent to printer is
     * this [0,7],[1,7],[2,7]
     * then [0,6],[1,6],[2,6]
     * then [0,5],[1,5],[2,5]
     * and so on
     *
     * This effectively rotates image CCW 90 deg.   
     *
     * @param data
     * @param x
     * @param y
     * @param bWidth
     * @param height
     * @return
     */
    public static byte get(byte[] data, int x, int y, int bWidth, int height){
        if( x>= bWidth ) return 0; //handle request beyond image for images not %24 print height
        int idx=y*bWidth+x;
        if (idx<0 || idx>=data.length)
            return 0;
        return data[idx];
    }
   
    public static int expectChar(InputStream is, int expectedChar)
        throws IOException
    {
        int readChar=is.read();
        if (readChar==expectedChar) return readChar;
       
        throw
            new IllegalStateException(String.format("Expected %c found %c",expectedChar, readChar));
    }
   
    public static int expectWhitespace(InputStream is)
            throws IOException
    {
        int readChar=is.read();
        if (isWhitespace(readChar)) return readChar;
        throw
            new IllegalStateException(String.format("Expected whitecpace found %c",readChar));
    }
   
    public static int readDecimal(InputStream is)
        throws IOException
    {
        int accumulator=0;
        while (true){
            int c=is.read();
            if (isWhitespace(c)) return accumulator;
            else
                if (c>='0' && c<='9') accumulator=accumulator*10+(c-'0');
                else
                    throw
                        new IllegalStateException(String.format("Expected digit or whitecpace found %c",c));
        }       
    }
   
    public static boolean isWhitespace(int c){
        for (int i=0; i<wsChars.length; i++){
            if (c==wsChars[i]) return true;
        }
        return false;
    }


    /** not tested */
    private static void printBitmap9pin(int width, int height, byte[] data,
            FileOutputStream fos) throws IOException {
        int bWidth=width%8 == 0 ? width / 8 : width / 8 +1;
        int hLow=height & 0x000000ff;
        int hHigh= ( height >> 8 )& 0x000000ff;
       
        for (int x=0; x< bWidth; x++){
            fos.write(new byte[]{0x1b,0x2a,0x00,(byte)hLow,(byte)hHigh});
            for (int y=height-1; y>=0; y--){
                fos.write(get(data,x,y,bWidth, height));
            }
            fos.write('\n');
        }
    }

    /** not tested */
    private static void printBitmap9pinDoubleDensity(int width, int height, byte[] data,
            FileOutputStream fos) throws IOException {
        int bWidth=width%8 == 0 ? width / 8 : width / 8 +1;
        int hLow=height & 0x000000ff;
        int hHigh= ( height >> 8 )& 0x000000ff;
       
        for (int x=0; x< bWidth; x++){
            fos.write(new byte[]{0x1b,0x2a,0x01,(byte)hLow,(byte)hHigh});
            for (int y=height-1; y>=0; y--){
                fos.write(get(data,x,y,bWidth, height));
            }
            fos.write('\n');
        }
    }
}








Graphical printing tool for POS printer, jehož autorem je LRA, podléhá licenci Creative Commons Uveďte původ-Zachovejte licenci 4.0 Mezinárodní .

.. on a trip to the C world again

Testing lua embedding and mongoose http server.
Lua is great to implement business logic in the C based app.
It lets you bee less woried about memory leaks, stack overruns, invalid pointer etc.

online collaboration in realtime

Some projects for online collaboration:
http://etherpad.org/ - realtime colaboration for text editing
http://socket.io/  - bidirectional realtime communication platform
http://XMPP.org - realtime presence & message protocol

Tune SSL on Apache 2.4/Windows

Today I spent some time to tune Apache SSL settings to be Grade A at https://www.ssllabs.com/ssltest/.
Finding the equilibrium point between compatibility and transport security took some time. To save yours, I'm sharing the final configuration here.

Some notes at first. I favored security over backward compatibility and so some older (very old in fact) browsers will fail to establish connection. I tried to cope with all failed tests, but not succseeded. There stil are some Failed tests. Those tests are not affecting the main purpose of the server.

1. Download and install the latest Apache 2.4 binaries to overcome known CVE
2. I'm using http://www.startssl.com/ as the server certification authority.
3. Tune SSL Protocol and Ciphersuites and some others at a server level (httpd.conf)
1. SSLProtocol all -SSLv2 -SSLv3
2. SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH EDH+aRSA !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS"
3. SSLHonorCipherOrder on
4. SSLUseStapling On
5. SSLStaplingCache shmcb:logs/ssl_stapling(32768)
4. Add HTTP Strict Transport Security
1. Enable headers module: 
1. LoadModule headers_module modules/mod_headers.so
    
2. Set header to require HSTS at the VirtualHost level
1. Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains"

And you are done.

... DIY electronics

From time to time I learn about new DIY electronic project. Here I will sumarize all of them in a short notice:

littleBits

Creations are constructed by placing bits&pieces together glued by magnetic force and connected by contacts. Cloud enabled.
SmartHome Kit - $249 per 14 modules -  $17/module.

SAM

Standalone bits equiped with bluetooth low energy connectivity connected by SW configuration. Battery in a module lasts from 3 weeks of operation (button) to 1 hour (motor). MicroUSB charger.
Cloud enabled.
SAM Pro  - GBP 349 per 11 modules -  GBP31/module

... backup with Areca

I'm giving try to Areca backup tool which seems as a good multiplatform (Java) tool for simple backup tasks on a workstation and small server.
It is featuring pretty usable GUI and command line interface for common backup tasks.

... and another Apache httpd story ... client authentication of proxied requests

Centos/RHEL 6.3, apache httpd 2.2.15
The goal is - let te httpd listens at port 80 plain unencrypted HTTP requests. The request will be forwared  to https://some.site/ where there is a HTTPS with client authentication.

Findings:

Client certificate key/cert file format (directive SSLProxyMachineCertificateFile):

• key + certificate need to be in single PEM file looking something like
  -----BEGIN RSA PRIVATE KEY-----
  MIIE...
  ...
  ...
  ...
  -----END RSA PRIVATE KEY-----
  -----BEGIN CERTIFICATE-----
  MII...
  
  ...
  .,
  ...
  -----END CERTIFICATE-----
• mind the RSA part (emphasized above)  of private key header - some openssl versions use the header with and some without the RSA letters
• without RSA
• my version of apache httpd wants it there

Re-negotioation at the server side is not supported. The server should be configured to protect every request with client auth. To avoid TLS renegotiation when entering protected resource, SSLClientVerify should be configured at VirtualHost level or server level.

After above changes were made server started and operated as expected.

References:
http://apache-http-server.18135.x6.nabble.com/Apache-fails-to-start-if-SSLProxyMachineCertificateFile-does-not-contain-RSA-td5009238.html

Apache httpd 2.4.6 hangs not servicing HTTPS

A lot of pain ... till solution found
# Apparently this fixes an issue with Apache 2.4.6 on Windows hanging
# when serving requests from Internet Explorer 10/11.
# see http://stijndewitt.wordpress.com/2014/01/10/apache-hangs-ie11/
AcceptFilter http none
AcceptFilter https none

... Google Authenticator or Doogee DG800 strange error

After purchasing new chinese Android phone I needed to install Google Authenticator. But had no luck. The application worked fine but ... generated codes did not work. I tried other phones and came to a strange conclusion. The other Doogee DG800 I had show she same codes. Bud all other Android phones shown different codes. For sure all the phones shared the same secret.  The strange thing was the codes were different in some digits only. Typical difference looked like this 605678 - 604356.

As the authenticator codes are derived using hash function (RFC 6238 - TOTP), in case of an error there should be completely different results.

Fortunately - authenticator is an open source software. I cloned the repository and built my own version for debugging.

And here is the source of difference. During the computation a piece of a hash value is taken, converted to integer and then divided by a power of 10 and the remainder is the code:

int code =  truncatedHas % (int)Math.pow(10, codeLength);

On the problematic phone the result of power computation was this:
 Math.pow(10, codeLength): 999999,99999999

After cast to integer the result was 999999.

This is the answer - wrong result from system runtime library because the Math.pow function computes doubles which can be not so precise.

It is rare but it happens.

I'm going to  send a patch which does not uses double computation.

UPDATE: Issue was discussed at Stack Overflow. The main cause is a bug in the phone platform library.

... firefox 30+ disabled NTLMv1 over insecure connection

NTLMv1 is known to be insecure but if you need it new setting was introduced:
network.negotiate-auth.allow-insecure-ntlm-v1 - defaults to false and NTLMv1 auth request is silently ignored when sent over HTTP. Set it to true to force NTLMv1authentication prompt.

Disclaimer: NTLMv1 over HTTP is considered insecure - you should know what are you doing

... nssm - non sucking service manager

Seems really usable - new to me.
Why I like it ... it is able to start any command as a service and has several options for service shutdown (Send Ctrl-C, send WM_CLOSE,kill process)

http://nssm.cc/usage

Chrome native messaging on windows

Problem:
When developing extension using native messaging you need to develop so called native messaging host - exe which is registered with chrome and  spawned by chrome when requested by the extension.
Chrome uses pipes to communicate with the host. The pipe is connected to stdin/stdout descriptors in the host process.
There is a drawback on windows - default stdin mode is TEXT. But in this mode once 0x1A is passed through, the C library closes the pipe and the host is terminated because 0x1A id end of file marker.

Solution:
Before reading attempt from the pipe set the mode to binary (defaults to text) using following code snippet:

_setmode( _fileno( stdin ), _O_BINARY )

Reference: http://msdn.microsoft.com/cs-cz/library/tw4k6df8.aspx

Convert SSH public key to a form for .authorized_hosts

Evary time i need this command it takes me minutes to retrieve the right form from my memory. Let's take a note :)

ssh-keygen -i -f pem -f ~/.ssh/id_rsa.pub

Error 'LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt - Solved

No Visual Studio 2012 installed just SP1 of Visual Studio 2010.

Found this (no solution, but direction): http://stackoverflow.com/questions/10888391/error-link-fatal-error-lnk1123-failure-during-conversion-to-coff-file-inval

But it did not help.

Found this(almost solusion): http://social.msdn.microsoft.com/Forums/vstudio/en-US/d10adba0-e082-494a-bb16-2bfc039faa80/vs2012-rc-installation-breaks-vs2010-c-projects?forum=vssetup

It pointed me to the root cause - bad version od cvtres.exe which depends on missing msvcr100_clr0400.dll.

Solution: downloaded http://www.dll-files.com/dllindex/dll-files.shtml?msvcr100_clr0400

I downloaded the right version (32bit) of the file and stored it next to cvtres.exe and BINGO!!! compiling with no error.

Sahme on MS ... and shame on MS problem resolution culture. Searching the root cause is seldom seen. Uninstall/Reinstall is the only advice in most of dicussions.