KB: Install and Config SoftEther VPN Bridge on OpenWrt Router

Compile softethervpn Package

See https://openwrt.org/packages/pkgdata_lede17_1/softethervpn for the available compiled package. Compile package for your target architecture if not available. Reference:

  Install dependency libraries

apt-get update
apt-get install -y subversion make gcc g++ libncurses5-dev libghc-zlib-dev libreadline-dev libssl-dev gawk bzip2 patch xz-utils git unzip

  Get OpenWrt and SoftEther VPN source code

git clone https://git.openwrt.org/openwrt/openwrt.git ./openwrt
cd openwrt
git checkout openwrt-18.06

./scripts/feeds update
./scripts/feeds install softethervpn

  Compile

make defconfig
make menuconfig

make prepare
make package/softethervpn/compile V=99

 

Install – Standard Package

Installing package “softethervpn” will start all the softethervpnbridge, softethervpnserver and softethervpnclient services and may make the router unresponsive. Prepare and run the following shell script to stop the services and disable from autostart in next reboot.

#!/bin/sh

while :
do
        /etc/init.d/softethervpnbridge disable
        /etc/init.d/softethervpnserver disable
        /etc/init.d/softethervpnclient disable
        /etc/init.d/softethervpnbridge stop
        /etc/init.d/softethervpnserver stop
        /etc/init.d/softethervpnclient stop
        sleep 1
done

Install package softethervpn.

Install – For router with 8MB ROM

The pre-compiled package includes all client, bridge and server which can’t fit into router with 8MB ROM. Workaround by installing only softetherbridge.

Step 1: Install dependency packages manually

#!/bin/sh
opkg install libc
opkg install libpthread
opkg install librt
opkg install libreadline
opkg install libopenssl
opkg install libncurses
opkg install kmod-tun
opkg install zlib

Step 2: Expand the pre-compiled softethervpn package

tar xvfs softethervpn_4.22-9634-1_mips_24kc.ipk

Step 3: Remove softetherclient and softetherserver files from data.tar.gz

gunzip data.tar.gz
tar --delete --file data.tar ./usr/libexec/softethervpn/vpnclient
tar --delete --file data.tar ./usr/libexec/softethervpn/vpnserver
tar --delete --file data.tar ./usr/libexec/softethervpn/vpn_client.config
tar --delete --file data.tar ./usr/libexec/softethervpn/vpn_server.config
tar --list --file data.tar

Step 4: Upload data.tar to router and extract to root path /

cd /tar xvf /tmp/data.tar

 

Config VPN Bridge

Create a command file  (vpn_config.txt)

CascadeCreate hkvpn /SERVER:{vpnserver}:443 /HUB:{virtualhubname} /username:{username}
CascadePasswordSet {virtualhubname} /PASSWORD:{password} /TYPE:standard
CascadeOnline {virtualhubname}
BridgeCreate BRIDGE /DEVICE:{tap_devicename} /TAP:yes
CascadeList
BridgeList

Run vpncmd

vpncmd localhost:443 /SERVER /ADMINHUB:BRIDGE /IN:vpn_config.txt

 

Config Network Interface

{tbc}

 

 

 

 

KB: Install Python packages in a mint HDInsight Spark cluster

Method 1: Script Action

https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-spark-python-package-installation

Method 2: %%configure

https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-spark-jupyter-notebook-use-external-packages

Method 2: SSH

cd /usr/bin/anaconda/envs/py35/bin
./conda install [package]
./conda install -c conda-forge [package]

KB: Approximating Normal Distribution by Monte Carlo Integration

import numpy as np
import sympy as sp
import scipy.stats as stats

#X~N(0,1) , Pr(-1<x<1)
p=stats.norm.cdf(1)-stats.norm.cdf(-1)
pi=np.pi
print(p)

#by solving integral
x = Symbol('x')
pdf = 1/sqrt(2.0*pi) * exp(-1/2*x**2)
cdf=sp.integrate(pdf_norm)
p=cdf.replace(x,1)-cdf.replace(x,-1)
print(p)

#by Monte Carlo Integration
u=list(np.random.uniform(-1,1,10000))
h=list(map(lambda u: pdf.replace(x,u),u ))
p=np.mean(h) * (1-(-1))
print(p)


KB: Route Based Site-to-Site VPN with OpenWRT/LEDE and Ubuntu VM in Azure

Azure VM Confugruation

  • OS: Ubuntu Server 17, etc
  • Virtual Network/Subnet: 10.0.0.0/24
  • Private IP: 10.0.0.4
  • Public IP: 51.x.x.x
  • Inbound security rules
    • Allow UDP 500
    • Allow UDP 4500

Enable IP forwarding

  • Edit /etc/sysctl.conf and enable the followings
    net.ipv4.ip_forward=1
    net.ipv4.conf.all.accept_redirects = 0
    net.ipv4.conf.all.send_redirects = 0
    
  • Apply settings
    sudo sysctl -p
    

Install StrongSwan

sudo apt-get install strongswan

Config IPSec – /etc/ipsec.conf

# ipsec.conf - strongSwan IPsec configuration file

# basic configuration

config setup
# strictcrlpolicy=yes
# uniqueids = no

conn %default
        ikelifetime=60m
        keylife=20m
        rekeymargin=3m
        keyingtries=1
        authby=secret
        keyexchange=ikev2

conn 41D
        auto=add
        type=tunnel
        aggressive=no
        ike=aes256-sha1-modp1024,3des-sha1-modp1024
        esp=aes256-sha1,3des-sha1
        mark_in=42
        mark_out=42

        left=10.0.0.4
        leftsubnet=0.0.0.0/0
        leftid=51.x.x.x
        leftauth=psk
        leftfirewall=yes

        right=59.x.x.x
        rightsubnet=192.168.5.0/24
        rightauth=psk
        rightfirewall=yes

Config IPSec – /etc/ipsec.secrets

# /etc/ipsec.secrets - strongSwan IPsec secrets file
59.x.x.x 51.x.x.x : PSK "secret"

Config IPSec – /etc/strongswan.conf

# strongswan.conf - strongSwan configuration file
#
# Refer to the strongswan.conf(5) manpage for details
#
# Configuration changes should be made in the included files

charon {
        install_routes=no
        install_virtual_ip=no

        load_modular = yes
        plugins {
                include strongswan.d/charon/*.conf
        }
}

include strongswan.d/*.conf

Config IPSec – Reconnect

ipsec restart
ipsec statusall

Config VTI

ip tunnel add vti0 local 10.0.0.4 remote 59.x.x.x mode vti key 42
ip link set vti0 up

Config Routing

ip route add 192.168.5.0/24 dev vti0

Config SNAT and DNS Forwarding

iptables -t nat -A PREROUTING -p udp --dport 53 -j DNAT --to 8.8.8.8
iptables -t nat -A PREROUTING -p tcp --dport 53 -j DNAT --to 8.8.8.8
iptables -t nat -A POSTROUTING -s 192.168.5.0/24 -o eth0 -j SNAT --to-source 10.0.0.4

OpenWRT/LEDE Configuration

Install StrongSwan and VTI packages

strongswan-minimal
ip-full
kmod-ip-vti
vtiv4

Config IPSec – /etc/ipsec.conf

# ipsec.conf - strongSwan IPsec configuration file
# basic configuration

config setup
        # strictcrlpolicy=yes
        # uniqueids = no

conn %default
        ikelifetime=60m
        keylife=20m
        rekeymargin=3m
        keyingtries=1
        authby=secret
        keyexchange=ikev2

conn UKGW
        auto=start
        closeaction=restart
        type=tunnel
        aggressive=no
        ike=aes256-sha1-modp1024,3des-sha1-modp1024
        esp=aes256-sha1,3des-sha1
        mark_in=42
        mark_out=42
        left=59.x.x.x
        leftsubnet=192.168.3.0/24
        leftid=59.x.x.x
        leftauth=psk
        leftfirewall=yes

        right=51.x.x.x
        rightsubnet=0.0.0.0/0
        rightid=51.x.x.x
        rightauth=psk
        rightfirewall=yes

Config IPSec – /etc/ipsec.secrets

# /etc/ipsec.secrets - strongSwan IPsec secrets file
59.x.x.x 51.x.x.x : PSK "secret"

Config IPSec – /etc/strongswan.conf

# strongswan.conf - strongSwan configuration file
#
# Refer to the strongswan.conf(5) manpage for details
#
# Configuration changes should be made in the included files

charon {
        install_routes=no
        install_virtual_ip=no

        load_modular = yes
        plugins {
                include strongswan.d/charon/*.conf
        }
}

include strongswan.d/*.conf

Config IPSec – Reconnect

ipsec restart
ipsec statusall

Config VTI

ip tunnel add vti0 local 59.x.x.x remote 51.x.x.x mode vti key 42
ip link set vti0 up

Config Routing 1

ip route add 10.0.0.0/24 dev vti0

Config Routing 2 – set subnet default route to Azure VM

ip rule add from 192.168.3.0/24 table 1000
ip route add 0.0.0.0/0 dev vti0 table 1000
ip route show table 1000

iptables -A PREROUTING -p udp -s 192.168.3.0 --dport 53 -j DNAT --to 10.0.0.4
iptables -A PREROUTING -p tcp -s 192.168.3.0 --dport 53 -j DNAT --to 10.0.0.4

Others

  • Add interface and zone for vti0. Allow transfer to/from LAN.
  • Enable MSS Clamping for VTI interface.

KB: Connecting OpenWRT/LEDE router to Azure Virtual Network Gateway (IKEv2)

 

Step1: Install StrongSwan and other packages

  • strongswan-minimal
  • ip-full
  • kmod-ip-vti
  • vtiv4

Step 2: Config IPSec

/etc/ipsec.conf

# ipsec.conf - strongSwan IPsec configuration file

# basic configuration

config setup
# strictcrlpolicy=yes
# uniqueids = no

# Add connections here.

conn %default
ikelifetime=60m
keylife=20m
rekeymargin=3m
keyingtries=1
authby=secret
keyexchange=ikev2
mobike=no

conn LONDON
auto=start
type=tunnel
aggressive=no
ike=3des-sha1-modp1024
esp=3des-sha1
mark=42
mark_in=42
mark_out=42
left={router WAN IP}
leftsubnet=192.168.1.0/24
leftid={router WAN IP}
leftauth=psk
leftfirewall=yes

right={Azure Virtual Network Gateway public IP}
rightsubnet={Azure Virtual Network, e.g. 10.1.0.0/24}
rightid={Azure Virtual Network Gateway public IP}
rightauth=psk
rightfirewall=yes

/etc/ipsec.secret

# /etc/ipsec.secrets - strongSwan IPsec secrets file
{router WAN IP} {Azure Virtual Network Gateway public IP} : PSK "your_secret"

/etc/strongswan.conf

charon {
        install_routes=no
        install_virtual_ip=no

        load_modular = yes
        plugins {
                include strongswan.d/charon/*.conf
        }
}
include strongswan.d/*.conf

 

Step 3: Config VTI interface for routing

ip tunnel add vti0 local {router IP} remote {Azure gateway public IP} mode vti key 42
sysctl -w net.ipv4.conf.vti0.disable_policy=1
ip link set vti0 up
ip route add 10.1.0.0/24 dev vti0

 

Other useful commands:

ipsec restart
ipsec statusall
ip -s xfrm state
ip route list table 220


Uncomment the followings in /etc/sysctl.conf
net.ipv4.ip_forward=1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0

Apply changes
sysctl -p

KB: Notes for Using Windows Subsystem for Linux (WSL)

  • X server for Windows – Xming https://sourceforge.net/projects/xming/
    • $ export DISPLAY=:0
      $ gvim
  • Install essential libraries/packages
    • $ sudo apt-get install build-essential
      $ sudo apt-get install libcurl4-openssl-dev
      $ sudo apt-get install libssl-dev
      $ sudo apt-get install libxslt1.1
      $ sudo apt-get install default-jre
      
  • Things that work: (as of 2017/6/10)
    • Anaconda
    • Firefox
    • Scrapy – Need to install gcc first
    • PyCharm
    • R
      • Create /etc/apt/sources.list.d/CRAN.list and add the following line
      • deb http://cran.r-project.org/bin/linux/ubuntu xenial/
    • RStudio
      • sudo apt-get install libxslt1.1
        sudo apt-get install gdebi-core
        sudo gdebi rstudio-0.99.891-amd64.deb​​
        
        sudo apt-get install libpng16-dev
        
        

KB: Using DBSCAN in R with precomputed distance

File “points.txt” contains the X, Y coordinates of the points. We want to customize the distance calculation and feed into DBSCAN as a distance object.

Data file – points.txt

x    y
5    8
6    7
6    5
2    4
3    4
5    4
7    4
9    4
3    3
8    2
7    5

R Script

library(dbscan)

data=read.table("d:/temp/points.txt", sep="\t", header=TRUE)
distance=matrix(,nrow=nrow(data),ncol=nrow(data))

for(i in 1:nrow(data)){
    for(j in i:nrow(data)){
        dx=abs(data$x[i]-data$x[j])^0.5
        dy=abs(data$y[i]-data$y[j])^0.5
        distance[i,j]=(dx+dy)^2
        distance[j,i]=(dx+dy)^2
    }
}

result=dbscan(as.dist(distance), eps=4, minPts=3)
result$cluster