KB: Install Python packages in a mint HDInsight Spark cluster

Method 1: Script Action

https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-spark-python-package-installation

Method 2: %%configure

https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-spark-jupyter-notebook-use-external-packages

Method 2: SSH

cd /usr/bin/anaconda/envs/py35/bin
./conda install [package]
./conda install -c conda-forge [package]

Advertisements

KB: Approximating Normal Distribution by Monte Carlo Integration

import numpy as np
import sympy as sp
import scipy.stats as stats

#X~N(0,1) , Pr(-1<x<1)
p=stats.norm.cdf(1)-stats.norm.cdf(-1)
pi=np.pi
print(p)

#by solving integral
x = Symbol('x')
pdf = 1/sqrt(2.0*pi) * exp(-1/2*x**2)
cdf=sp.integrate(pdf_norm)
p=cdf.replace(x,1)-cdf.replace(x,-1)
print(p)

#by Monte Carlo Integration
u=list(np.random.uniform(-1,1,10000))
h=list(map(lambda u: pdf.replace(x,u),u ))
p=np.mean(h) * (1-(-1))
print(p)