1. Threading package actually slows down your system because of GIL
2. Multithreading packages make use of processes and can have the expected speed gain
If you want to pass global dictionary, use Manager.dict()
Flask app heroku binding fix
put the following to main
app.run(host='0.0.0.0',debug=True,port=int(environ.get("PORT", 5000)))
403 forbidden permission denied (publickey) error etc when pushing files to Github fix.
Tried to push a project onto github and faced sooooooo much annoying errors.
Fix:
1. Use ssh instead of https
ssh-keygen -t rsa -C "your_email@example.com"
2. Press enter all the way until the end.
3. Vi into your public key usually at ~/.ssh/id_rsa.pub
4. copy all text from public key and put it into your github account settings–ssh keys.
5. push using ssh you should be all set and you should not see any of the errors again.
human readable form linux disk space check
Was just trying to check for disk usage etc.
i.e. if I want to check how much space I am using in ~
then:
du -sh ~/
would return
11G /home/user_name/
If I want to check the total disk space:
df -h
would give me what I want
Filesystem Size Used Avail Use% Mounted on
/dev/ 241G 125G 104G 55% /
...
Install new version of python and set up virtualenvwrapper under your user account (e.g. you don’t have admin rights)
I ran into a problem today trying to install python libraries on server which I don’t have admin rights. Not happy but got a solution around. This way its easier to maintain the environment myself. Hopefully someone can make a software that manages software environment in user space on linux.
Install python 2.7.10 under your user/.localpython
Replace the USER_NAME with your own username.
mkdir ~/src
mkdir ~/.localpython
cd ~/src
wget https://www.python.org/ftp/python/2.7.10/Python-2.7.10.tgz
tar -zxvf Python-2.7.10.tgz
cd Python-2.7.10
./configure -prefix=/home/USER_NAME/.localpython
make
make install
Install virtualenv
cd ~/src
wget https://pypi.python.org/packages/source/v/virtualenv/virtualenv-13.1.2.tar.gz#md5=b989598f068d64b32dead530eb25589a
tar -zxvf virtualenv-13.1.2.tar.gz
cd virtualenv-13.1.2
~/.localpython/bin/python setup.py install
install per
cd ~/src
wget https://pypi.python.org/packages/source/p/pbr/pbr-1.8.1.tar.gz#md5=c8f9285e1a4ca6f9654c529b158baa3a
tar -zxvf pbr-1.8.1.tar.gz
cd pbr-1.8.1
~/.localpython/bin/python setup.py install
install virtualenvwrapper
cd ~/src
wget https://pypi.python.org/packages/source/v/virtualenvwrapper/virtualenvwrapper-4.7.1.tar.gz#md5=3789e0998818d9a8a4ec01cfe2a339b2
tar -zxvf virtualenvwrapper-4.7.1.tar.gz
cd virtualenvwrapper-4.7.1
~/.localpython/bin/python setup.py install
install stevedore (dependency for virtualenvwrapper
cd ~/src
wget https://pypi.python.org/packages/source/s/stevedore/stevedore-1.9.0.tar.gz#md5=53e2bc3b49dd9c920cfce7f63822b1a5
tar -zxvf stevedore-1.9.0.tar.gz
cd stevedore-1.9.0
~/.localpython/bin/python setup.py install
Now last step:
Edit your ~/.bashrc file so your python distribution is the one when you type which python
command. Add the following lines at the top.
export PATH="/home/USER_NAME/.localpython/bin:$PATH"
export WORKON_HOME=~/.virtualenvs
export PROJECT_HOME=~/Devel
source /home/USER_NAME/.localpython/bin/virtualenvwrapper.sh
exit your server and log in again. You should be good to go to make mkvirtualenv and do whatever you want.
The above guide is tested on my University cluster where python2.6 was installed however i don’t have control over custom libraries. I could use virtualenv though.
Check cass healthiness every hour and send out notification if down
reset cassandra database
Got a corrupted cass table. I unfortunately set the replication factor to 1 and have no way of getting it back.
From stackoverflow, the easist way is just to delete <data dir>/data/*
<data dir>/commitlog/*
<data dir>/saved_caches/*
from all nodes.