Cheng's Blog

Run the following command to copy the Packages and Installed Packages folder to your dropbox.

mkdir $HOME/Dropbox/sublime-text-3/
mv "$HOME/Library/Application Support/Sublime Text 3/Packages" "$HOME/Dropbox/sublime-text-3/"
mv "$HOME/Library/Application Support/Sublime Text 3/Installed Packages" "$HOME/Dropbox/sublime-text-3/"

Then, on a different computer, remove its Packages and Installed Packages folder and make a symbolic link to the copies in the Dropbox folder.

DSTPATH="$HOME/Library/Application Support/Sublime Text 3"
DROPBOX_PATH="$HOME/Dropbox/sublime-text-3"
rm -rf "$DSTPATH/Installed Packages"
rm -rf "$DSTPATH/Packages"
mkdir -p "$DSTPATH"
ln -s "$DROPBOX_PATH/Packages" "$DSTPATH/Packages"
ln -s "$DROPBOX_PATH/Installed Packages" "$DSTPATH/Installed Packages"

seen on: http://www.hongkiat.com/blog/sublime-text-tips/

After updating pip using pip install -U pip,

$ pip
/usr/bin/pip: No such file or directory

pip can no longer be found:

$ which pip
/usr/local/bin/pip

$ pip
-su: /usr/bin/pip: No such file or directory

$ type pip
pip is hashed (/usr/bin/pip)

So pip is definintely in /usr/local/bin/pip but it is been cached as in /usr/bin/pip, thanks to the Stackoverflow question, the solution is very simple:

$ hash -r

When the cache is clear, pip is working again.

Linters are used to highlight syntax and style errors in your source code. It is very useful to have when writing python, js, css, YAML, etc.

To enable linting in Sublime text 3, a few things need to be installed to get it working.

1) Install Sublime Linter first. This is a framework rather than an actual linter that can inspect your code. When we install other linters, they will depend on this framework. Thus, it is important to install this 'mother base' first.

  - open up package control, cmd+shift+p on Mac OS X, ctrl+shift+p on Linux/Windows
  - type 'install' and select 'Package Control: Install Packages' 
  - type 'SublimeLinter'

2) Install pep8 and pyflake. These python libraries will be used by the linters installed in the next step. Open up a console window, and install them by:

pip3 install pep8 pyflakes

The pep8 package is for style checking and pyflakes is for syntax error checking.

NOTE: make sure you are not installing these packages while using virtualenv. This is the mistake that I made. These pacakges need to be available system wide.

3) Install the actual linters: Sublime Linter-pyflakes, Sublime Linter-pep8:

• open up package control, cmd+shift+p on Mac OS X, ctrl+shift+p on Linux/Windows
• type 'install' and select 'Package Control: Install Packages'
• type 'pep8' and find 'SublimeLinter-pep8'
• type 'pyflakes' and find 'SublimeLinter-pyflakes'

4) After everything has been installed, restart sublime text 3

5) Now you should see error highlights in the gutter and around the code. To set when linting should be active, right click anywhere in Sublime Text 3 and select 'SublimeLinter' > 'Lint Mode'

6) It is very likely that pep8 will point out a few styling errors. It is anonying to fix them by hand. We can use another plugin to autoformat python code for us. Install Python PEP8 Autoformat via package control.

After this is installed, press 'ctrl + shift + r' to perform auto format.

7) pep8 linter is likely to complain about using 4 spaces instead of tabs when writing Python code. You can change this in Sublime's setting File. Go to 'Sublime Text > Preferences > Settings - User', add the following lines:

"tab_size": 4,
"translate_tabs_to_spaces": true,

Now, when the tab key is pressed, 4 spaces are inserted instead.

“If you don’t design your own life plan, chances are you’ll fall into someone else’s plan. And guess what they have planned for you? Not much.” — Jim Rohn

When trying to access either the MEDIA_URL or STATIC_URL in the template, it is important to add the following to the template context processor in settings.py:

TEMPLATE_CONTEXT_PROCESSORS += (
    "django.core.context_processors.media",
    "django.core.context_processors.static",
)

Setting the MEDIA_URL or the STATIC_URL in settings.py is not enough.

Sometimes, we need to dump a database from production server and import it into development machine for testing purpose, to do that:

1. use the following command to dump the database first:

su - postgres
pg_dump database_name > file_name

For instance, if I have a database called note, I would dump it as (this is done in shell not in psql):

pg_dump note > note.db.bak

Before importing note.db.bak into your local postgres, a few things need to be taken care of first:

1) You have to have the same list of users on your local machine as on your server. Use \du inside psql to list users on your server. Then, recreate them on your local machine in psql

create user user_name;

Users created on your local machine do not have to mirror the same privileges they have on the server.

2) Create the database that you want to import the backup into. In psql,

create database note_server_bak;

Now, you can import the data with the following command in shell:

psql -1 note_server_bak < note.db.bak

Note that the option after psql is 'dash one' not 'dash L'. This option means while importing, everything will be wrapped in a transaction so that if the import fails, nothing will be stored in note_server_bak, in other words, note_server_bak stays clean.

If you don't use -1 and an error occurs during importing, all of the data before the error will be imported into the database, which leaves your database with incomplete data. When you import the next time, you are going to see even more errors because of this.

I was running a Django server on Ubuntu 12.04 and saw a lot of errors logged in gunicorn's error log file:

error: [Errno 111] Connection refused

One line of the error messages caught my eye, send_mail. Gunicorn is trying to send my the error message via the send_mail() function but failed. I realized that I didn't setup the email settings in settings.py.

I searched online for a solution and found two:

1. send email via Gmail's smtp
2. setup your mail server

Option 1 seems like a quick and dirty way to get things done, but it has a few drawbacks:

1. Sending email via Gmail's smtp means the FROM field will be your Gmail address rather than your company or whatever email address you want it to be.
2. To access Gmail, you need to provide your username and password. This means you have to store your gmail password either in the settings.py file or to be more discreet in an environment variable.
3. You have to allow less secured app to access your gmail. This is a setting in your gmail's account.

I decided to setup my own mail server because I don't want to use my personal email for contacting clients. To do that, I googled and found that there are two mail servers that I can use:

• postfix
• sendmail

Since postfix is newer and easier to config, I decided to use it.

1. Install postfix and Find main.cf

Note: main.cf is the config for postfix mail server

sudo apt-get install postfix

postfix is already the newest version.
postfix set to manually installed.

Great, it is already installed. Then, I went to /etc/postfix/ to find main.cf and it is not there! Weird, so I tried to reinstall postfix:

sudo apt-get install --reinstall postfix

After installation, I saw a message:

Postfix was not set up.  Start with
  cp /usr/share/postfix/main.cf.debian /etc/postfix/main.cf
.  If you need to make changes, edit
/etc/postfix/main.cf (and others) as needed.  To view Postfix configuration
values, see postconf(1).

After modifying main.cf, be sure to run '/etc/init.d/postfix reload'.

I see. So, I followed the instruction and copied the main.cf file to /etc/postfix/:

cp /usr/share/postfix/main.cf.debian /etc/postfix/main.cf

Add the following lines to main.cf:

mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mydestination = localhost

Then, reload this config file:

/etc/init.d/postfix reload

Now, let's test to see if we can send an email to our mailbox via telnet:

telnet localhost 25

Once connected, enter the following line by line:

mail from: whatever@whatever.com
rcpt to: your_real_email_addr@blah.com
data (press enter)
type whatever content you feel like to type
. (put an extra period on the last line and then press enter again)

If everything works out, you sould see a comfirmation message resemables this:

250 2.0.0 Ok: queued as CC732427AE

It is guaranteed that this email will end up in the spam box if you use Gmail. So take a look at your spam inbox to see if you received the test mail (it may take a minute to show up).

If you recevied the test email, then it means postfix is working properly. Now, let's config Django to send email via postfix.

First, I added the following line to my settings.py file:

EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_PORT = 25
EMAIL_HOST_USER = ''
EMAIL_HOST_PASSWORD = ''
EMAIL_USE_TLS = False
DEFAULT_FROM_EMAIL = 'Server <server@whatever.com>'

Then, I opened up Django shell to test it:

./manage.py shell

>>> from django.core.mail import send_mail
>>> send_mail('Subject here', 'Here is the message.', 'from@example.com',
    ['to@example.com'], fail_silently=False)

Again, check your spam inbox. If you received this mail, then it means Django can send email via postfix, DONE!

references:
http://stackoverflow.com/questions/5802189/django-errno-111-connection-refused
http://stackoverflow.com/questions/4798772/postfix-its-installed-but-how-do-i-test
http://stackoverflow.com/questions/26333009/django-postfix

There are times when you are not worried about user authentication but still want to have each user sees only his/her stuff. Then you need a way to login a user without password, here is the solution posted on stackoverflow

Normally, when logging a user with password, authenticate() has to be called before login():

username = request.POST['username']
password = request.POST['password']
user = authenticate(username=username, password=password)
login(request, user)

What authenticate does is to add an attribute called backend with the value of the authentication backend, take a look at the line of the source code. The solution is to add this attribute yourself:

#assuming the user name is passed in via get
username = self.request.GET.get('username')
user = User.objects.get(username=username)

#manually set the backend attribute
user.backend = 'django.contrib.auth.backends.ModelBackend'
login(request, user)

That's it!

When a user request a page view from a website (powered by Django), a cookie is returned along with the requested page. Inside this cookie, a key/value pair is presented:

Cookie on the user's computer

Key          Value
---          -----
sessionid    gilg56nsdelont4740onjyto48sv2h7l

This id is used to uniquely identify who's who by the server. User A's id is different from User B's etc. This id is not only stored in the cookie on the user's computer, it is also stored in the database on the server (assuming you are using the default session engine). By default, after running ./manage.py migrate, a table named django_session is created in the database. It has three columns:

django_session table in database

session_key        session_data        expire_date
---------------------------------------------------
y5j0jy3l4v3        ZTJlMmZiMGYw        2015-05-08 15:13:28.226903

The value stored in the session_key column matches the value stored in the cookie received by the user.

Let's say this user decides to login to the web service. Upon successfully logged into the system, a new sessionid is assigned to him/her and a different session_data is stored in the database:

Before logging in:

session_key        session_data        expire_date
---------------------------------------------------
437383928373        anonymous           2015-05-08 15:13:28.226903

After logging in:

session_key        session_data        expire_date
---------------------------------------------------
218374758493        John               2015-05-08 15:13:28.226903

*I made up this example to use numbers and usernames instead of hash strings. For security reasons, these are all hash strings in reality.

As we can see here, a new session_key has been assigned to this user and we now know that this user is 'John'. Form now on, John's session_key will not change even if he closes the browser and visit this server again. Thus, when John comes back the next day, he does not need to login again.

Django provides a setting to let developers to specify this behaviour, in settings.py, a variable named SESSION_SAVE_EVERY_REQUEST can be set:

SESSION_EXPIRE_AT_BROWSER_CLOSE = False # this is the default value. the session_id will not expire until SESSION_COOKIE_AGE has reached.

If this is set to True, then John is forced to login everytime he visits this website.

Since saving and retrieving session data from the database can be slow, we can store session data in memory by:

#Assuming memcached is installed and set as the default cache engine
SESSION_ENGINE = 'django.contrib.sessions.backends.cache'

The advantage of this approach is that session store/retrival will be faster. But the downside is if the server crashes, all session data is lost.

A mix of cache & database storage is:

SESSION_ENGINE = 'django.contrib.sessions.backends.cached_db'

According to django's documentation:

every write to the cache will also be written to the database. Session reads only use the database if the data is not already in the cache.

This approach is slower than a pure cache solution but faster than a purse db solution.

Django's offical document did warn to not use local-memory cache as it doesn't retain data long enough to be a good choice.

By default the session data for a logged in user lasts two weeks in Django, users have to log back in after the session expires. This time period can be adjusted by setting the SESSION_COOKIE_AGE variable.

I had to deploy a django project on CentOS 6.5 and here are the steps to set up Python and then Django. The problem with CentOS 6.5 is that it comes with python 2.6.6 and it's package installer yum relies on it.

Although we can install python 2.7.x and soft link 2.7.x as the system default, it will break yum and who knows what else relies on 2.6.6. Thus, I am installing python 2.7.x via pyenv, which is suggested by this blog post.

This solution is cleaner as it does not temper with the OS default python 2.6.6.

1. Update all of yum's package

yum -y update 

1. Install pyenv, which works like rbenv:

curl -L https://raw.githubusercontent.com/yyuu/pyenv-installer/master/bin/pyenv-installer | bash

Put the following lines into your .bashrc file:

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

Then, reload bashrc:

. .bashrc

This will enable the pyenv command in bash.

1. Install these packages before installing python:

yum install zlib-devel bzip2 bzip2-devel readline-devel sqlite sqlite-devel mysql-devel 

1. Use this command to list all of the 2.7.x version of python (grep 3.x for python 3):

pyenv install -l | grep 2.7 

1. Install the python version you prefer:

pyenv install 2.7.10 

1. Use the newly installed version of python:

pyenv local 2.7.10

You can list all python versions abailable to pyenv by:

pyenv versions

1. Update pip to its latest version:

pip install -U pip

Now, we have everything setup and ready to go. The natural thing to do next is to install virtualenv and virtualenvwrapper:

Installing virtualenv and virtualenvwrapper

WARNING: DO NOT install virtualenv and virtualenvwrapper !!!
Let me tell you what happend after I installed both via pip. I put the following lines in .bashrc

export WORKON_HOME=$HOME/.virtualenvs
source ~/.pyenv/shims/virtualenvwrapper.sh  #virtualenvwrapper.sh got installed to this dir instead of /usr/local/bin/

Then, I was disconnected from the cloud server with the following warning:

pyenv: -bash: command not found

It doesn't matter what I try, I cannot connect to the cloud server anymore because every time I login successfully, I get disconnected from the server with the same error above. I cannot even login using the VNC connection provided by the cloud service provider. The only option I had was to reinstall the image on the cloud server...

I cannot find the cause of this issue on Google, but from the look of it, I messed out the path to bash so everytime I logged in, cannot find bash -> disconnect

Don't worry you can still use virtualenv

If you look under the .pyenv/plugins directory, you can see a directory named pyenv-virtualenv. This is what we can use to create virtual environments.

To create a test environment:

pyenv virtualenv test  # create a virtualenv named test based on the current local or system version of python

Since we have already set our local version of python to be 2.7.10, this test env inherits from that. But if you did not set the local version, the test env will inherit from the system version of python which may not be what you want. (Note: while running the command above, the 'virtualenv' package is installed to your local version of python)

You can also specify which version of python to use (by putting a version number after 'pyenv virutalenv'):

pyenv virtualenv 2.6.6 test # use version 2.6.6 for this environment

Once the test environment has been setup, you can verify it by:

pyenv versions

You should see something similar to this:

  system
* 2.7.10 (set by /.python-version)
  test

You can switch to the test env by:

pyenv activate test

To deactivate:

pyenv deactivate

To list virtualenvs:

pyenv virtualenvs

To delete existing virtualenvs:

pyenv uninstall my-virtual-env

If you have seen pyenv-virtualenv's github page, you have probably noticed something named pyenv-virtualenvwrapper. Don't install it unless you know what it is. I don't have the time to figure out what it is ATM. But it is definitely NOT the virtualenv + virtualenvwrapper combination you are familiar with. So be cautious!