over 2 years ago

“If you don’t design your own life plan, chances are you’ll fall into someone else’s plan. And guess what they have planned for you? Not much.” — Jim Rohn

 
over 2 years ago

When trying to access either the MEDIA_URL or STATIC_URL in the template, it is important to add the following to the template context processor in settings.py:

TEMPLATE_CONTEXT_PROCESSORS += (
    "django.core.context_processors.media",
    "django.core.context_processors.static",
)

Setting the MEDIA_URL or the STATIC_URL in settings.py is not enough.

 
over 2 years ago

Sometimes, we need to dump a database from production server and import it into development machine for testing purpose, to do that:

  1. use the following command to dump the database first:
su - postgres
pg_dump database_name > file_name

For instance, if I have a database called note, I would dump it as (this is done in shell not in psql):

pg_dump note > note.db.bak

Before importing note.db.bak into your local postgres, a few things need to be taken care of first:

1) You have to have the same list of users on your local machine as on your server. Use \du inside psql to list users on your server. Then, recreate them on your local machine in psql

create user user_name;

Users created on your local machine do not have to mirror the same privileges they have on the server.

2) Create the database that you want to import the backup into. In psql,

create database note_server_bak;

Now, you can import the data with the following command in shell:

psql -1 note_server_bak < note.db.bak

Note that the option after psql is 'dash one' not 'dash L'. This option means while importing, everything will be wrapped in a transaction so that if the import fails, nothing will be stored in note_server_bak, in other words, note_server_bak stays clean.

If you don't use -1 and an error occurs during importing, all of the data before the error will be imported into the database, which leaves your database with incomplete data. When you import the next time, you are going to see even more errors because of this.

 
over 2 years ago

I was running a Django server on Ubuntu 12.04 and saw a lot of errors logged in gunicorn's error log file:

error: [Errno 111] Connection refused

One line of the error messages caught my eye, send_mail. Gunicorn is trying to send my the error message via the send_mail() function but failed. I realized that I didn't setup the email settings in settings.py.

I searched online for a solution and found two:

  1. send email via Gmail's smtp
  2. setup your mail server

Option 1 seems like a quick and dirty way to get things done, but it has a few drawbacks:

  1. Sending email via Gmail's smtp means the FROM field will be your Gmail address rather than your company or whatever email address you want it to be.
  2. To access Gmail, you need to provide your username and password. This means you have to store your gmail password either in the settings.py file or to be more discreet in an environment variable.
  3. You have to allow less secured app to access your gmail. This is a setting in your gmail's account.

I decided to setup my own mail server because I don't want to use my personal email for contacting clients. To do that, I googled and found that there are two mail servers that I can use:

  • postfix
  • sendmail

Since postfix is newer and easier to config, I decided to use it.

  1. Install postfix and Find main.cf

Note: main.cf is the config for postfix mail server

sudo apt-get install postfix
postfix is already the newest version.
postfix set to manually installed.

Great, it is already installed. Then, I went to /etc/postfix/ to find main.cf and it is not there! Weird, so I tried to reinstall postfix:

sudo apt-get install --reinstall postfix

After installation, I saw a message:

Postfix was not set up.  Start with
  cp /usr/share/postfix/main.cf.debian /etc/postfix/main.cf
.  If you need to make changes, edit
/etc/postfix/main.cf (and others) as needed.  To view Postfix configuration
values, see postconf(1).

After modifying main.cf, be sure to run '/etc/init.d/postfix reload'.

I see. So, I followed the instruction and copied the main.cf file to /etc/postfix/:

cp /usr/share/postfix/main.cf.debian /etc/postfix/main.cf

Add the following lines to main.cf:

mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mydestination = localhost

Then, reload this config file:

/etc/init.d/postfix reload

Now, let's test to see if we can send an email to our mailbox via telnet:

telnet localhost 25

Once connected, enter the following line by line:

mail from: whatever@whatever.com
rcpt to: your_real_email_addr@blah.com
data (press enter)
type whatever content you feel like to type
. (put an extra period on the last line and then press enter again)

If everything works out, you sould see a comfirmation message resemables this:

250 2.0.0 Ok: queued as CC732427AE

It is guaranteed that this email will end up in the spam box if you use Gmail. So take a look at your spam inbox to see if you received the test mail (it may take a minute to show up).

If you recevied the test email, then it means postfix is working properly. Now, let's config Django to send email via postfix.

First, I added the following line to my settings.py file:

EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_PORT = 25
EMAIL_HOST_USER = ''
EMAIL_HOST_PASSWORD = ''
EMAIL_USE_TLS = False
DEFAULT_FROM_EMAIL = 'Server <server@whatever.com>'

Then, I opened up Django shell to test it:

./manage.py shell

>>> from django.core.mail import send_mail
>>> send_mail('Subject here', 'Here is the message.', 'from@example.com',
    ['to@example.com'], fail_silently=False)

Again, check your spam inbox. If you received this mail, then it means Django can send email via postfix, DONE!

references:
http://stackoverflow.com/questions/5802189/django-errno-111-connection-refused
http://stackoverflow.com/questions/4798772/postfix-its-installed-but-how-do-i-test
http://stackoverflow.com/questions/26333009/django-postfix

 
over 2 years ago

There are times when you are not worried about user authentication but still want to have each user sees only his/her stuff. Then you need a way to login a user without password, here is the solution posted on stackoverflow

Normally, when logging a user with password, authenticate() has to be called before login():

username = request.POST['username']
password = request.POST['password']
user = authenticate(username=username, password=password)
login(request, user)

What authenticate does is to add an attribute called backend with the value of the authentication backend, take a look at the line of the source code. The solution is to add this attribute yourself:

#assuming the user name is passed in via get
username = self.request.GET.get('username')
user = User.objects.get(username=username)

#manually set the backend attribute
user.backend = 'django.contrib.auth.backends.ModelBackend'
login(request, user)

That's it!

 
over 2 years ago

When a user request a page view from a website (powered by Django), a cookie is returned along with the requested page. Inside this cookie, a key/value pair is presented:

Cookie on the user's computer

Key          Value
---          -----
sessionid    gilg56nsdelont4740onjyto48sv2h7l

This id is used to uniquely identify who's who by the server. User A's id is different from User B's etc. This id is not only stored in the cookie on the user's computer, it is also stored in the database on the server (assuming you are using the default session engine). By default, after running ./manage.py migrate, a table named django_session is created in the database. It has three columns:

django_session table in database

session_key        session_data        expire_date
---------------------------------------------------
y5j0jy3l4v3        ZTJlMmZiMGYw        2015-05-08 15:13:28.226903

The value stored in the session_key column matches the value stored in the cookie received by the user.

Let's say this user decides to login to the web service. Upon successfully logged into the system, a new sessionid is assigned to him/her and a different session_data is stored in the database:

Before logging in:

session_key        session_data        expire_date
---------------------------------------------------
437383928373        anonymous           2015-05-08 15:13:28.226903
After logging in:

session_key        session_data        expire_date
---------------------------------------------------
218374758493        John               2015-05-08 15:13:28.226903

*I made up this example to use numbers and usernames instead of hash strings. For security reasons, these are all hash strings in reality.

As we can see here, a new session_key has been assigned to this user and we now know that this user is 'John'. Form now on, John's session_key will not change even if he closes the browser and visit this server again. Thus, when John comes back the next day, he does not need to login again.

Django provides a setting to let developers to specify this behaviour, in settings.py, a variable named SESSION_SAVE_EVERY_REQUEST can be set:

SESSION_EXPIRE_AT_BROWSER_CLOSE = False # this is the default value. the session_id will not expire until SESSION_COOKIE_AGE has reached.

If this is set to True, then John is forced to login everytime he visits this website.

Since saving and retrieving session data from the database can be slow, we can store session data in memory by:

#Assuming memcached is installed and set as the default cache engine
SESSION_ENGINE = 'django.contrib.sessions.backends.cache'

The advantage of this approach is that session store/retrival will be faster. But the downside is if the server crashes, all session data is lost.

A mix of cache & database storage is:

SESSION_ENGINE = 'django.contrib.sessions.backends.cached_db'

According to django's documentation:

every write to the cache will also be written to the database. Session reads only use the database if the data is not already in the cache.

This approach is slower than a pure cache solution but faster than a purse db solution.

Django's offical document did warn to not use local-memory cache as it doesn't retain data long enough to be a good choice.

By default the session data for a logged in user lasts two weeks in Django, users have to log back in after the session expires. This time period can be adjusted by setting the SESSION_COOKIE_AGE variable.

 
over 2 years ago

I had to deploy a django project on CentOS 6.5 and here are the steps to set up Python and then Django. The problem with CentOS 6.5 is that it comes with python 2.6.6 and it's package installer yum relies on it.

Although we can install python 2.7.x and soft link 2.7.x as the system default, it will break yum and who knows what else relies on 2.6.6. Thus, I am installing python 2.7.x via pyenv, which is suggested by this blog post.

This solution is cleaner as it does not temper with the OS default python 2.6.6.

  1. Update all of yum's package
yum -y update 
  1. Install pyenv, which works like rbenv:
curl -L https://raw.githubusercontent.com/yyuu/pyenv-installer/master/bin/pyenv-installer | bash

Put the following lines into your .bashrc file:

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

Then, reload bashrc:

. .bashrc

This will enable the pyenv command in bash.

  1. Install these packages before installing python:
yum install zlib-devel bzip2 bzip2-devel readline-devel sqlite sqlite-devel mysql-devel 
  1. Use this command to list all of the 2.7.x version of python (grep 3.x for python 3):
pyenv install -l | grep 2.7 
  1. Install the python version you prefer:
pyenv install 2.7.10 
  1. Use the newly installed version of python:
pyenv local 2.7.10

You can list all python versions abailable to pyenv by:

pyenv versions
  1. Update pip to its latest version:
pip install -U pip

Now, we have everything setup and ready to go. The natural thing to do next is to install virtualenv and virtualenvwrapper:

Installing virtualenv and virtualenvwrapper

WARNING: DO NOT install virtualenv and virtualenvwrapper !!!
Let me tell you what happend after I installed both via pip. I put the following lines in .bashrc

export WORKON_HOME=$HOME/.virtualenvs
source ~/.pyenv/shims/virtualenvwrapper.sh  #virtualenvwrapper.sh got installed to this dir instead of /usr/local/bin/

Then, I was disconnected from the cloud server with the following warning:

pyenv: -bash: command not found

It doesn't matter what I try, I cannot connect to the cloud server anymore because every time I login successfully, I get disconnected from the server with the same error above. I cannot even login using the VNC connection provided by the cloud service provider. The only option I had was to reinstall the image on the cloud server...

I cannot find the cause of this issue on Google, but from the look of it, I messed out the path to bash so everytime I logged in, cannot find bash -> disconnect

Don't worry you can still use virtualenv

If you look under the .pyenv/plugins directory, you can see a directory named pyenv-virtualenv. This is what we can use to create virtual environments.

To create a test environment:

pyenv virtualenv test  # create a virtualenv named test based on the current local or system version of python

Since we have already set our local version of python to be 2.7.10, this test env inherits from that. But if you did not set the local version, the test env will inherit from the system version of python which may not be what you want. (Note: while running the command above, the 'virtualenv' package is installed to your local version of python)

You can also specify which version of python to use (by putting a version number after 'pyenv virutalenv'):

pyenv virtualenv 2.6.6 test # use version 2.6.6 for this environment

Once the test environment has been setup, you can verify it by:

pyenv versions

You should see something similar to this:

  system
* 2.7.10 (set by /.python-version)
  test

You can switch to the test env by:

pyenv activate test

To deactivate:

pyenv deactivate

To list virtualenvs:

pyenv virtualenvs

To delete existing virtualenvs:

pyenv uninstall my-virtual-env

If you have seen pyenv-virtualenv's github page, you have probably noticed something named pyenv-virtualenvwrapper. Don't install it unless you know what it is. I don't have the time to figure out what it is ATM. But it is definitely NOT the virtualenv + virtualenvwrapper combination you are familiar with. So be cautious!

 
over 2 years ago

首先确保你安装了Homebrew, 然后:

brew install rbenv ruby-build

安装成功后, 编辑你的.bash_profile文件, 将一下两行命令添加在文件的最后:

export PATH="$HOME/.rbenv/bin:$PATH"
eval "$(rbenv init -)"

添加完后,重新执行一遍bash_profile:

cd ~
. .bash_profile

然后,用rbenv来安装一个你想要的ruby版本,比如你要安装2.2.2版本的话:

rbenv install 2.2.2
rbenv gloabl 2.2.2   

查看一下我们安装ruby是通过rbenv安装的,而不再是Mac系统自带的ruby:

which ruby

输出的结果应该包含(.rbenv/shims/ruby):

/Users/cheng/.rbenv/shims/ruby

然后更新gem到最新版本:

gem update --system

现在就可以使用我们的ruby而不使用系统自带的ruby了。如果要继续安装其它的gem,而且你又在国内,确保你换个安装源。因为国内防火墙的原因,无法直接使用gem自带的源来安装。

总结一下整个安装的过程:

  1. 为了避免和Mac系统自带的ruby发生冲突,我们通过安装rbenv来绕过系统自带的ruby。
  2. 通过brew来安装rbenv和ruby-build。(rbenv需要ruby-build)
  3. 将rbenv的执行目录放在系统自带的ruby目录前面来保证接下来运行的ruby命令是从rbenv来的。
  4. 通过rbenv选择一个先要安装的ruby版本进行安装。用gloabl命令将这个版本的ruby定为全局使用的默认ruby版本。
  5. 更新gem。
  6. 替换gem的安装源。

参考:
https://robots.thoughtbot.com/using-rbenv-to-manage-rubies-and-gems
http://stackoverflow.com/questions/10940736/rbenv-not-changing-ruby-version
http://stackoverflow.com/questions/14607193/installing-gem-or-updating-rubygems-fails-with-permissions-error

 
over 2 years ago

I had the following code:

from django.core.urlresolvers import reverse

class UserProfileView(FormView):
    template_name = 'profile.html'
    form_class = UserProfileForm
    success_url = reverse('index')

When the code above runs, an error is thrown:

django.core.exceptions.ImproperlyConfigured: The included urlconf 'config.urls' does not appear to have any patterns in it. If you see valid patterns in the file then the issue is probably caused by a circular import.

There are two solutions to this problem, solution one:

from django.core.urlresolvers import reverse_lazy 

class UserProfileView(FormView):
    template_name = 'profile.html'
    form_class = UserProfileForm
    success_url = reverse_lazy('index') # use reverse_lazy instead of reverse

Solution 2:

from django.core.urlresolvers import reverse

class UserProfileView(FormView):
    template_name = 'profile.html'
    form_class = UserProfileForm
    
    def get_success_url(self): # override this function if you want to use reverse

        return reverse('index')

According to Django's document, reverse_lazy should be used instead of reverse when your project's URLConf is not loaded. The documentation specifically points out that reverse_lazy should be used in the following situation:

  • providing a reversed URL as the url attribute of a generic class-based view. (this is the situation I encountered)

  • providing a reversed URL to a decorator (such as the login_url argument for the django.contrib.auth.decorators.permission_required() decorator).

  • providing a reversed URL as a default value for a parameter in a function’s signature.

It is unclear when URLConf is loaded. At least I cannot find the documentation on this topic. So if the above error occurs again, try reverse_lazy

 
over 2 years ago

I didn't expect creating a radio input in a form using Bootstrap3's style could have some many pitfalls.

Here is a model I have:

class UserProfile(models.Model):
    GENDER_CHOICES = (('M', 'Male',), ('F', 'Female',))
    fullname = models.CharField(u'Name', max_length=10)
    sex = models.CharField(u'Gender', max_length=1, choices=GENDER_CHOICES)
    # ... and 11 other fields

Here is the corresponding ModelForm:

from django.forms.widgets import RadioSelect

class UserProfileForm(ModelForm):
    class Meta:
        model = UserProfile
        widgets = {
            'sex': forms.RadioSelect()
        }

I have a sex field which let users to pick their gender. When running the code above, I get three choices instead of two:

  • '--------' # django automatically insert this 'default' option
  • 'Male'
  • 'Female'

This is of course not what we want. There should only be two options, 'Male' and 'Female'. So to correct this issue, I had to add TWO extra parameters to the Model:

sex = models.CharField(u'Gender', max_length=1, choices=GENDER_CHOICES, blank=False, default='M')

Note that I added blank=False AND default='M'. If you only include blank=False, you will still see the '--------' option. It is important to have both of them set.

Now, I want to style my forms using Bootstrap3. To do this, I recommend using crispy-forms

from crispy_forms.helper import FormHelper
from crispy_forms.layout import Submit
from crispy_forms.bootstrap import InlineRadios

class UserProfileForm(forms.ModelForm):
    def __init__(self, *args, **kwargs):
        super(UserProfileForm, self).__init__(*args, **kwargs)
        self.helper = FormHelper(self)
        self.helper.method = 'post'
        self.helper.form_action = 'index'
        self.helper.layout.append(Submit('save', 'Save'))

    class Meta:
        model = UserProfile

Crispy-form creates a Bootstrap3 style input field for every single field defined in the model when this line runs:

self.helper = FormHelper(self)

The UserProfile model has 13 fields. It is nice that I don't have to specify each one of the 13 fields. But here comes the problem, when the radio input is been rendered. The male and female options are listed vertically. I want them to be listed horizontally. Crispy-form does provide an inline option for rendering radio inputs horizontally. This is nice, but how can I replace the default Layout of the gender field with the suggested inline Layout?

self.helper[1] = InlineRadios('sex')

Crispy-forms allows you to access the Layout of each field by using index.

With the tweaks above, now I have a wonderful radio input for gender selection :) So many small things need to be taken care of for something seems so trivial.