In this part of the post, I would like to document the steps needed to run an existing Django project on gunicorn as the backend server and nginx as the reverse proxy server. (Please refer to Part 1 of this post)
Disclaimer: this post is based on this site but I added a lot of details that wasn't explained.
Setup nginx
Before starting the nginx server, we want modify its config file. I installed nginx via brew on my machine, and the conf file is located here:
/usr/local/etc/nginx/nginx.conf
We can modify this file directly to add configurations for our site, but this is not a good idea. Because you may have multiple web apps running behind this nginx server. Each web app may need its own configuration. To do this, let's create another folder for storing these site-wise configuration files first:
cd /usr/local/etc/nginx
mkdir sites-enabled
We can store our site specific config here but wouldn't it be better if we store the config file along with our project files together? Now, let's navigate to our project folder. (I named my project testproject and stored it under /Users/webapp/Apps/testproject)
cd /Users/webapp/Apps/testproject
touch nginx.conf
Here is my config file:
server {
listen 80;
server_name your_server_ip;
access_log /Users/webapp/logs/access.log; # <- make sure to create the logs directory
error_log /Users/webapp/logs/error.log; # <- you will need this file for debugging
location / {
proxy_pass http://127.0.0.1:9000; # <- let nginx pass traffic to the gunicorn server
}
location /static {
root /Users/webapp/Apps/testproject/vis; # <- let nginx serves the static contents
}
}
Let me elaborate on the '/static' part. This part means that any traffic to 'your_server_ip/static' will be forwarded to '/Users/webapp/Apps/testproject/vis/static'. You might ask why doesn't it forward to '/Users/webapp/Apps/testproject/vis'?(without '/static' in the end) Because when using 'root' in the config, it will append the '/static' part after it. So be aware! You can fix this by using alias instead of root and append /static to the end of the path:
location /static {
alias /Users/webapp/Apps/testproject/vis/static;
}
Here is the folder structure of my project :
/Users/webapp/Apps/testproject/
manage.py <- the manage.py file generated by Django
nginx.conf <- the nginx config file for your project
gunicorn.conf.py <- the gunicorn config file that we will create later, just keep on reading
testproject/ <- automatically generated by Django
settings.py
urls.py
wsgi.py <- automatically generated by Django and used by gunicorn later
vis/ <- the webapp that I wrote
admin.py
models.py
test.py
urls.py
template/
vis/
index.html
static/ <- the place where I stored all of the static files for my project
vis/
css/
images/
js/
All of the static files are in the /testproject/vis/static folder, so that's where nginx should be looking. You might ask that the static files live in their own folders rather than right under the /static/ path. How does nginx know where to fetch them? Well, this is not nginx's problem to solve. It is your responsibility to code the right path in your template. This is what I wrote in my template/vis/index.html page:
href="{% static 'vis/css/general.css' %}
It is likely that you won't get the path right the first time. But that's ok. Just open up Chrome's developer tools and look at the error messages in the console to see which part of the path is messed up. Then, either fix your nginx config file or your template.
To let nginx read our newly create config file:
cd /usr/local/etc/nginx/
nano nginx.conf
Find the ' http { ' header, add this line under it:
http{
include /usr/local/etc/nginx/sites-enabled/*;
This line tells nginx to look for config files under the 'sites-enabled' folder. Instead of copy our project's nginx.conf into the 'sites-enabled' folder, we can simply create a soft link instead:
cd /usr/local/etc/nginx/site-enabled
ln -s /full_path/to/your/django_project a_name
# in my case, this is what my link command looks like:
# ln -s /Users/webapp/Apps/testproject/nginx.conf testproject
# this would create a soft link named testproject which points to the real config file
Once this is done, you can finally start up the nginx server:
To start nginx, use
sudo nginx
To stop it, use
sudo nginx -s stop
To reload the config file without shutting down the server:
sudo nginx -s reload
Please refer to this page for a quick overview of the commands.
Setup gunicorn
Setting up gunicorn is more straight forward (without considering optimization). First, let's write a config file for gunicorn. Navigate to the directory which contains your manage.py file, for me, this is what I did:
cd /Users/webapp/Apps/testproject
touch gunicorn.conf.py # yep, the config file is a python script
This is what I put in the config file:
bind = "127.0.0.1:9000" # Don't use port 80 becaue nginx occupied it already.
errorlog = '/Users/webapp/logs/gunicorn-error.log' # Make sure you have the log folder create
accesslog = '/Users/webapp/logs/gunicorn-access.log'
loglevel = 'debug'
workers = 1 # the number of recommended workers is '2 * number of CPUs + 1'
Save the file and to start gunicorn, make sure you are at the same directory level as where the manage.py file is and do:
gunicorn -c gunicorn.conf.py testproject.wsgi
The ' -c ' option means read from a config file. The testproject.wsgi part is actually referring to the wsgi.py file in a child folder. (Please refer to my directory structure above)
Just in case if you need to shutdown gunicorn, you can either use Ctrl + c at the console or if you lost connection to the console, use [1]:
kill -9 `ps aux | grep gunicorn | awk '{print $2}'`
Actually, a better way to run and shutdown gunicorn is to make it a daemon process so that the server will still run even if you log out of the machine. To do that, use the following command:
gunicorn -c gunicorn.conf.py testproject.wsgi --pid ~/logs/gunicorn.pid --daemon
This command will do three things:
- run gunicorn with the configuration file named gunicorn.conf.py
- save the process id of the gunicorn process to a specific file ('~/logs/gunicorn.pid' in this case)
- run gunicorn in daemon mode so that it won't die even if we log off
To shutdown this daemon process, open '~/logs/gunicorn.pid' to find the pid and use (assuming 12345 is what stored in '~/logs/gunicorn.pid'):
kill -9 12345
to kill the server.
This is it! Enter 127.0.0.1 in your browser and see if your page loads up. It is likely that it won't load up due to errors that you are not aware of. That's ok. Just look at the error logs:
tail -f error.log
Determine if it is an nginx, gunicorn or your Django project issue. It is very likely that you don't have proper access permission or had a typo in the config files. Just going through the logs and you will find out which part is causing the issue. Depending on how you setup your Django's logging config, you can either read debug messages in the same console which you started gunicorn, or read it form a file. Here is what I have in my Django's settings.py file:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format' : "[%(asctime)s] %(levelname)s [%(name)s:%(lineno)s] %(message)s",
'datefmt' : "%d/%b/%Y %H:%M:%S"
},
'simple': {
'format': '%(levelname)s %(message)s'
},
},
'handlers': {
'null': {
'level': 'DEBUG',
'class': 'logging.NullHandler',
},
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'verbose'
},
'logfile':{
'level':'DEBUG',
'class':'logging.handlers.WatchedFileHandler',
'filename': "/Users/webapp/gunicorn_log/vis.log",
'formatter': 'verbose',
},
},
'loggers': {
'django.request': {
'handlers': ['logfile'],
'level': 'DEBUG',
'propagate': True,
},
'django': {
'handlers': ['logfile'],
'propagate': True,
'level': 'DEBUG',
},
'vis': {
'handlers': ['console', 'logfile'],
'level': 'DEBUG',
'propagate': False,
},
}
}
Bad Request 400
Just when you thought everything is ready, and you want the world to see what you have built...BAM! Bad Request 400. Why??? Because when you turn DEBUG = False in the settings.py file, you have to specify the ALLOWED_HOSTS attribute:
DEBUG = False
ALLOWED_HOSTS = [
'127.0.0.1',
]
Just don't put your port number here, only the ip part. Read more about this issue here: http://stackoverflow.com/questions/19875789/django-gives-bad-request-400-when-debug-false
Conclusion
Setting up nginx, gunicorn and your Django project can be a very tedious process if you are not familiar with any of them like me. I document my approach here to hopefully help anybody who has encountered the same issue.
