about 3 years ago

用select_related来提取数据和不用select_related来提取数据的差别。

我有一个Model(用户每天会填写一些问题,这个Model用来收集用户每天填写的问题的答案):

class Answer(models.Model):
    day_id = models.IntegerField()
    set_id = models.IntegerField()
    question_id = models.IntegerField()
    answer = models.TextField(default='')
    user = models.ForeignKey(User)

    def __unicode__(self):
        result = u'{4}:{0}.{1}.{2}: {3}'.format(self.day_id, self.set_id, self.question_id, self.answer, self.user.id)
        return result

这个Model除了自带的Field外还有一个外键 user,用于引用Django自带的User表。

我需要将所有用户的每天的答案都输出到一个文件里, 所以首先要提取出所有的答案:

answers = Answer.objects.all()

for answer in answers:
    print answer

运行这个指令会对数据库进行981次请求。数据库里有980行数据,Answer.objects.all()算是一次请求。那么剩下的980次请求是在干嘛呢?问题出在:

result = u'{4}:{0}.{1}.{2}: {3}'.format(self.day_id, self.set_id, self.question_id, self.answer, self.user.id)

当打印answer这个Object的时候,除了会打印出自身的Field外,还会打印出user.id. 由于Answer这个表中没有user.id这个Field,所以要单独请求User的表给出相应的id。这个就是造成980次请求的原因。每次打印一个answer,都要请求一遍User表来获取相对应的用户id。

这种提取数据的方式明显是不合理的,为了能一次完成任务,可以使用select_related:

answers = Answer.objects.all().select_related('user')

for answer in answers:
    print answer

这样,数据库请求只有一次。select_related()的作用其实是把Answer和User这两张表给合并了:

SELECT ... FROM "answer" INNER JOIN "auth_user" ON ( "answer"."user_id" = "auth_user"."id" )

这样页面的载入时间就会被大大减少。

 
about 3 years ago

最近在搭建服务器时需要每天对数据库里的内容进行备份,所以简单学习了下crontab这个命令的使用方式。

目标

  • 每天凌晨1点对数据库进行备份
  • 每次备份后将数据库文件加上时间戳并存储到指定文件夹
  • 备份完成后在log里写入当前时间和备份文件名

先写一个bash脚本

#! /bin/bash

cur_time=$(date +%Y_%m_%d_%k_%M_%S)    #获取当前时间,并按照 年_月_日_小时_分钟_秒 的格式存储

file_name=./db_backup/db.sqlite3.bak_$cur_time    #将刚获取的时间接在备份文件名的后面

sqlite3 db.sqlite3 .dump > $file_name    #对一个sqlite数据库进行备份

now=$(date +'%x %X')    #获取现在的时间

output=$(printf "[ %s ] db backed @ %s" "$now" "$file_name")    #写一行log

echo $output >> ~/logs/cron.log    #将log写入指定文件

将上述文件保存为一个脚本,比如backup.sh

再来添加定时重复的工作

在Linux下使用crontab这个命令可以做到:

  1. crontab -e | 编辑你的定时工作脚本
  2. crontab -l | 查看你的工作脚本
  3. crontab -r | 删除你的工作脚本
  4. crontab -l > 文件名 | 将你的脚本备份到一个文件里
  5. crontab 文件名 | 将指定文件的内容复制到你的工作脚本

编辑定时工作脚本

首先通过下面的命令来创建一个脚本,如果你有设置系统默认的文本编辑器的话,则该文件会在默认的编辑器下打开

crontab -e

如果输入完上述命令后你得到的结果是:

no crontab for root - using an empty one
888

则说明你还没有设置默认的文本编辑器,可以通过下面的命令来设置:

export EDITOR=nano

上面例子中使用了nano,但你可以随意替换成pico、vi等任意编辑器。设置好后,再次执行crontab -e

我希望每天凌晨一点自动备份一下数据库,那么我会在crontab的文件里这么写:

0 1 * * * /home/cheng/backup.sh

第一个0代表0分,1代表1点,后面的三个*分别代表日,月,周日。最后的文件路径是你需要执行的脚本。

前五个数值的范围分别为:

  • 分 0 - 59
  • 小时 0 - 23
  • 日 1 - 31
  • 月 1 - 12
  • 周日 sun, mon, tue, wed, thu, fri, sat (可以用这些缩写表示一周中的那一天,也可以用数字 0 - 7 表示, 注意0和7都是周日的意思)

下面是一些简单的示例:

每周日下午五点执行一次:

0 17 * * sun  /scripts/script.sh

每6小时执行一次:

0 */6 * * * /scripts/script.sh

每分钟执行一次:

* * * * * /scripts/script.sh

找到一个很好的参考网站: http://tecadmin.net/crontab-in-linux-with-20-examples-of-cron-schedule/

Crontab文件的存储位置

每个用户通过crontab -e创建的脚本文件会被存储在:

cd /var/spool/cron/crontabs/

这个文件夹下每个用户的脚本文件会以该用户的用户名进行区分存储。

Reference:
http://www.ibm.com/developerworks/library/l-job-scheduling/

 
about 3 years ago

I wrote two simple scripts for starting and stopping a gunicorn server that hosts a Django project.

Create a run.sh:

#!/bin/bash


source `which virtualenvwrapper.sh`  #assuming you use virtualenv

workon your_env_name # if you do not use virtualenv, remove this line the line above

gunicorn -c gunicorn.conf.py your_project_name.wsgi --daemon 

Create a stop.sh:

#!/bin/bash


kill -9 `ps aux |grep gunicorn |grep your_app_name | awk '{ print $2 }'`  # will kill all of the workers

Once these scripts are created, don't forget to set the permission:

chmod 755 run.sh
chmod 755 stop.sh

To run them:

./run.sh
./stop.sh
 
about 3 years ago

Bootstrap offers a great Accordion element but it needs a few patches:

  1. An up and down arrow beside the accordion's title would be nice.
  2. The clickable area is centered around the title text. If you click between the text and the background border, the accordion will not show or hide.
  3. A hide all or show all method would be nice.

I created a JSFiddle: http://jsfiddle.net/p1ok0t51/4/

A few key points to point out,

  1. the up/down arrow switch is enabled by the CSS.

  2. I added the following to both the div.panel-heading and h4 element, so that the clickable area is the entire title div:

data-toggle="collapse" data-target='#collapse2'

The data-toggle="collapse" attribute makes your element clickable, by adding it to the div.title element, the entire div becomes clickable. Thus, when users click in between the title text and the border of the div.title element, the accordion will still respond.

The data-target="#collapse2" attribute will add and remove a class named collapsed when clicked. The CSS in the JSFiddle has different styles when collapsed appear and disappear.

  1. I removed the data-parent="#accordion" attribute to allow show all and hide all. If this attribute is added like in the Bootstrap's example, then only one section can be expanded at anytime.

  2. A great website for looking up Bootstrap icons (if you don't want to use the up/down arrow button), click here: http://glyphicons.bootstrapcheatsheets.com/

Reference:
http://stackoverflow.com/questions/18325779/bootstrap-3-collapse-show-state-with-chevron-icon

 
about 3 years ago

I used Jasny's image upload plugin for Bootstrap to allow users upload images to a server. Later on, I had to populate a page with these uploaded images. Here is how I did it:

<!-- fileinput-exists is used instead of fileinput-new -->
<div class="fileinput fileinput-exists" data-provides="fileinput">
  <div class="fileinput-preview thumbnail" data-trigger="fileinput" style="width: 200px; height: 150px;">
  <!-- add the following line -->
  <img src="static/thumb.jpg">
  </div>
  <div>
    <span class="btn btn-default btn-file"><span class="fileinput-new">Select image</span><span class="fileinput-exists">Change</span><input type="file" name="..."></span>
    <a href="#" class="btn btn-default fileinput-exists" data-dismiss="fileinput">Remove</a>
  </div>
</div>

That's it :)

 
about 3 years ago

Just found a great tool for debugging Django applications, the Django debug toolbar provides a toolbar to your webpage when DEBUG=True. It looks like this:

https://github.com/django-debug-toolbar/django-debug-toolbar

The toolbar will provides a ton of information, such as:

  • The number of database queries made while loading the apge
  • The amount of time it took to load the page (similar to Chrome dev tools' Timeline feature)
  • The content of your Settings file
  • Content of the request and response headers
  • The name and page of each static files loaded along with the current page
  • Current page's template name and path
  • If Caching is used, it shows the content of the cached objects and the time it took to load them
  • Singals
  • Logging messages
  • Redirects

To install it:

pip install django-debug-toolbar

Then, in your settings.py:

INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',  # <- automatically added by Django, make sure it is not missing
    'debug_toolbar',                             # <- add this
    'myapp',                                           # <- your app 
)

# static url has to be defined
STATIC_URL = '/static/'

# pick and choose the panels you want to see
DEBUG_TOOLBAR_PANELS = [
    'debug_toolbar.panels.versions.VersionsPanel',
    'debug_toolbar.panels.timer.TimerPanel',
    'debug_toolbar.panels.settings.SettingsPanel',
    'debug_toolbar.panels.headers.HeadersPanel',
    'debug_toolbar.panels.request.RequestPanel',
    'debug_toolbar.panels.sql.SQLPanel',
    'debug_toolbar.panels.staticfiles.StaticFilesPanel',
    'debug_toolbar.panels.templates.TemplatesPanel',
    'debug_toolbar.panels.cache.CachePanel',
    'debug_toolbar.panels.signals.SignalsPanel',
    'debug_toolbar.panels.logging.LoggingPanel',
    'debug_toolbar.panels.redirects.RedirectsPanel',
]

That's it. Start your server by

python manage.py runser

Load up your page in the web browser, you should see a black vertical toolbar appearing on the right side of your page.

 
about 3 years ago

A very useful GUI for browsing sqlite DB either on Mac or Windows:

http://sqlitebrowser.org/

sqlite browser let you:

  • browse DB content
  • modify DB content
  • execute SQL commands

One thing to note, you can edit the content of your DB using this tool. After you make some modifications, make sure click on 'Write Changes' on the top to apply the changes to the DB.

 
about 3 years ago

RStudio的制作公司提供了一个叫RPubs的网站,该网站可以让用户在RStudio内将自己制作的R Markdown文档上传至该网站以便共享。

分享的方法是:

  1. 在Rstudio里穿件一个R Markdown文档
  2. 文档编写好后,点击上面工具栏里的 Knit HTML (注:确保这个文档的保存地址里没有中文出现。不要使用中文文件夹或者文件名) 3.当HTML生成好后,点击上面的Publish

如果出现下面错误:

Error in function (type, msg, asError = TRUE) : 
SSL certificate problem: unable to get local issuer certificate
Calls: rpubsUpload ... <Anonymous> -> .postForm -> .Call -> <Anonymous> -> fun
Execution halted

官方给出了解决方法

  1. 在Rstudio的命令行里输入
    file.edit('~/.Rprofile')
    
    这时,一个叫.Rprofile的文件会被打开。在里面输入:
    options(rpubs.upload.method = "internal")
    
    然后保存,Session -> Restart R,将R重启。然后再次打开生成好的HTML并Publish就可以了。(别忘了先去官网上注册个账号并登陆)。

P.S. 注意: 所有上传的文档都是公开的,所有人都能看到。

 
about 3 years ago

主题

Soda theme


Predawn theme

Spacegray theme

字体

Dejavu
编程推荐使用它的Sans Mono字体类型

Source code pro
Adobe推出的免费字体,主要用于用户界面上的文字。

 
about 3 years ago

最近刚刚接触到R,在学习过程中发现一个问题。我有一组数据大概是这样:

Year     Percentage
1950        20%
1955        21%
1960        22%
...

在我画图时报错:

> ggplot(china, aes(x = year, y = n)) +
  geom_line()

geom_path: Each group consist of only one observation. Do you need to adjust the group aesthetic?

通过检查变量类型,发现年份数据是Factor类型而不是Numeric类型:

$ year   : Factor w/ 21 levels "1950","1955",..: 1 2 3 4 5 6 7 8 9 10 ...

后来在Stackoverflow上发现了解决方案:

as.numeric(as.character(year))

year这个变量被做了两次转换,第一次是把它从Factor类型转换成Character类型,然后再从Character类型转换成Numeric类型。为什么不能将year直接转换成Numeric类型呢?我尝试了直接转换:

> as.numeric(china$year)
 [1]  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21

原本1950, 1955, 1960 ... 被转换成了1, 2, 3... 这样的问题在于,当你绘图时,X轴上原本应该显示年份,但却被1, 2, 3所代替:

所以为了避免出现这种情况,需要对数据进行两次转换。


刚刚发现了另外一种方法:

ggplot(china, aes(x = year, y = n)) +
  geom_line(aes(group=1))

但需要注意的是,如果要加入X轴坐标的话,需要使用:

scale_x_discrete  #不能使用scale_x_continuous,因为年份数据不是连续性的