3 months ago

The behavior of passing an integer, string, slicing, list of integers, list of strings to a DataFrame:

df[0] # error

df['col_0'] # A Series of col_0

# The following behaviors are downright confusing

df[0:1] # A DataFrame of row 0

df[[0]] # A DataFrame of col 0

df[['col_0']] # A DataFrame of col 0


  • If a slicing or a list is passed, a DataFrame object is returned

    • If a slicing is passed, a DataFrame of rows are returned
    • If a list is passed, a DataFrame of cols are turned
  • If a String is passed, a Series of a column is returned

3 months ago

The general format of selecting items within a Dataframe looks like this:

df.loc[row_index, col_index] ## or

df.iloc[row_index, col_index]

When both row_index and col_index are specified, the methods loc and iloc differ in the following ways:

  • You cannot use slicing or a list of integers for the col_index in the loc method.
  • You cannot use labels (i.e. string) for the col_index in the iloc method.

You can use slicing or a list of integers for the row_index in both loc and iloc, but they differ:

df.loc[0:2,] ## returns row 0,1,2 and all columns

df.iloc[0:2,] ## returns row 0,1 and all columns

Returned data type

Depending on what you passed into the methods, you get different data types back:

df.iloc[0, 1] ## returns the data of cell row 0 and col 1

df.iloc[[0], 1] ## returns a Series of row 0 and col 1

df.iloc[0:1, 1] ## returns a Series of row 0 and col 1

df.iloc[0, [1]] ## returns a Series of row 0 and col 1

df.iloc[0, 1:2] ## returns a Series of row 0 and col 1

df.iloc[[0], [1]] ## returns a DataFrame of row 0 and col 1

df.iloc[0:1,1:2]  ## returns a DataFrame of row 0 and col 1

The same rule applies to the loc method. So to sum it up:

  • If both row_index and col_index are a list or slicing, then you will get a DataFrame back
  • If either row_index or col_index is a list or slicing, then you will get a Series back
  • If none of row_index or col_index is a list or slicing, then you will get whatever is stored in that cell back

If only the row_index is specified, the same rule applies, an integer will return a Series object, a list or slicing will return a DataFrame object:

df.iloc[0] ## returns a Series of row 0

df.iloc[[0]] ## returns a DataFrame of row 0

df.iloc[0:1] ## returns a DataFrame of row 0

When to use which method

  • If all of your indexes are integers, slicing or a list of integers, use iloc (that is perceisily what the i stands for)
  • If you want to use Strings for the column names, use loc

The only caveat is when using slicing for the row_index in loc is that 0:2 yeilds 0,1,2, but for the iloc methods, it yeilds 0,1 (which is how slicing works everywhere else in python)


Selecting row 0,1,2 and all columns
df.loc[0:2,] ## note it is 2 here not 3

Selecting row 0 and 2 and all columns
Selecting row 0 and 2, and column 1 and 2
df.loc[[0,2], ['col1', 'col2']] ## has to use column names for col 1 & 2

df.iloc[[0,2], 1:3]
Selecting row 0 as Series
Selecting row 0 as DataFrame
Selecting a cell located at row 0 and column 2
7 months ago

I was trying to do this in Python 3:

t = {'1': 1, '2':2}

#Trying to sort the dictionary t by its value

       key=lambda (k,v): v,  # the left bracket raises syntax error, it works in python 2 but not 3


It turns out that in Python 3, tuple parameter unpacking is not supported anymore.

So if more than one parameter is passed into a lambda function, we have to access it like a list:

lambda parameters: parameters[0], parameters[1], parameters[2]

So for the issues I have, the correct code would be:

       key=lambda k_v: k_v[1],  
10 months ago

When I was trying to use Chinese characters for a chart's title, matplotlib displayed a bunch of squares. Here are the steps to solve it.

Step 1: Find matplotlibrc

I am using virtualenv, so for me the path is:


The key part is /lib/python3.5/site-packages/matplotlib/mpl-data

Step 2: Modify matplotlibrc

Uncomment font.family and font.sans-serif, then add the following fonts before Bitstream Vera Sans:

PingFang SC, Hiragino Sans GB, Microsoft YaHei,

It should look like this:

font.sans-serif     : PingFang SC, Hiragino Sans GB, Microsoft YaHei, Bitstream Vera Sans, Lucida Grande, Verdana, Geneva, Lucid, Arial, Helvetica, Avant Garde, sans-serif

If you have a ipyton notebook open, you have to close and reopen it to make this change effective. Otherwise this is it :D

11 months ago

Install Python3

  1. Install brew: http://brew.sh/
  2. Install Python3 via brew: brew install python3
  3. Vertify Python3 is installed: which python3 (should be located at: /usr/local/Cellar/python3/)

Install virtualenv and virtualenvwrapper

  1. pip3 install virtualenv virtualenvwrapper
  2. create a fold to store all of the virtual environments:
cd ~
mkdir .envs

`3. setup virtualenvwrapper by putting the following into .bash_profile:

export WORKON_HOME=~/.envs # point to the .envs fold created above
export VIRTUALENVWRAPPER_PYTHON=/usr/local/Cellar/python3/3.5.2_1/bin/python3   # make sure this is the python version you want to point to
export VIRTUALENVWRAPPER_VIRTUALENV=/usr/local/bin/virtualenv
source /usr/local/bin/virtualenvwrapper.sh

Then run .bash_profile to reflect the chagnes:

. ~/.bash_profile

Now everythiing should be set to go:

mkvirtualenv env_name


11 months ago

I have actually found a bug in Django:

Here is the issue: You cannot have a cookie which key contains either the character '[' or ']'

I discovered the solution following @Todor's link, then I found out about this SO post. Basically there was a bug in python 2.7.x that does not parse cookies with ']' in the value. The bug was fixed in 2.7.10.

I thought it would be good to just confirm this issue. So I dug through all of the cookies and found one with the following key/value:

key: BDRCVFR[feWj1Vr5u3D]
val: I67x6TjHwwYf0

So I inserted the following cookie locally and submitted to the server:

key: test
val: BDRCVFR[feWj1Vr5u3D]

The login page worked, which means 2.7.10 indeed fixed the bug.

But then I realized that the square brackets are actually in the key name not in the value, so I did the following tests:

key: [
val: I67x6TjHwwYf0


val: I67x6TjHwwYf0

Both cookies break the login process and django displays:

CSRF cookie not set

If a invalid cookie comes before the csrftoken cookie in the Cookie field of the http request header, then Django will ignore everything after the invalid cookie, which makes Django think that the CSRF cookie is not set

I filed a bug report to Django. This bug is fixed in version 1.10 so make sure you update Django when 1.10 is released.

12 months ago

Found a great SO post on this topic:

When the [[Construct]] property for a Function object F is called, the following steps are taken:

  1. Create a new native ECMAScript object.
  2. Set the [[Class]] property of Result(1) to "Object".
  3. Get the value of the prototype property of F.
  4. If Result(3) is an object, set the [[Prototype]] property of Result(1) to Result(3).
  5. If Result(3) is not an object, set the [[Prototype]] property of Result(1) to the originalObject` prototype object as described in
  6. Invoke the [[Call]] property of F, providing Result(1) as the this value and providing the argument list passed into [[Construct]] as the argument values.
  7. If Type(Result(6)) is Object then return Result(6).
  8. Return Result(1).

Let me translate that into code:

function Person() {}
var p1 = new Person();

//step 1

var newObj = {}

//step 2: I don't know what [[Class]] property is and I cannot find any reference so I will skip this step

//step 3:

var functionProto = Person.prototype;

if (typeof functionProto === 'object'){
    //step 4:

    newObj.__proto__ = functionProto;
} else {
    //step 5:

    newObj.__proto = Object.prototype;

//step 6:

Person.call(newObj, arguments);

if (typeof newPerson === 'object'){
    //step 7:

    return newPerson;
} else {
    //step 8:

    return newObj;

You can add an return statement in the constructor to hijack what it returns:

function Person(){
    return {};

var p1 = new Person(); // p1 is just a plain object not an instance of Person

If the returned object is not an object, then step 8 kicks in:

function Person(){
    return 2;

var p1 = new Person(); // p1 is an instance object of Person instead of 2
12 months ago

Originally, I thought there is only one prototype property per function until I read about __proto__. Take a look here:

function Person(){
  this.name = 'John';

Person.prototype // Object{}

                 //    > constructor: function Person()

                 //    > __proto__: Object

Person.__proto__ // function() {}

I was totally confused until I read this stackoverflow post. First of all, this graph helps to clarify things a bit:

The accept answer nails it:

__proto__ is the actual object that is used in the lookup chain to resolve methods, etc. prototype is the object that is used to build __proto__ when you create an object with new:

( new Foo ).__proto__ === Foo.prototype
( new Foo ).prototype === undefined

As one of the users commented:

Ah! So prototype is not available on the instances themselves, but only on the constructor functions.

Here is a table to summarize the idea, considering the code blow:

var obj = {}

function Person(){

var p1 = new Person()
__proto__ prototype
obj Object undefined
Person() Function Object {
  constructor: function Person()
  __proto__: Object
p1 Object {
  constructor: function Person()
  __proto__: Object

Note: obj.__proto__ is the Object.prototype and Person._proto__ is Function.prototype

12 months ago

The value of this depends on how the function is called. There are four ways to call a function:

  • function invocation
  • method invocation
  • constructor invocation
  • apply/call invocation

Function Invocation

function add(){
  console.log(this); // Window object


add(); // function invocation

In this case, this inside the function add is always bound to the global variable of the environment. (It is the Window object in this case because I run the code inside a browser)

A special case of function invocation is closure:

function add(){
  function plus(){
    console.log(this); // Window object

  plus(); // function invocation


add(); // function invocation

It is tempting to think the this value inside the plus() function is tied to add. However, that is not the case as it is explained in this post. plus is still triggered via function invocation, so this still points to the global variable.

Method Invocation

var person = {
    name: 'John',
    sayName: function(){

person.sayName(); // Method invocation, which output "John"

When ever the dot notation is used, a method invocation happens. In this case, the value of this equals to the object before the dot. So in this case, this == person. Because person.name = "John", this.name == "John".

A special case is nested objects:

var foo = {
  bar: {
    sayName: function() {...}

foo.bar.sayName(); // this == bar not foo

whoever is closest to the function call is the value of this.

Constructor Invocation

function Person(){
  this.name = 'John'

console.log((new Person()).name); // Constructor invocation, outputs "John"

In this case, a new object has been created. The keyword this points to the newly created object. The function setName() is equivelent to the following code:

function Person() {
  var this = {}; // this is not valid, just for illustration

  this.__proto__ = Person.prototype;
  this.name = 'John
  return this;

Apply/Call Invocation

By using Apply or Call (or bind), you can specify the value of this:

function setName() {
  this.name = "John";
setName();                // function invocaiton

console.log(this.name);   // John

console.log(window.name); // John

var p1 = {};
setName.apply(p1);    // apply invocation

console.log(p1.name); // John

When setName() is called the first time, name is created on the window object. If we want to attach the name property to a different object, we can use apply or call (or bind).

12 months ago

Found a really good example that explains the differences among bind, call and apply.

Both call and apply attaches this into function and executes the function immediately:

var person = {  
  name: "James Smith",
  hello: function(thing) {
    console.log(this.name + " says hello " + thing);

person.hello.call(person, "world"); // output: "James Smith says hello world"

The difference is that apply can take in an array as the parameters but call has to list them individually:

call(obj, 'arg1', 'arg2', 'arg3');

var args = ['arg1', 'arg2', 'arg3'];
apply(obj, args);

bind only attaches this with a function but id does not execute the function:

var person = {  
  name: "James Smith",
  hello: function(thing) {
    console.log(this.name + " says hello " + thing);

var helloFunc = person.hello.bind(person);
helloFunc("world");  // output: "James Smith says hello world"

You can also pass parameters to bind:

bind(obj, arg1, arg2 ,arg3);

bind is often used to express the intention of "this function is invoked with a particular this".