Python: test if a dictionary contains a specific key

def contains(dict, key):
    return key in dict

dict={}
dict['a']=1
dict['b']=2
dict['c']=3

print(contains(dict, 'a')) # True
print(contains(dict, 'd')) # False

Python: delete a item from a list

• remove the first matching element:

  if e in list:
      list.remove(e)

• remove all occurrences:

  while e in list:
      list.remove(e)

• EAFP style: remove the first occurrence:

  try:
      list.remove(e)
  except ValueError:
      # not in the list
      pass
  

• EAFP style: remove all occurrences:

  while True:
      try:
          list.remove(e)
      except ValueError:
          break
  

EAFP

Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is characterized by the presence of many try and except statements. The technique contrasts with the LBYL style common to many other languages such as C.

Python: del or assign None?

>>> x=1
>>> print x
1
>>> del x
>>> print x
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'x' is not defined

>>> x=1
>>> print x
1
>>> x=None
>>> print x
None

see also

• stackoverflow

synchronized method in python

Java code

class AClass {
    private int count =0;

    public synchronized inc(){
        count++;
    }

    public synchronized dec(){
        count--;
    }
}

Python code

import threading

class AClass(object):
    def __init__(self):
        self.__count = 0;
        self.__lock = threading.RLock()

    def inc(self):
        with self.__lock:
            self.__count += 1

    def dec(self):
        with self.__lock:
            self.__count -= 1

see also

• Python threads synchronization: Locks, RLocks, Semaphores, Conditions, Events and Queues

synchronized static method in python

Java code

class AClass {
    private static int _count = 0;

    public static synchronized inc(){
        _count ++;
    }
    
    public static synchronized dec(){
        _count --;
    }
}

Python code

import threading

class AClass(object):
    __count = 0
    __lock = threading.RLock()

    @classmethod
    def inc(cls):
        with cls.__lock:
            cls.__count += 1

    @classmethod
    def dec(cls):
        with cls.__lock:
            cls.__count -= 1

Python: check if file or directory exists

• os.path.exists to check if file or directory exists.

  import os
  os.path.exists('/tmp/1.txt')
  os.path.exists('/tmp')
  

• os.path.isfile to check if file exists.

  import os
  os.path.isfile('/tmp/1.txt')
  

Python: Get file absolute path

Use os.path.abspath function:

import os
os.path.abspath('abc/abc.txt')

Python: get the index of a char in a string

find and index method are available. The difference is that find returns -1 when what you're looking for isn't found, index throws an exception.

s='abc'
print(s.index('b')) # 1
print(s.find('b'))  # 1
print(s.find('d'))  # -1
print(s.index('d')) # exception

Python: string types in 2.x and 3.x

• In Python 2.x, basestring is the base class for both str and unicode, while types.StringType is str. If you want to check if something is a string, use basestring. If you want to check if something is a ascii (bytestring), use str and forget about types.
• Since Python 3.x, types not longer has StringType; str is always unicode. basestring no longer exists.

python: return inside with block

if returns value inside with statement, will the file be closed eventually?
The answer is YES

with open('myfile.txt') as f:
    return [line for line in f if len(line)>80]

python: iterables and iterators

A class with __iter__() method or __getitem__() method is iterable. The __iter__() method returns an iterator. A iterator object has a next() method (or __next__() method in python 3.0).

class Iterator(object):
    def next(self):
        pass

class Iterable(object):
    def __iter__(self):
        return Iterator()

see also

• Build a Basic Python Iterator
• What makes something iterable in python
• Understanding Python's iterator, iterable, and iteration protocols — what exactly are they?

python: len() function and __len__() method

len is a function to get the length of a collection. It works by calling an object's __len__ method.

class AClass(object):
    def __init__(self):
        self._list = [1,2,3]

    def __len__(self):
        return len(self._list)

a=AClass()
print(a.__len__()) # 3
print(len(a))      # 3

python: call another static method

class AClass(object):
    def __init__(self):
        pass

    @staticmethod
    def a():
        AClass.b()

    @staticmethod
    def b():
        pass

see also

• Python static methods - how to call a method from another method

Python: int to hex string

print(hex(255))

Result:

'0xff'

hex(255)[2:]

Result:

'ff'

Python: generates crc32 and adler32 checksum for big files

import zlib
import sys
import urllib2

def __zlib_csum(url, func):
    if isinstance(url, basestring if sys.version_info[0] < 3 else str):
        url = urllib2.Request(url)
    f = urllib2.urlopen(url)
    csum = None
    try:
        chunk = f.read(1024)
        if len(chunk)>0:
            csum = func(chunk)
            while True:
                chunk = f.read(1024)
                if len(chunk)>0:
                    csum = func(chunk, csum)
                else:
                    break
    finally:
        f.close()
    if csum is not None:
        csum = csum & 0xffffffff
    return csum
    

def crc32(url):
    return __zlib_csum(url, zlib.crc32)

def adler32(url):
    return __zlib_csum(url, zlib.adler32)

if __name__ == '__main__':
    print(hex(crc32('file:/tmp/111.zip')))
    print(hex(adler32('file:/tmp/111.zip')))

Python: read and write binary files

def copy(if_path, of_path):
    with open(if_path, 'rb') as in_file, open(of_path, 'wb') as out_file:
        while True:
            chunk = in_file.read(1024)
            if chunk:
                out_file.write(chunk)
                out_file.flush()
            else:
                break

Python: mutable byte array as buffer

bytearray function can be used to make a mutable byte array as buffer:

import struct
buffer = bytearray(8)
buffer[0]=1
buffer[1]=1
struct.pack_into('>ih', buffer, 2, 100, 200)
# to convert the array to str
print(bytes(buffer))

---

see also

• struct.unpack_into

python: join lists

list1 = ['a', 'b']
list2 = ['c', 'd']

list3 = list1 + list2

Python: encode integers to byte string

import struct
def int_to_bytes(value, size, big_endian=False):
    bs = []
    for i in range(size):
        offset = i * 8
        b = chr((value & (0xff << offset)) >> offset)
        if big_endian:
            bs.insert(0, b)
        else:
            bs.append(b)
    return b''.join(bs)

def int32_to_bytes(value, big_endian=False):
    return int_to_bytes(value, 4, big_endian)

def int64_to_bytes(value, big_endian=False):
    return int_to_bytes(value, 8, big_endian)

if __name__ == '__main__':
    # the two commands below do the same
    print(repr(int64_to_bytes(65536, big_endian=True)))
    print(repr(struct.pack('>Q', 65536)))

---

see also

• Python: struct.pack

python: print a string as hex bytes

':'.join(x.encode('hex') for x in 'Hello World!')

---

Executes in python CLI:

>>> ':'.join(x.encode('hex') for x in 'Hello World!')
'48:65:6c:6c:6f:20:57:6f:72:6c:64:21'

join binary strings in Python

b''.join(['abc', '\x01\x02\x03', 'efg'])

---

Execute it in python cli:

>>> b''.join(['abc', '\x01\x02\x03', 'efg'])
'abc\x01\x02\x03efg'

Python: get the length of a utf-8 encoded string

len(s.encode('utf-8'))

Enum in python

• Before Python 3.4 (PEP 435),

  class Animal:
      DOG = 1
      CAT = 2
  
  x = Animal.DOG
  

  or

  class Animal:
      DOG, CAT = range(2)
  
  x = Animal.DOG
  

• Since Python 3.4 (PEP 435),

  class Animal(Enum):
      DOG = 1
      CAT = 2
  
  x = Animal.DOG
  

  or equivalently:

  from enum import Enum
  
  Animal = Enum('Animal', 'DOG CAT')
  
  x = Animal.DOG
  

python: int and long are unified

Since Python 2.4, int and long are unified. Furthermore, from Python 3, int is the only type for integer, which has the capacity of long in Python 2.

sys.maxint returns the maximum integer number that Python can hold.

see also

• How does Python manage int and long?

python: parse boolean from string

def parse_bool(s):
    return s.lower() in ("yes", "true", "T", "1")

NOTE:

bool('foo') # True
bool('') # False

You can not use the above function to parse boolean from string.

python: check if a variable is a integer

if isinstance(var, int):
    print("It is integer")

On python 2:

if is instance(var, (int, long)):
    print("It is integer")

python: check if a variable is a string

import sys

if isinstance(var, basestring if sys.version_info[0]<3 else str):
    print("it is string") # var is string

see also

• stackoverflow

python: conditional expression

x = true_value if condition else false_value

if isinstance(s, basestring if sys.version_info[0]<3 else str):
    print('It is a str.')

see also

• 1 PEP 308: Conditional Expressions

python: dictionary comprehension

dict = { 'a':1, 'b':2 }
dict = { k: str(dict[k]) for k in dict.keys() }
print(dict)

• http://stackoverflow.com/questions/14507591/python-dictionary-comprehension

main function in python module

class A:
    def __init__(self):
        pass


def main():
    print("Hello World")


if __name__ == '__main__':
    main()

see also

• stackoverflow

Google coding style guide for Python

Google code style for python.

Python doc generator: sphinx docstring examples

• https://pythonhosted.org/an_example_pypi_project/sphinx.html
• https://pythonhosted.org/an_example_pypi_project/sphinx.html#full-code-example
• https://pythonhosted.org/an_example_pypi_project/pkgcode.html

Python: Code completion in pydev

Code Completion

Type hinting with docstrings/comments

New on PyDev 2.8.0

It's possible to provide hints for code-completion with docstrings by commenting types with the Sphinx/Epydoc format.
Below are some examples of how to provide type-hints.

Return type with Sphinx

class MyClass:

    def method(self):
        ':rtype unittest.TestCase'

Parameter type with Sphinx

class MyClass:

    def method(self, a):
        ':type a: TestCase'
        #Note that just the class name is accepted, but in this case,
        #it'll search for a TestCase class in the whole workspace

Parameter type with Sphinx inline

class MyClass:

    def method(self, a):
        ':param TestCase a:'

Local variable with Sphinx

class MyClass:

    def method(self, lst):
        #Can be on the same line
        for a in lst: #: :type a: GUITest
            a.;

Local variable with Sphinx

class MyClass:

    def method(self, lst):
        #Or on the line before
        #: :type a: GUITest
        for a in lst:
            a.;

Local variable with Sphinx

class MyClass:

    def method(self, lst):
        #If commented as a docstring must be on the
        #line after
        for a in lst:
            ': :type a: GUITest'
            a.;

Return type with Epydoc

class MyClass:

    def method(self):
        '@rtype unittest.TestCase'

Parameter type with Epydoc

class MyClass:

    def method(self, a):
        '@type a: TestCase'
        #Note that just the class name is accepted, but in this case,
        #it'll search for a TestCase class in the whole workspace

Python: equivalent to Java toString()

In Java, overrides toString() method in the your class can change the textual representation of the object. The equivalent in Python is __str__ and __repr__. The important differences are:

• the goal of __repr__ is to be unambiguous
• the goal of __str__ is to be human-readable.
• str(object) calls the object's __str__ method
• repr(object) calls the object's __repr__ method
• by default, if __repr__ is defined and __str__ is not, calls to __str__ will be redirected to __repr__
• __repr__: representation of python object usually eval will convert it back to that object
• __str__: is whatever you think is that object in text form
• str(list) calls the list elements' _repr_ methods. Therefore, to make a list of objects human-readable, you need to override __repr__, or use list comprehension like:

  [str(elem) for elem in list]

  ..

see also

• Stackoverflow discussions

Python: module file name conflicts with the package name

I have a python module file named xml.py, in the module

import xml.etree.ElementTree as ET
... ...

When executing the file, it raises the following error:

ImportError: No module named etree.ElementTree

According the stackoverflow post, it was caused by the moudle file name xml.py, rename the xml.py to something else resolved the issue.

Python: ElementTree XML element to string

import xml.etree.ElementTree as ET

# element is an xml.etree.ElementTree.Element object
xmlstr = ET.tostring(element, encoding='utf8', method='xml')

See also

• Python ElementTree API

Python: parse xml using ElementTree

• parse from a file:

  import xml.etree.ElementTree as ET
  tree = ET.parse('/home/wilson/my.xml')
  root = tree.getroot()
  

• parse from a string:

  import xml.etree.ElementTree as ET
  root = ET.fromstring('1')
  

see also

• Python ElementTree API

Python: class method vs static method

class A:
    def object_m(self,x):
        print "executing object_m(%s,%s)"%(self,x)

    @classmethod
    def class_m(cls,x):
        print "executing class_m(%s,%s)"%(cls,x)

    @staticmethod
    def static_m(x):
        print "executing static_m(%s)"%x    

According to stackoverflow:

A staticmethod is a method that knows nothing about the class or instance it was called on. It just gets the arguments that were passed, no implicit first argument. It is basically useless in Python -- you can just use a module function instead of a staticmethod.

A classmethod, on the other hand, is a method that gets passed the class it was called on, or the class of the instance it was called on, as first argument. This is useful when you want the method to be a factory for the class: since it gets the actual class it was called on as first argument, you can always instantiate the right class, even when subclasses are involved.

Python: check argument type

• for objects:

  def f(arg):
      if isinstance(arg, ClassA):
          print('A')
      elif isintance(arg, ClassB):
          print('B')
      elif issubclass(arg, ClassC):
          print('subclass of C')
      else:
          print("D")
  

• for built-in types:

  def f(arg):
      if type(arg) is str:
          print "str"
      elif type(arg) is int:
          print "int"
      elif type(arg) is dict:
          print "dict"
      else:
          print "unsupported"
  

Python: get the sub-elements of a xml element

According to Python XML ElementTree API,

• get all sub-elements(descendants):

  all_descendants = list(element.iter())

• get direct sub-elements(children):

  children = list(element)

Python: check if a list is empty

    if not a:
        print('empty')

Python: iterate over dictionary

for name in dict.keys():
    print(name)
    print(dict[name])

values = [ dict[name] for name in dict.keys() ]

Python: list comprehension

• To transform a list to another:

  list2 = [ unicode(e.strip()) for e in list1 ]
  

• To transform a list to another conditionally:

  list2 = [ unicode(e.strip()) if e is not None else '' for e in list1 ]