Interesting Python Standard Library Modules

Python developers use pypi to find packages. You need to search, filter some packages, read reviews from other developers, read the documentation and explore examples if exists. Anyone can build and deploy a package to pypi so you really don’t know if the package you found is working ok.

Python standard library is full of useful modules so before looking outside, it is important to be familiar with it.

Pretty Printer – pprint

If you program print information or you are using the print function for debugging this module will be very helpful. For example, if you have the following dictionary:

dict1={'Avi':[123,('Haifa','Male','Student')],
       'Rina':[456,('Washington','Female','Instructor')],
       'John':[765, ('Paris', 'Male', 'Student')],
        4:'count'}

print(dict1)

the output looks like:

{'Avi': [123, ('Haifa', 'Male', 'Student')], 'Rina': [456, ('Washington', 'Female', 'Instructor')], 'John': [765, ('Paris', 'Male', 'Student')], 4: 'count'}

One long line and not so easy to read. With pprint module:

import pprint

dict1={'Avi':[123,('Haifa','Male','Student')],'Rina':[456,('Washington','Female','Instructor')],'John':[765, ('Paris', 'Male', 'Student')],4:'count'}

pprint.pprint(dict1)

the output:

{4: 'count',
 'Avi': [123, ('Haifa', 'Male', 'Student')],
 'John': [765, ('Paris', 'Male', 'Student')],
 'Rina': [456, ('Washington', 'Female', 'Instructor')]}

It can be used on any python structure, in this case, the dictionary is sorted by key. You can control indentation, width, depth and more using the PrettyPrinter object. See the docs for details

Working with temporary files – tempfile

While working with data, sometimes we need to create a temporary file. The module tempfile helps to create temp files and directories based on the running OS for example:

import tempfile

f = tempfile.mkstemp("_sufix","prefix_")
print (f)

the result is a tuple with the file descriptor as the first element (or other OS file handle) and the full path as the second

(4, '/var/folders/v3/xp0wxj9d11b7qz001p81xxr00000gn/T/prefix_n1mavo16_sufix')

the next module is used to access the file

Operating System Specific – os

os module is very useful, it contains many os specific routines for working with processes, files, IPC and other OS objects.

If for example, you want to use the tempfile generated file descriptor for read and write you can use os module:

import tempfile
import os

f = tempfile.mkstemp("_sufix","prefix_")
fd = f[0]

os.write(fd,bytes("hello","utf8"))
os.lseek(fd,0,0)
s=os.read(fd,5);

print (s)

Tiny shell with shutil

If you need to manage files – copy, delete, move and more, this module offers some high-level operations that can help

For example to copy a folder recursively

import shutil

shutil.copytree("/Users/liran/py1/","/Users/liran/py2")

To delete a folder recursively :

shutil.rmtree("/Users/liran/py2")

You can use this module with help of os module to create a tiny shell using python

Working with signals

Signals is a good way for communication between processes. One application can register a signal handler and the other can send it. While a signal received in the process, one of its threads will stop and handle that signal. You can also use signals to output current status on a long processing operation.

For example:

import signal
import time
num = 0;

def handler(sig, frame):
    global num
    print("CTRL+C Num=",num)


signal.signal(signal.SIGINT, handler)

while 1:
     num+=1;
     time.sleep(1)

In this example, the script registers a signal handler for CTRL+C, then it runs an infinite loop and increments a counter every second. To print the current counter value, press CTRL+C

You can register a user define signal using SIGUSR1, SIGUSR2

signal.signal(signal.SIGUSR1, handler)

to send a signal from other process use os.kill

os.kill(pid,signal.SIGUSR1)

The best way to create this is to create parent-child processes using fork. See details here

Handling date and time

Date and time manipulation is a common task in many programs. There are three related modules in the Standard Library which help

datetime
calendar
time

datetime module includes datetime objects for extracting date and time in a different format, timedelta for diff calculations, time zones, formatting and more

import datetime
import time

s=datetime.date.fromtimestamp(time.time())

print(s) # prints 2018-01-03

calendar module includes some useful iterators

For example, iterate over all days in Feb 2016:

import calendar

c=calendar.Calendar()
for n in c.itermonthdays(2016,2):
    print(n)

time module supports mktime(), localtime(), gmtime(), strftime(), sleep() and more

The platform module

used to identify the platform we are using:

import platform

print("Platform:",platform.platform())
print("Compiler:",platform.python_compiler())
print("Python  :",platform.python_version())
print("System :",platform.system())
print("Release :",platform.release())
print("System :",platform.processor())

output:

Platform: Darwin-17.3.0-x86_64-i386-64bit
Compiler: GCC 4.2.1 (Apple Inc. build 5666) (dot 3)
Python  : 3.6.2
System : Darwin
Release : 17.3.0
System : i386

Network Programming with socket module

The socket module is a network programming interface. The API also contains a set of Python-specific extensions.

Code written using this module is portable. You can communicate easily between python and other languages using socket. For secure connection use ssl module

Server example:

import socket

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
sock.bind(("", 2000))
sock.listen(5)

while True:
    sktup = sock.accept()

# usually we open a new thread or process to handle the client
    bMessage = sktup[0].recv(1024, 0)
    print("Msg Recieved: ", bMessage.decode())

Client side:

sock  = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
sock.connect(('localhost',2000))

sock.send('hello')

There are many more important modules in Python standard library – see the docs