8 Python Modules For Files Handling

Python has input/output features which are very easy to use. Files are accessed through file objects and you can open, read, write and close files using a very simple and easy to use functions from the standard library. To manage files and deal with special file formats (xml, json..) python provides special packages that make the developer life ever easier.

Filtering Files – inputfile module

If you write scripts for automation , you are probably familiar with commands like grep, head, tail and more. These commands gets a file (or more) as input and filter its content to the output. If you want to write a similar task in python, the inputfile module is very helpful

run this script with a list of filenames as parameters or patterns:

The script loops over all the lines in all txt files, printing the file name, line content and line number. To write a filter program, just add  conditions:

Using the script: (searching for lines with string hello)

 

Serialization with pickle

Pickle module converts Python objects into a stream of bytes usually written to a file, or across a network. To use pickle, open a binary file and dump/load your objects.

and load:

Types supported by pickle:

  • All primitive types
  • strings, bytearrays
  • collections with pickable objects (set, list, tuple, dictionary)

custom types:

Note the pickle protocol attribute (see docs)

 

File Compression with bz2, gzip

With bz2, gzip modules , you can create an archived compressed file

 

Pickling on a large scale with shelve

if you are going to use pickle on a large scale, the shelve module uses a database to store pickle objects by a string key:

 

Configuration Files – INI files

Windows uses ini files for settings and configurations. The configparser module helps you writing and parsing those files:

the generated file:

 

JSON Files

JSON files are very useful for saving data offline, saving configuration and more. In the following example we use the json module to dump a dictionary to a JSON file

the generated file: (note the conversions)

 

XML Files

You can find many modules and packages for handling and parsing XML files. One simple module is minidom:

Parsing XML string:

Parsing XML file:

And navigating in the DOM object:

And many more

 

CSV Files

CSV files are used to store tables. All database systems can import and export data in CSV format. Use the csv module to handle CSV files:

The DictReader converts each row to a dictionary

 

 

 

 

 

 

 

Tagged

3 thoughts on “8 Python Modules For Files Handling

  1. Yaml is another great choice for config files and it works well in many languages.

  2. Thank you very much for this great tutorial.
    It’s helpful to me.

  3. For XML, I would bring up the xml.etree.ElementTree as well. The DOM model is cumbersome and ElementTree makes a lot of things much easier.

Leave a Reply

Your email address will not be published. Required fields are marked *