Python is the ideal language to do data processing, and it’s one of the reasons it’s used so much for data science. Python supports data formats like CSV, JSON, and XML out of the box. However, there are also many external libraries on PyPI to aid you with processing data, like PyYAML for YAML files and pandas for spreadsheets.
In this section, you’ll learn how to read, change and write the data formats you’ll run into most. Each article starts with the basics and has examples you can run yourself.
Python data processing topics
- Processing plain text files: open, read, write and append to files, and handle the errors that come with it. If you just want the quickest way to read a file, see 4 ways to read a text file.
- Working with JSON: parse JSON strings and files with the built-in
jsonmodule, write JSON, and pretty-print it on the command line. - Querying JSON with JMESPath: pull exactly the values you need out of a large JSON document with a short query instead of nested loops.
- Reading and writing YAML in Python: load and dump YAML with PyYAML, handle files with multiple documents, and convert YAML to JSON.
- Processing CSV data: read and write CSV with the built-in
csvmodule, and when to switch to NumPy or pandas instead. - Reading and Writing Excel files with Python: create, read, update and merge Excel sheets with pandas.
Related topics
Prefer the command line? jq lets you filter and highlight JSON right in your terminal. When your data outgrows a few files, the data science section takes it further with NumPy, pandas and Jupyter notebooks.