Python CAN Bus API - Automate CAN/LIN Processing [Open Source]

Python-CAN API MDF MF4 Data Lake

Need to automate your CAN bus data processing via Python?

Vehicles and machines generate tons of CAN bus data - and GUI software is often not enough to extract the insights you need.

Here, Python scripts can help accelerate your data processing.

Here, you'll learn how to process CAN/LIN data via Python. In particular, we outline how to record/decode CAN data, key features of our Python API, use cases and case studies.

Tip: Try our Python API with J1939 data & DBC samples here.

How to record & decode CAN bus data?

The Controller Area Network (CAN bus) is used in most automotives today (cars, trucks, tractors etc), as well as maritime vessels and machines.

To record CAN data, you can use a CAN bus data logger. For example, the CANedge1 lets you record raw CAN data to an SD card. Further, with the CANedge2, your data can be auto-pushed via WiFi to your own server.

The devices record CAN frames, which need to be decoded to physical values for analysis - typically via a DBC file.

Once the CAN bus data is DBC decoded, you can perform e.g. graphical visualization and statistical analysis.

In many cases, you may simply want to open a specific log file and analyze the data via a software GUI tool. For example, you might know that an issue occurred in a truck and you wish to review data related to the event manually.

For such use cases, various GUI software tools can be used. For the CANedge, you can either use the free asammdf GUI or load data into your favorite software tools:

asammdf gui free CAN bus software tool open source

asammdf GUI

This is a general-purpose GUI software that lets you load MF4 log files, DBC decode them and plot the physical values.

asammdf intro
ASAM MDF4 Measurement Data Format Converters ASC TRC CSV

MF4 converters

If you prefer using software like CANalyzer, PCAN-Explorer etc. you can convert your MF4 files to the relevant format.

mf4 converters intro
CAN bus data logging and DBC decoding

Why use Python for CAN bus data processing?

GUI tools are great for small scale and ad hoc data analysis. However, cases exist where GUIs become impractical:

CAN bus big data script api processing

Too much CAN bus data

If you "record everything" from e.g. a truck, you're looking at 1 GB+ per day. If you wish to analyze 3 months of data for 10 trucks, that's almost 1 TB - not suited for GUI processing.

Automated vehicle data processing API

Too much manual work

If you are performing the same steps many times to e.g. extract a specific report/result from a GUI tool, you'll typically benefit from automating the process via scripts.

Automotive data OBD2 J1939 custom API integration

Integration required

If you need to transform/process your data to enable advanced analyses - or integrate your data with a custom system/database - scripting is typically required.

Python CAN bus fast processing predictive maintenance

Need for speed

If you need to extract results from your CAN data in "near real-time", you cannot rely on manual processes or GUIs - in particular if your use case needs to scale

You can use various scripting languages to process your CAN bus data. Python is popular as it enables efficient processing of data at scale, while also providing many plug & play packages suited for CAN data processing (e.g. our Python API modules).

Other popular options within the automotive sector include MATLAB. In particular, MATLAB's Vehicle Network Toolbox plugin lets you load MF4 log files from the CANedge natively for quick analysis, making it a powerful alternative if your team prefers to work within MATLAB. Further, MATLAB natively has great support for Parquet data lakes, meaning it's great for working with data that has already been DBC decoded via our MF4 decoders. On the other hand, Python has the benefit of being 100% free.

If you're new to Python, we recommend that you go through a few 'get started' tutorials. While not strictly necessary, it's a good investment to ensure you have the basics in place - and it'll definitely save you time in the medium/long run.

You can find several tutorials online, but we in particular recommend the Python tutorial videos by Mosh:

Once you've gone through the basics, you can try out some of our Python CAN API examples.

CAN bus Python API - key features

We offer multiple options for processing your CAN/LIN data in Python:

MF4 support load CAN data LOAD RAW DATA

Load log files directly into python-can for flexible processing ...


... or create Parquet data lakes for large scale analysis via Python

DBC decode CAN data DBC DECODE

Use DBC file(s) to efficiently decode raw CAN/LIN data to physical values

Open Source Windows Linux Python CAN bus API 100% FREE

Modules are 100% free and supported on both Windows, Linux

Python CAN bus API examples API EXAMPLES

Multiple simple-to-use code snippets are available to get you started quickly

Light weight Python CAN bus API LIGHT WEIGHT

Modules are efficient, yet extremely light with minimal dependencies

Load your log files directly ...

The CANedge log file format can be loaded directly in the popular python-can library. This lets you directly analyse raw CAN/LIN frames, replay log files onto USB-to-CAN interfaces or e.g. selectively DBC decode data into physical values. It is extremely flexible and ideal for specialized use cases.

Python API python-can iterator

Python API Parquet data lake

... or start with a Parquet data lake

As an alternative, you can use our MF4 decoders to DBC decode your log files into a Parquet data lake. By pre-processing the data you dramatically reduce the Python coding required in order to extract your insights. In particular, this approach is useful if your goal is to perform large-scale data analysis, where Python offers great Parquet support. Learn more about creating your own Parquet data lake in the MF4 decoder intro.

The above methods are currently the recommended and maintained methods for analysing your CANedge data in Python. In particular, our integration with python-can now replaces our previous Python API, which consisted of below sub modules. You can still continue using this legacy API, but it will not be further developed:

Our legacy Python CAN API is comprised of three separate Python modules - learn more below:

List MF4 log files CAN data API canedge_browser

This module provides you with a list of CANedge log files based on a start/stop timestamp. The files can be listed from local disk or from an S3 server

MF4 support load CAN data mdf_iter

This lets you load MF4 logs (or MFC/MFE/MFM) with CAN/LIN data into a pandas dataframe / iterator for processing

DBC decode CAN data can_decoder

This module lets you decode raw CAN/LIN data using DBC files (or manually entered decoding rules) - incl. support for J1939, NMEA 2000, OBD2

In practice, most users deploy a combination of the above modules - and we generally refer to this combination as 'our Python API'. You can of course also use each of the modules in isolation for more specialized use cases. Today, however, we recommend to use our python-can integration or the MF4 decoders.

The above options are great for most CAN related use cases involving data from the CANedge. But if your goal is specifically to output modified MF4 files, then the asammdf Python API may be relevant.

The asammdf Python API lets you perform all of the steps you can do through the asammdf GUI, but through Python scripts. This means that you can load log files, DBC decode them and get the output into pandas dataframes - similar to our Python API tools.

For many use cases, you can thus use the asammdf API, though there are some distinct advantages to our API if you're working with data from the CANedge:

  • Listing data: The asammdf API does not provide any methods for listing log files in a structured manner. In contrast, our Python API is designed to enable time-based listing of log files across both local disk and S3 (for use with the CANedge2)
  • Support for MF4/MFC/MFE/MFM: The asammdf API lets you natively load CANedge log files in the MF4 format. However, if you wish to compress and/or encrypt your log files, you cannot natively load these log files via the asammdf API, but instead you'll need to first use the 'mdf2finalized' MF4 converter to get the files into the MF4 format
  • Dependencies: The asammdf Python API enables a lot of features not included in our Python API. This also means that the tool is far heavier in terms of dependenices.
  • AWS Lambda: If you wish to set up automated processing of incoming log files from the CANedge2, we provide ready-to-use AWS Lambda 'ARN layers' for enabling easy automation using our Python API

On the other hand, the asammdf API is better suited when your use case involves loading MF4 log files with CAN bus data and exporting the data into e.g. DBC decoded and/or concatenated MF4 log files. Similarly, the asammdf API also makes it easy to export data into MATLAB compatible formats. If in doubt which API to use, contact us.

Below we list a couple of other Python CAN tools that you may find useful. Note that most of the tools listed are intended for use with CAN bus interfaces (i.e. real-time streaming of data via USB), in contrast to our Python API and the asammdf API (intended for processing log files):

  • canmatrix: This module lets you load CAN databases (e.g. DBC files) for use in scripts or for exporting to other formats. This module is used in our API and the asammdf API
  • cantools: Like canmatrix, cantools enables loading of DBC files (and e.g. KCD, SYM, ARXML), encoding/decoding of CAN frames and basic graphical plots. It can e.g. be used with python-can to provide DBC decoding of raw CAN bus data streams
  • python-can: Supports many CAN bus interfaces (incl. SocketCAN), enabling real-time streaming/transmission of raw CAN frames. Useful for diagnostics, testing etc. The module also supports various popular CAN bus file formats like Vector ASC, BLF, TRC

For more Python CAN tools, see this overview.

Use case examples: CAN bus + Python API

Visualize your CAN data via telematics dashboards

You can use Python within e.g. AWS Lambda functions to perform DBC decoding via the CANedge Python API or using our MF4 decoders - enabling easy integration with e.g. Grafana dashboards.

Python CAN bus telematics dashboard Grafana

Handle ISO TP multiframe across J1939, NMEA 2000 and UDS

Our API examples provide basic Python classes for 'transport protocol' handling across popular CAN protocols - incl. combining frame sequences and decoding the data to physical values.

ISO TP Transport Protocol J1939 Multiframe Decoding BAM

Integrate your CAN bus data into any system/database

Using the Python API, you can easily extract relevant CAN signals into pandas dataframes - making it easy to integrate with BI systems, databases etc. via standard Python APIs.

CAN bus data integration custom system

Perform advanced statistical analysis & diagnostics

Once you have your DBC decoded data available in a pandas dataframe, you can utilize a vast range of open source Python packages to help perform advanced statistical analysis.

CAN bus statistical analysis Python script

Handle huge amounts of CAN data - e.g. for ML

Need to handle gigabytes or terabytes of CAN data - e.g. for machine learning? Python is ideal for this - in particular if you pre-process your data into a Parquet data lake.

Python Machine Learning Vehicle Data Telematics AI

Set up predictive maintenance to reduce breakdowns

Reduce breakdowns via simple predictive models (e.g. based on threshold alerts) and use event triggers (e.g. AWS Lambda functions) to auto-trigger the processing.

Vehicle Predictive Maintenance Python

Reduce time spent on data processing through automation

Automate repetitive steps in your CAN bus analyses - for example creating PDF reports, extracting statistical results, performing CAN diagnostics or identifying events of interest.

Process automation CAN bus data analysis

Export data to custom formats for use in 3rd party tools

Need to convert CAN log files into other formats? If your format is not supported by our MF4 converters or decoders, Python can be used to load log files and export them to custom formats

MF4 convert via Python

Want to try working with our Python API in practice?

Check out our Python script examples - with free sample data and DBC files!

Case studies: Python API + CAN bus

Many of our end users deploy Python scripts for their CAN processing - see e.g. below examples from our 50+ case studies

Case study: CAN telematics

HAVELSAN logo case study

Learn how HAVELSAN use the CANedge2 to collect CAN bus data from their UGVs via SD/WiFi - and visualize it via dashboards.

"The CANedge2 is an autonomous way to collect, transfer & analyse data. Set it up - and watch the data from your office."

full case study

CAN bus WiFi Telematics & Dashboards

Predictive Maintenance Heavy Duty Vehicle OEM

Case study: Predict issues

Learn how one of our larger OEM customers use the CANedge2 to collect CAN bus data from heavy machinery in the field - and use it to perform Python based predictive maintenance.

"Both the hardware and support from CSS Electronics have been outstanding!"

full case study

Case study: Dash dashboards

Nuvera Fuel Cells Logo

Learn how Nuvera Fuel Cells uses the CANedge2 and Python to perform automated diagnostics and reporting - with custom dashboard visualization.

"The CANedge2 is a very affordable and reliable solution for acquiring, elaborating and visualizing CAN bus data"

full case study

CAN bus Python API dashboards reporting

Kverneland Case Study

Case study: ISO TP dashboards

Kverneland logo case study

Kverneland uses our Python API to process 2 x J1939 from the CANedge2, uploaded to their Azure cloud. In particular, they use our 'transport protocol' examples to handle J1939 ISO TP messages.

"The CANedge2 is a high quality product, but the support from CSS makes it an absolutely unbeatable solution!"

full case study

Ready to process your CAN data via Python?

Get your CANedge today!

Recommended for you