GUIDE: MERGE & DOWNSAMPLE YOUR CAN BUS DATA
CANvas provides a number of great options for customizing your conversion.
Click ‘Options’ in the ‘Output’ tab to specify basic things like save path or real-time clock output format.
But there are far more powerful options available as well:
1# CAN BUS LOG FILE MERGING
In the Options, you can enable Log File Merging.
This is a really cool feature:
Imagine you set up your logger to split files in 100 MB chunks - and that you log data from e.g. 10 trucks.
Over a month, this can lead to tons of individual log files!
How do you manage this?
With CANvas, you just load all your log files and enable Log File Merging to compile it all into one big output file.
To distinguish e.g. LoggerIDs (i.e. what data from what truck), simply include the ID as an output column via the options.
With e.g. Excel pivots it’s then easy to filter your data for fast analysis!
But what if this creates a massive file that is way too heavy for e.g. Excel?
In that case, just enable ‘Dynamic File Splitting’. This allows you to specify a max #rows in the final output file, automatically breaking the merged file into chunks sized as you see fit.
E.g. if you want to limit a file to a max of 20,000 rows, simply enter this in the 'Rows limit' - and your merged "mega file" will be output in blocks of 20,000 rows each.
2# CAN BUS LOG FILE DOWNSAMPLING
Downsampling is another great feature:
For some analyses you need the high frequency - but for others you just want to look at e.g. a daily or hourly trend perspective.
Example: Imagine you’ve set up your CL3000 to store high frequency data from a tractor over several months and you've now got 16 GB of data.
For your analysis you may just need the long-term trend data - and 16 GB of high frequency data can be impractical.
By enabling Downsampling in your data conversion, you can aggregate your CAN data to a specified frequency (in miliseconds).
This lets you turn super frequent data into one observation per Signal per interval (e.g. 1 observation per hour per Signal).
But wait, there’s more!
The average may not be enough for your analysis.
Maybe you also need the standard deviation, min, max or observation count per interval.
No problem - you can just add this information as separate columns in your output file!
Together, Log File Merging and Downsampling provide a powerful combination:
They allow you in a few clicks to go from tons of log files and data - to a single, optimized output file with just the data you need.
Sliced and diced, ready to provide you with insight!
3# Excel & Pivot Friendly Timestamp & Header Format (NEW)
We've recently made it a lot easier to work with the output files in various 3rd party programs - incl. e.g. Excel and Pivot tables.
In particular, we've added a "timestamp split" feature that allows you to split each part of the timestamp in separate columns:
I.e. one column for each of the following: YYYY, MM, DD, HH, MM, SS, kkk.
This makes the data a lot easier to utilize in Excel Pivot tables and many other programs.
Further, by excluding the info header from the output file, you'll be able to load the data directly into an Excel PowerPivot.
This lets you manipulate data sets far larger than what Excel normally handles (we've tested with 20 GB+ files).
4# Customize Output File Column Structure (NEW)
Often you'll want to exclude certain columns from your final output to compress or simplify.
Now we've made this really easy with our customized header selection!
Simply select the columns in Output Settings you wish to exclude and move them - and that's it:
5# Create Accumulating Output Datasets (NEW)
If you're constantly getting new, small log files (e.g. from a fleet of FTP pushing WiFi CL3000 units), you'll want an easy way to combine them into a dataset as they arrive on your PC or cloud server.
Luckily, we've now got you covered:
In CANvas you can now specify to save an output file onto an existing output file, which will "append" it to the end of the file.
In this way, you can e.g. enable CANvas "auto conversion" and continuously expand your output database.
In turn, this can then be referenced by e.g. SQL or an Excel PowerPivot.
This is ideal e.g. if you're doing frequent updates of dashboards and always need the freshest data together with historical data.
Note that the output of course has to be structured consistently when using this method.
For all the above features, you can save your option settings and load them next time you work.
Further, you can also use Log File Merging if you use CANvas to re-transform your raw CLX000 data files into e.g. Vector's CANalyzer or PEAK's PCAN-View formats.
We hope you like these great options in CANvas - if you've got ideas for new features, please let us know!
To learn more about the CLX000 series, click here.