Grafana-Athena: 95% Lower Cost CAN Dashboards [News]
news / 2024-01
Grafana-Athena is our new CANedge dashboard integration at 95% lower cost: Lambda functions DBC decode uploaded log files into an S3 Parquet data lake, which Grafana queries via Amazon Athena for scalable CAN/LIN dashboards. Plus: A recap of our 9 big upgrades of 2023.
Grafana-Athena: New CANedge dashboard integration
Want to visualize your CANedge CAN/LIN data in Grafana dashboards?
With Grafana-Athena, this is now much easier, more scalable - and 95% lower cost.
Below is a typical Grafana-Athena setup:
1) A CANedge uploads raw CAN/LIN data to an AWS S3 bucket
2) When a file is uploaded it triggers a Lambda function
3) The Lambda function DBC decodes the data to Parquet files
4) The Parquet files are written to an AWS S3 'output bucket'
5) Grafana visualizes this 'data lake' via an Amazon Athena plugin
Note: You can use Grafana-Athena with any CANedge (1, 2, 3). You can simply skip the Lambda automation and e.g. upload DBC decoded Parquet files from a CANedge1 directly to an AWS S3 output bucket.
Key new benefits of Grafana-Athena dashboards
The 'magic sauce' of Grafana-Athena is the S3 data lake approach:
⚙️ AUTOMATED & SCALABLE
Our new 'MF4 decoders' enable the creation of DBC decoded Parquet data lakes. Deploying this with Lambda functions lets you easily automate and scale your setup from 1 unit to 1,000+ units.
💰 95% LOWER COST
Storing your data as an S3 Parquet data lake is 95%+ lower cost. As illustrated in our high level cost calculator, a 100 GB use case would cost ~150$ / month with Grafana-InfluxDB vs. ~3$ / month with Grafana-Athena.
⚡️ EXTREMELY FAST
Athena is a server less tool that dynamically allocates resources and performs query parallelization. This lets you refresh dashboards across GBs of data in 2-4 seconds - meaning you don't sacrifice any practical speed.
👍 EASY TO DEPLOY
Our step-by-step guide provides 'CloudFormation stacks' that auto-deploy everything in a few clicks. The integration can be set up in <15 min - and if any questions, we offer fast technical support as always. Try it yourself right now with our sample data!
🚚 IDEAL FOR CAN/LIN
CAN/LIN data often involves GB or TB of data, making a data lake ideal. Further, with Grafana-Athena you retain the original timestamp resolutions of every message.
🛠 MULTI PURPOSE
Once Grafana-Athena is deployed, the data lake can be used in hundreds of other tools - incl. Python, MATLAB, Excel, PowerBI and Tableau (more on this in future updates).
We believe this makes Grafana dashboard visualization much more viable at scale!
9 big upgrades in 2023
2023 has probably been one of the most 'high impact' years we've had at CSS.
In terms of numbers, we've grown sales by 25%+, making 2023 our 8th consecutive growth year - with 2,000+ new companies now using our products.
More importantly, we radically improved our products and software.
Below we recap the top 9 upgrades in 2023:
⚡️ MF4 converters are now 7x faster
⚡️ New CANedge enclosure with flanges (and 15% smaller)
⚡️ New CANedge1/2 incl. GPS/IMU: 25% lower cost and 70% smaller
⚡️ New CANedge3 with 3G/4G + GPS/IMU: 25% lower cost and 80% smaller
⚡️ New features: 50% reduced file size, cross channel triggers, CAN based RTC sync
⚡️ New optional Super SIM enabling 90% lower 3G/4G costs
⚡️ New MF4 decoders enabling easy creation of Parquet data lakes
⚡️ 10x faster deployments via AWS CloudFormation stacks
⚡️ New Grafana-Athena dashboard integration enabling 95% lower costs
Importantly, many of these upgrades 'stack' - which can save thousands of EUR/year for many users (see our example below).
We look forward to introducing new end user improvements in 2024 - if you have specific requests/suggestions/feedback, please reply to this mail.
This update was sent as part of our newsletter on 30 January 2024. You can also download it as a PDF.
Want updates like this in your inbox? Join our newsletter - or see all news.
