Perhaps you’ve heard some buzz about Wavefront but don’t quite know what it is or how it can be used. I was in the same boat for a while, but then I saw a demo that blew my mind. Imagine a platform that could ingest time series data (or metrics) from various sources, retain all data points with no roll up, and allowed IT operations and developers to quickly analyze the data for spotting and visualizing trends affecting application performance. How could we use that data to make the business better, and what kind of data actually gets collected? Moreover, how and why are people using this tool? I hope this post will provide some answers.
Wavefront Integration with Google Cloud Platform
The Wavefront website shows a number of integrations, but what does that really mean? Wavefront has pre-built methods for metric collection from certain applications and will guide you through the setup process. There are other ways to get data into Wavefront, but integrations make the process easier.
I happen to have a Google Compute Engine instance running a single Linux virtual machine, so it made sense to play with the Google Cloud Platform integration. This blog post gives some detail on the GCP integration, but once I read it, I wanted to see it for myself. Sign up for a free 30-day trial of Wavefront and a free year of GCP to follow along.
We’ll start by logging into a Wavefront with the proper e-mail address and password.
After logging into Wavefront, there are a number of tutorials available. To get started on our mission, click the Integrations option on the top bar as shown here.
There are numerous Wavefront integrations available, but we’re only concerned with Google Cloud Platform in this post.
After clicking on Google Cloud Platform, you will see something similar to the screenshot shown here. Since Wavefront is not yet connected to GCP, the METRICS and CONTENT labels below show gray information symbols (no metrics are reporting, no content is installed). Keep in mind the integration with GCP will bring in metrics for all of the following (if applicable): Google Computing Engine, Google Container Engine, Google App Engine, and Google Cloud Datastore. Read about the different metrics collected if you like, but click Setup to continue.
Since there are no integrations with GCP yet, click SET UP INTEGRATION as shown here. Clicking ADVANCED takes you to a list of all cloud integrations setup with your Wavefront instance.
I’ve given the integration a name at this point (Nick’s GCP Metric Playground) but need to get some information from my GCP instance before I register the integration. Let’s see if the directions work as expected.
Dive into GCP
After logging into my GCP account, I followed step 2 shown above from the GCP dashboard.
When following step 3, make sure a project is selected. In the screenshot shown here, the project I selected was called My First Project. Then choose CREATE SERVICE ACCOUNT.
The service account name from step 4 above can be anything, but I chose to stick with wavefront-integration. Notice the service account ID is in the form of an e-mail address. It will be an address unique to your GCP instance. The Role field below shows as Viewer and not Project Viewer even after following the instructions. I’ve included an additional screenshot below showing the dropdown selection process to verify we have completed all steps correctly.
Steps for selecting the Project Viewer Role are shown here.
After clicking CREATE on the service account menu above, you will see a message stating the service account and key were created, and the proper JSON file gets downloaded to your computer.
And Now Back to Wavefront
Now we’re ready to go back to the GCP integration page in Wavefront. Open the JSON file in an editor like Notepad, and paste all of its text in the JSON key field shown here.
After pasting the JSON key, you have a choice to fetch all metric categories or narrow the scope (i.e. just pull metrics for App Engine and Compute, etc.). I suggest leaving all categories selected. The service refresh rate is how often Wavefront will reach back to the GCP instance to pull in new data.
At the bottom of this page, click REGISTER to save the integration, and hopefully you see a success message similar to the one shown here. Wavefront is now connected to my GCP instance and should be pulling in data.
At this point, we are directed back to the Google Cloud Platform Integration Page in Wavefront.
Our integration can be edited at any time (i.e. to collect data more frequently or collect a smaller subset of metrics) by clicking its description in the Name column. The Project ID is something that comes from the GCP project for which you created a service account. If you had multiple GCP projects, you could follow the same process to connect them all.
How Can I See My Metrics?
The integration is complete, and you’re chomping at the bit to see metrics. Go back to the Integrations menu, and you’ll see the GCP integration now has a green check mark on it. This means the integration is actively in use. Click on it to continue.
This looks different. We have metrics! The metrics Wavefront is collecting are being ingested at 1.433 points per second. The metrics collected are listed by name in the left-hand tree.
But that’s not all. What if we click on Dashboards? There are some canned dashboards for GCE, so let’s see what we have. This is only one of several ways to get to these dashboards (search for GCP in Dashboards -> All Dashboards for another option).
In the few minutes the integration has been setup, look at the data I have. I’m showing you 4 out of 36 different charts in the dashboard giving overview stats (8), network stats (12), and storage stats (16) for my GCE workloads (just a single VM at the moment). And in a day, a week, a month, etc. all of this data will be retained and searchable should I want to do some comparative analysis.
I can click on the name of any chart to show a single metric and zoom in to any time period for which we have metric data. For example, clicking on the STORAGE IOPS chart above, I see a default 2 hour chart of the average storage IOPs across my GCE instance that can be changed to my liking.
How Does This Integration Help Me?
All of the metrics seen in Wavefront are just those exposed via API from Google. If we look at the native monitoring tools inside Google Compute Engine for CPU utilization of my Linux VM, I have a few options for time windows, but that’s it. Is that going to help me if performance isn’t so great? Maybe.
What if I wanted to look at CPU utilization for a specific date range or zoom into a window of say 15 minutes when I knew there was a performance spike? That’s where Wavefront is far superior. The range below shows a 2 hour period, but I can switch to any time period with ease.
What if I wanted to…
- Create my own chart for a specific metric?
- Mark events like code pushes on a chart to locate and visualize a performance bottleneck?
- Create a new dashboard?
- Setup an intelligent alert?
- Do analysis to correlate 2 metrics?
- Use query language to create a chart?
My cloud provider does not have the tools for these tasks, but with Wavefront, I can use the metrics collected to make informed decisions based on the results of my detailed analysis.
This is one of many integrations to explore. Will you catch the wave?