Sep 14, 2020
Market Metrics Launch
This announcement article is one I’ve been looking forward to writing for quite sometime, and for us at LinkPool its been the feature we’ve been most excited about. That feature is Market Metrics, and we’re happy to be launching it today.
I’ve got to use the original announcement banner for what is a significant overhaul.
To begin, Market Metrics has been the product of many months of development and business thought. It has required us to make architectural changes to our infrastructure and software with some real thinking about what data is most important to collect, how we can display it and how the wider community can use this data to better their understanding around the day to day operation of the Chainlink network.
When we first started to build out the metrics capability, the scope of it wasn’t anywhere close to the product we have now. It was a lot simpler, more just charting of data for each Chainlink feed within the Market with an emphasis on individual responses and latencies. Although, once we fleshed it out more, it became apparent how far we could take it and how useful it would actually be to the wider community.
To dive in, Market Metrics consists of two core components:
- market.link overhaul, displaying masses of data around how feeds, nodes and the network is performing, all in real time.
- An Open API (Prometheus), giving everybody in the community the power to build their own queries and display data how they choose fit. Even in your own Grafana dashboards.
The overhaul to the Market is something we’ve recently announced and demoed for the first time at SmartCon, and we were incredibly pleased to hear the great feedback. As we’ve been building the Market over time, one area that became apparent to us was the need to show more data to make the listings we have increasingly more analytical. We’ve had the ability to give people a place to list the items they’re offering on the Chainlink network, but there wasn’t much data to show how what they were providing was benefiting the network.
The latest overhaul completely changes that, giving a full top-down and in-detail view of how every aspect of the network is operating. From the nodes response times, revenue, profit, gas prices and data quality; to the feeds latest answers, individual responses, latency answer deviation and individual profit and loss.
The latest landing page for the Market, showing the most important data points, instantly.
When first browsing to the updated Market, you’ll now be greeted with what we think are the most critical data points to display on a landing page. Having a high level view of the latest updates to the price feeds, to how many LINK has been earned a day in rewards overlaid with the transaction counts and panels for ETH gas costs.
Although, when you dive deeper into the app, it displays a more granular view of network activity.
Network metrics provides a high-level overview off all network activity
For the very first time, you can now watch activity across all Chainlink feeds in real-time. Within the Network Metrics section of the app, you can see how many active feeds there are within Chainlink, the number of daily transactions and how much LINK is earned each day or week.
You can also simply leave the page open and watch the network operate in real time within the “Feed Updates” table, showing every update as it happens with all critical pieces of information along side it. Even internally, we’ve been using this page when volatility is high to watch to see how Chainlink reacts, how gas prices spike and how quickly nodes perform. We like to think that if this is a core tool for us internally, the wider Chainlink observers will find it just as useful.
One area of the network that has always been incredibly difficult to track and display in a simple fashion is the breakdown of each node. This update changes that, providing an incredibly detailed view of every node in the network that is operational, nodes that help to secure billions within DeFi protocols.
We like easy to read, but informative charts
Never before have you been able to see how much LINK a Chainlink node is earning and how much of that is profit relative to the gas price with a breakdown per each feed. Match that with the ability to compare nodes per hour or per day over three months, and you have a extremely powerful tool that is as informative to the DeFi protocol creators and the community alike.
To give a more detailed breakdown about how the USD figures are generated, we didn’t just take the LINK figure and the gas spent then multiply that by the current price, as that would be inaccurate. Instead, we’ve created queries that give increasingly more accurate amounts by calculating the prices at the time of the transaction using the historical data from Chainlink price feeds. This is one of the core benefits of us choosing to produce a metrics API powered by Prometheus. Even though the data in its most basic form is rather rudimental, the way you can then perform calculations on that data with its in-built query language drastically expands the data you can formulate.
It’s easy to get lost in the rabbit hole of data (for completely free). You can waste time by comparing many nodes together, seeing how they respond compared to each other and network averages, how quickly they react to gas spikes and how some node operators choose lower gas prices for a higher margin, where some prefer high prices to prioritise latency.
To discuss the final large section of the metrics capability, we’ve fleshed out metrics for each individual feed within the Market. These metrics are common across every type of feed, and will keep up-to-date with developments in the future. Always including any types of new contracts that are developed and used by Chainlink in production, for example: off chain reporting.
What isn’t shown on this chart is response time charts, LINK rewards and P&L.
What’s important about the feed metrics, is it shows the sheer quality and integrity of the price feeds that Chainlink operates. You can clearly view how any individual node deviations don’t adjust the aggregated answer, how the nodes themselves all leverage different data feeds and the latency of each node to respond that effects the time it takes to form consensus. For DeFi builders who want to verify the core principles of Chainlink, then the Market provides all the data to do just that; while actually having the ability to hook this into their own alerting/graphing tooling to watch in real-time and organise that data how best suits them.
As mentioned, it’s important for us to expand the metrics capability to all types of feeds and developments going forward. An exciting prospect to consider is how these metrics will change over time when areas develop. For example: with the arrival of threshold signatures, you’ll see the amount of ETH spent per day will be a fraction of the cost now; you could then see how the rate of update within these feeds increases and how the time to form consensus also drastically falls. We’ll also be including detailed staking metrics, when the time comes.
The charts we display on the Market are powered by our new Metrics API that we’re releasing alongside this update. As previously mentioned, the API is powered by a high-availability, long-term storage, autoscaling Prometheus setup powered by Cortex.
For all of the charts shown within the node and feed metrics, there’s a “Get API” button that will take you to our Metrics API page to preview that query and to experiment for yourself.
Querying for the individual responses on the ETH/USD feed
The page above gives you a playground to experiment with the query language resulting in queries for the exact parts of the data that you want to extract and manipulate. Since we’re opening a direct Prometheus API, you can also directly use all aspects of PromQL, which really unlocks the capability of what’s possible.
We’re offering up to 90 days of history, 60 steps in resolution and 7200 queries an hour without the need for an API key. Since this is Prometheus, you can also directly integrate this into your own Grafana dashboards by adding
https://market.link/v1/metrics as a data source.
We’ve also provided example dashboards for people to use locally to get quickly familiar with the metrics capability, see our repository and documentation:
To conclude, I’d like to thank the Chainlink team for supporting us and making this feature possible with the grant program. Without it, we wouldn’t of had the capability to create something as feature rich.
Thanks for taking the time to read the article, we hope you find our latest announcement to be beneficial to the ecosystem as a whole. We can’t wait for you to try it out and to hear your feedback.