Data and analytics
Last edit: , Invalid DateTime
As utilization of the network continues to grow, an increasing amount of valuable information will exist in the on-chain data. As the volume of data rapidly increases, calculating and aggregating this information to report upon or drive a dapp can become a time and process heavy endeavor.
Leveraging existing data providers can expedite development, produce more accurate results, and reduce ongoing maintenance efforts. This will enable a team to concentrate on the core functionality their project is trying to provide.
You should understand the basic concept of Block Explorers in order to better understand using them in the data analytics context. In addition, familiarize yourself with the concept of an index to understand the benefits they add to a system design.
In terms of architectural fundamentals, understanding what an API(opens in a new tab) and REST(opens in a new tab) are, even in theory.
Many Block Explorers offer RESTful(opens in a new tab) API(opens in a new tab) gateways that will provide developers visibility into real-time data on blocks, transactions, miners, accounts, and other on-chain activity.
Developers can then process and transform this data to give their users unique insights and interactions with the blockchain. For example, Etherscan(opens in a new tab) provides execution and consensus data for every 12s slot.
The Graph Network(opens in a new tab) is a decentralized indexing protocol for organizing blockchain data. Instead of building and managing off-chain and centralized data stores to aggregate on-chain data, with The Graph, developers can build serverless applications that run entirely on public infrastructure.
Using GraphQL(opens in a new tab), developers can query any of the curated open APIs, known as sub-graphs, to acquire the necessary information they need to drive their dapp. By querying these indexed sub-graphs, Reports and dapps not only get performance and scalability benefits but also the built in accuracy provided by network consensus. As new improvements and/or sub-graphs are added to the network, your projects can rapidly iterate to take advantage of these enhancements.
Client diversity is important for the overall health of the Ethereum network because it provides resilience to bugs and exploits. There are now several client diversity dashboards including clientdiversity.org(opens in a new tab), rated.network, execution-diversity.info(opens in a new tab) and Ethernodes(opens in a new tab).
Dune Analytics(opens in a new tab) pre-processes blockchain data into relational database (PostgreSQL and DatabricksSQL) tables, allows users to query blockchain data using SQL and build dashboards based on query results. On-chain data are organized into 4 raw tables:
logs and (call)
traces. Popular contracts and protocols have been decoded, and each has its own set of event and call tables. Those event and call tables are processed further and organized into abstraction tables by the type of protocols, for example, dex, lending, stablecoins, etc.
- Graph Network Overview(opens in a new tab)
- Graph Query Playground(opens in a new tab)
- API code examples on EtherScan(opens in a new tab)
- Beaconcha.in Beacon Chain explorer(opens in a new tab)
- Dune Basics(opens in a new tab)