reading time

Article

reading time

Oct 8, 2022

Direct Request Automation Framework for Testing (DRAFT)

This post is an edited version of a long-form piece available on GitHub and originally presented as a live workshop at Chainlink SmartCon 2022. The video of this SmartCon workshop is available on LinkPool’s YouTube channel: Direct Request Automation Framework for Testing (DRAFT) SmartCon Workshop.

separator
reading time

10 minutes

reading time

Víctor Navascués

Cover image of LinkPool Direct Request Automation Framework for Testing (DRAFT).

Cover image of LinkPool Direct Request Automation Framework for Testing (DRAFT).

Direct Request Automation Framework for Testing (DRAFT) is a framework that we (LinkPool Integrations Team) developed to support our Chainlink Direct Request (DR) integration work. This article will discuss how DRAFT can support node operators (NodeOps) and developers on their DR integrations. We’ll first introduce what DR integration is and its most common integration lifecycle and pain points. Then, we’ll explain the motivation for creating DRAFT at LinkPool and provide a high level overview of its main components. Finally, we’ll walk through a few of its basic uses and few time-based automated strategies for testing jobs. 

DRAFT is an open-source (link below) framework for automating end-to-end (E2E) testing of Chainlink DR integrations. But, before forking it and giving it a try, be aware of a few specifics:

  • The framework was developed on the go to suit the specific needs of LinkPool’s Integrations Team.
  • Its use is focused on on-demand/periodical E2E DR job testing. It is not a substitute for an integration-specific Consumer contract or an E2E DR job test with the final Consumer contract.
  • It has not been audited or fully optimised.

What Is Direct Request?

Direct Request (aka Any API) is a Chainlink product that allows requests of any API & service from a blockchain where the request is triggered by an on-chain event. A high level overview includes the following steps:

High level architecture diagram of Chainlink's basic request model.

  1. A NodeOp adds a directrequest Job on its node.
    • A directrequest job specification (job spec) is a TOML file that defines a set of tasks to be executed upon an OracleRequest event emitted by the Oracle or Operator contract. 
    • These tasks determine how to request APIs, process their response, and submit the result on-chain. 
    • Some API integrations (request requirements, processing the response, etc.) are too complex for their built-in core tasks (e.g. http, jsonparse) and an External Adapter (EA) must be used.
    • An EA is an integration-specific server that bridges a job run with an API and provides enhanced request and response processing capabilities. 
    • Both jobs and EAs can be implemented by anyone (NodeOp, client, anonymous dev, etc.) but to be used they must be available in the Chainlink node first.
  2. The NodeOp shares the Job details with the customer who implements the Consumer contract (Consumer) using the ChainlinkClient library.
  3. To request the job, the Consumer transfers LINK to the Oracle and attaches the Chainlink.Request data as payload.
  4. The Oracle emits the OracleRequest event (including the payload) after receiving the LINK payment.
  5. Then, the Chainlink node subscribed to the event triggers a job run.
  6. Responses are processed and the result is submitted on-chain back to the Consumer via the Oracle thus fulfilling the request.

Resources:

Direct Request Integration Life Cycle

The phases of a Direct Request integration life cycle are, in theory, the same as those of the Software Development Life Cycle (SDLC): planning, defining, designing, building, testing, deployment, maintenance. In reality, however, a few are grouped and streamlining the process is difficult.

Designing, Building, Testing

Validating the data transmission between the Consumer (on-chain) and the Job (off-chain) is the most important step of the process. It is very common to face unknowns about the type and format of request parameters in the Chainlink.Request, how to process them in the job run and/or EA, and the type and format of the response submitted on-chain. Key reasons for validating all data transmissions include:

  • Demand for increasingly complex integrations for all kinds of industries, sectors, and Data Providers (DPs) translating into Chainlink requests with more complex request parameters, results, and scenarios.
  • Continuous improvements on the Chainlink framework (multi-variable responses, new built-in core tasks).
  • The lack of existing patterns and code to provide inspiration and/or copy. There’s little information on StackOverflow, Ethereum Stack Exchange, or discord…we really are pioneering the process of putting real-world data on-chain!

Therefore, it is difficult to separate these 3 phases and, despite the fact TDD & BDD practices already blurred the boundaries of these phases in web2, DR integrations involve frequent E2E job testing. The most common method is:

  1. Having the off-chain infrastructure running locally: a Chainlink node with the job(s) and in nearly all circumstances with bridged EA(s) running locally also.
  2. A web browser with Remix, Metamask, and an Etherscan-like explorer to process the on-chain element, interact with the Consumer (deploy, set up, manage funds, verify), and inspect Consumer & Oracle transactions. 
    • It is also common to have tabs for multiple online data converters that allows us to convert human-readable data into Solidity friendly data types to be minimal, non-opinionated, and consume less gas.
  3. Request the Job until a successful request and successful fulfillment validate the data transmission. 
    • Otherwise, amend any of the elements and rinse, lather, repeat. 
    • A DR integration may require more than one job and each job may have more than one scenario.

Testing, Deployment, Maintenance

Ideally, every job is tested on each listed network upon deployment and periodically while in production. Troubleshooting a DR integration may also require an E2E test of a job under specific scenarios if the problem is not on the DP and/or the EA side.

Direct Request Integrations At Scale

Direct Request (along with a Webhook with an External Initiator) is not plug & play. It is the most complex Chainlink service NodeOps provide and DR integrations keep them busy beyond simply providing basic data types. 

At LinkPool, the Integrations Team regularly extends existing integrations and builds 1-2 new DR integrations every 2 weeks on average. These jobs are monitored, maintained and periodically tested. At the same time we look out for any Chainlink Direct Request product releases. This work is challenging, time consuming, and very interesting. All in all that’s a lot of E2E jobs testing!

Rationale Behind DRAFT Design

Given the DR integration life cycle, what if we had:

  • A generic Consumer that:
    • Facilitates prototyping the Job spec & Consumer by quickly validating the data transmission between them.
    • Requests any job (unless it has whitelisted consumers) and fulfills any request.
    • Eases the contract management by having a single instance per network (reducing the amount of dust LINK on consumers), and standardises access controls, request/fulfillment methods, fund/withdraw methods, events, etc.
    • Standardises on-demand/scheduled E2E jobs testing, and allows for testing of all jobs listed in market.link.
  • A framework that:
    • Provides CLI interaction.
    • Allows requests and interprets the results in a human-readable way.
    • Deals with data conversions and repeated inputs across consumers on the network.
    • Eases troubleshooting past DR integrations and their context switching.

What Is DRAFT?

DRAFT is a framework composed of two parts: one on-chain and one off-chain. The core of the on-chain element is the GenericConsumer.sol contract (GenericConsumer or GC), while the off-chain part is a set of Hardhat tasks that interact with the on-chain code.

The main on-chain features are:

  • Reduce the contract management per network across the DR integration life cycle. 
    • Request any job on the network (unless the job whitelists consumers) and any request with a single contract. 
  • Prevent LINK dust amounts across networks and hold internal LINK balances for consumers. 
  • Standardise methods and strategies for scheduled jobs testing.

The main off-chain features are:

Understanding how the generic request and fulfillment methods work in detail not only makes the usage of DRAFT easier but also unlocks its customisation potential. We’ll share only a couple points in this post; a deeper look is available on the DRAFT GitHub repository.

A Generic Request Method

An integration-specific consumer contract implies a request method with specific function arguments and CBOR encoding (using their related ChainlinkClient methods) to build the Chainlink.Request. It leverages the setBuffer method and allows us to set the CBOR payload directly on the Chainlink.Request by executing the CBOR encoding off-chain and passing it as a bytes calldata _buffer parameter. 

DRAFT also provides a tool to CBOR encode off-chain data using JSON with a specific format (documentation). 

See Generic Request Method code snippets on GitHub.

A Generic Fulfillment Method

An integration-specific consumer contract implies a fulfillment method with specific function arguments and a particular function body.

The generic fulfillment method leverages a fallback function, executed by the Oracle on every attempt to fulfill a request and the function signature set in the Chainlink.Request is not found. The fallback function then processes the msg.data with the same guarantees as recordChainlinkFulfillment

See Generic Fulfillment Method code snippets on GitHub.

Basic Usages for DRAFT

In this section we’ll run through a high-level showcase of a few of DRAFT’s basic use cases. See the requirements of how to use DRAFT on the ETH Goerli network. 

  1. Log GenericConsumer detail to log the current state of a GenericConsumer by reading its storage. 
  2. Fund a consumer address with LINK to pre-fund any address that might later want to request a job. 
  3. Request a job via task arguments to request a job by passing all GenericConsumer.requestData or GenericConsumer.requestDataAndForwardResponse method arguments. 
  4. Request a job from JSON file to request a job without having to pass all request method arguments via CLI and instead use a reference to a documented job request.
  5. Collect & decode a fulfilled request to interpret the submitted on-chain response data in the most human-readable way.
  6. Withdraw the funds to withdraw all or part of a LINK balance.

Time-Based Automation With DRAFT

One use case worth special mention is using DRAFT to schedule job requests using automated time-based strategies

Chainlink cron job

The simplest way to start the process of testing jobs periodically is by using Solidity Cron Jobs on your Chainlink node. But, this does have certain limitations like centralisation and poor reliability.

See how to do it on GitHub

Time-based Automation

Chainlink Automation Job Scheduler is a decentralised and more reliable version of the Chainlink cron job with the added benefit of a guide to help streamline the process. A full walkthrough is available on GitHub.

Summary

DRAFT streamlines DR integrations by addressing the pain points developers commonly encounter during the development, testing and maintenance phases. It validates data transmissions between the Consumer and the Job faster, simplifies the Consumer contract management, and provides clear and standardized strategies for on-demand and scheduled E2E jobs testing. DRAFT is also fully customisable and open-source making it an ideal tool for anyone building Chainlink Direct Request integrations. Give it a try today!

This blog post was originally presented as a live, deep dive technical workshop at Chainlink SmartCon 2022 and published as a DRAFT research paper on GitHub. This blog post has been condensed from the GitHub repo. You can view the original video presentation on LinkPool’s YouTube channel: Direct Request Automation Framework for Testing (DRAFT) SmartCon Workshop and you can follow Victor on Twitter at @vnavascues1.

Related Articles