Skip to content

Latest commit

 

History

History
281 lines (234 loc) · 9.24 KB

boardroom-api.mdx

File metadata and controls

281 lines (234 loc) · 9.24 KB
Raw

Boardroom Governance API

The repository already contains the OpenAPI specification and the API pipeline manifest needed to create a RAG API pipeline. This pipeline generates a knowledge base from any DAO/Protocol hosted by the Boardroom Governance API.

Pre-requisites

To use this API, you'll need an API key. Request one from Boardroom's developer portal. You can run the rag-api-pipeline setup command to set the REST API Key, or your can directly store the key in the config/secrets/api-key file. A less secure option is to provide it using the --api-key CLI argument.

Defining the RAG API Pipeline Manifest

This pipeline will extract data related to protocol metadata (/protocols/aave), DAO proposals (/protocols/aave/proposals) and discussion posts from the Discourse forum site (discourseTopics, discourseCategories and discourseTopicPosts) if there's any.

Basic Configuration

The manifest starts by defining the API name, parameters and requests settings. You can visit this link to get the list of all DAO protocols in Boardroom. This example focuses on the Aave Governance DAO:

api_name: "aave_boardroom_api"

api_parameters:
  cname: "aave"
  protocol: "aave"

api_config:
  request_method: "get"
  content_type: "application/json"
  response_entrypoint_field: "data"

Connector Specification

The manifest then defines some metadata and the request parameters needed for making calls to the API:

spec:
  type: Spec
  documentation_url: https://docs.airbyte.com/integrations/sources/boardroom
  connection_specification:
    $schema: http://json-schema.org/draft-07/schema#
    title: Boardroom API Spec
    type: object
    required:
      - api_key
      - cname
      - protocol
    additionalProperties: true
    properties:
      api_key:
        type: string
        description: "Boardroom API Key. See <a href='https://docs.boardroom.io/docs/api/05c1fb6d88a07-governance-api'>here</a> for details."
        airbyte-secret: true
      cname:
        type: string
        description: "Protocol ID a.k.a cname"
        examples:
          - aave
          - hopprotocol
      protocol:
        type: string
        description: "Alias for cname. Required for endpoints using 'protocol' as parameter ID"
        examples:
          - aave
          - metacartel

API Request Configuration

Then, the requester_base defines the how connector should make requests to the API. Here, an ApiKeyAuthenticator schema is required and gets the api_token value from the config object:

definitions:
  requester_base:
    type: HttpRequester
    url_base: "https://api.boardroom.info/v1"
    http_method: "GET"
    authenticator:
      type: ApiKeyAuthenticator
      api_token: "{{ config['api_key'] }}"
      inject_into:
        type: RequestOption
        field_name: "key"
        inject_into: request_parameter

Record Selection and Pagination

Data records returned by the API are always wrapped in the data field, while pagination is handled using a Cursor-based approach:

definitions:
  selector:
    type: RecordSelector
    extractor:
      type: DpathExtractor
      field_path: ["data"]

  paginator:
    type: DefaultPaginator
    pagination_strategy:
      type: CursorPagination
      cursor_value: "{{ response.get('nextCursor', '') }}"
      stop_condition: "{{ 'nextCursor' not in response }}"
    page_token_option:
      type: RequestOption
      field_name: "cursor"
      inject_into: "request_parameter"

Endpoint Configuration

Now it's time to define the target endpoints with their respective schemas. Below is an example for the proposals endpoint:

endpoints:
  "/protocols/{cname}/proposals":
    id: "proposals"
    primary_key: "refId"
    responseSchema: "#/schemas/Proposals"
    textSchema:
      $ref: "#/textSchemas/Proposal"

Schema Definitions

The responseSchema reference from above defines the complete unwrappd data schema that is returned by the API endpoint:

schemas:
  Proposals:
    type: object
    $schema: http://json-schema.org/draft-07/schema#
    properties:
      refId:
        type: string
      id:
        type: string
      title:
        type: string
      content:
        type: string
      protocol:
        type: string
      adapter:
        type: string
      proposer:
        type: string
      totalVotes:
        type: integer
      blockNumber:
        type: integer
      externalUrl:
        type: string
      startTime:
        type: object
        properties:
          timestamp:
            type: integer
      endTime:
        type: object
        properties:
          timestamp:
            type: integer
      startTimestamp:
        type: string
      endTimestamp:
        type: string
      currentState:
        type: string
      choices:
        type: array
        items:
          type: string
      results:
        type: array
        items:
          type: object
          properties:
            total:
              type: number
            choice:
              type: integer
      events:
        type: array
        items:
          type: object
          properties: {}
      type:
        type: string
      indexedResult:
        type: array
        items:
          type: object
          properties:
            total:
              type: string
            choice:
              type: string
      summary:
        type: string
      privacy:
        type: string
      indexedAt:
        type: integer
      txHash:
        type: string
      quorum:
        type: integer

On the other hand, the endpoint's textSchema reference specifies the list of fields for text parsing. Note that all properties are also listed in the responseSchema. In this case, title, content, and summary will be parsed as texts, while other fields will be included as metadata properties in a JSON object:

textSchemas:
  Proposal:
    type: object
    properties:
      title:
        type: string
      content:
        type: string
      summary:
        type: string

Chunking Parameters

This section set the settings to be used when applying text chunking to the extracted content:

chunking_params:
  mode: "elements"
  chunking_strategy: "by_title"
  include_orig_elements: true
  max_characters: 1500
  new_after_n_chars: 1024
  overlap: 0
  overlap_all: false
  combine_text_under_n_chars: 0
  multipage_sections: true

Using the RAG Pipeline to generate a Knowledge Base for Aave

RAG Pipeline CLI

  1. Make sure to setup the pipeline initial settings by running the rag-api-pipeline setup command.
  2. Execute the following command:
rag-api-pipeline run all config/boardroom_api_pipeline.yaml config/boardroom_openapi.yaml

The processed data and knowledge base snapshot for Aave will be available in the output/aave_boardroom_api folder. You can also find a public knowledge base snapshot on Hugging Face.

Import the KB Snapshot into a Gaia Node

  1. Locate the generated snapshot in output/aave_boardroom_api/ (named aave_boardroom_api_collection-xxxxxxxxxxxxxxxx-yyyy-mm-dd-hh-mm-ss.snapshot.tar.gz) or download it from the HuggingFace link above.
  2. Follow the official knowledge base selection guide
  3. Configure your node using the recommended settings from the node deployment guide. Do not forget to update the custom prompts to mention aave as the target DAO protocol.

Example user prompts

  • Asking what information the RAG bot is able to provide

intro_prompt

proposal1_prompt

proposal1_prompt

Customizing for Other DAOs

To generate a knowledge base for a different DAO, you just need to modify the api_name and api_parameters values in the boardroom_api_pipeline.yaml manifest file.