All pages
Powered by GitBook
1 of 4

Loading...

Loading...

Loading...

Loading...

Adding New Block via Extension

Extensions provide a powerful way to augment the capabilities of the system by exporting blocks to the block manager. These blocks are loaded on startup, enabling you to create custom components and patches. In this guide, we'll outline how to use extensions to create and export components.

Creating Components with the Composition API

Follow Creating Components with the Composition API , you can use the composition API from the mercs_rete library to create components. First, make sure to import the necessary modules:

Then, you can create components and patches using methods provided by OAIBaseComponent.

Example:

Adding Inputs/Outputs

Follow , you can also define and add inputs/outpus to the component:

Configuring ESBuild to Avoid Errors

To prevent any errors from being thrown and to avoid bundling mercs_shared, which the server already has, you'll need to externalize mercs_rete. You can do this by adding --external:mercs_rete to the ESBuild in your package.json. This step ensures a smooth building process (because it already is loaded in server memory, no need to bundle it).

Export Components and Patches

Add the supports: ["blocks:v2"] property to the extension.yaml to tell the server to try to load the blocks:

An extension must export an object containing hooks and a createComponents() function. Once you have defined your components and patches, combine them into an array and return them within the createComponents() function as described. The system will automatically load these during startup, incorporating them into the available set of components.

Building the Extension

After making all the necessary updates to your extension code, run the following command to build the extension:

This command compiles your code and generates the extension.js file, which contains the entire extension ready for use. If you are using TypeScript for your extension development, make sure to update the tsconfig.json file with the appropriate configuration for your project.


By following these steps, you can create complex and customizable components and integrate them into the system via extensions. Make sure to follow the guidelines for each type of component or patch and consult the specific documentation related to each class or method for more details.

import { OAIBaseComponent, WorkerContext, OmniComponentMacroTypes, Composer } from 'mercs_rete';
Creating Components with the Composition API
let component = OAIBaseComponent
  .create(NS_ONMI, "redactPll")
  .fromScratch()
  .set('title', 'PII Redaction')
  .set('category', 'Compliance')
  .set('description', 'Redact Personal Information from text')
  .setMethod('X-CUSTOM')
  .setMeta({
    source: 'summary',
    summary: 'A PII redactor built using the solvvy/redact-pii library, implementing regex based PII reduction along with support for Google Cloud',
    links: {
      'What is PII?': 'https://www.cloudflare.com/learning/privacy/what-is-pii/',
      'redact-pii github': 'https://github.com/solvvy/redact-pii',
      'google cloud DLP': 'https://cloud.google.com/d1pr',
      'Solvy': 'https://www.solvvy.com/'
    }
  });
const inputs = [
  { name: 'text', type: 'string', description: 'A string', customSocket: 'text' },
  // More input definitions...
  { name: 'usSocialSecurityNumber', type: 'boolean', default: true }
];

inputs.forEach(({ name, type, customSocket, description, default: defaultValue }) => {
  component.addInput(
    component.createInput(name, type, customSocket)
      .set('description', description)
      .setDefault(defaultValue)
      .toOmniIO()
  );
});
supports:
    - "blocks:v2"
export default {
  hooks: extensionHooks,
  createComponents: () => {
    blocks: [ /*array of OmniComponentFormat*/ ],
    patches: [ /*array of OmniComponentPatch*/ ]
  }
}
yarn build

Create New Blocks

Adding new Block via OpenAPI Spec

Importing an Existing OpenAPI Specification

  1. Locate the API Specification: Determine if the API specification exists at a known URL. For example, OpenAI's specification can be found at https://raw.githubusercontent.com/openai/openai-openapi/master/openapi.yaml.

  2. Validate the Specification: Use https://editor.swagger.io/ to validate the OpenAPI spec.

  3. Add to System:

    • Navigate to packages/omni-server/extensions/omni-core-blocks/server/apis/[NAMESPACE]/[NAMESPACE].yaml.

    • Replace [NAMESPACE] with your desired namespace (e.g., openai).

Example Configuration for OpenAI

Creating an OpenAPI Spec

  1. Generate Draft Spec: If the specification doesn't exist, use tools like "OpenAPI Spec Generator" or request ChatGPT 4 to draft an initial spec.

  2. Validate the Specification: Use https://editor.swagger.io/ to validate the OpenAPI spec.

  3. Add to System:

Example Configuration for GetImg

Configure API in YAML

Patch Authentication

We support the following authentication type:

If auth is not defined globally in the original OpenAPI spec, you can patch it in the API yaml /omni-core-blocks/server/apis/[NAMESPACE].yaml

Filter APIs

Configure Blocks in YAML

Basic Metadata

Filter Inputs/Outputs

Transform

Hoist?

Control

Input

type:

customSocket:

Socket Format Options

Base64 image socket option:

Array:

Allow multiple connect

Rename

Run block to Generate Choices

Use JSONATA to Manipulate the format

Patch Security Scheme

Default Result

how to patch when response doesn't have property?

Special Cases

When properties have mutually exclusive or dependencies

Property name is Case-sensitive

When request content type is multipart/form-data, need file type to carry mimetype

Update the api section as below.

Go to packages/omni-server/extensions/omni-core-blocks/server/apis/[NAMESPACE]/api/[NAMESPACE].yaml.

  • Replace [NAMESPACE] with your desired namespace (e.g., getimg).

  • Update the api section as below.

  • namespace: openai
    api:
      url: https://raw.githubusercontent.com/openai/openai-openapi/master/openapi.yaml
      basePath: https://api.openai.com/v1
      componentType: OAIComponent31
    ...
    namespace: getimg
    api:
      spec: ./api/getimg.yaml
      basePath: https://api.getimg.ai
    title: getimg
    http_basic' | 'http_bearer' | 'apiKey' | 'oauth2
    namespace: elevenlabs
    api:
      url: https://api.elevenlabs.io/openapi.json
      basePath: https://api.elevenlabs.io
      auth:
        type: apiKey
        requireKeys:
          - id: xi-api-key
            displayName: xi-api-key
            type: string
            in: header
    title: elevenlabs
    namespace: openai
    api:
      url: https://raw.githubusercontent.com/openai/openai-openapi/master/openapi.yaml
      basePath: https://api.openai.com/v1
      componentType: OAIComponent31
      auth:
      
    filter:
      operationIds:
        - createChatCompletion
        - createCompletion
        - createImage
        - createModeration
        - createTranscription
        - createTranslation
        - createImageVariation
        - createImageEdit
        - createEdit
        - listModels
        - createEmbedding
    title: openai
    category: Text-to-Speech
    description: >-
      Text to Speech Synthesis using the ElevenLabs API, supporting a variety of
      monolingual (english) and multilingual voices.
    meta:
      source:
        title: 'ElevenLabs: Text To Speech'
        links:
          Website: https://beta.elevenlabs.io/speech-synthesis
          Subscription: https://beta.elevenlabs.io/subscription
          API Reference: https://docs.elevenlabs.io/api-reference/quick-start/introduction
          Documentation: https://docs.elevenlabs.io/welcome/introduction
          Voice Lab: https://beta.elevenlabs.io/voice-lab
        summary: >-
          Text to Speech Synthesis using the ElevenLabs API, supporting a variety of
          monolingual (english) and multilingual voices.
    title: Text To Speech
    apiNamespace: elevenlabs
    apiOperationId: Text_to_speech_v1_text_to_speech__voice_id__post
    displayNamespace: elevenlabs
    displayOperationId: simpletts
    scripts:
      hideExcept:inputs:
        - prompt
        - temperature
        - model
        - top_p
        - seed
        - max_tokens
        - instruction
        - images
      hideExcept:outputs:
        - text
    scripts:
      transform:inputs:
      transform:outputs:
    scripts:
        hoist:input
    controls:
      preview:
        type: AlpineImageGalleryComponent
        displays: output:image
        opts:
          readonly: true
    placeholder?
    image:
        customSocket: image
    socketOpts:
          format: base64
    socketOpts:
          format: base64_withHeader
    socketOpts:
          array: true
    allowMultiple: true
    inputs:
      'n':
        title: Number of Images
      model:
        type: string
        customSocket: text
        choices:
          block: getimg.listModels
          cache: global
          args: 
            pipeline: face-fix
            family: enhancements
          map:
            title: name
            value: id
    messages:
        scripts:
          jsonata: >-
            [{"role":"system", "content": $string(instruction) }, {"role":"user",
            "content": $string(prompt) }]
          delete:
            - prompt
            - instruction
        hidden: true
    securitySchemes
    outputs:
      _omni_result:
        hidden: true
    image_strength:
        scripts:
          jsonata: >
            $exists(init_image_mode) and init_image_mode = "IMAGE_STRENGTH" ? image_strength : undefined
      step_schedule_end:
        scripts:
          jsonata: >
            $exists(init_image_mode) and init_image_mode = "STEP_SCHEDULE" ? step_schedule_end : undefined
      step_schedule_start:
        scripts:
          jsonata: >
            $exists(init_image_mode) and init_image_mode = "STEP_SCHEDULE" ? step_schedule_start : undefined
    Accept:
        hidden: true
        default: application/json
      Organization:
        hidden: true
    image:
        customSocket: file 

    Add a New Block via Composition API

    Step 1: Import Necessary Components

    Import the necessary components

    Step 2: Create a New Component Instance

    Instantiate the ComponentComposer and initialize your component by defining its namespace and operationId using the .create(displayNamespace, displayOperationId) method.

    Set the namespace to const NS_OMNI

    Step 3: Define Component Characteristics

    Further define the characteristics of your component using provided methods. You can set the title, description, method, and category.

    Instead of using the fromScratch() method, you can also utilize the fromJSON() method with a valid Partial<OmniComponentFormat> JSON. This must include both the displayNamespace/operationID and apiNamespace/operationId.

    Step 4: Create and Add Inputs and Outputs

    For inputs and outputs, you'll first create an IOComposer using the .createInput(name, type, customSocket) or .createOutput(name, type, customSocket) methods. You can further define their properties, including setting up controls which can be automatically selected or overridden.

    Inputs always have controls associated with them, and they can be defined directly within the input creation process. Here's an example of defining an input with a control:

    Note:

    When using .allowMultiple() in conjunction with { array: true }, the input array is flattened into a single array:

    You can define which characters are used for joining (or separating on input) an array (default is \n) via a custom setting (specific to the Text Socket):


    Alternatively, you can also define inputs and outputs in an array:

    You also have the option to directly write the JSON if that's your preference.

    Step 5: Create and Add Controls (Optional)

    Controls are created using the ControlComposer and are added similarly to inputs/outputs.

    Or

    Step 6: Define Execution Macro

    You can define the behavior of your component using a macro. In this case, the OmniComponentMacroTypes.EXEC macro type is used.

    Step 8: Export the Component

    Finally, convert the component to JSON and export it.

    Step 9: Add the Component to App Blocks

    Note: extensions don’t need to call addBlock. they just need to export the createComponent function.

    Optional Settings:

    Composing Patches

    To compose a patch with a valid OmniComponentPatch, the same format applies:

    Dependencies

    Use .dependsOn(string[]) to specify dependencies, indicating if a block relies on other blocks (for instance, when using runblock internally). For example:

    import { OAIBaseComponent, type OmniComponentFormat, WorkerContext, OmniComponentMacroTypes } from "./path_to_file"
    const NS_OMNI = 'your_namespace'
    let component = OAIBaseComponent.create(NS_OMNI, 'your_operationId')
    component
        .fromScratch()
        .set('description', 'Your description')
        .set('title', 'Your title')
        .set('category', 'Your category')
        .setMethod('Your Method')
        .setMeta({
            source: {
                summary: 'A standard text input component with built-in URL fetching, enabling it to be connected to File (Image/Audio/Document) sockets',
                authors: ['Mercenaries.ai Team'],
                links: {
                    'Mercenaries.ai': 'https://mercenaries.ai'
                }
            }
        })
    const partialComponentFormat: Partial < OmniComponentFormat > = {
        displayNamespace: 'your_display_namespace',
        displayOperationId: 'your_display_operationId',
        apiNamespace: 'your_api_namespace',
        apiOperationId: 'your_api_operationId',
        // other properties can be added as needed
    };
    component.fromJSON(partialComponentFormat)
    const input = component.addInput(
      component.createInput('input_name', 'input_type', 'input_x-type')
        .set('title', 'Input title')
        .set('description', 'Input description')
        .setDefault('default value')
        .setConstraints(min value, max value)
        .setChoice([])
        .setRequired(true)
        .allowMultiple(true) // enable an input to accept multiple connections. 
        .setControl({
            controlType: 'alpine control type' 
          }) // Override the automatically selected control
        .toOmniIO()
    );
    component.createInput('input_name', 'input_type', 'input_x-type', {array:true})
    {array: true, customSettings: {array_separator = '-------' }};
    const inputs = [
      { name: 'text', type: 'string', description: 'A string', customSocket: 'text' },
      // More input definitions...
      { name: 'usSocialSecurityNumber', type: 'boolean', default: true }
    ];
    
    inputs.forEach(({ name, type, customSocket, description, default: defaultValue }) => {
      component.addInput(
        component.createInput(name, type, customSocket)
          .set('description', description)
          .setDefault(defaultValue)
          .toOmniIO()
      );
    });
    let controlComposer = component.createControl('controlName')
    
    component
      .addControl(
        controlComposer
          .setRequired(true)
          .setControlType('alpineControlType')
          .toOmniControl()
      );
    component.addControl(
      component
        .createControl("controlName")
        .setRequired(true)
        .setControlType("alpineControlType")
        .toOmniControl()
    );
    component.setMacro(OmniComponentMacroTypes.EXEC, (payload: any, ctx: WorkerContext) => {
      // define your function here
    })
    const YourComponent = component.toJSON() 
    export default YourComponent
    app.blocks.addBlock(component)
    let patch= OAIBaseComponent
      .createPatch(NS_ONMI, "text_replace")
      .fromComponent(apiNamespace, apiOperationId)
      ....
      .toJSON() //<-- OmnicomponentPatch
    
    app.blocks.addPatch(patch)
    component.dependsOn(['automatic1111.interrogateapi_sdapi_v1_interrogate_post', 'automatic1111.simpleImage2Image']);