Skip to main content

The Manifest

The App manifest file​

The manifest.yaml is a required file which defines how the app integrates closely with the user's context and actions.

The manifest is a yaml file listing some simple key/value settings (such as title and description ) and possibly a set of functionalities: filters , transforms , and services as described below. These functionalities work together to expose powerful context-based extensions.

A simple manifest.yaml​

name: Hello World
version: 1.0.0
description: A very simple iframe type application
icon: fa-hand

The manifest above provides a custom name, title, and icon. This app uses the default docker for a context-free app.

name: Credit Check
version: 1.0.0
image: octostar/code-streamlit
description: Shows a live credit check for the selected person
icon: fa-magnifying-glass-dollar
engine: k8s
entry_point: bash /app/main.sh

memory_limit: 8Gi # App will be killed for OOM and restarted by K8s
memory_requested: 250Mi  # App won't even deploy if less memory is available
storage_limit: 10Gi # App will hit "No space left on device"
storage_requested: 1Gi # App will not be even deployed if less storage is available

semantic_bindings:
    record_viewer:
        - person

watchers:
    - name: "check_updates_for_person"
      entry_point: bash /app/watcher.sh x
      description: "Check if a person has new credit events & notify"
      semantically_bound: [ person ]
      interval: "20m"

filters:
  persons:
    type: semantically_bound
    concepts:
      - person

transforms:
  open_credit_check_iframe:
    description: opens an iframe
    type: iframe
    title: '{{ app.folder.os_item_name }}'
    icon: fa-magnifying-glass-dollar

  enhance_item:
    description: Adds some_new_attribute to item
    type: javascript
    code: |
      function transform(item){
        item.some_new_attribute = 'hello'
        return {item};
      }

services:
  open_background_check:
    role: context_menu
    group: open
    label: Credit Check
    icon: fa-magnifying-glass-dollar
    description: Opens the streamlit app passing the person as context for credit check
    accepts:
      - persons
    transforms:
      - enhance_item
      - open_credit_check_iframe

Dissecting the manifest.yaml​

Simple key/value settings​

name: credit_check
version: 1.0.0
description: Shows a live credit check for the selected person
icon: fa-magnifying-glass-dollar
image: scarduzio/code-streamlit
entry_point: bash /app/main.sh

The name , title and icon for the app are provided. Other options include:

  • image to specify a specific docker image used for the app.
  • entry_point the command to bootstrap the frontend app
  • no_archive if true the app zip file is not uploaded/expanded on the server. The commands from main.sh will be directly executed in the container.
  • url_from_logs if true, the octostar web application will try to extract a url (maybe it contains a token) from the app logs, and include it on the app url
  • engine the default is k8s; other options include html and react which require no kubernetes job
  • run_as_user can give the numeric user_id to use if not the default.
  • startup_timeout can give the number of seconds to wait if different from the default
  • container_port can give the docker container http port if different from the default (8080)
  • template the name of a template to open as the frontend. This can be useful to give a user interface to a fastAPI backend, for example. The template must exist within the templates folder of the app.

Semantic Binding​

The optional semantic_binding section provides a way to register the app to feature within the some contexts. Beneath each context are listed the ontology concepts supported by the app.

The following contexts are supported:

  • record_viewer : The app will appear as a tab in the record viewer.

In this example for a map app, the concepts of person , vehicle , mobile_phone and place are semantically bound to the record_viewer

The semantic binding for person includes priority and maximize properties recognised by the record viewer. In this case if the binding has the highest priority (lower number means higher priority), the app will show in full tab view due to the maximize: true property.

(Note for the record_viewer the record details have a priority of 200 and media gallery has priority of 100)

semantic_bindings:
    record_viewer:
        - person:
            priority: 50
            maximize: true
        - vehicle
        - mobile_phone
        - place

Watchers​

The watchers section contains the list of watcher functions the app exposes. When an app offers a watcher function, users are free to create local entities of type watch_intent in the workspaces containing a reference to the specific watcher function.

watchers:
    - name: "check_updates_for_person"
      entry_point: bash /app/watcher.sh x
      description: "Check new credit events & notify"
      semantically_bound: [ person ]
      interval: "20m"

In this case, the watcher function named check_updates_for_person, and it can process only entity of type "person". It will run every 20 minutes (minimum interval is 5 minutes).

An example of a watch_intent local entity created by an app or by octostar to reserve the periodic execution of the above watcher function about an entity of type person.

 {
            "entity_id": "3a96da2f-96ee-40d6-9c5b-882a99829c56",
            "entity_type": "os_watch_intent",
            "entity_label": "Check new credit events & notify - credit_check - Thomas Jefferson",

            "description": "<Normally null, if value is 'STOPPED', this execution will be skipped>", 
            "previous_run_output": "<base64 encoded value, useful to compare results and finding for differences (optional)>",
            "os_last_updated_at": "2024-03-20 11:25:56",
            "os_created_by": "operator@octostar.com",
            "app_id": "oj-c2-ea-e9e",
            "os_last_updated_by": "operator@octostar.com",
            "os_workspace": "c43ae201-d175-406e-8f04-f8d1d8b7e402",
            "last_run": null,
            "watcher_name": "credit_check",
            "os_item_type": "os_watch_intent",
            "os_entity_uid": "3a96da2f-96ee-40d6-9c5b-882a99829c56",
            "arguments": "<base64 encoded JSON representing the arguments for the watcher function (optional), e.g. twitter handle>",
            "interval": "20m",
            "os_created_at": "2024-03-20 11:25:56"
        },

The octostar-python-client pip dependency offers a practical helper to implement watcher functions without worrying much about the details, here is how to use it with an example:

from octostar.utils.watchers import process 


def monitor_watcher(w_intent:WatcherIntent) -> Optional[BaseException]:
    if w_intent.app_name == "social_monitor":
        search_filter = base64_to_dict(w_intent.arguments)
        last_ts = base64_to_dict(w_intent.previous_run_output)["last_ts"]
        if last_ts:
            last_day = last_ts.split(" ")[0]
            search_filter["from_date"] = last_day
        tweets = get_tweets_from_sl(search_filter)
        if tweets:
            tweets_df = tweets_records_to_df(tweets, parser=True)
            if last_ts:
                tweets_df = tweets_df[tweets_df["creation_date"] > last_ts]
            last_ts = max(tweets_df["creation_date"])
            save_records(tweets_df, [w_intent.os_workspace], w_intent.watcher_name)
            if search_filter["notify"]:
                notify_about_new_tweets(w_intent, len(tweets_df))
        save_new_latest_timestamp.sync(w_intent, last_ts)

process.sync(processor=monitor_watcher)

Filters​

The filters section, if provided, lists one or more named re-usable selectors which return a true/false value to accept an offered item/items.

The property names for a filter depend on the filter type . In the example, a semantically_bound filter has a concepts property, listing the concepts which are accepted by the filter. In this case, just one concept is provided: person . Any person record will be accepted by the filter, including records of a type which are a subclass of person , for example student or fisherman.

  persons:
    type: semantically_bound
    concepts:
      - person

The following filter types are supported:

semantically_bound​

As noted above, the semantically_bound filter has a concepts property, which lists one or more accepted concepts.

javascript​

Defines an accepts function in javascript code.

  is_requirements_txt:
    type: javascript
    code: |
      function accepts(item) {
        return item.os_item_name === "requirements.txt";
      }

A javascript filter has a code attribute which defines an accepts function which is evaluated in the browser.

In the example above we see the filter returns a true value for an item if the os_item_name attribute equals requirements.txt

python​

Defines an accepts function in python code.

  is_a_folder:
    type: python
    code: |
      def accepts(item):
        return item["os_item_type"] == "os_folder"

A python filter has a code attribute which defines an accepts function which is evaluated in the browser.

In the example above we see the filter returns a true value for an item if the os_item_type attribute equals os_folder .

For both javascript and python filters, the possible parameters which may be used by the accepts function include:

item : the workspace_item under consideration

items : an array the workspace_item objects under consideration

entities : a list of entities under consideration

workspace : the workspace to which the item or items belong. The full list of items in the workspace can be found in workspace.items property

ee : the EventEmitter

graph : an object having nodes (records / entities) and edges

Transforms​

Transforms are operations taking the accepted context as input, and producing as output either a modification to the context, a visible change to the user interface, or both.

From the example:

  enhance_item:
    description: Adds some_new_attribute to item
    type: javascript
    code: |
      function transform(item){
        item.some_new_attribute = 'hello'
        return {item};
      }

Above we see a transform named enhance_item . The transform defines in javascript ( type: javascript ) a transform function which adds some_new_attribute to the item parameter, then returned as the item property of the output. In this way the item is effectively and silently transformed: the context to any downstream transform is modified with the updated item parameter.

Also from the example:

  open_credit_check_iframe:
    description: opens an iframe
    type: iframe
    title: '{{ app.folder.os_item_name }}'
    icon: fa-magnifying-glass-dollar

Above we see a transform of type iframe . The transform opens a new tab in which the app is opened with the default app url (an alternate url could be provided). An icon and title for the tab are given. The iframe transform allows powerful integration with the user interface, for example by using streamlit apps which can communicate directly with the Octostar desktop or server using our APIs.

The full list of available transforms are below:

python​

Executes a python transform function in the browser. The function may be provided either inline using a code attribute, or in a file referenced using the file attribute.

javascript​

Executes a javascript transform function in the browser. The function may be provided either inline using a code attribute, or in a file referenced using the file attribute.

nunjucks​

Executes a nunjucks template transform in the browser. The template may be provided either inline using a template attribute, or in a file referenced using the file attribute.

iframe​

Opens a tab to the given url attribute, or the default app url. Exposes the browser api to the app using window.message event api as exposed with the octostar streamlit api.

  • Note the item(s) exposed within the iframe are referenced as record and records .
  • Each record will open in a different tab. To use the same tab include reuse: true

http​

Without opening a tab, invokes a server http operation which can be a service offered by the app (for example a service in a flask app), or an absolute url to an external service. The following optional parameters may be provided:

  • headers an object for which any attribute can be a nunjucks template or a string
  • data an object for which any attribute can be a nunjucks template or a string
    • note that if the parameter is a string and matches an entry in the transform context, then that context entry is returned. The example below sends the item object as the body of the http POST request.
  • url can be relative to the app url /credit_rating or an absolute url eg:. https;//example.com/credit_rating
  • method default is GET, but POST, PUT etc may be used.

Example

Here is an example of posting an item to a flask service

  post_to_flask:
    description: Upload the file to flask
    type: http
    method: POST
    url: "{{ app.url }}/upload"
    headers:
      Content-Type: "application/json"
    data: item

Below is an example of a server side flask service which receives the item from the above transform, and returns a notification object:

@app.route('/upload', methods=['POST'])
def notification():
    item = request.get_json()
    return jsonify({
            "notification": {
                    "message": "Upload Successful!",
                    "description": f"""The item '{item["entity_label"]}' was successfully uploaded""",
                    "level": "info"
                }
    })

emit_event​

The emit_event transform provides a powerful way to integrate with the octostar event emitter api.

Emits an event on the EventEmitter. The following parameters are required:

  • topic a string or nunjucks template returning the topic for the event
    • TODO Make reference to the built-in event topics
    • TODO: support shorthand topic names for built-in event topics
  • args a list of strings or nunjucks templates returning the arguments for the event emitter.
    • Note that if the parameter is a string and matches an entry in the transform context, then that context entry is returned. The example below sends the notification object as the parameter to the showNotification event

Example

  emit_notification:
    description: Emit notification received from the flask app
    type: emit_event
    topic: octostar:desktop:builtins:showNotification
    args: [ notification ]

Parameters​

The following parameters may be available to the accept filter and transform functions. The accept filter must check that the parameters it needs are present and acceptable then return true, otherwise it must return false.

    • item: the workspace item under consideration
  • items: the list of workspace items under consideration
  • workspace: the relevant Workspace object
  • graph: the graph of nodes (Entity[]) and edges (EdgeSpec[])
  • dataTransfer: a dataTransfer like object
  • entities: the Entity[] to be considered

The following parameters are available to the transforms (but not to the accept filter)

    • OntologyAPI: the OntologyAPI (todo documentation)
  • DesktopAPI: the DesktopAPI (todo documentation)
  • onTaskCompleted: a function which can be invoked with the task result
  • onTaskCanceled: a function which can be invoked to cancel the task
  • any other attributes assigned by previous tranform(s)

Services​

Services connect the Octostar user interface with the transforms by the app.

From the example:

  open_background_check:
    role: context_menu
    group: open
    label: Credit Check
    icon: fa-magnifying-glass-dollar
    description: Opens the streamlit app passing the person as context for credit check
    accepts:
      - persons
      - legal_person
    transforms:
      - enhance_item
      - open_credit_check_iframe

Here we have a service open_background_check in the context_menu role (the default). The service appears under the open actions on the context menu, having the label Credit Check .

The service is available whenever the context (what was right clicked on) is accepted by any of the listed filters in the accepts section. Each name given in the accepts section must be either either:

    • The name (key) of a filter defined in the filters section of the manifest
  • OR the name of a concept, eg legal_person in the example (a shortcut semantic binding filter).

When the user select the operation, two transforms are performed in sequence. First the enhance_transform then the open_credit_check_iframe

Other roles (including record_viewer and set_viewer will become available soon).

The group attribute defines where the service appears in the context menu

    • open
  • edit
  • transform
  • general
  • danger

The subgroup attribute may be included for transform services.

The optional show_progress parameter can be set to false (default is true ). When false the service progress bar is not displayed.

A params key can also be included on the service (not shown in the example) to include any additional parameters to be passed to the transform.

Another Full Example:

This example provides a transform service which uses the http example above to post the item to flask, then send the result from flask to the event emitter.

name: Send Folder To Flask
version: 1.0.0
description: Example posting from context menu to flask

filters:
  is_a_folder:
    type: semantically_bound
    concepts:
      - os_folder

transforms:

  post_to_flask:
    description: Upload the file to flask
    type: http
    method: POST
    url: "{{ app.url }}/upload"
    headers:
      Content-Type: "application/json"
    data: item

  emit_notification:
    description: Emit notification received from the flask app
    type: emit_event
    topic: octostar:desktop:builtins:showNotification
    args: [ notification ]

services:
  send_to_flask:
    group: transform
    subgroup: Folder
    label: Post to Flask
    icon: fa-car
    description: Posts it to flask
    accepts:
      - is_a_folder
    transforms:
      - post_to_flask
      - emit_notification