Author: Hotrod
Summary: A development template for testing pipe actions
Keywords: development, testing
Categories: development

Development Pipe Template

This is a development pipe template. It is intended as a quick-start for testing and developing pipe actions.

When you want to build a new pipe for a data stream, often you want to build the transformation part of the pipe first, using sample data, before hooking up the pipe to an actual data source.

This can be achieved with the echo input, which generates a fixed (hardcoded) event data payload for a pipe to operate on.

Similarly, the print output is used to capture pipe output. This causes output events to be logged to the STDOUT file descriptor of the process where the pipe is executed - typically Hotrod Server's internal agent, or another dedicated Hotrod agent.

This pipe contains some very common actions for removing, adding and renaming fields, as well as adding a timestamp to the output event.

It's worth nothing that when using the pipe editor in Hotrod Server's web frontend, sample input can be provided with any input. This mechanism uses echo and print magic behind the scenes.

Pipe Definition

name: dev-template
input:
  echo:
    json: true
    ignore-linebreaks: false
    event: |
      {"field1": "someValue", "field2" : "someValue"}

actions:
  - remove:
      fields:
        - field1

  - rename:
      fields:
        - field2: field3

  - add:
      output-fields:
        - field3: true

  - time:
      output-field: timestamp
      output-timezone: UTC
      output-format: default_iso

output:
  print: STDOUT