Skip to main content

Pipe Language Types Reference

This is an exhaustive list of input values used by the Pipe Language.

If you are unsure about how a particular field's value should look like, the examples in this type reference will assist.

(string, string) map​

Map containing string keys and string values

Example:

generate:
  let:
    two: '1+1'
    three: '1+2'

(string,string) pair​

A Key: Value pair

Example:

marker: HERE

array of (field,expression) pairs​

Array of field: expression pairs

Example:

- one_plus_two: one + two
- three_plus_four: three + four

array of (field,field) pairs​

todo

Example:

todo

array of (field,regex) pairs​

An array of fields and regular expressions

Example:

patterns:
- name: ^Jane\s+

array of (field,type) pairs​

Array of field: type (num,str,etc) pairs

Example:

convert:
   - m1: num
   - m5: num
   - m15: num

array of (field,value) pairs​

Array of field: value pairs

Example:

script:
  set:
    - three: 3
    - four: four

array of JSON Pointers​

An array (in JSON Pointer RFC6901 format) that should be skipped.

Example:

- /path/to/field
- /toplevelpath

array of fields​

An array of field names

Example:

output-fields:
  - field1
  - time_field

array of key-value pairs​

An array of maps from string to any kind of value

Example:

output-fields:
  -  first: 1
  -  second: two
  -  3rd: number three

array of maps​

An array of dictionaries

Example:

- Encoding: JSON/UTF-8
- Content-Type: application/json

array of path​

An array of OS Paths

Example:

include:
  - /var/log
  - /tmp

array of strings​

An array of plain strings

Example:

input-formats:
  - default_iso
  - epoch_secs

array of time-format​

Array of time format specifiers

Example:

- "%Y-%m-%dT%H:%M:%S%.3fZ"
- epoch_secs
- default_iso
- epoch_frac_secs

array of time-range​

Array of time-range specifiers

Example:

when:
  - 'mon-fri:09:00-17:00'
  - 'sat:09:00-13:00'

array of two strings​

An array of plain strings

Example:

hash-value: [new_hash, cat]

bool​

A boolean value: true or false

Example:

true

connection-string​

for 'sqlite', a path to a file, for 'postgres' "postgres://name:password@host/table"

Example:

- sqlite:///var/log/dpkg.db
- postgres://user:password@host/table

cron​

A cron spec, but with seconds added

Example:

- 10h15 everyday: '0 15 10 * * *'
- every 3s in Jan, from Wed to Fri: '0/3 * * * jan wed-fri'
- every 1st, 5th, and 15th second, on 16th of every month: '1,5,15 * * 16 * *'
- 10h15 everyday, in 2020: '0 15 10 * * * 2020'
- midnight, first day of the month: '0 0 0 1 * *'

delta​

todo

Example:

todo

duration​

A time duration

Example:

1ms 1s 1m 1h 1d 1y

duration or 'document'​

todo

Example:

todo

encoding​

A text encoding (we default to UTF-8)

Example:

input:
  files:
    path: latin-1.xml
    encoding: ISO-8859-1
    stop-reading-after: true

enum​

Only one of the options can be specified at a time

Example:

selection 1

expression​

A LUA expression

Example:

script:
  condition: num == 1
  let:
    - is_one: "true"

fanout|topic​

todo

Example:

todo

field​

A JSON field name

Example:

input-data

filename​

A path to a file on the filesystem

Example:

/path/to/file

get|post​

todo

Example:

todo

gzip​

todo

Example:

todo

host:port​

A host:port pair

Example:

0.0.0.0:443

iana-timezone​

IANA timezone specifiers, see: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones

Example:

Europe/Berlin

integer​

A number without a fraction component

Example:

1 10 100

integer or the 'document'​

todo

Example:

todo

interval​

todo

Example:

todo

iso time field​

A field containing a ISO compliant time (e.g. 2019-07-10T18:45:00.000+0200)

Example:

time_field: timestamp

path​

A file system path

Example:

/var/log/dpkg.*

previous|next​

either "previous" or "next"

Example:

todo

regex​

a regular expression

Example:

\S+\)$

single|container​

todo

Example:

todo

string​

A plain (unstructured) text value

Example:

some text

string or (string,string)​

single string or key: value string

Example:

"one"
"two: two"

templated text​

A section of text that can contain template expansions

Example:

add:
  template: |
    {
      "another-one": ${one},
      "tags": ["yoyo",${one}]
    }

time​

A time in the format: 2019-07-10 18:45:00.000 +0200

Example:

start_time: '2019-07-10 18:45:00.000 +0200'

time-format​

Time format specifiers

Example:

%Y-%m-%dT%H:%M:%S%.3fZ

url​

A url including scheme, host, and path

Example:

https://www.example.com/path/to/resource