5. Web Socket
Broker offers access to the publish/subscribe layer via WebSocket in order to make its data accessible to third parties.
WebSocket clients are treated as lightweight peers. Each Broker
endpoint can be configured to act as a WebSocket server by either (1)
setting the environment variable BROKER_WEB_SOCKET_PORT
; (2)
setting broker.web-socket.port
on the command line or in the
configuration file; or (3) from C++ by calling
endpoint::web_socket_listen()
. When running inside Zeek, scripts
may call Broker::listen_websocket()
to have Zeek start listening
for incoming WebSocket connections.
Note
Broker uses the same SSL parameters for native and WebSocket peers.
5.1. JSON API v1
To access the JSON API, clients may connect to
wss://<host>:<port>/v1/messages/json
(SSL enabled, default) or
ws://<host>:<port>/v1/messages/json
(SSL disabled). On this WebSocket
endpoint, Broker allows JSON-formatted text messages only.
5.1.1. Handshake
The broker endpoint expects a JSON array of strings as the first message. This array encodes the subscriptions as a list of topic prefixes that the client subscribes to. Clients that only publish data must send an empty JSON array.
After receiving the subscriptions, the Broker endpoint sends a single ACK message:
{
"type": "ack",
"endpoint": "<uuid>",
"version": "<broker-version>"
}
In this message, <uuid>
is the unique endpoint ID of the WebSocket server
and <broker-version>
is a string representation of the libbroker
version, i.e., the result of broker::version::string()
. For example:
{
"type": "ack",
"endpoint": "925c9110-5b87-57d9-9d80-b65568e87a44",
"version": "2.2.0-22"
}
5.1.2. Protocol
After the handshake, the WebSocket client may only send Data Messages. The Broker endpoint converts every message to its native representation and publishes it.
The WebSocket server may send Data Messages (whenever a data message matches the subscriptions of the client) and Error Messages_ to the client.
5.1.3. Data Representation
Broker uses a recursive data type to represent its values (see Data Model). This data model does not map to JSON-native types without ambiguity, e.g., because Broker distinguishes between signed and unsigned number types.
In JSON, we represent each value as a JSON object with two keys: @data-type
and data
. The former identifies one of Broker’s data types (see below) and
denotes how Broker parses the data
field.
5.1.3.1. None
There is only exactly one valid input for encoding a none
:
{
"@data-type": "none",
"data": {}
}
5.1.3.2. Boolean
The type boolean
can take on exactly two values and maps to the native JSON
boolean type:
{
"@data-type": "boolean",
"data": true
}
{
"@data-type": "boolean",
"data": false
}
5.1.3.3. Count
A count
is a 64-bit unsigned integer and maps to a (positive) JSON
integer. For example, Broker encodes the count 123
as:
{
"@data-type": "count",
"data": 123
}
Note
Passing a number with a decimal point (e.g. ‘1.0’) is an error.
5.1.3.4. Integer
The type integer
maps to JSON integers. For example, Broker encodes the
integer -7
as:
{
"@data-type": "integer",
"data": -7
}
Note
Passing a number with a decimal point (e.g. ‘1.0’) is an error.
5.1.3.5. Real
The type real
maps to JSON numbers. For example, Broker encodes -7.5
as:
{
"@data-type": "real",
"data": -7.5
}
5.1.3.6. Timespan
A timespan
has no equivalent in JSON and Broker thus encodes them as
strings. The format for the string is <value><suffix>
, whereas the value
is an integer and suffix is one of:
- ns
Nanoseconds.
- ms
Milliseconds.
- s
Seconds.
- min
Minutes
- h
Hours.
- d
Days.
For example, 1.5 seconds may be encoded as:
{
"@data-type": "timespan",
"data": "1500ms"
}
5.1.3.7. Timestamp
Like timespan
, Broker uses formatted strings to represent timestamp
since there is no native JSON equivalent. Timestamps are encoded in ISO 8601 as
YYYY-MM-DDThh:mm:ss.sss
.
For example, Broker represents April 10, 2022 at precisely 7AM as:
{
"@data-type": "timestamp",
"data": "2022-04-10T07:00:00.000"
}
5.1.3.8. String
Strings simply map to JSON strings, e.g.:
{
"@data-type": "string",
"data": "Hello World!"
}
5.1.3.9. Enum Value
Broker internally represents enumeration values as strings. Hence, this type also maps to JSON strings:
{
"@data-type": "enum-value",
"data": "foo"
}
5.1.3.10. Address
Network addresses are encoded as strings and use the IETF-recommended string format for IPv4 and IPv6 addresses, respectively. For example:
{
"@data-type": "address",
"data": "2001:db8::"
}
5.1.3.11. Subnet
Network subnets are encoded in strings with “slash notation”, i.e.,
<address>/<prefix-length>
. For example:
{
"@data-type": "subnet",
"data": "255.255.255.0/24"
}
5.1.3.12. Port
Ports are rendered as strings with the format <port-number>/<protocol>
,
whereas <port-number>
is a 16-bit unsigned integer and protocol
is one
of tcp
, udp
, icmp
, or ?
. For example:
{
"@data-type": "port",
"data": "8080/tcp"
}
5.1.3.13. Vector
A vector
is a sequence of data
. This maps to a JSON array consisting of
JSON objects (that in turn each have the @data-type
and data
keys
again). For example:
"@data-type": "vector",
"data": [
{
"@data-type": "count",
"data": 42
},
{
"@data-type": "integer",
"data": 23
}
]
5.1.3.14. Set
Sets are similar to vector
, but each object in the list may only appear
once. For example:
"@data-type": "set",
"data": [
{
"@data-type": "string",
"data": "foo"
},
{
"@data-type": "string",
"data": "bar"
}
]
5.1.3.15. Table
Since Broker allows arbitrary types for the key (even a nested table), Broker
cannot render tables as JSON objects. Hence, tables are mapped JSON arrays of
key-value pairs, i.e., JSON objects with key
and value
.
For example:
{
"@data-type": "table",
"data": [
{
"key": {
"@data-type": "string",
"data": "first-name"
},
"value": {
"@data-type": "string",
"data": "John"
}
},
{
"key": {
"@data-type": "string",
"data": "last-name"
},
"value": {
"@data-type": "string",
"data": "Doe"
}
}
]
}
5.1.4. Data Messages
Represents a user-defined message with topic and data.
A data message consists of these keys:
type
Always
data-message
.topic
The Broker topic for the message. A client will only receive topics that match its subscriptions.
@data-type
Meta field that encodes how to parse the
data
field (see Data Representation).data
Contains the actual payload of the message.
Example:
{
"type": "data-message",
"topic": "/foo/bar",
"@data-type": "count",
"data": 1
}
5.1.5. Error Messages
The error messages on the WebSocket connection give feedback to the client if the server discarded malformed input from the client or if there has been an error while processing the JSON text.
An error message consists of these keys:
type
Always
error
.code
A string representation of one of Broker’s error codes. See Section 2.1.5.
context
A string that gives additional information as to what went wrong.
For example, sending the server How is it going?
instead of a valid data
message would cause it to send this error back to the client:
{
"type": "error",
"code": "deserialization_failed",
"context": "input #1 contained malformed JSON -> caf::pec::unexpected_character(1, 1)"
}
5.1.6. Encoding of Zeek Events
Broker encodes Zeek events as nested vectors using the following structure:
[<format-nr>, <type>, [<name>, <args>, <metadata (optional)>]]
:
format-nr
A
count
denoting the format version. Currently, this is always1
.type
A
count
denoting the encoded Zeek message type. For events, this is always1
. Other message types in Zeek are currently not safe for 3rd-party use.name
Identifies the Zeek event.
args
Contains the arguments for the event in the form of another
vector
.metadata
Contains a
vector
of key-value pairs (represented as furthervectors
of size 2) for which the first element is acount
for identification purposes and the second element any supported Broker data type. This vector can be used to attach arbitrary metadata to events.Zeek version 6.0 and up always includes the network time of an event as metadata. The key for a network timestamp is
1
and the data type for the value is atimestamp
.Broker endpoints are free to use counts starting with 200 to identify and exchange metadata of their own choosing. Within a network of Broker nodes, individual endpoints need to agree on the meaning and type of metadata attached to events.
For example, an event called event_1
that has been published to topic
/foo/bar
with an integer argument 42
and a string argument test
without attached metadata would be render as:
{
"type": "data-message",
"topic": "/foo/bar",
"@data-type": "vector",
"data": [
{
"@data-type": "count",
"data": 1
},
{
"@data-type": "count",
"data": 1
},
{
"@data-type": "vector",
"data": [
{
"@data-type": "string",
"data": "event_1"
},
{
"@data-type": "vector",
"data": [
{
"@data-type": "integer",
"data": 42
},
{
"@data-type": "string",
"data": "test"
}
]
}
]
}
]
}
An event including with NetworkTimestamp
metadata event render as follows,
having the args
vector followed by another vector containing the network
timestamp of the event:
{
"type": "data-message",
"topic": "/foo/bar",
"@data-type": "vector",
"data": [
{
"@data-type": "count",
"data": 1
},
{
"@data-type": "count",
"data": 1
},
{
"@data-type": "vector",
"data": [
{
"@data-type": "string",
"data": "event_1"
},
{
"@data-type": "vector",
"data": [
{
"@data-type": "integer",
"data": 42
},
{
"@data-type": "string",
"data": "test"
}
]
},
{
"@data-type": "vector",
"data": [
{
"@data-type": "vector",
"data": [
{
"@data-type": "count",
"data": 1
},
{
"@data-type": "timestamp",
"data": "2023-04-18T14:13:14.000"
}
]
}
]
}
]
}
]
}