S3

The S3 plugins allows to stream packets to the given S3 buckets.

Configuration

AWS credentials

Before running PacketStreamer, AWS credentials need to be configured by one of the following ways:

• AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables
• ~/.aws/config file - it can be created by aws configure

The first way might be more convenient when running as root (required when running a sensor).

Configuration scheme

S3 plugin configuration has the following syntax:

output:
  plugins:                         # optional
    s3:
      bucket: _string_
      region: _string_
      totalFileSize: _file_size_   # optional; default: 10 MB
      uploadChunkSize: _file_size_ # optional; default: 5 MB
      uploadTimeout: _timeout_     # optional; default: 1m
      cannedACL: _acl_             # optional; default: Bucket owner enforced

Sensor configuration

If you want to stream locally captured packets from sensor to S3, you can use the following example configuration from contrib/config/sensor-s3.yaml:

output:
  plugins:
    s3:
      region: eu-west-1
      bucket: foo-pcap
      totalFileSize: 10MB
      uploadChunkSize: 5MB
      uploadTimeout: 1m
      cannedACL: bucket-owner-full-control
pcapMode: all

And run PacketStreamer with it:

sudo packetstreamer sensor --config ./contrib/config/sensor-s3.yaml

Receiver configuration

If you want to stream packets from receiver to S3, you can use the following example configuration from [contrib/config/receiver-s3.yaml]

input:
  address: 0.0.0.0
  port: 8081
output:
  plugins:
    s3:
      region: eu-west-1
      bucket: foo-pcap
      totalFileSize: 10MB
      uploadChunkSize: 5MB
      uploadTimeout: 1m
      cannedACL: bucket-owner-full-control

packetstreamer receiver --config ./contrib/config/receiver-s3.yaml