> ## Documentation Index
> Fetch the complete documentation index at: https://docs.permutive.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Setting Up AWS S3 Streaming Routing

> Configure near real-time event streaming to Amazon S3 with Hive-style partitioning

## Overview

Set up S3 Streaming routing to export your first-party event data to your AWS S3 bucket in near real-time as GZIP-compressed JSONL files. This guide provides context for choosing S3 Streaming and what to expect after setup.

S3 Streaming is the recommended option for new S3 routing integrations.

<Info>
  **Prerequisites:**

  * AWS account with S3 bucket creation permissions
  * Permission to attach a bucket policy to that bucket
  * S3 bucket in the appropriate AWS region
</Info>

<Note>
  You do **not** need to create an IAM user or share AWS access keys with Permutive. Access is granted by attaching a bucket policy that allows a Permutive-owned IAM role to write to your bucket.
</Note>

## When to Choose S3 Streaming

**Best for:**

* Organizations using AWS as their primary cloud provider
* Publishers who need data for a specific workspace rather than the whole organization (configuring a parent workspace also includes its child workspaces)
* Teams needing raw event files for custom processing pipelines
* Organizations requiring data in S3 for ingestion into other AWS services (Athena, Redshift, EMR)
* Organizations preferring file-based data over database connections

**Consider alternatives if:**

* You prefer automatic schema management in a database (consider BigQuery or Snowflake)
* You need immediate SQL query access without additional setup

## Setup Steps

S3 Streaming routing requires coordination with Permutive support.

<Steps>
  <Step title="Create Your S3 Bucket">
    Create a bucket with the following settings:

    * **Bucket name:** must be globally unique across all of Amazon S3
    * **Object Ownership:** ACLs disabled (Bucket owner enforced)
    * **Block Public Access:** Block all public access
    * **Default encryption:** SSE-S3 (`AES256`), the default for new buckets, which requires no additional configuration — see [AWS default bucket encryption](https://docs.aws.amazon.com/AmazonS3/latest/userguide/default-bucket-encryption.html)

    <Note>
      Use a region-specific location (e.g., `us-east-1`, `eu-west-1`) rather than generic regions.
    </Note>
  </Step>

  <Step title="Attach a Bucket Policy">
    Attach a policy to the bucket granting Permutive's IAM role `arn:aws:iam::941252478151:role/permutive-s3-routing` permission to write objects.

    ```json theme={"dark"}
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "PermutiveRoutingWrite",
          "Effect": "Allow",
          "Principal": {
            "AWS": "arn:aws:iam::941252478151:role/permutive-s3-routing"
          },
          "Action": [
            "s3:PutObject",
            "s3:GetObject",
            "s3:DeleteObject"
          ],
          "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/YOUR_PREFIX/*"
        },
        {
          "Sid": "PermutiveRoutingList",
          "Effect": "Allow",
          "Principal": {
            "AWS": "arn:aws:iam::941252478151:role/permutive-s3-routing"
          },
          "Action": "s3:ListBucket",
          "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME"
        }
      ]
    }
    ```

    <Warning>
      If you scope the policy to a prefix, the `Resource` must end with `/*` after the prefix — for example `arn:aws:s3:::my-bucket/permutive/*`. A wildcard does not match across `/`, so `.../permutive*` will reject every write.
    </Warning>

    <Note>
      Attach this at the **bucket** level (S3 console → your bucket → Permissions → Bucket policy), not to an IAM user or role in your own account.
    </Note>
  </Step>

  <Step title="Contact Permutive Support">
    Email [technical-services@permutive.com](mailto:technical-services@permutive.com) with:

    * **Bucket Name**
    * **Bucket Region** (e.g., `us-east-1`)
    * **Bucket Prefix** (optional, e.g., `permutive/` — include the trailing `/`)
    * **Routing Mode:** Streaming

    Confirm that the bucket policy above has been attached.
  </Step>

  <Step title="Setup Completion">
    Permutive will configure your routing instance and notify you when the integration is live.
  </Step>
</Steps>

## Understanding S3 Streaming Data Structure

S3 Streaming uses Hive-style partitioning to organize data efficiently:

### Folder Structure

```
s3://bucket/prefix/
├── type=events/
│   ├── year=2026/
│   │   ├── month=01/
│   │   │   ├── day=15/
│   │   │   │   ├── hour=14/
│   │   │   │   │   └── 2026-01-15T14:00:00.000000Z-abc123-worker1.jsonl.gz
│   │   │   │   └── hour=15/
├── type=sync_aliases/
│   └── year=2026/month=01/day=15/hour=14/...
└── type=segment/
    └── timestamp-hash-worker.jsonl.gz
```

### File Format

* **Format:** Newline-delimited JSON (JSONL)
* **Compression:** GZIP (`.gz`)
* **Extension:** `.jsonl.gz`
* **Encoding:** UTF-8

### Data Types Exported

| Data Type | Description | Partitioned |
| - | - | - |
| `events` | User behavioral events | Yes (hourly) |
| `sync_aliases` | Identity synchronization data | Yes (hourly) |
| `segment` | Segment metadata snapshots | No |

See the [S3 integration documentation](/integrations/data-collaboration/data-warehouses/aws-s3#streaming-schema) for detailed schema information.

## Common Considerations

<Note>
  **Latency:** S3 Streaming has approximately 5-minute latency from event collection to file availability in S3.
</Note>

<Tip>
  **Bucket Prefix:** Use a bucket prefix (e.g., `permutive/`) to organize Permutive data separately from other data in your bucket. The prefix should NOT include a leading `/` or the bucket name, and should end with a trailing `/`.
</Tip>

<Warning>
  **Encryption:** Permutive does not specify an encryption algorithm when writing, so objects are encrypted using your bucket's default encryption setting. SSE-S3 requires nothing further. If your bucket's default is a **customer-managed** KMS key, contact [Technical Services](mailto:technical-services@permutive.com) before setup — our role (`arn:aws:iam::941252478151:role/permutive-s3-routing`) will need `kms:Encrypt` and `kms:GenerateDataKey` on the key, and your own users will need `kms:Decrypt` to read the files. Buckets encrypted with an **AWS-managed** key (`aws/s3`) are not supported: AWS does not permit resources encrypted under an AWS managed key to be shared with other accounts.
</Warning>

## What Happens After Setup

Once routing is active:

1. **Files stream to S3** in near real-time with approximately 5-minute latency
2. **Hive-style partitions** are created automatically by hour
3. **Event data** is written as GZIP-compressed JSONL files
4. **File naming** follows the pattern `{timestamp}-{hash}-{worker_id}.jsonl.gz`

## Next Steps

<CardGroup cols={2}>
  <Card title="S3 Integration" icon="aws" href="/integrations/data-collaboration/data-warehouses/aws-s3">
    View full integration documentation
  </Card>

  <Card title="Back to Routing" icon="arrow-left" href="/products/connectivity/routing">
    Return to Routing overview
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.