# AWS Glue

Tableflow can automatically register tables in Glue and update a table's metadata location to point to the latest snapshot.

### Prerequisites

Tableflow uses the AWS SDK for Go v2 to access Glue, so credentials can be specified using one of the supported ways in the SDK’s [default credentials chain](https://docs.aws.amazon.com/sdk-for-go/v2/developer-guide/configure-gosdk.html#specifying-credentials). To use STS AssumeRole instead, set the `WARPSTREAM_TABLEFLOW_ASSUME_ROLE_ARN` environment variable on the Agent (and optionally `WARPSTREAM_TABLEFLOW_ASSUME_ROLE_DURATION_MINUTES`). In either case, the same identity is used to access Glue and S3.

The Glue integration requires the Agent version to be at least [version 710](/warpstream/overview/change-log#release-v710). Additionally, the Agents must have the appropriate IAM policy for Glue attached. Specifically, the following permissions are needed on the catalogs, databases and tables that you want Tableflow to manage:

* `glue:GetTable`
* `glue:CreateTable`
* `glue:UpdateTable`
* `glue:GetTableVersions`
* `glue:BatchDeleteTableVersion`

The IAM policy should look like the following:

```json
{
  "Version":"2012-10-17",		 	 	 
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "glue:CreateTable",
        "glue:GetTable",
        "glue:UpdateTable",
        "glue:GetTableVersions",
        "glue:BatchDeleteTableVersion"
      ],
      "Resource": [
        "arn:aws:glue:<region>:<account-id>:catalog",
        "arn:aws:glue:<region>:<account-id>:database/<database-name>",
        "arn:aws:glue:<region>:<account-id>:table/<database-name>/<table-name>"
      ]
    }
  ]
}

```

### Configuration

To enable this feature, update the configuration YAML with the following:

```yaml
tables:
    - source_topic: "example_json_logs_topic"
      name: '<glue-table-name>'
      ...
      aws_glue_table_config:
        enabled: { true | false }
        catalog_id: '<glue-catalog-id>'
        database_name: '<glue-database-name>'
      schema:
      ...
```

Make sure that a table name is provided under the `name` option in the table configuration section. This will both be used as the Glue table name and also the name used by Tableflow for the table.

**Required parameters**

`enabled`

Specifies whether the Glue integration should run.

`database_name`

Specifies the database in which to create the table. This database needs to exist already as Tableflow will not try to create one automatically.

**Optional parameters**

`catalog_id`

Specifies the ID of the Data Catalog in which to create the Table. If none is supplied, the AWS account ID will be used.

Note that the `database_name` and `table_name` parameter should match the resources from the IAM policy.