file-scrubber

Command Info

Tool to scrub sensitive data from WarpStream formatted data files.

Command Usage

Usage of file-scrubber:
  -detailed-footer
      print detailed information about the footer
  -file-url string
      the file url to scrub

Example

$ warpstream cli file-scrubber --file-url s3://my-s3-bucket/warpstream/vci_2b277e04_f805_45de_8a23_2b5b49028864/00002/0000000000000002

Header:
version: 1
content type: CONTENT_RECORDS

Scrubbed 1 records in page 1 offsets long for stream 1 shard 0
Scrubbed 1 records in 1 batches in this file

Footer:

stream ids: [1]
shard ids: [0]
offsets: []
counts: [1]


batch page counts: [1]
batch page offsets: [0]
batch offsets: [15]


uncompressed size hints: [23]
total size: 0MB




Scrubbing finished, scrubbed file is available at /tmp/file_0000000000000002.scrubbed