Finding public S3 objects

There's lot of concern over public objects in S3 buckets. Now Amazon gives you a way to lock down entire buckets - but what if you legitimately have a mix of public and private objects?

Ruby script to find public S3 objects

First get 2 gems:

source 'https://rubygems.org' do
  gem 'aws-sdk'
  gem 'thread'
end

Then the script itself:

# find_public_s3_objects.rb
require 'aws-sdk-s3'
require 'thread/pool'
BUCKET = ARGV[0] or raise("expected bucket")
s3 = Aws::S3::Resource.new(region: 'us-east-1')
count = 0
pool = Thread.pool 8
mutex = Mutex.new
s3.bucket(BUCKET).objects.each do |object|
  pool.process do
    grants = object.acl.grants
    mutex.synchronize do
      count += 1
      if count % 100 == 0
        $stderr.write "#{count}..."
      end
    end
    if grants.map { |x| x.grantee.uri }.any? { |x| x =~ /AllUsers/ }
      mutex.synchronize do
        puts object.key
      end
    end
  end
end
pool.shutdown

Then you run it like this:

bundle exec ruby find_public_s3_objects.rb my-bucket-name

It's much faster than the bash-based example on StackOverflow.