Finding public S3 objects
There's lot of concern over public objects in S3 buckets. Now Amazon gives you a way to lock down entire buckets - but what if you legitimately have a mix of public and private objects?
Ruby script to find public S3 objects
First get 2 gems:
source 'https://rubygems.org' do
gem 'aws-sdk'
gem 'thread'
end
Then the script itself:
# find_public_s3_objects.rb
require 'aws-sdk-s3'
require 'thread/pool'
BUCKET = ARGV[0] or raise("expected bucket")
s3 = Aws::S3::Resource.new(region: 'us-east-1')
count = 0
pool = Thread.pool 8
mutex = Mutex.new
s3.bucket(BUCKET).objects.each do |object|
pool.process do
grants = object.acl.grants
mutex.synchronize do
count += 1
if count % 100 == 0
$stderr.write "#{count}..."
end
end
if grants.map { |x| x.grantee.uri }.any? { |x| x =~ /AllUsers/ }
mutex.synchronize do
puts object.key
end
end
end
end
pool.shutdown
Then you run it like this:
bundle exec ruby find_public_s3_objects.rb my-bucket-name
It's much faster than the bash-based example on StackOverflow.