Uploading Segments and Bid Models to S3
If a segment file is over 10 Mb in size, the file needs to be uploaded to a Beeswax S3 bucket. To upload your segment files or bid model files to S3, you need to complete the following steps:
1. Provide your Beeswax account representative with your AWS Identity & Access Management (IAM) Amazon Resource Name (ARN). ( Link to AWS documentation). Ensure the ARN isn't for a "group", or for "root", only "user" and "role" ARNs will work.
2. Beeswax will notify you once your permissions have been added to the Beeswax bucket policy.
NOTE: Please do not try uploading the files before receiving notification from your Beeswax account representative as Beeswax will have to update permissions as well. Once notified, you can start writing to your designated S3 folder.
3. Beeswax will provide an ACL policy to apply to your user. It will look like:
{ "Sid": "AllowAssumeBeeswaxRole", "Effect": "Allow", "Action": "sts:AssumeRole", "Resource": "arn:aws:iam::164891057361:role/customer-s3-<user-list or bid-models>-<customer_buzz_key>" }
Note: The placeholder <user-list or bid-models> must be replaced depending on what file type you are uploading. The placeholder <customer_buzz_key> must be replaced with your actual buzz key.
4. You will assume the role from your ACL policy in order to access the Beeswax S3 bucket. ( Link to AWS documentation). We recommend that you create a new profile in ~/.aws/config that uses your AWS user provided in step 1 to assume the specified role. You will use this profile when executing steps 5 and 6.
5. Upload the segment file to the Beeswax S3 path
Example s3 path: s3://beeswax-data-us-east-1/<user-list or bid_models>/<customer_buzz_key>/
Note: Please be sure your file is formatted per the instructions in the Segment Uploads documentation
Note: The placeholder <user-list or bid_models> must be replaced depending on what file type you are uploading. The placeholder <customer_buzz_key> must be replaced with your actual buzz key. If you are uploading files to EMEA or APAC please use the corresponding bucket <beeswax-data-eu-west-1> or <beeswax-data-ap-northeast-1>.
Note: You should be using the --profile option with the name of the profile you created in step 4.
Note: Ensure you have not forgotten the forward-slash at the end when specifying the bucket in your command/code. Omitting this may result in the Access Denied error.
6. Perform the following command via AWS Command Line Interface (CLI) to provide proper permissions to Beeswax for the file:
aws s3api put-object-acl \ --profile <assumeBeeswaxRoleRrofile> \ --bucket beeswax-data-us-east-1 \ --key <user-list or bid_models>/<customer_buzz_key>/file_name.txt \ --acl bucket-owner-full-control \ --region us-east-1
Note: The placeholder <user-list or bid_models> must be replaced depending on what file type you are uploading. The placeholder <customer_buzz_key> must be replaced with your actual buzz key. Please ensure the bucket and region values are replaced when uploading files to EMEA or APAC. The placeholder <assumeBeeswaxRoleRrofile> should be replaced with whatever you named your profile you created in step 4.
curl -X POST "https://<customer_buzz_key>.api.beeswax.com/rest/segment_upload" -b cookies.txt -d '{"segment_file_list":["s3://beeswax-data-us-east-1/user-list/<customer_buzz_key>/file_name.txt"],"account_id":2,"file_format":"DELIMITED","user_id_type":"OTHER_MOBILE_ID","segment_key_type":"DEFAULT"}'
FAQ
Q: Can bid models uploaded to a regional AWS bucket be utilized globally?
A: Yes, when a bid model is uploaded Beeswax will replicate that bid model data and load into every region where the customer has a bidder active.