use env variables for AWS creds

[pjfanning]

see #832

not yet for merge - contains some build hacks to test the changes in a PR

[pjfanning]

@jtjeferreira the tests have access to the the AWS creds in this PR

The KinesisITTest fails because we don't have the specific AWS permissions to create the Kinesis resources needed for that test.

[jtjeferreira]

The KinesisITTest fails because we don't have the specific AWS permissions to create the Kinesis resources needed for that test.

Can you take care of that?

[pjfanning]

The KinesisITTest fails because we don't have the specific AWS permissions to create the Kinesis resources needed for that test.

Can you take care of that?

Is there any chance that we could start with using localstack or something like that to test with an AWS-like setup for our Kinesis related testing?

I can put some details in #655 and we can follow up on that later. I'd prefer if an ASF volunteer with an AWS account was able to take that task on. It's easier to sort that out by testing with a personal AWS setup and then being able to go to the ASF Infra team with a detailed setup that they can implement for us.

[jtjeferreira]

Is there any chance that we could start with using localstack or something like that to test with an AWS-like setup for our Kinesis related testing?

I was gonna say that we already have that, because I think I wrote something like that in the past, but I might be mixing up. I can do that but it wont help me with this puzzle #830

I'd prefer if an ASF volunteer with an AWS account was able to take that task on. It's easier to sort that out by testing with a personal AWS setup and then being able to go to the ASF Infra team with a detailed setup that they can implement for us.

I have access to an AWS account, but I am not sure what the ASF infra team needs? How was the S3 setup done? Was that discussion public? If yes can you link me to it?