Retry file status (ls) operations, which can fail with FileNotFound due to eventual consistency in S3

[ddaniels]

Fixes issue #309.

[rangadi]

I think the default should be 0.

btw, we can avoid this getFileStatus() call entirely. The output stream can keep track of number of bytes written or just not even bother deleting index file for small lzo files (with a config).

[ddaniels]

Any of those sound good. Which would you prefer?

[rangadi]

Keeping track of number of bytes written sounds best.. (not sure sure what S3 does when you delete file before it actually 'appears').

Otherwise, you could set the default based on filesystem (20 for S3 and 0 for others).

[ddaniels]

Sounds good. I'm going to close this pull request and open a new one with the bytes written strategy.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 2 committers have signed the CLA.

❌ Doug Daniels
❌ MarkRoddy

---

Doug Daniels seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}