Downloading results is very slow

[jkleint]

It seems that when running a query in batch mode, the results are downloaded via the API in small chunks, which is very slow. For instance, using athena-cli to --execute a query that returned 19 MB (50000 rows) took 81 sec (about 250k / sec), but the actual Athena finished in 3 seconds and aws s3 cp downloads the results in 1 second.

Any reason you don't just use S3 to fetch the results?

[satterly]

Yes, because the AWS SDK for Athena doesn't work that way.

If downloading results is slow with the athena CLI then you should just download the output directly from the Athena S3 bucket as mentioned here #25 (comment)

[jkleint]

In the case of batch queries, would it be possible to get the url of the result from the Athena SDK, then use the S3 API to fetch?

[satterly]

Yes, I suppose so. PR's welcome.