Repository navigation
Long scans fail on ExpiredToken when using AWS-IRSA #2255
Description
🐞 Bug report
Describe the bug
When using S3 with auth type of AWS-IRSA, the S3 connection is initiated on Scan creation.
AWS-IRSA credentials are limited to 1 hour, and thus, when a scanner takes over 1 hour to complete, the lurker fails with ExpiredToken error (error code 400).
The S3 connection is only relevant once the scanner container finishes running, but it is initiated on setup.
Steps To Reproduce
- Use S3 with
authType: aws-irsain operatorvalues.yaml. - Run a scan that takes over an hour
- See scanner container completing
- Lurker container fails on
ExpiredToken
Expected behavior
Scanner completes after an hour, lurker uploads file successfully, rest of the flow is executed.
System (please complete the following information):
secureCodeBox version: 4.0.1 (couldn't see anything related to it in newer versions' release notes, related code looks identical)
Kubernetes version:
Client Version: v1.27.4
Kustomize Version: v5.0.1
Server Version: v1.28.4-eks-8cb36c9
Docker version:
Docker version 20.10.18, build b40c2f6b5d
Screenshots / Logs
2024/02/01 06:10:12 File upload returned non 2xx status code (400)
2024/02/01 06:10:12 Failed Request:
2024/02/01 06:10:12 HTTP/1.1 400 Bad Request
Connection: close
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Thu, 01 Feb 2024 06:10:11 GMT
Server: AmazonS3
X-Amz-Id-2: mt...
X-Amz-Request-Id: HV...
5c8
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>ExpiredToken</Code><Message>The provided token has expired.</Message><Token-0>IQ..=</Token-0><RequestId>HV...</RequestId><HostId>mtU5...</HostId></Error>
0
2024/02/01 06:10:12 Lurker failed to upload scan result file. File upload returned non 2xx status code (400)
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
J12934 commentedon Feb 5, 2024
Hi @nierz
Thanks for reporting this.
Also impressive that you got it to work at all as AWS IRSA is barely documented on our side as we still consider the support to be experimental in the operator (in large parts due to this limitation)
Do you have any link confirming this 1 hour expiration?
My current understanding is that the IRSA tokens are limited to 8 hours and are refreshed one hour before their expiry.
This also comes down to (in the worst case) 1 hour of token validity when the presigned url is created right before the token is refreshed, as the presigned url can't be valid any longer than the token.
Is that what you meant with the expires after one hour? Because if the acutal IRSA token would only be valid for one hour this would be a way bigger problem as the tokens then might only be valid for a couple of minutes in the worst case.
My hope was that we could that we could find out a way to tell the golang minio client to refetch the tokens early. e.g. after 4 hours to ensure that the is enough time for the scans.
I'd personally not want to change the time when the presigned urls are created, I don't see a easy way to fit this into the architecture which would still be easy & secure to run.
nierz commentedon Feb 6, 2024
Hi @J12934,
Thanks for your quick, detailed reply!
I might have mixed the 1 hour limit a bit - looks like that's true for
GetSessionTokenspecifically.As for the token caching / refreshing - isn't that only true if the remaining token duration is lower than the
ExpiryWindow, on the NEXT request made? That is, when the initial presigned URL is created, that specific URL will only be valid for the minimum between the duration set for it and the token's duration - assuming no additional requests are issued that would make this token refresh.As a quick workaround - do presigned URLs created when using credentials for S3 (
access-secret-keyauth type, rather thanaws-irsa) last the full 12 hours? The errors I mentioned happened on a scan that took 1 hour and 10 mins.Appreciate the help.
oles-visible commentedon Aug 27, 2026
I am using the same integration, IRSA+S3. Default session for AWS Roles 3600s == 1h. You can extend it with
max_session_duration = NNNHowever, my ZAP scanning takes 20+ min and I am getting the same error. I will try to research.