Verify endpoint health during scan #729

Activity

1. EndPositive

   added

   enhancementNew feature or requestNew feature or request

   on Oct 14, 2021
2. EndPositive commented on Oct 19, 2021

   

   EndPositive

   on Oct 19, 2021

   Last edited by J12934

   ContributorAuthor

   More actions

   With further investigation, I propose that we reuse the built-in Kubernetes lifeness probes since they supposedly also work on remote targets.

   We could define a health configuration on a scan:

   ...
   spec:
     scanType: nmap
     parameters:
       - example.com
     livenessProbe:
       httpGet:
         host: example.com
         path: /
         port: 443
       initialDelaySeconds: 3
       periodSeconds: 3

   Since a lifenessProbe is attached to a single container, we could do one of the following:

   1. Attach it to lurker (shares complete lifecycle with job)
   2. Attach to scanner container (shares partial lifecycle with job)
   3. Add user-configurable sidecar container such that users can set up exec probes (difficult lifecycle management).

   We should probably also make sure that the job is not restarted automatically by Kubernetes by adding:

   backoffLimit: 0
   restartPolicy: Never

   Any thoughts from core team?
3. J12934 commented on Oct 19, 2021

   

   J12934

   on Oct 19, 2021

   Member

   More actions

   Hi 👋

   That is a interesting proposal.
   What happens in the case that the remote target goes down during the scan is currently undefined / completly dictated by the behavior of the scan tool. My expectations of a good tool would be that they retry / backoff to a certain extend by themself and error when they can't do that in a reasonable timespan.

   If a scan tool does not do that that would be a issue / improvement that should ideally be done in the tool itself. As we integrate a lot 3rd party tools we don't always have influence in these things so a workaround like the one proposed here might be necessary.

   Though I'm not sure using the livenessProbe is a clean option to do it, as these are meant to test the containers themselfs not the remote endpoint (even thought it is technically possible to use them for it).

   What I'm thinking about, might not be possible for every scanner, is to add an additional sidecar to the scanners which talks to the scanners api (e.g. ZAP) and monitors their progress (e.g. how many requests were performed in the last 5 min). These could be a simple python / js / bash script. If the progress has stalled, the sidecar could exit and the pod would be marked as failed and restarted by kubernetes (if allowed by the restart backoff limit).