[Feature]: Expose pilot waiting time metric for scheduling (pilot placement - site director) optimization

[mezzeddinee]

Feature Description

Pilot waiting times on sites/queues can vary significantly depending on site load, resource availability, and other factors outside DIRAC control. However, this information is very important for scoring and ranking sites/queues.

It would be valuable to expose pilot waiting time:

PilotWaitingTime = PilotStartTime - PilotSubmissionTime

This metric could be used to score sites/queues based on expected waiting time, where sites with lower predicted waiting time are favored.

In our case, we hope to use this in the context of multi-objective optimization, where sites are favored based on a weighted scoring function combining both expected pilot waiting time and site greenness metrics such as carbon intensity

This would help improve scheduling decisions by balancing turnaround time and environmental impact.

[fstagni]

Hi, I have few critiques on this idea:

• There's no exact relation between jobs and pilots. I think what you care most is how long a job takes to run.
• Apart from few factors, all pilots are the same, they are "disposable". We can send (too) many or just send "enough" to fill the queue
• Simple ML techniques can predict the waiting time, but not the many existing conditions affecting the actual waiting.