6 scheduling#36

Merged

wvengen merged 28 commits intoq-m:mainfrom

vlerkin:6-scheduling

Apr 1, 2025

Commits on Mar 4, 2025

extracted resource watching logic into a separate class; implemented logic for observer in a RecourceWatcher class; added method to stop a thread gracefully

vlerkin
authored and
wvengen
committed
add kubernetes scheduler class that subscribes to the event observer that handles the logic to unsuspend jobs and get the next in order according to the creation timestamp; modify schedule endpoint…

vlerkin
authored and
wvengen
committed
finish the observer logic extraction; merge changes from main; add resource watcher instance to enable_joblogs to subscribe to the event watcher if the log feature is configured; delete logic about…

vlerkin
authored and
wvengen
committed
change enable_joblogs method signature in docker

vlerkin
authored and
wvengen
committed
refactor api.py and launcher/k8s.py to keep api.py launcher agnostic

vlerkin
authored and
wvengen
committed
add implementation for docker to check the number of running containers and run more from the queue of created jobs when capacity is available; add backgroung thread that sleeps for 5 sec and trigg…

vlerkin
authored and
wvengen
committed
remove update from RBAC because we perform patching of the resource and patch is sufficient

vlerkin
authored and
wvengen
committed
add number of retry attempts and exponential backoff time to the reconnect loop for event watcher; make number of reconnect attempts, backoff time and a coefficient for exponential growth configura…

vlerkin
authored and
wvengen
committed
create a helper finction _filter_jobs that accepts a filter function and a label selector to make the code in listjobs, get_running_jobs and list_suspended_jobs DRY; refactor listjobs to use the he…

vlerkin
authored and
wvengen
committed
change the signature of the filtering method to also accept a parse function because list jobs uses a different logic

vlerkin
authored and
wvengen
committed
add number of retry attempts and exponential backoff time to the reconnect loop for event watcher; make number of reconnect attempts, backoff time and a coefficient for exponential growth configura…

vlerkin
authored and
wvengen
committed
add logic to reset number of reconnect attempts and backoff time when connection to the k8s was achieved so only sequential failures detected; add exception handling to watch_pods to handle failure…

vlerkin
authored and
wvengen
committed
added a CONFIG.md file with detailed explanations about parameters used to re-establish connection to the Kubernetes wather

vlerkin
authored and
wvengen
committed
move section about config file from README.md to CONFIG.md; add a link to the CONFIG.md in the README.md; remove variables for reconnection_attempts, backoff_time and backoff_coefficient fron the s…

vlerkin
authored and
wvengen
committed
make number of job limitation optional by separating k8s_scheduler in a package that has an enable function in launcher/k8s.py which also part of resource watcher initialization; initialize the sch…

vlerkin
authored and
wvengen
committed
implement the feature to limit running jobs to be optional for Docker; remove max_proc from config file since by default we want to runn all scheduled jobs in parallel; add a section about max_proc…

vlerkin
authored and
wvengen
committed
add exceptions to the KubernetesScheduler class, for the init method, propagate the exception

vlerkin
authored and
wvengen
committed
add custom exceptions to the class for log handling; add exceptions to handle and propagate the critical ones

vlerkin
authored and
wvengen
committed
modify method that fetches nez job id to be unsuspended to add creation_timestamps to the jobs that do not have them, so they are proccessed at the end of the queue; add unit tests for k8s_schedule…

vlerkin
authored and
wvengen
committed
make the proper test structure according to the standards; add tests for docker implementation of the scheduler

vlerkin
authored and
wvengen
committed
rebased to main

vlerkin
authored and
wvengen
committed
change the test test_listversions_ok to check the exact list

vlerkin
authored and
wvengen
committed
add integration tests for scheduler with two different configs

vlerkin
authored and
wvengen
committed

Commits on Mar 5, 2025

Adapt integration testing setup
wvengen
committed

Commits on Mar 11, 2025

grouped unit tests under classes; added parametrization to avoide douplicate code
vlerkin
committed

Commits on Mar 28, 2025

Commits on Apr 1, 2025

edited config to two params, removed those params from the sample config, added an empty line in api.py
vlerkin
committed