-
Notifications
You must be signed in to change notification settings - Fork 208
Description
Follows #2967
dstack can provision new instance into existing fleet when applying a run. Currently, only the run spec is considered when choosing instance offers in that case, and fleet spec applies only to pre-provisioning on fleet apply. This means provisioned instances may contradict the fleet spec. This is a problem especially for elastic fleets (#2967)
Solution: merge fleet and run specs and get offers according to the merged requirements. For most properties, the merged property will be an intersection. For some, such as tags, it may be a union.
process_submitted_jobs can merge run spec with the fleet spec of the chosen optimal fleet. It's a question what offers should be returned in run plan. Currently those are pool offers + run spec offers. If we respect the fleet specs, we should either 1) get and merge offers for all fleets 2) or show offers of the most likely fleet. Option 1 can be slow and complicated, so probably we'll have to go with 2.